I have just completed my involvement in a research review which was carried out by the University of Oklahoma. The project involved the development of a set of criteria for evaluating learning management systems. The process was an interesting one as it took place over 3 weeks and involved an expert panel. Each week, as panel members we downloaded a PDF document and gave feedback using an online survey – we didn’t know who the other panel members were and had no contact with them.
In the first week, we were presented with a set of criteria which had been collated from elsewhere and gave feedback on their completeness. In week 2, a revised set of criteria was supplied and we rated each on importance. In week 3, we reviewed the resulting new set of criteria as a complete set. Now we’re awaiting the final results.
Although I felt the process could have benefited from some group interaction between panel members, I thought the process was well designed and managed and could be a very useful model for an organisation looking to undertake evaluation of a system prior to purchase or development. It provided a very structured and focused process for consulting stakeholders – I’ll be able to use my experience as a panel member as a model in the future.
There is more information at the project site www.hqranch.net/ but the surveys etc are not openly available.