verso.co.nz

I often hear from educators and others wanting to set up a simple, small-scale e-learning platform to support their activities. While such a site has the potential to be more flexible and adaptable than a large institutional setup, it’s important to start it off on a sound footing to avoid wasting time and money.

My suggestions:

1. Start with and maintain a strong focus on using the web to support learning, not to merely publish information. This should underlie all the decisions you make on technical matters.
2. Don’t spend any money on arranging hosting until you have made decisions about the software platform (eg LMS or CMS) that will best meet your needs. Some platforms require much more robust hosting than others do.
3. Don’t spend any money on a software platform until you have made some decisions about exactly how you are going to use the site to support learning. In fact, once you have done this it’s very likely that you can identify free and open source software (FOSS) that will be very suitable.
4. Get help to identify your needs and make decisions. Not just because you may not have all the knowledge required, but also because an outside perspective is invaluable.
5. Seek out independent help and advice. Talk to lots of people, but be aware that many of the people who will offer advice may not  have an objective understanding of what you want to achieve.
6. Spend money on what matters – apart from ongoing payments for hosting, you probably have a small budget for initial set up. Since it’s very likely you won’t need to spend anything on software (see point 3 above), set aside at least some of your budget for buying the time of someone independent to help you identify needs and make decisions.

To my surprise, I’m still recommending PBWorks to teachers as a good way to get started using a wiki. That’s because the Moodle wiki is still not a very effective tool, and PBWorks is easy-to-use and provides some good features for developing and formatting content. It’s proprietary, of course, so it has to be used with caution, but it’s a good way to get started.

When I’m introducing teachers to the potential of wikis and other web tools, I naturally start by getting them to set up and work with wikis themselves. It seems to me like a set of core skills – how to plan and put together a collection of linked pages. This can be applied in reflective individual writing or as a collaborative exercise.

Here’s a 3-page PDF document on how to get started with PBWorks. It’s covered by the by-nc-sa licence so you are welcome to download and use it as you see fit provided it’s not used commercially and my authorship is attributed.

If you modify or adapt it, please add a comment to this post with a link to the new version.

In a purely social community, collaboration and communication can be very open and unstructured. But when incorporating online community approaches into courses, we normally need to adopt a more structured approach to meet needs of the curriculum and the learners.

Here are some tips for this situation:

1. The start of the activity is important – so welcome people and set a positive tone. Don’t overload people with information – start gently!
2. Focus on participant needs – so exploring and sharing existing knowledge is often a great place to start.
3. Give clear instructions and information about the activity – break the activity into digestible chunks so participants can focus on one thing at a time. Move supplementary information (ie info which is not key to the activity) to subsidiary pages.
4. Use triggers to build engagement – eg provide a video or a link to an online survey, then follow with a directly-related question.
5. Use open questions – eg ‘why do you think…?‘ These are more likely to generate higher-level thinking and in-depth discussion.
6. Be a bit provocative – a trigger or question which is controversial is likely to generate engagement. But avoid topics which are too risky as these can lead to dangerous conflict – if you’re inexperienced as a facilitator this can be hard to deal with.
7. Consider the flow of the activity – it is often preferable to deal with one question at a time, allowing participants to focus on one question at a time. You can start with prior experience or knowledge, then move in a developmental sequence to more in-depth questions. If you pose multiple questions from the beginning, especially with smaller groups, the community focus can be easily dissipated and lose momentum.
8. The end of the activity is important – you should at least farewell participants and thank them for taking part. Consider also how the key points that arose can be summarised and published – true collaboration is not just discussing but generating new and powerful knowledge that should not be lost. You might ask for a volunteer to do this.

Image: Augustin Théodule Ribot: The Conversation

BigBlueButton is an open source tool that provides:

• Text chat
• Webcam support
• Audio discussion
• Shared desktop
• Shared presentation

You don’t need to install anything to check it out, there’s a working demo available:

I did find what seems to be a small bug using Firefox (Mac OS): the placement of minimised tools is not quite right, the thumbnail is pushed too far to the left of the screen and is hard to find. Other than that, it seems to work really well, and it seems to have lots of potential.

What I really like about BigBlueButton:

• It’s very simple – the interface is much more straightforward than (say) Wimba Classroom. Admittedly it appears to have fewer features but it has enough to be a very effective tool for synchronous learning and teaching.
• There’s a Moodle integration available – that’s a big plus that should ensure its uptake.
• The moderator and presenter roles are clearly defined. My first impression was that ‘viewers’ had not enough control and were limited to passive learning, but it seems easy to share control of the various tools and enable active learning modes.
• There are clear tutorials and help files available.
• Did I mention it’s open source? And the authors seem to be doing everything right with the project: there’s a published API and open issue tracking and version control.

Given its simplicity of use, I think learners will have fewer problems in using it than they currently do with some of the proprietary systems. Overall, this seems a great tool and one I’ll be following closely.

The purpose of evaluation

A key question for anyone managing or facilitating an online community is how to make it sustainable. Sustainable communities need to maintain (and grow) an active and engaged membership. Structured online events or activities can play a very important role in engaging community members and ensuring their regular and active involvement. However, these events must be effective – badly planned and/or facilitated events can turn members off and lead to the failure of the community.

The facilitator’s overall impression of whether an event is effective or not is useful. But without a more rigorous evaluation, we can miss underlying issues which have the potential to damage members’ ongoing engagement in the community. So some kind of evaluation process is vital to its ongoing success and sustainability.

This article focuses on evaluating community events: not to check that they meet a minimum standard, but as a way of engaging in a process of ‘continuous improvement’.

The evaluation process

Many educational and management approaches employ cyclical models involving reflection or evaluation – eg Kolb’s experiential learning cycle and the action research cycle. The PDSA Cycle shown here represents the stages of Deming’s approach to quality improvement in business, but can be adapted to provide a model useful in relation to online community events and activities: PLAN: plan the community event DO: facilitate the community event STUDY: evaluate the event ACT: feed the evaluation results back into further community development

Carrying out the evaluation

Most of the evaluation will take place as part of the STUDY phase. But during the event (the DO phase) the facilitator should keep notes on what’s going well and what isn’t. If it’s not too big a group, it can also be really helpful to keep notes on the level of engagement of each member – not as a form of assessment, but as data that may be useful later.

During the STUDY phase:

Decide how you will gather feedback

Without its members, the community does not exist. So feedback from participants in an event is an essential component of evaluation:

• Online tools such as surveymonkey or polldaddy are quick to complete, so community members are usually happy to complete a survey. But make the survey brief, and tell them how long it’ll take.
• Individual interviews: you may get much more meaningful information about the event’s effectiveness if you personally approach participants. This may be a phone call or skype, or using an asynchronous method such as email. Ideally, a neutral 3rd party will gather the data, since participants may be unwilling to open up to the facilitator. You may be able to set up a reciprocal arrangement with another facilitator to gather data from each other’s participants.
• Collaborative feedback: you could set up a wiki that participants can use to record their feedback. Or you could set up a forum or a synchronous discussion space that participants use to discuss the event. Ideally this would be anonymous, so you may appoint one member to gather the raw data and provide you with a summary. Again, you may be able to arrange with another facilitator to manage this and gather feedback.

Develop evaluation questions and tools

• Communicate clearly what you are evaluating – point the participants back to the activity if possible so they can reflect on it. Be specific – make sure the questions clearly identify aspects of the activity that you want feedback on.
• Focus on how well the activity met their needs, not just on how much they enjoyed the process. If possible, ask for feedback at the higher levels of Kirkpatrick’s model – eg has the activity had a positive result, has it made a difference, have they been able to apply what they learned during the activity?
• Focus separately on the design of the activity and its facilitation: they are distinct, and the effectiveness of each is essential.
• Include questions that are open and qualitative so you can find out why things happened the way they did.

Gather feedback from community members.

• Tell participants how you will make use of the feedback: if you focus on improvement and they believe you are sincerely interested in making things better next time, they are more likely to engage in the evaluation process.
• Gather feedback not just from those that took part in the event, but also from those who chose not to. Asking those who didn’t take part why they didn’t can tell you a lot about the design of the event and the way it was communicated to community members!

Reflect and evaluate
Feedback from participants is just one form of information on which to base the evaluation. Your own reflection is another essential component.

• You may want to reflect on the activity before you gather data from participants – that way your own thoughts won’t be overly influenced by feedback. But once you have gathered the data, that’s a chance to reflect on and learn from the feedback from participants.
• In your reflection, avoid placing blame on participants. Assume you can do better and learn from mistakes. If people go wrong, ask yourself how could I have communicated more clearly?
• Use your own recollections and notes from during the activity to triangulate – compare them with what participants have told you about the activity, what happened and why.
• Focus on improvement – even where you think the activity was effective, try to identify specific things you could do to make it better next time.
• Share your evaluation with the participants – it’s a community, right? Sharing the evaluation means it becomes part of the larger community collaboration and conversation, and can enhance member commitment. But… if there are comments or conclusions that you feel are private or could offend members, leave them out of the published version.

Image: A Midnight Modern Conversation by William Hogarth.

Bibliography

Davies, C (nd). Kolb Learning Cycle Tutorial. Downloaded 30 May 2010 from http://www.ldu.leeds.ac.uk/ldu/sddu_multimedia/kolb/static_version.php

Unknown author, Business Performance Ltd (nd). Why Measure Training Effectiveness? Downloaded 30 May 2010 from http://www.businessperform.com/workplace-training/evaluating_training_effectiven.html

Unknown author, Carpenter group (nd). The Deming Cycle. Downloaded 30 May 2010 from http://www.quality-improvement-matters.com/deming-cycle.html

Unknown author, Warwick University (2008). Action Research. Downloaded 30 May 2010 from http://www2.warwick.ac.uk/services/ldc/resource/evaluation/tools/action/