e-assessment (f)or learning

PMatch is a new Moodle question type (based on OpenMark’s pattern matching question type that is currently in use at the Open University for the short-answer free text questions that I have written). There is more information here.

Follow the links and you’ll see that it is pretty simple and, despite the fact that I have no computer programming experience, I found it easy to use. If you wanted to, you could use PMatch simply to look for keywords – however, for most of the questions that I have written that would not be sufficient. Word order and negation can be very important, especially when students really do give answers that are ‘opposite’ to a correct one, as in the examples that follow. So, in one question, you need to be able to mark ‘kinetic energy is converted to gravitational potential energy’ (and synonyms) as right but ‘gravitational potential energy is converted to kinetic energy’ (and synonyms) as wrong. In another question, you need to be able to mark ‘the forces are balanced’ and ‘There are no unbalanced forces’ as right (note that students very often use double negatives in their answers) whilst marking ‘The forces are not balanced’ and ‘The forces are unbalanced’ as wrong.

(more…)

At CAA 2012 I gave a paper with the title ‘Short-answer e-assessment questions : five years on’  in which I discussed OU work in this area. There was a  lot of interest in what I said, especially concerning evaluation findings. However I wanted to get a discussion going on the reasons why more people don’t use assessment items of this type, and this didn’t really happen. So I’m trying again here. (My CAA 2012 paper is at Open Research Online if you want more background information.)

(more…)

Last week I attended the International Computer Assisted Assessment Conference in Southampton. This is the third consecutive year I’ve attended this conference and I enjoyed it, even if it was sometimes challenging to the point of being depressing.

So what is there to be depressed about?  Bobby Elliott from the Scottish Qualifications Agency said ‘CAA2002 would be disappointed in CAA 2012′ – not because of the conference itself, but because computer aided assessment has not achieved as much as was hoped 10 years ago.  Sue Timmis from the University of Bristol summed up the problem by saying by saying that, in reviewing the literature relating to the use of digital technologies in assessment, she and colleagues have not yet found evidence of a transformative effect. Steve Draper from the University of Glasgow and the Keynote speaker, raised another issue in saying that there is not much evidence of the effectiveness of feedback given from tutor to student.

So, on one level, has all of our work been a waste of time? I think I’d be slightly more optimistic if only because most of the conference attendees were interested in these issues, rather than talking about a wish to use technology whether or not that is the best solution from the students point of view. So at least our focus is on learning and teaching and we are looking for evidence of effectiveness rather than sailing on regardless – now we just have to get it right!

One good thing that came out of the conference is that John Kleeman told me about his Assessment Prior Art wiki – do take a look.