e-assessment (f)or learning

Listening to the radio this morning, my attention was drawn to a new medical test with an accuracy of ‘more than 99%’. I was left thinking ‘well, is that good enough?’ and then ‘so is a marking accuracy of 99% good enough for e-assessment questions?’ (Actually the situation with the medical test is not as simple as I’d thought – the idea is that this test would just be used to give a first indication. That’s better.)

Returning to e-assessment. Well no, on one level 99% accuracy is not good enough. What about the 1%? But the e-assessment is likely to be considerably more accurate than human markers. And students are more likely to be confused by something in the question (which may be as simple as reading an addition sign as a division) than they are to be marked inaccurately by the system. Clearly this is something that we must continue to monitor, and to do our best to improve accuracy of marking (whether by humans or computers). And it matters even in purely-formative use, because if our answer matching is not correct we will not be giving the correct feedback message to the students.

I gave a webinar yesterday in the ‘Transforming Assessment’ series, on “Short-answer e-assessment questions: six years on”. The participants were lovely and I was especially pleased that there were lots of people I didn’t know. There is a recording at http://bit.ly/TA5J2013 if you’ve interested.

The response yesterday was very encouraging, but I remain concerned that more people are not using question types like this. However I rest my case that you need to use real student responses in developing answer matching for questions of the type we have written. That’s fine for us at the OU, with large student numbers, but not necessarily for others.

Then, in the feedback from participants, someone suggested that they would value training in writing questions and developing answer matching. I would so much like to run training like this, but simply don’t have the time.

But, thanks to Tim Hunt, we have another suggestion. I have recently used the Moodle Pattern Match question type to write very much simpler questions, which require a tightly constrained single word answer – like the one shown below.

If you are interested in using Pattern Match, writing questions like this would give a simple way in - and you’d probably get away with developing the questions without the need for student responses beforehand (though I would still monitor the responses you do get).