e-assessment (f)or learning

I’m still thinking about guidelines for checking questions. Except this is a guideline for what not to do…

My husband has been checking some Moodle questions for colleagues today and has mentioned two things to me. First of all he told me that when he’s checking the questions there’s a ‘helpful’ button he can press to reveal the correct answer. I’m sure it is meant to be helpful. But if we want the best questions for our students, I’m afraid I don’t think this is it. It encourages checkers to be lazy and to just check that the question ‘seems right’. Maybe I’m being mean, but I’d prefer checkers to be forced to attempt all questions as if they were a student.

The second thing Richard told me was about a problem he’d found in a particular question – and I doubt he’d have found it if he’d used the ‘helpful button’. I suspect that the question author has done just that. It’s a drag and drop question where you have to fill in the blanks in a sentence. The problem is that, unless you are being incredibly pedantic, some of the dragable terms meant as an option for one place in the sentence are  interchangeable with those meant for elsewhere, and if you get the wrong one e.g. ‘lower than’ instead of  ‘less than’ you get marked wrong, with no targeted feedback. This is the sort of thing that gives interactive computer-marked assessment a bad name.

At last week’s Quality Enhancement Seminar, I was asked for guidance on checking interactive computer-marked assignment (iCMA) questions and for guidance on writing questions that are easy to check.

We have detailed but rather OU- and OpenMark-centric guidelines for question checkers (if anyone from the OU would like a copy, please email me) but this has made me think about the more general points.

Firstly, checking iCMA questions is vitally important and I think it is reasonable that this might take as long as writing the questions in the first place. Checking all assessment material is important, but with tutor-marked assignments, you don’t need to check the answer matching, and there are tutors to mediate any unexpected answers. Not that this should be an excuse for not doing everything you can to make the questions as good as they possibly should be before students see them. There are two general points that apply to checking both computer-marked and tutor-marked questions : (1) try the questions for yourself as if you were a student; (2) if possible get someone else to check your questions too – a new pair of eyes will often see ambiguities etc. that you have missed. (more…)