Day 3. eAssessment design. I’ll start thinking about question type soon enough; in the meantime, this post is about how you run the questions. In OpenMark, usual practice is to give students multiple (usually three) attempts at each question, with increasingly directive feedback, as shown in the example below. After a first unsuccessful attempt, students are usually just told that they are wrong; after a second unsuccessful attempt they are given a more detailed hint, wherever possible tailored to the mistake that has been made; after a third unsuccessful attempt they are given a worked solution.
I can’t take credit for this design (Phil Butcher had a lot to do with it) but over the years I have analysed its effectiveness and thought about the theoretical underpinning. The instantaneous delivery of feedback coupled with the opportunity for another attempt, gives students the opportunity to respond to the feedback while is still fresh in their mind. Provided the student understands the feedback and considers it to be relevant to them, we have found this approach to work well. However, it is vitally important that the student understands the feedback and its relevance to them. If they don’t, they can be sent on a ‘wild goose chase’, perhaps believing that their first attempt answer was completely incorrect, not juat wrong in some small detail – for this reason I find myself more and more likely to give targeted feedback after an unsuccessful first attempt. If a student doesn’t understand the feedback they can also lose confidence in the marking software, assuming that the computer has not ‘understood’ their answer.
Moodle questions can also be run in ‘interactive mode’ so that a student has multiple attempts with immediate increasing feedback. In summative use, the student usually gains less credit the more they have been helped. It is also possible to give partial credit for partially correct answers. An alternate approach is to allow the student to break the question down into several constitutent ‘steps’, if they are not able to tackle the problem as a whole – again typically for a smaller amount of credit. This is the approach taken by the use of ‘STEPS’ by Cliff Beevers and others at Heriot Watt University in Edinburgh.
Perhaps the most sophisticated way of using eAssessment questions is in adaptive mode. At its most advanced (and we don’t do this at the OU – yet!) this means that the questions offered to a student depend in some way on their performance on previous questions. What we do already do that the OU is to sometimes give students feedback on the basis of their responses to several questions, in addition to or instead of the question-by-question feedback. So, in a diagnostic quiz, we might combine question scores to tell students about their preparedness in several different areas (e.g. maths, basic English).