e-assessment (f)or learning

At CAA 2012 there were several very welcome papers that addressed ways in which technology (e.g. forums, wikis, blogs) can be used to make assessment more authentic. At first sight (to me, an oldy who has never really ‘understood’ computer games) the final symposium ‘Games and simulations in e-assessment’ seemed to be encouraging a step in the opposite direction. Games don’t seem very authentic! However after listening to the presentations and discussion (and a very interesting presentation from Jean Phua : What do secondary school students think about multimedia science computer assisted assessment?) I think I may have been wrong.

I think the point here is that if students can ‘have a go’ at a simulation of an experiment or whatever at the same time as doing an assignment (and then recheck when they receive feedback), learning will be enhanced. So this is a bit like us at the OU requiring students to use a computer-based activity as part of the assessment of a module (incidentally, I have long been arguing that we should have direct links – in both directions – between our assignments and our activities). And if we can make the activities as authentic as possible (e.g. by using ‘interactive screen experiments’ , based on photographs of a real experiment in progress, rather than simulations) so much the better.

Sadly, the final presentation of S154 Science starts here has now ended. It was a 10-credit module so didn’t fit well with the 30-credit study intensity that is necessary for English students to get funding. But it was a lovely little module – popular with students and tutors alike, and highly effective in preparing students to study the longer S104 Exploring science.

S154′s assessment strategy was written to be complementary to that of S104, so it was at first something of a mystery when a ‘different’ student behaviour was observed on S154. The figures below show the way in which three individual (but typical) students interacted with a lightly-weighted  S154 iCMA. The red dot indicates the date on which the student first engaged with the iCMA and the blue dots indicate subsequent interactions.

‘Student 1′ is typical of many students, on S154 and all other modules – the last-minute merchants! (spot the iCMA’s cut-off date…). ‘Student 2′ is also typical of many students on many modules – the student starts the iCMA and completes the questions that they can; then over the next week or so, they use the feedback to improve on their answers to the other questions.

‘Student 3′ is typical for S154 but not for other modules. How odd I thought! But it turns out that there is a very simple explanation. S154 was an introductory module, with careful scaffolding. When students had completed Chapter 2 they were advised to attempt the first 4 questions on the iCMA; when they had completed Chapter 3 they were advised to attempt the next 2 questions; when they had completed Chapter 4 they were advised to attempt the final 4 questions. And that was exactly what they did! If surprises some people, but I have found on many occasions that students actually do what they think that they are expected to do.

Yesterday morning I particated in a wonderful webinar on Peerwise (http://peerwise.cs.auckland.ac.nz/), led by Paul Denny from the University of Auckland. The more I see of it, the more I am impressed by Peerwise – yesterday I attempted to write questions for myself for the first time, and also reviewed other people’s questions. It was tremendous fun and we all agreed that students would be likely to learn by authoring questions and attempting and reviewing other students’ questions.

Someone asked Paul if has plans to add question types other than multiple choice (the answer is yes, but not many). However this led to an interesting point – Paul explained that for student authoring of questions, multiple choice is good, because they have to think about the distractors. He could be right!