e-assessment (f)or learning

Last week I participated in a fascinating Transforming Assessment webinar on ‘The semi-automatic generation of assessment items: objectives, challenges, and perspectives’ . The presenter was Muriel Foulonneau from the Henri Tudor Research Centre, Luxembourg, and I was left feeling very much an amateur in a ‘room’ full of very clever professionals. The way in which assessment items and distractors are automatically generated is really very clever – with my ‘day job’ and so many wide-ranging interests in assessment, I feel I will never be able to keep up with what is going on.

However I was absolutely amazed to discover that one of the drivers for this work is the cost of producing test items – $1500-$2500 per item. How so? We are only talking about simple multiple-choice items here, how can it possibly cost that much?

In describing a presentation by Margit Hofler of the Institute for Information Systems and Computer Media at Graz University of Technology, Austria, the CAA 2011 Conference Chair Denise Whitelock used the words ‘holy grail’ and this is certainly interesting and cutting-edge stuff.  The work is described in the paper ‘Investigating automatically and manually generated questions to support self-directed learning’ by Margit and her colleagues at http://caaconference.co.uk/proceedings/

An ‘enhanced automatic question creator’ has been used to create questions from a piece of text, and the content quality of 120 automatically created test items has been compared with 290 items created by students. (more…)