We’ve now had two more talks as part of the OU Institute for Educational Technology’s ‘Refreshing Assessment’ series. First we had Lester Gilbert from the University of Southampton on ‘Understanding how to make interactive computer-marked assessment questions more reliable and valid: an introduction to test psychometrics’. Then, yesterday, Don Mackenzie from Professional e-Assessment Services (which I think is a University of Derby spin-off) , with the title ‘From trivial pursuit to serious e-assessment: authoring and monitoring quality questions for online delivery’
Both were thoughtful, echoing some of my own findings and views, giving me good ideas for things that we could do at the OU, and challenging some of my entrenched ideas. Don showed some questions that we could (technically) deliver easily, but doing things slightly differently – and lots of his examples were from geology, which is an area where our questions could be better. I particularly liked the idea of getting students to draw a line on a figure to represent direction of dip and the idea of using a simple multiple-response question (with lots of options for selection) requiring students to pick correct and relevant points for inclusion in an essay.
Both speakers touched on issues of reliability and validity and it has made me realise that some of my ‘anti psychometrics’ views are perhaps a little unfair. Clearly I am still a beginner in this area, and of course my own use of e-assessment, primarily for its function in supporting learning, means that I have worried about its summative role rather less than others have done. So I’m back to remembering the need to stop and think ‘why’ at every stage of the assessment design process.