As well as the reading that has sparked my recent posts on Learning Outcomes and Revolution or Evolution?, I’ve been reading articles about multiple-choice questions and about assessing practical work. I’m fairly sure that I’ll be saying more about both of those topics during 2013, if not sooner. But there’s a common theme. Honest, there is.
When you want to assess practical work, it can be very easy to assess the underlying subject knowledge rather than the practical skills, and there are decisions to be taken about whether you want to assess practical experimental skills or report writing skills or both. If you chose to use multiple-choice assessment questions, irrespective of whether you like MCQs or loathe them per se, again it is sensible to stop and think about what you are actually assessing. An interesting paper that I read today:
Martinez, M.E. (1999) Cognition and the question of test item format. Educational Psychologist, 34(4), 207-218.
points out that ‘data bearing on the hypothesis of equivalence between constructed response and multiple-choice questions have been equivocal.’ Some people think the two types of questions can be equivalent, others disagree. For simple items, where a constructed response question is assessing recall, a multiple-choice question is assessing recognition. So although there may be correlation between scores obtained on tests comprising multiple-choice questions and tests comprising constructed response questions, the two tests are unlikely to be actually assessing the same thing.
My common theme is that you need to think carefully about WHAT you want to assess, and check that you are actually assessing this in the tasks that you produce for your students. And I think the easiest way to do this is to think in terms of learning outcomes. What is it that you hope your students have learnt: recall or recognition?; practical skills, report-writing skills or knowledge?