e-assessment (f)or learning

I’ve talked about students’ difficulties with units before – on 10th and 15th March 2011. In addition to the deeper problems that students encounter, they frequently give the incorrect case when writing abbreviations for units. When they write Kg instead of kg for kilograms in an answer to an iCMA question, I have a certain amount of sympathy (especially since K and k are completely impossible to distinguish in many people’s handwriting – including mine!). However Kg is just plain wrong, so I would be unhappy to give full credit for an answer that included this error. It is really important to give targeted feedback on errors of this type.

Students also get the case wrong when the correct case of a unit is more obvious e.g. they write j instead of J for joules. And they make similar errors in writing the case 0f variables e.g. in answer to a high-stakes question where the correct answer is nRT, more than 1% of responses were only incorret because of case, with many responses of nrt, NRT etc. I find it slightly disappointing that students don’t seem to appreciate that things like this matter (even though we have explained this in the teaching text).

OpenMark e-assessment questions were used for the first time in a little 10-credit module called Maths for Science  that has been running since 2002. I did some analysis years ago into the mistakes that students make, but I’m about to start writing a second edition of the module, so I’m revisiting this work. One of the things that amazed me when I first did the analysis and continues to amaze me now is that students are surprisingly good at rearranging equations. However, they are surprisingly bad at substituting values into equations to give a final result, complete with correct units and appropriate precision. (more…)

If we start from the premise that we want assessment to encourage and support learning, then one measure of the assessment’s effectiveness is better performance on later summative tasks. Mundeep Gill and Martin Greenhow (Gill, M. and Greenhow, M. (2008) How effective is feedback in computer-aided assessments? Learning Media and Technology, 33(3), 207-220) report on work where the introduction of  computer-aided assessment had positive impact (by this measure) in all areas but one.

The problem hinged round the presence of correct units with students’ numerical answers, so we might accept an answer of 10 metres, 10 meters, 10 m, but not 10 M, 10 kg, 10 m s^-1 or just 10. Like most physicists, this is something on which I have extremely strong views – I regard the unit as a crucial part of the answer. The problem is that the answer-matching for many e-assessment systems doesn’t allow you to check for correct values and correct units at the same time (let alone whether the answer has been expressed to the correct precision etc.). Fortunately this is something that OpenMark handles well – we can check numbers, units etc. etc. and give appropriate targeted feedback on any aspect(s) of an answer that are incorrect.

In the system Mundeep and Martin were using this was not possible, so units were provided for ths students outside the input box; all students had to input was a number. Unfortunately, over the two year period of the investigation, students were observed to be more likely to omit units from their written work. This is another one of those unintended consequences – not exactly a positive outcome for e-assessment. Thank you Mundeep and Martin for your honesty in reporting this; others would be advised to take note.