OpenMark e-assessment questions were used for the first time in a little 10-credit module called Maths for Science that has been running since 2002. I did some analysis years ago into the mistakes that students make, but I’m about to start writing a second edition of the module, so I’m revisiting this work. One of the things that amazed me when I first did the analysis and continues to amaze me now is that students are surprisingly good at rearranging equations. However, they are surprisingly bad at substituting values into equations to give a final result, complete with correct units and appropriate precision.
Consider the question illustrated on the left, which is in high-stakes summative use and which exists in a number of variants. For a combined data set of 1639 users of the question, 670 (40.9%) were correct at first attempt, 387 (23.6%) were correct at second attempt, 163 (9.9%) were correct at third attempt, and 419 (25.6%) were still not correct after three attempts. This resulted in a total of 3190 responses, of which just 1220 (38.2%) were completely correct.
The most common problem, as you’ve probably guessed from the title of this post, was with the units. In general, it is not that people omit units entirely; units are given, but sadly they are wrong. The students are trying very hard, but clearly they struggle in this area. I’m struggling too – I fear that my teaching on this can’t be very good, but I don’t know what to do better. Students also have problems getting the number of significant figures correct.
The most common incorrect responses (for all variants of this question) were as follows:
- There were 205 responses (6.4% of all responses) which gave an otherwise correct answer with units of m2.
- There were 111 responses (3.5%) which gave correct units and a numerically correct answer, but expressed to three significant figures.
- There were 80 responses (2.5%) which gave a numerically correct value to two significant figures, but without any units. (Which could either be that they forgot the units, or thought that they cancelled out to give a dimensionless number.)
- There were 49 responses (1.5%) which gave an otherwise correct answer with units of m3.