The screenshots below show two variants of the same question. I should start by emphasising that the question is only used formatively – it’s on the Maths for Science Practice Assignment.
The two variants behave very differently and the reason for this has very little to do with student’s understanding. 63% of responses to the variant on the right-hand side selected the correct response but only 49% of responses to the variant on the left-hand side selected the correct response . The difference can be explained by the relatively large proportion of responses (12% just below and 14% just above) to the left-hand side variant which gave responses adjacent to the correct one. Students who calculate a value close to the correct one are likely to find their ‘almost right’ answer amongst the available options. In contrast, for the right-hand side variant, the incorrect options are more different from the correct one, so students whose calculated answer is a little way off are more likely to select the correct option as the nearest.
I can remember why I asked this question as a multiple choice question (to do with the difficulty in drawing a tangent and estimating the gradient of a curve) however I don’t think I made the right decision and this is not a question of which I’m proud! I should have asked students to estimate the gradient and give their answer as free-text number. I could then have accepted a range of values, monitored the actual values given and adjusted the range accepted.
There are of some lessons here about the design of multiple-choice questions and about question design in general.