![\"\"](\"https:\/\/www.open.ac.uk\/blogs\/SallyJordan\/wp-content\/uploads\/2012\/02\/Capture1.png\" "\\"Capture\\"")
<\/a>For the \u00a0question shown (which is one of a series of linked questions on the Maths for Science <\/em>\u00a0formative-only practice assessment), 62% of students are right at the first attempt but 22% remain incorrect after the allowed two responses. At the response level, whilst 60.2% of responses are correct, the other options are selected approximately equal numbers of time. The details are below:<\/p>\nP<\/em>>0.1\u00a0\u00a012.4% of responses<\/p>\n 0.1>P<\/em>>0.05 14.0% of responses<\/p>\n 0.05>P<\/em>>0.01 60.2% of responses<\/p>\n P<\/em><0.01 13.5% of responses<\/p>\n So what’s this saying? I think it is reasonable to assume that many of the 12.4%-14.0% responses which\u00a0selected the incorrect option were guesses. It is also then reasonable to assume that a similar number of responses which selected the correct option were also guesses. So overall, it is possible that as many as 50% of all responses were guesses (and most of the rest were right). And you wonder why I don’t like the use of multiple-choice questions without some way to discourage guessing!<\/p>\n In ‘e-assessment (f)or learning’, I have described two things which seem to imply less than full engagement with a question. The first of these (which occurs in selected response type questions, like multiple-choice) is as described here i.e.incorrect distractors being selected an almost equal – and large – number\u00a0of times.\u00a0The second is blank and repeated responses, as reported in a previous post.<\/a> This occurs mostly for constructed response (i.e. free-text) type questions – so when students can’t guess,\u00a0some of them \u00a0just don’t bother. Not great.<\/p>\n Not surprisingly,\u00a0both of these effects occur more for formative-only questions; indeed they are quite rare for summative questions.\u00a0The effects\u00a0are also more common for difficult questions and for questions that require more effort (e.g. doing a complex calculation, accessing a calculator, filling in multiple boxes in one question).<\/p>\n Students say that even if they don’t engage seriously with the question, they still use the feedback provided. But do they? The problem is, we don’t really know.<\/p>\n","protected":false},"excerpt":{"rendered":" Depressingly, this post\u00a0reports a similar finding to the last one. For the \u00a0question shown (which is one of a series of linked questions on the Maths for Science \u00a0formative-only practice assessment), 62% of students are right at the first attempt … Continue reading