{"id":405,"date":"2011-03-17T15:21:42","date_gmt":"2011-03-17T15:21:42","guid":{"rendered":"https:\/\/www.open.ac.uk\/blogs\/SallyJordan\/?p=405"},"modified":"2011-04-27T16:40:29","modified_gmt":"2011-04-27T16:40:29","slug":"can-formative-only-assignments-tell-you-about-student-misunderstandings","status":"publish","type":"post","link":"https:\/\/www.open.ac.uk\/blogs\/SallyJordan\/?p=405","title":{"rendered":"Can formative-only assignments tell you about student misunderstandings?"},"content":{"rendered":"<p><a href=\"https:\/\/www.open.ac.uk\/blogs\/SallyJordan\/wp-content\/uploads\/2011\/03\/pa-q13-v0.png\"><img loading=\"lazy\" decoding=\"async\" class=\"alignleft size-medium wp-image-408\" title=\"pa-q13-v0\" src=\"https:\/\/www.open.ac.uk\/blogs\/SallyJordan\/wp-content\/uploads\/2011\/03\/pa-q13-v0-276x300.png\" alt=\"\" width=\"276\" height=\"300\" srcset=\"https:\/\/www.open.ac.uk\/blogs\/SallyJordan\/wp-content\/uploads\/2011\/03\/pa-q13-v0-276x300.png 276w, https:\/\/www.open.ac.uk\/blogs\/SallyJordan\/wp-content\/uploads\/2011\/03\/pa-q13-v0.png 359w\" sizes=\"auto, (max-width: 276px) 100vw, 276px\" \/><\/a>When I was doing the original analysis of student responses to <em>Maths for Science<\/em> assessment questions, I concentrated on questions that had been used summatively and also on questions that required students to input an answer rather than selecting a response. I reckoned that summative free-text questions would give me more reliable information on student misunderstandings than would be the case for formative-only and multiple-choice questions. Was this a valid assumption?\u00a0I&#8217;ll start by\u00a0considering student responses to a\u00a0question that is similar to the one I have been discussing in the previous posts, but which is in use on the <em>Maths for Science<\/em>\u00a0\u00a0formative-only &#8216;Practice assessment&#8217;. The question is shown on the left. The Practice assessment is very heavily used and students can (and do) re-attempt questions as often as they would like to, being given different variants for extra practice.<!--more--><\/p>\n<p>The first thing to note is that, not surprisingly, this question is less well answered than its equivalents in the summative end-of-module assignments.\u00a0 Of 6732 uses, 1513 (22.5%)\u00a0were correct\u00a0at the first attempt, 1076 (16.0%) were correct at the second attempt and \u00a0527 (7.8%) were correct at the third attempt, leaving 3616 (53.7%) who didn&#8217;t get a completely correct answer, even after three attempts. It is also noteworthy (and common for non-trivial questions in formative-only uses of OpenMark) that there was a high percentage of blank responses and of responses that were repeated at second and third attempt without the correction of previous errors.\u00a0 After filtering out these responses, we\u00a0were left with 13480 responses, of which 3116 (23.1%) were correct.<\/p>\n<p>An analysis of the 100 most commonly given responses (a total set of 8054 responses) enables us to\u00a0identify the following common themes:<\/p>\n<p>Of the 8054 responses, 3111 (38.6%) were marked as correct.<\/p>\n<p>A total of 1652 responses (20.5%) were numerically correct, but gave incorrect units. In more detail:<\/p>\n<ul>\n<li>There were 698 responses (8.7%) which were numerically correct but gave the units as m<sup>-2<\/sup><\/li>\n<li>There were 571 responses (7.1%) which were numerically correct but gave the units as m<sup>2<\/sup><\/li>\n<li>There were 205 responses (2.5%) which were numerically correct but gave the units as m<sup>-1<\/sup><\/li>\n<li>There were 178 responses (2.2%) which were numerically correct but gave the units as W m<sup>-2<\/sup><\/li>\n<\/ul>\n<p>And what of other errors?:<\/p>\n<ul>\n<li>There were 1296 responses (16.1%) which were numerically correct but did not include any units.<\/li>\n<li>There were 435 responses (5.4%) where the student appeared to have rounded incorrectly.<\/li>\n<li>There were 324 responses (4.0%)\u00a0where the student\u00a0had calculated <em>d<\/em><sup>2<\/sup> rather than <em>d,<\/em> i.e.\u00a0they had not taken the square root. One of the most pleasing things that I noticed was the fact that many of these people gave the units as m<sup>2 <\/sup>(which is of course consistent with their numerical answer).<\/li>\n<li>\u00a0There were 234 responses (2.9%) which were numerically correct but who gave the answer to three significant figures (and quite a lot of those who gave three significant figures also omitted the units&#8230;).<\/li>\n<\/ul>\n<p>So what can we conclude? The blank and repeated responses, and also the number of responses that have no units at all, gives some indication that students are not trying so hard on a purely formative assessment. Perhaps this means that we should have less confidence in the results of the analysis of errors, but I still think it has some useful things to tell us.<\/p>\n<p>The fact that m<sup>-2<\/sup> and m<sup>2<\/sup> were the most common incorrect units suggests that\u00a0students \u201cforget\u201d to take the square root of the units, or perhaps they think\u00a0that the square root doesn\u2019t apply to the units.<\/p>\n<p>The fact that giving units of \u00a0m<sup>-2<\/sup>\u00a0was the single most common mistake, and also that m<sup>-1<\/sup>\u00a0was\u00a0on the list, suggests that\u00a0students also have trouble seeing that the reciprocal of a negative power is a positive power. This is consistent with findings from other questions &#8211; students have problems with fractions, reciprocals and powers.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>When I was doing the original analysis of student responses to Maths for Science assessment questions, I concentrated on questions that had been used summatively and also on questions that required students to input an answer rather than selecting a &hellip; <a href=\"https:\/\/www.open.ac.uk\/blogs\/SallyJordan\/?p=405\">Continue reading <span class=\"meta-nav\">&rarr;<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[111,110,7],"tags":[11,369,368,346],"class_list":["post-405","post","type-post","status-publish","format-standard","hentry","category-mathematical-misunderstandings","category-question-analysis","category-student-engagement","tag-formative","tag-mathematical-misunderstandings","tag-question-analysis","tag-student-engagement"],"_links":{"self":[{"href":"https:\/\/www.open.ac.uk\/blogs\/SallyJordan\/index.php?rest_route=\/wp\/v2\/posts\/405","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.open.ac.uk\/blogs\/SallyJordan\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.open.ac.uk\/blogs\/SallyJordan\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.open.ac.uk\/blogs\/SallyJordan\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.open.ac.uk\/blogs\/SallyJordan\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=405"}],"version-history":[{"count":5,"href":"https:\/\/www.open.ac.uk\/blogs\/SallyJordan\/index.php?rest_route=\/wp\/v2\/posts\/405\/revisions"}],"predecessor-version":[{"id":470,"href":"https:\/\/www.open.ac.uk\/blogs\/SallyJordan\/index.php?rest_route=\/wp\/v2\/posts\/405\/revisions\/470"}],"wp:attachment":[{"href":"https:\/\/www.open.ac.uk\/blogs\/SallyJordan\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=405"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.open.ac.uk\/blogs\/SallyJordan\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=405"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.open.ac.uk\/blogs\/SallyJordan\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=405"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}