{"id":621,"date":"2011-08-02T14:38:37","date_gmt":"2011-08-02T14:38:37","guid":{"rendered":"https:\/\/www.open.ac.uk\/blogs\/SallyJordan\/?p=621"},"modified":"2011-08-14T19:14:34","modified_gmt":"2011-08-14T19:14:34","slug":"does-a-picture-paint-a-thousand-words","status":"publish","type":"post","link":"https:\/\/www.open.ac.uk\/blogs\/SallyJordan\/?p=621","title":{"rendered":"Does a picture paint a thousand words?"},"content":{"rendered":"<p>One of the things that Matt Haigh looked at when considering the impact of item format (see <a href=\"https:\/\/www.open.ac.uk\/blogs\/SallyJordan\/?p=616\">previous post<\/a>) was whether the presence of a picture made the question easier or harder. He\u00a0started with a\u00a0very simple multiple choice item:<\/p>\n<p>Which of the following is a valid argument for using nuclear power stations?<\/p>\n<ul>\n<li>for maximum efficiency, they have to be sited on the coast<\/li>\n<li>they have high decommissioning costs<\/li>\n<li>they use a renewable energy source<\/li>\n<li>they do not produce gases that pollute the atmosphere<\/li>\n<\/ul>\n<p>All the students received this question, but half had a version which showed a photograph of a nuclear power station. Not surprisingly, the students liked the presence of the photograph. The version with the photograph also\u00a0had a slightly lower item difficulty \u00a0(when\u00a0calcualted by\u00a0either Classical Test Theory and Item Response Theory paradigms), but not significantly so.<\/p>\n<p>When compared with aspects of my work that I&#8217;ve \u00a0already described\u00a0briefly (it&#8217;s the dreaded sandstone question again &#8211; see <a href=\"https:\/\/www.open.ac.uk\/blogs\/SallyJordan\/?p=354\">Helpful and unhelpful feedback : a story of Sandstone<\/a>) it is perhaps surprising that the presence of the photograph does not confuse people and so make the question more difficult. <!--more-->I thought that this was what we found with the Sandstone question, though it is difficult to be sure because we were not doing systematic research at the time, rather just trying to make the questions as good as possible &#8211; and\u00a0this resulted in us altering more than one thing at once (oops!).<\/p>\n<p>The question as used in the presentation of \u00a0S104 <em>Exploring science<\/em> that started in February 2008\u00a0 (known as the 08B presentation) is shown below:<\/p>\n<p><a href=\"https:\/\/www.open.ac.uk\/blogs\/SallyJordan\/wp-content\/uploads\/2011\/08\/Picture1.png\"><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-full wp-image-624\" title=\"Picture1\" src=\"https:\/\/www.open.ac.uk\/blogs\/SallyJordan\/wp-content\/uploads\/2011\/08\/Picture1.png\" alt=\"\" width=\"376\" height=\"556\" srcset=\"https:\/\/www.open.ac.uk\/blogs\/SallyJordan\/wp-content\/uploads\/2011\/08\/Picture1.png 376w, https:\/\/www.open.ac.uk\/blogs\/SallyJordan\/wp-content\/uploads\/2011\/08\/Picture1-202x300.png 202w\" sizes=\"auto, (max-width: 376px) 100vw, 376px\" \/><\/a><\/p>\n<p><a href=\"https:\/\/www.open.ac.uk\/blogs\/SallyJordan\/wp-content\/uploads\/2011\/08\/08b-sandstone-score.png\"><img loading=\"lazy\" decoding=\"async\" class=\"alignleft size-medium wp-image-627\" title=\"08b-sandstone-score\" src=\"https:\/\/www.open.ac.uk\/blogs\/SallyJordan\/wp-content\/uploads\/2011\/08\/08b-sandstone-score-300x200.png\" alt=\"\" width=\"300\" height=\"200\" srcset=\"https:\/\/www.open.ac.uk\/blogs\/SallyJordan\/wp-content\/uploads\/2011\/08\/08b-sandstone-score-300x200.png 300w, https:\/\/www.open.ac.uk\/blogs\/SallyJordan\/wp-content\/uploads\/2011\/08\/08b-sandstone-score.png 675w\" sizes=\"auto, (max-width: 300px) 100vw, 300px\" \/><\/a>This version of the question was attempted by 852 students and had a mean score of\u00a0 1.78. The proportions of students getting the question right at the first, second and third attempt\u00a0is as shown on the left.\u00a0<\/p>\n<p>For the following (October 2008 start &#8211; or 08J) presentation\u00a0we removed the photograph and altered the wording slightly (see below):<\/p>\n<p><a href=\"https:\/\/www.open.ac.uk\/blogs\/SallyJordan\/wp-content\/uploads\/2011\/08\/sandstone-before.png\"><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-full wp-image-630\" title=\"sandstone-before\" src=\"https:\/\/www.open.ac.uk\/blogs\/SallyJordan\/wp-content\/uploads\/2011\/08\/sandstone-before.png\" alt=\"\" width=\"296\" height=\"272\" \/><\/a><\/p>\n<p><a href=\"https:\/\/www.open.ac.uk\/blogs\/SallyJordan\/wp-content\/uploads\/2011\/08\/08j-sandstone-score.png\"><img loading=\"lazy\" decoding=\"async\" class=\"alignright size-medium wp-image-631\" title=\"08j-sandstone-score\" src=\"https:\/\/www.open.ac.uk\/blogs\/SallyJordan\/wp-content\/uploads\/2011\/08\/08j-sandstone-score-300x200.png\" alt=\"\" width=\"300\" height=\"200\" srcset=\"https:\/\/www.open.ac.uk\/blogs\/SallyJordan\/wp-content\/uploads\/2011\/08\/08j-sandstone-score-300x200.png 300w, https:\/\/www.open.ac.uk\/blogs\/SallyJordan\/wp-content\/uploads\/2011\/08\/08j-sandstone-score.png 675w\" sizes=\"auto, (max-width: 300px) 100vw, 300px\" \/><\/a>This version of the question was attempted by 1065 students and the mean score was now\u00a02.08. The proportions of students getting the question right at the first, second and third attempt\u00a0is as shown on the right.\u00a0There appears to be a significant difference between the behaviour of 08B and 08J version of the question, with\u00a0a significantly\u00a0different proportion of students getting the answer right at the first attempt at 08J (so changes to feedback are irrelevant). An inspection of the responses indicated that students were less likely to just write everything they knew about the geology of Dawlish, and instead to answer the question.<\/p>\n<p><a href=\"https:\/\/www.open.ac.uk\/blogs\/SallyJordan\/wp-content\/uploads\/2011\/08\/09b-sandstone-score.png\"><img loading=\"lazy\" decoding=\"async\" class=\"alignleft size-medium wp-image-632\" title=\"09b-sandstone-score\" src=\"https:\/\/www.open.ac.uk\/blogs\/SallyJordan\/wp-content\/uploads\/2011\/08\/09b-sandstone-score-300x200.png\" alt=\"\" width=\"300\" height=\"200\" srcset=\"https:\/\/www.open.ac.uk\/blogs\/SallyJordan\/wp-content\/uploads\/2011\/08\/09b-sandstone-score-300x200.png 300w, https:\/\/www.open.ac.uk\/blogs\/SallyJordan\/wp-content\/uploads\/2011\/08\/09b-sandstone-score.png 675w\" sizes=\"auto, (max-width: 300px) 100vw, 300px\" \/><\/a>The two compounding factors were\u00a0(1) the fact that this change coincided with the introduction of the 20-word filter on word-length (described in more detail in the\u00a0\u00a0<a href=\"https:\/\/www.open.ac.uk\/blogs\/SallyJordan\/?p=354\">Helpful and unhelpful feedback : a story of Sandstone<\/a>\u00a0post) and (2) the fact that the students were from different populations. However the facts that (1) a version of the question very slightly modified from 08J was used in 09B too and that 1007 students got a mean score of 1.98, with the proportion correct at first, second and third attempt (shown left)\u00a0being noticably\u00a0different from that found for 08B but very similar to that found for 08J and (2)\u00a0the lack of variability in the\u00a0performance from another question that (fortuitously!) was on the same iCMA for the 08B, 08J and 09B presentations point towards the possibility that the Sandstone question might have been genuinely easier without the picture.<\/p>\n<p>This is certainly worthy of further investigation.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>One of the things that Matt Haigh looked at when considering the impact of item format (see previous post) was whether the presence of a picture made the question easier or harder. He\u00a0started with a\u00a0very simple multiple choice item: Which &hellip; <a href=\"https:\/\/www.open.ac.uk\/blogs\/SallyJordan\/?p=621\">Continue reading <span class=\"meta-nav\">&rarr;<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[140,143,86],"tags":[142,66,144,360],"class_list":["post-621","post","type-post","status-publish","format-standard","hentry","category-item-format","category-picture","category-question-difficulty","tag-item-format-2","tag-matt-haigh","tag-pictures","tag-question-difficulty"],"_links":{"self":[{"href":"https:\/\/www.open.ac.uk\/blogs\/SallyJordan\/index.php?rest_route=\/wp\/v2\/posts\/621","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.open.ac.uk\/blogs\/SallyJordan\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.open.ac.uk\/blogs\/SallyJordan\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.open.ac.uk\/blogs\/SallyJordan\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.open.ac.uk\/blogs\/SallyJordan\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=621"}],"version-history":[{"count":12,"href":"https:\/\/www.open.ac.uk\/blogs\/SallyJordan\/index.php?rest_route=\/wp\/v2\/posts\/621\/revisions"}],"predecessor-version":[{"id":663,"href":"https:\/\/www.open.ac.uk\/blogs\/SallyJordan\/index.php?rest_route=\/wp\/v2\/posts\/621\/revisions\/663"}],"wp:attachment":[{"href":"https:\/\/www.open.ac.uk\/blogs\/SallyJordan\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=621"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.open.ac.uk\/blogs\/SallyJordan\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=621"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.open.ac.uk\/blogs\/SallyJordan\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=621"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}