{"id":667,"date":"2011-08-16T09:07:14","date_gmt":"2011-08-16T09:07:14","guid":{"rendered":"https:\/\/www.open.ac.uk\/blogs\/SallyJordan\/?p=667"},"modified":"2012-01-18T09:19:27","modified_gmt":"2012-01-18T09:19:27","slug":"the-testing-effect","status":"publish","type":"post","link":"https:\/\/www.open.ac.uk\/blogs\/SallyJordan\/?p=667","title":{"rendered":"The testing effect"},"content":{"rendered":"

This will be my final post\u00a0that picks up a\u00a0theme\u00a0from CAA 2011 <\/a>, but\u00a0the potential implications of this one are massive.\u00a0For the past few weeks I have been trying to get my head around the significance of the ideas I was introduced to by John Kleeman’s presentation ‘Recent cognitive psychology research shows strongly that quizzes help retain learning’. I’m ashamed to admit\u00a0that the ideas John was presenting were mostly new to me. The ideas\u00a0echo with a lot of what we do at the UK Open University in encouraging students to learn actively, but they go further. Thank you John!<\/p>\n

John has written more on the Questionmark blog – go to http:\/\/blog.questionmark.com\/<\/a>\u00a0and search for ‘Roediger’; much of the recent work has been done Professor Roddy Roediger and his team at Washington University in St Louis (for\u00a0a complete bibliography of Professor Roediger’s work see http:\/\/psych.wustl.edu\/memory\/publications\/<\/a>). \u00a0I’ve done some reading around the subject, but still have a lot more to do, so what follows is just a summary of the points that have struck me so far:<\/p>\n

Retrieval practice i.e. practising retrieving something (as in an online test) aids long-term retention more than further studying.<\/p>\n

Thus, the assumption that\u00a0underpins\u00a0much summative\u00a0testing, that testing \u00a0is ‘neutral’ (measuring a students learning without affecting that learning) is false.<\/p>\n

This is not a newly discovered effect. I’ve found references to the work of\u00a0 Gates (1917), Jones (1923) and Spitzer (1939). And apparently Aristotle said ‘Exercise in repeatedly recalling a thing strengthens the memory’.<\/p>\n

The effect has been found both in the pyschological laboratory (in impressively controlled experiments) and in the classroom, and for students of all ages.<\/p>\n

The effect persists even after allowing for the fact that testing provides extra exposure to course content, applies to the learning of concepts as well as facts, and the testing effect appears to be more effective than getting students to do a concept mapping exercise after studying (Karpicke and Blunt, 2011). The effect also persists when the questions asked in the final test are different (in both format and content) from the questions used in the intermediate test. And although early work only looked at the impact on retention a few hours or days after the intermediate test, more recent work has found the same effect after a much longer time period (McDaniel et al, 2007).<\/p>\n

In general, it appears that short answer tests are more effective than multiple choice tests, and that providing feedback is helpful. Several authors introduce the importance of feedback as a means of stopping students from remembering distractors (delightfully called ‘lures’ by American authors) in multiple choice questions. However the findings here are slightly confusing –\u00a0it seems that feedback is actually more useful when used with short answer questions (perhaps because students are less likely to get them right)\u00a0and one recent paper (Butler & Roediger, 2007) reports that feedback made no difference. Another paper (Kang, McDermott & Roediger, 2007) appears to report that, without feedback, multiple choice questions are at least as effective as short answer questions.<\/p>\n

The reasons for the effectiveness of the testing effect are also a bit confusing, at least for a non-psychologist\u00a0like me. It appears that it’s the effort that goes into retrieval that’s important (which is why short answer questions are more effective than multiple choice questions) – a concept of ‘desirable difficulty’ is introduced. \u00a0But\u00a0 I struggle a bit to see how the role of feedback (where ‘feedback’ here seems to mean giving the correct answer) fits into this argument.<\/p>\n

So I’m still thinking around the details – and work is ongoing e.g. looking at the optimum number of intermediate tests and the spacing between them. But the basic effect and its implications for educational practice are mind blowing and some of the areas that are less clear cut may have links to other aspects of my work (e.g. use of short answer free-text e-assessment questions, investigation into what students actually do with the feedback we provide). It’s exciting stuff.<\/p>\n

Since this is my final reflection on CAA 2011, I’ll close with a photograph of me looking rather intense at\u00a0the conference,\u00a0with Matt Haigh, John Dermo and Simon Cross and others. Until next year!<\/p>\n

\"\"<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"

This will be my final post\u00a0that picks up a\u00a0theme\u00a0from CAA 2011 , but\u00a0the potential implications of this one are massive.\u00a0For the past few weeks I have been trying to get my head around the significance of the ideas I was … Continue reading →<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[133,158],"tags":[134,32,159,160,381],"class_list":["post-667","post","type-post","status-publish","format-standard","hentry","category-conferences","category-testing-effect","tag-caa-2011","tag-caa-conference","tag-john-kleeman","tag-roddy-roediger","tag-testing-effect"],"_links":{"self":[{"href":"https:\/\/www.open.ac.uk\/blogs\/SallyJordan\/index.php?rest_route=\/wp\/v2\/posts\/667","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.open.ac.uk\/blogs\/SallyJordan\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.open.ac.uk\/blogs\/SallyJordan\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.open.ac.uk\/blogs\/SallyJordan\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.open.ac.uk\/blogs\/SallyJordan\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=667"}],"version-history":[{"count":9,"href":"https:\/\/www.open.ac.uk\/blogs\/SallyJordan\/index.php?rest_route=\/wp\/v2\/posts\/667\/revisions"}],"predecessor-version":[{"id":808,"href":"https:\/\/www.open.ac.uk\/blogs\/SallyJordan\/index.php?rest_route=\/wp\/v2\/posts\/667\/revisions\/808"}],"wp:attachment":[{"href":"https:\/\/www.open.ac.uk\/blogs\/SallyJordan\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=667"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.open.ac.uk\/blogs\/SallyJordan\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=667"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.open.ac.uk\/blogs\/SallyJordan\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=667"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}