{"id":1545,"date":"2014-01-31T07:14:07","date_gmt":"2014-01-31T07:14:07","guid":{"rendered":"https:\/\/www.open.ac.uk\/blogs\/SallyJordan\/?p=1545"},"modified":"2014-01-31T07:14:07","modified_gmt":"2014-01-31T07:14:07","slug":"victorian-clergymen","status":"publish","type":"post","link":"https:\/\/www.open.ac.uk\/blogs\/SallyJordan\/?p=1545","title":{"rendered":"Victorian clergymen"},"content":{"rendered":"

\"\"<\/a>This is more ‘rant of the day’ than ‘quote of the day’ but I’d like to start with a quote from my own ‘Maths for Science’ (though I’m indebted to my co-author Pat Murphy who actually wrote this bit):<\/p>\n

” It is extremely important to appreciate that even a statistically significant correlation between two variables does not<\/em> prove that changes in one variable cause<\/em> changes in the other variable.<\/p>\n

Correlation does not imply causality.<\/strong><\/p>\n

<\/strong>A time-honoured, but probably apocryphal, example often cited to illustrate this\u00a0point is the statistically significant positive correlation reported for the late 19th Century between the number of clergymen in England and the consumption of alcoholic spirits. Both the increased number of the clergymen and the increased consumption of spirits can presumably be attributed to population growth (which is therefore regarded as a ‘confounding variable’) rather than the increase in the number of clergymen being the cause<\/em> of the increased consumption of spirits of vice versa.”<\/p>\n

Jordan, S., Ross, S. and Murphy, P. (2013) Maths for Science<\/em>. Oxford: Oxford University Press. p. 302.<\/p>\n

Now, my fellow educational researchers, have you understood that point? Correlation does not imply causality. In the past week I have read two papers, both published in respectable peer-reviewed journals and one much cited (including, I’m sad to say, by one publication on which I’m a co-author), which make the mistake of assuming that an intervention has been the cause of an effect.<\/p>\n

In particular, if you offer students some sort of non-compulsory practice quiz, those who do the quiz will do better on the module’s summative assessment. We hope that the quiz\u00a0has \u00a0helped them, and maybe it has – but we can’t prove this fact\u00a0just from the fact that they have done better in a piece of assessed work.\u00a0\u00a0What we mustn’t forget that it is the keen, well motivated, students who do the non-compulsory activities – and\u00a0 these students are more likely to do well in the summative assessment, for all sorts of reasons (they may actually have\u00a0studied the course\u00a0materials\u00a0for a start…).<\/p>\n

One of the papers I’ve just mentioned tried to justify the causal claim by saying that the quiz was particularly effective for “weaker” students. The trouble is that a little investigation showed me that this claim made the logical inconsistency even worse! Firstly it assumed that weaker students are less well motivated. That may be true, but no proof was offered. Secondly, I was puzzled about where the data came from and discovered that the author was using score on the first quiz that a student did, be that formative or summative, as an indicator of student ability. But students try harder when the mark counts and their mark on a summative assignment is very likely to be higher for that reason alone. The whole argument is flawed. Oh dear…<\/p>\n

I am deliberately not naming the papers I’m referring to, partly because I’m not brave enough and partly because I fear there are many similar cases out there. Please can we all try a little harder not to make claims unless we are sure that we can justify them.<\/p>\n","protected":false},"excerpt":{"rendered":"

This is more ‘rant of the day’ than ‘quote of the day’ but I’d like to start with a quote from my own ‘Maths for Science’ (though I’m indebted to my co-author Pat Murphy who actually wrote this bit): ” … Continue reading →<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[299,31],"tags":[300,301,349],"class_list":["post-1545","post","type-post","status-publish","format-standard","hentry","category-research-methods","category-statistics","tag-causality","tag-clergyman","tag-statistics"],"_links":{"self":[{"href":"https:\/\/www.open.ac.uk\/blogs\/SallyJordan\/index.php?rest_route=\/wp\/v2\/posts\/1545","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.open.ac.uk\/blogs\/SallyJordan\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.open.ac.uk\/blogs\/SallyJordan\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.open.ac.uk\/blogs\/SallyJordan\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.open.ac.uk\/blogs\/SallyJordan\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=1545"}],"version-history":[{"count":1,"href":"https:\/\/www.open.ac.uk\/blogs\/SallyJordan\/index.php?rest_route=\/wp\/v2\/posts\/1545\/revisions"}],"predecessor-version":[{"id":1547,"href":"https:\/\/www.open.ac.uk\/blogs\/SallyJordan\/index.php?rest_route=\/wp\/v2\/posts\/1545\/revisions\/1547"}],"wp:attachment":[{"href":"https:\/\/www.open.ac.uk\/blogs\/SallyJordan\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=1545"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.open.ac.uk\/blogs\/SallyJordan\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=1545"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.open.ac.uk\/blogs\/SallyJordan\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=1545"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}