The title of this post is copied from another New Scientist article, this time by Sonia van Gilder Cooke, and published in Issue number 3069 (16th April 2016) on pages 39-41. The article starts “Listening to When I’m Sixty-Four by The Beatles can make you younger. This miraculous effect, dubbed ‘chronological rejuvenation’ was revealed in the journal Pyschological Science in 2011. It wasn’t a hoax, but you’d be right to be suspicious. The aim was to show how easy it is to generate statistical evidence for pretty much anything, simply by picking and choosing methods and data in ways that researchers do every day.”
The article is wider ranging than the one that I’ve just posted about here. However, what is most worrying is that it goes on to point out that dubious results are alarmingly common in many fields of science. The summary of causes of bias includes some things that I suspect I have been guilty of:
- Wishful thinking – unconsciously biasing methods to confirm your hypothesis
- Sneaky stats – using the statistical analysis that best supports your hypothesis
- Burying evidence – not sharing research data so that results can be scrutinised
- Rewriting history – inventing a new hypothesis in order to explain unexpected results
- Tidying up – ignoring inconvenient data points and analyses in the write-up
I will discuss one cause that isn’t explicitly mentioned in the summary, namely our wish to only publish ‘positive’ results, in my next post in this morning’s trilogy:
The article goes on to suggest a number of fixes:
- Pre-registration – publicly declaring procedures before doing a study
- Blindfolding – deciding on a data analysis method before the data are collected
- Sharing – making methods and data transparent and available to others
- Collaboration – working with others to increase the rigour of experiments
- Statistical education – acquiring the tools required to assess data meaningfully