Poor quality assessment – inescapable and memorable

David Boud famously said ‘Students can, with difficulty, escape from the effects of poor teaching, they cannot (by definition if they want to graduate) escape the effects of poor assessment.’

Boud, D. (1995) Assessment and learning: contradictory or complementary? In P. Knight (ed) Assessment for learning in higher education. Kogan Page in association with SEDA. pg 35.

Poor assessment is also memorable. I was reminded of this when reading Rick Trebina’s  ‘Lateral thoughts’ in the August 2011 issue of Physics World. The story is best reported in Rick’s own words – he tells it better than I could:

Because healthy employees are more likely to be safe employees, another aspect of [safety agenda at the lab where he worked] was a computerized multiple-choice, fill-in-the-bubbles-with-a-number-two-pencil health test. I took this test enthusiastically. But a few weeks later when my test results arrived, I was shocked to learn that they declared me pathologically unhealthy. Apparently, I was at high risk for a broad spectrum of decidedly unhealthy events, including auto accidents, heart attacks, strokes, cirrhosis of the liver and a disastrous personal life.

It turned out that all of these conclusions followed from my answer to a single test question: “How many glasses of wine do you drink per day?” Because I drink less than three glasses a week, I had rounded to the nearest integer and filled in the “zero” bubble in the column of ones-digit bubbles. But I had unthinkingly left all the tens-digit bubbles blank, thus leaving the computer to decide which tens digit to ascribe to my wine drinking. And unfortunately, it chose the “nine” bubble, recording me as drinking, not 0 glasses of wine per day, but 90.

The test-result memo went on to list numerous classes I would be required to take to deal with my severe alcoholism and the many dangers I obviously posed to myself and others. It also suggested that, in the meantime, perhaps I could begin to rehabilitate myself by cutting my daily wine consumption to a mere 45 glasses.

Fortunately, when I pointed out the error, I was allowed to skip the courses. But the management regretted that removing the test results from my medical file would violate their employee-health-testing-procedure, so the results would have to remain on my file for all time.

Rick goes on to say that he decided at the time that this was a battle not worth fighting, but a few years later, when he applied for disability insurance he was turned down. In his words again:

I was quickly rejected and placed on an industry-wide blacklist of chronic alcoholics, so other insurance companies could immediately also reject me.

A good story. A particularly unfortunately piece of assessment design, and completely mindless ‘health and safety’. But Rick lives and works in the United States – it wouldn’t happen in the UK…or would it? Actually it’s not far removed from recent experiences (mine and those of friends and colleagues) in the UK.  For example,  when I discovered that I could no longer read the screen on my desktop computer at work, I got a free eye-test and a free pair of computer glasses – unfortunately  the two new pairs of variofocal specs that I also needed cost me more than £300! Ah well. The icing on the cake was having to re-do the multiple choice self-assessment of my display screen set-up. After completing n questions for which the correct answer was always the third option on the list I got bored and ticked a different option. You’ve guessed it – I was wrong!

So when I talk about poor assessment design in education I guess it could be worse. These occupational health type tests really need a rethink – in both the US and the UK.

This entry was posted in quality and tagged , , , , . Bookmark the permalink.

Leave a Reply

Your email address will not be published. Required fields are marked *