e-assessment (f)or learning

Back in the early days of this blog, I muttered about our misunderstanding of the word ‘feedback’ in the context of assessment and gave examples of positive and negative feedback from everday life. I talked about the thermostat on a central heating system as an example of negative feedback.

The trouble is, it isn’t quite that simple. Whereas a thermostat acts in a predictable way to close the gap between the actual and reference temperature (by causing the heating to switch off and on as appropriate),  humans respond to feedback interventions in complex (if understandable) ways. For example, Kluger and deNisi point out that if a student does well on an assignment, he or she may ‘raise the bar’  i.e. make their target level higher, effectively making the gap that has to be closed larger than would otherwise be the case.

Mind you, feedback in other contexts is also more complicated than may initially appear to be the case – global warming is a case in point!

I’ve just finished reading Kluger & DeNisi’s meta-analysis of the impact of feedback interventions on performance. (Kluger, A.N. & DeNisi, A. (1996) The effects of feedback interventions on performance : a historical review, a meta-analysis, and preliminary feedback intervention theory, Psychological Bulletin, 119 (2), 254-284. It’s rather theoretical and I didn’t understand all of it, but there are some clear messages.

Firstly,  ’feedback interventions’ (which they define as actions taken by external agent(s) to provide information regarding some aspect(s) of one’s task performance) are not the same as feedback. I think that’s a useful distinction.

Secondly, in the 131 papers included in the meta-analysis (representing 607 effect sizes, 12,652 participants and 23,663 observations), whilst on average feedback interventions had a moderate positive effect on performance, over 38% of the effects were negative. These reported negative effects are frequently ignored, as we continue to assume that feedback interventions  are always beneficial. The paper goes on to develop a ‘feedback intervention theory’, seeking to identify ways in which the effectiveness of feedback interventions can be improved. There is a crying need for more work in this area.

With apologies, I thought I had put links on this blog to some Open University eAssessment questions, but this was not the case. I’ll add these links to the ‘E-assessment at the Open University’ link (right-hand side of blog) in a minute.

It’s actually a bit tricky, because we can’t make summative questions generally available. However the diagnostic quiz, Are you ready for S104? (https://students.open.ac.uk/openmark/science.level1ayrf.s104/) should give you the idea.

In addition, if you are specifically interested in our short-answer free-text questions, go to  https://students.open.ac.uk/openmark/omdemo.pm2009/ (though again note that our best questions are in summative use!).

The title of this post may sound contradictory. If we give students marks, the assessment is summative. Right? Not necessarily. It is perfectly possible to tell students their ‘mark’ for an assignment but for that mark not to count towards the final outcome (so this assessment is purely formative). Similarly, it is possible for a grade from an assignment to count towards the final outcome (and so be summative) even if the student is not told this grade. (more…)