e-assessment (f)or learning

‘The formative assessment for Anne was not a supportive step toward summative assessment, but a significant hurdle in its own right; a moment of judgement of her aptitude for higher education and her identity. Therefore, for Anne, the formative process was one of anxious torment’ [pg515]

‘For our participants, as one assessment hurdle is jumped, another looms darkly in the distance.’ [pg517]

Cramp, A., Lamond, C., Coleyshaw, L & Beck, S. (2012). Empowering or disabling? Emotional reactions to assessment amongst part-time adult students. Teaching in Higher Education, 17(5), 509-521 [pg515]

Yesterday was a busy day! After the STACK webinar I jumped in my car and drove to Milton Keynes to speak at an OU meeting about assessment, this run for and by the Childhood, Youth and Education Programme. This was somewhat nerve-wracking – these guys are experts in things educational.

For me, the highlight of the day was listening to the guest speaker, Graham Gibbs, whose work I am sure that most people reading this Blog will know. I worked with Graham on the FAST Project (well that might be stretching a point a bit – I was a very junior player) and it was lovely to see him again. Graham’s title was ‘How assessment supports student learning, how assessment is changing in UK universities and how assessment at the OU might change.’ Much of what he said was familiar, but that doesn’t make it any less important. If we’d all heard it before and acted on it, perhaps we wouldn’t still have an ‘assessment problem’.

I absolutely agree with some of the things Graham suggested, in particular that we should use more oral feedback and plan a progression of assessment across modules and programmes.  In the OU context, I am still uneasy about the use of  unmarked assignments. When we talk about formative thresholded assessment in the Science Faculty, we are still giving marks as an indicator of progress. However I do agree that it might be a good idea to separate the formative and the summative and, in particular, to alter our culture so that students can submit early drafts of projects for (formative) feedback before they are resubmitted for (summative) marks.

I’ve posted before about the fact that whilst students usually engage quite well with formative-only iCMA questions, when the going gets tough, they are inevitably more likely to guess than is the case when the mark counts. When I eventually get to the end of my course writing (and associated preference for blogging about things to do with maths misunderstandings, on the basis that this is relevant for the course writing too), I will talk about our changes of assessment strategy in the Open University Science Faculty. For now, I just want to reflect on the size of the formative vs summative effect. Don’t treat this too seriously, but I think the answer to ‘how big is the effect’ may be 3%. Read on.

Conside the question shown on the right. Variants of this question occur both in the formative practice assessment and in one of the summative end-of-module assessments.

In the practice assessment, 74.3% of responses were correct whilst 6.2% gave the number given in the question (so for this variant, they gave an answer of 4 – presumably guessing). In the summative equivalent, 77.7% of responses were correct whilst 3.3% gave the number given in the question.

I promise that this will be my last post about the difference between summative and formative assessment per se.  It seems to be something that bothers people; maybe I’ve caught the bug!

I used to imagine a continuum that had formative assessment at one end and summative assessment at the other, and the  debate hinged around whether you believed that the formative (‘assessment for learning’) or summative (‘assessment of learning’) function was more important i.e. where on this continuum you sat. Bull and McKenna talked about a blurring of the boundaries of formative and summative assessment, and I agreed.

But actually I think you can go further than that. Surely summative and formative assessment are just different things. Summative assessment is about measuring; formative assesment is about learning. Surely both can happen at the same time (or neither! - on the basis that assessment isn’t formative just because it is meant to be). Formative assessment can be summative; summative assessment can be formative.

While I’ve been absent from this blog I’ve been doing lots of analysis of student responses to our questions. More on that to follow. I’ve also been reading a variety of stuff, including the report on the  JISC-funded project ’Scoping a vision for formative e-assessment’ and a selection of papers about the same work. I don’t agree with everything in this report, but I do absolutely agree that ‘no assessment technology is in itself formative, but almost any technology can be used in a formative way – if the right conditions are set in place’. I also agree that problems arise when practices are driven by ‘state-of-the-art technological know-how rather than pedagogy’.

The report identifies design ‘patterns’ of processes in formative assessment that can be supported by software tools. (more…)

Given tha amount of work that we put into formative assessment, why don’t students do better in summative assessment?

This was one of the recurring themes at the EARLI/Northumbria Assessment Conference, first raised in Liz McDowell’s keynote. Liz wondered if the explanation might be that we sometimes norm-reference our summative assessment, even when we claim we don’t do this.

Sue Bloxham (in discussion after Liz’s paper) and Gordon Stobart (in the final keynote of the Conference) saw the problem as a lack of alignment between formative and summative assessment.

If our formative interventions are not having any obvious impact, we certainly ought to be investigating this further.

I’ve just read a paper by Pachler et al (Computers & Education 54 (2010) pp715-721) which describes aspects of the JISC-funded project ‘Scoping a vision of formative e-assessment’. The paper starts by considering different perspectives on the ‘nature and value of formative e-assessment’. I’m sure it should have occurred to me before (but hadn’t!) that ‘formative (e)assessment’ can mean a range of different things. The emphasis might be on

• practice for summative assessment
• the provision of feedback
• a means of providing self-reflection

The paper goes on to make the key point that ‘no technology-based assessment is in itself formative, but almost any technology can be used in a formative way – if the right conditions are set in place.’ In other words, the technology isn’t the thing that makes learning happen, it’s student engagement that matters. Amen to that.

Before I get side-tracked down any more avenues related to assessment, I feel I should say a bit more about my choice of title for this blog. My first idea for a title was (e)assessment (f)or learning (with the bracketed ‘e’ indicating that whilst most of my work is in e-assessment, I am interested in more general issues in assessment) but that looked too messy, hence the current title, e-assessment (f)or learning.

(more…)