e-assessment (f)or learning

If learning that lasts is active and independent, integrative and experiential, assessment must judge performance in contexts related to life roles.

If learning that lasts is self-aware, reflective, self assessed and self-regarding, assessment must include explicitness of expected outcomes, public criteria and student self assessment.

If learning that lasts is developmental and individual, assessment must include multiplicity and be cumulative and expansive.

If learning that lasts is interactive and collaborative, assessment must include feedback and external perspectives as well as performance.

If learning that lasts is situated and transferable, assessment must be multiple in its mode and context.

Mentkowski, M. (2006) Accessible and adaptable elements of Alverno student assessment-as-learning: strategies and challenges for peer review. In Innovative assessment in Higher Education ed. Bryan, C & Clegg, K.V., London U.K., Routledge, pp48-63.

‘When the cook tastes the soup it is formative evaluation; when the dinner guest tastes the soup it is summative evaluation’

Harvey, J. (1998) Evaluation cookbook. Edinburgh: Learning Technology Dissemination Initiative. http://www.icbl.hw.ac.uk/ltdi/cookbook/ pg 7.

Note: this is about formative and summative evaluation, but gives some interesting ideas for assessment too.

‘We are just beginning to have a small idea of the real extent of the possibilities for online assessment.’

Howell, S. C. and Hricko, M. (eds) (2006) Online assessment and measurement: case studies from higher education, K-12 and corporate. Information Science Publishing. pg xvii

‘[there] are tensions associated with e-assessment in which practices are driven by state-of-the-art technological know-how rather than pedagogy.’

Daly, C., Pachler,N., Mor, Y. & Mellar, H. (2010) Exploring formative e-assessment; using case stories and design patterns. Assessment & Evaluation in Higher Eduction, 35(5), 619-636.

‘Assessment is the ‘tail that wags the dog’…’

Dysthe, O. (2008) The challenges of assessment in a new learning culture. In Havnes, A. and McDowell, L. (eds).  Balancing dilemmas in assessment and learning in contemporary education. Routledge/Taylor and Francis. pp15-28.

‘Assessment does not stand outside teaching and learning but stands in dynamic interaction with it.’

Gipps, C.V. (1994) Beyond testing: towards a theory of educational assessment. London, Falmer. pg15.

‘..the interaction between assessment and learning was likened to a three legged race, in which neither partner can make progress without the others’ contribution.’

Harding, R. and Raikes, N. (2002) ICT in assessment and learning: the evolving role of an external examinations board. Maths CAA Series (http://ltsn.mathstore.ac.uk/articles/maths-caa-series/feb2002/index.shtml)

I was looking for something lighter for a Sunday afternoon and came across this. I haven’t read the original paper, just a reference to it, but I reckon that ‘Orangoutang score’ is more fun than ‘random guess score’.

‘The Orangoutang score is that score on a standardised reading test that can be obtained by a well-trained Orangoutang under these special conditions. A slightly hungry Orangoutang is placed in a small cage that has an oblong window and four buttons. The Orangoutang has been trained that every time the reading teacher places a neatly typed multiple choice item in a reading test in the oblong window, all that he (the Orangoutang) has to do to get a bit of banana is to press a button, any of these buttons, which, incidentally, are labelled A, B, C and D.’

Fry, E. (1971) The Orangoutang score. Reading Teacher, 24, 360-2.

‘The formative assessment for Anne was not a supportive step toward summative assessment, but a significant hurdle in its own right; a moment of judgement of her aptitude for higher education and her identity. Therefore, for Anne, the formative process was one of anxious torment’ [pg515]

‘For our participants, as one assessment hurdle is jumped, another looms darkly in the distance.’ [pg517]

Cramp, A., Lamond, C., Coleyshaw, L & Beck, S. (2012). Empowering or disabling? Emotional reactions to assessment amongst part-time adult students. Teaching in Higher Education, 17(5), 509-521 [pg515]

‘Assessment is not working, or at least it is not working as it should. In our attempt to generate forms of assessment capable of addressing all the purposes for whish we use assessment, we have produced a Frankenstein that preys on the educational process, reducing large parts of teaching and learning to mindless mechanistic process whilst sapping the transformative power of education.’

Broadfoot, P. (2008) Assessment for learners: assessment literacy and the development of learning power. In Havnes, A. and McDowell, L. (eds).  Balancing dilemmas in assessment and learning in contemporary education. Routledge/Taylor and Francis. pp213-224.

Reading through my notes on some of the many assessment papers I have read, I’m finding a few of those ‘sit up and take note’ quotes; things (sometimes very obvious) that other people somehow manage to say so much better than I can. So, I bring you the first of an occasional series of ‘Quote of the day’:

…’ summative assessment is itself ‘formative’. It cannot help but be formative. This is not an issue. At issue is whether that formative potential of summative assessment is lethal or emancipatory. Does summative assessment exert its power to discipline and control, a power so possibly lethal that the student may be wounded for life?’

Barnett, Ronald (2007) Assessment in higher education: an impossible mission? In Boud, David and Falchikov, Nancy (eds) Rethinking assessment in higher education. London, Routledge. pg37.

I have had an interesting debate with colleagues about whether questions in which you have to drag one or more markers to appropriate places on an image (see example below) are selected response or constructed response questions.

I am of the opinion that this is a constructed response question, because students are not given clues as to where the markers should go. It is fundamentally different to the question below (‘drag and drop onto image’) which is selected response because there are only a number of places where the labels can go.

However, during this debate, my colleague pointed out that the boundaries between constructed and selected response question types are not that clear cut. In a sense the top image is selected response because there are a finite number of pixels in the image. Similarly if you ask a numerical question in which you want an answer that’s in integer between 1 and 9, there are actually only 9 options available to you. For what it’s worth, I still think both of these are constructed response questions, but the debate is an interesting one.

This morning I’ve been reading an oldish paper (Sambell & McDowell, 1998) about work on the ‘hidden curriculum’, an even older phenomenon (Snyder, 1971). 

The hidden curriculum can be thought of in terms of the distinction between ‘what is meant to happen’ i.e. the curriculum stated officientally by the educational system or institution, and what students actually experience ‘on the ground’. Assessment is very important in determining the hidden curriculum.

Sambell & McDowell take this one step further. They point out that every student has a different hidden curriculum; the same assessment is interpreted differently not just by ‘staff’ and ‘students’ but by individuals. Students bring with them different range experiences, motivations and perspectives which influence their response.

In the light of this, how are we do work to improve the assessment experience (and hence the hidden curriculum) for all our students, especially in a large and diverse University such as the Open University? Dealing with each individual student’s previous experiences and perceptions is challenging. However I think we are still missing some very obvious tricks. We assume that our students share our understanding of what assessment is for, but many probably don’t. I am interested in doing some work to improve that understanding, perhaps by running online tutorials for students before their first assignment, to discuss the assessment process not the assessment itself. Then we might to similarly after students have got the first piece of work back, to discuss what they have learnt from it and from the feedback.

Anyone want to join in the fun?

Sambell, K. & McDowell, L (1998) The construction of the hidden curriculum: messages and meanings in the assessment of student learning, Assessment and Evaluation in Higher Education, 23(4), 391-402.

Snyder, B.R. (1971) The hidden curriculum. New York: Alfred A. Knopf.