e-assessment (f)or learning

This one comes from Carl Wieman who won the Nobel Prize for physics in 2001. I’ll start with a quote which gives the broader flavour of the paper:

[pg10] [we should] ‘approach the teaching of science like a science. That means applying to science teaching the practices that are essential components of scientific research and that explain why science has progressed at such a remarkable pace in the modern world.
The most important of these components are:
• Practices and conclusions based on objective data rather than—as is frequently the case in education—anecdote or tradition.This includes using the results of prior research, such as
work on how people learn.
• Disseminating results in a scholarly manner and copying and building upon what works. Too often in education, particularly at the postsecondary level, everything is reinvented, often in a highly flawed form, every time a different instructor teaches a course. (I call this problem “reinventing the square wheel.”)
• Fully utilizing modern technology. Just as we are always looking for ways to use technology to advance scientific research, we need to do the same in education.’
[I'm not sure I necessarily agree with the final point - I'd use technology when, and only when, that is beneficial to the student experience.]

Relative to this, the point I want to emphasise sounds timid:

[pg13] ‘Even the most thoughtful, dedicated teachers spend enormously more time worrying about their lectures than they do about their homework assignments, which I think is a mistake.’

But it is oh so true – certainly in my own institution, relative to the time and effort that goes into developing our (excellent) teaching resources, we put so little time and effort into getting assessment right. I think that’s a mistake! Your institution may be different of course, but I doubt that many are.

Wieman, C. (2010). Why not try a scientific approach to science education? Change: The Magazine of Higher Learning, 39(5), 9-15

I’m not sure about the detail of this one, but as a physicist I couldn’t resist the analogy between quantum mechanics and the testing effect!

‘The testing effect represents a conundrum, a small version of the Heisenberg uncertainty principle in psychology: Just as measuring the position of an electron changes that position, so the act of retrieving information from memory changes the mnemonic represention underlying retrieval – and enhances later retention of the tested information.’
Roediger, H. L., III, & Karpicke, J. D. (2006). The power of testing memory: Basic research and implications for educational practice. Perspectives on Psychological Science, 1, 181-210. pg 182.

‘Assessment is a moral activity. What we choose to assess and how shows quite starkly what we value.’

Knight, P. (ed) (1995) Assessment for learning in Higher Education. Kogan Page in association with SEDA. pg13 (introduction)

If learning that lasts is active and independent, integrative and experiential, assessment must judge performance in contexts related to life roles.

If learning that lasts is self-aware, reflective, self assessed and self-regarding, assessment must include explicitness of expected outcomes, public criteria and student self assessment.

If learning that lasts is developmental and individual, assessment must include multiplicity and be cumulative and expansive.

If learning that lasts is interactive and collaborative, assessment must include feedback and external perspectives as well as performance.

If learning that lasts is situated and transferable, assessment must be multiple in its mode and context.

Mentkowski, M. (2006) Accessible and adaptable elements of Alverno student assessment-as-learning: strategies and challenges for peer review. In Innovative assessment in Higher Education ed. Bryan, C & Clegg, K.V., London U.K., Routledge, pp48-63.

‘When the cook tastes the soup it is formative evaluation; when the dinner guest tastes the soup it is summative evaluation’

Harvey, J. (1998) Evaluation cookbook. Edinburgh: Learning Technology Dissemination Initiative. http://www.icbl.hw.ac.uk/ltdi/cookbook/ pg 7.

Note: this is about formative and summative evaluation, but gives some interesting ideas for assessment too.

‘We are just beginning to have a small idea of the real extent of the possibilities for online assessment.’

Howell, S. C. and Hricko, M. (eds) (2006) Online assessment and measurement: case studies from higher education, K-12 and corporate. Information Science Publishing. pg xvii

‘[there] are tensions associated with e-assessment in which practices are driven by state-of-the-art technological know-how rather than pedagogy.’

Daly, C., Pachler,N., Mor, Y. & Mellar, H. (2010) Exploring formative e-assessment; using case stories and design patterns. Assessment & Evaluation in Higher Eduction, 35(5), 619-636.

‘Assessment is the ‘tail that wags the dog’…’

Dysthe, O. (2008) The challenges of assessment in a new learning culture. In Havnes, A. and McDowell, L. (eds).  Balancing dilemmas in assessment and learning in contemporary education. Routledge/Taylor and Francis. pp15-28.

‘Assessment does not stand outside teaching and learning but stands in dynamic interaction with it.’

Gipps, C.V. (1994) Beyond testing: towards a theory of educational assessment. London, Falmer. pg15.

‘..the interaction between assessment and learning was likened to a three legged race, in which neither partner can make progress without the others’ contribution.’

Harding, R. and Raikes, N. (2002) ICT in assessment and learning: the evolving role of an external examinations board. Maths CAA Series (http://ltsn.mathstore.ac.uk/articles/maths-caa-series/feb2002/index.shtml)

I was looking for something lighter for a Sunday afternoon and came across this. I haven’t read the original paper, just a reference to it, but I reckon that ‘Orangoutang score’ is more fun than ‘random guess score’.

‘The Orangoutang score is that score on a standardised reading test that can be obtained by a well-trained Orangoutang under these special conditions. A slightly hungry Orangoutang is placed in a small cage that has an oblong window and four buttons. The Orangoutang has been trained that every time the reading teacher places a neatly typed multiple choice item in a reading test in the oblong window, all that he (the Orangoutang) has to do to get a bit of banana is to press a button, any of these buttons, which, incidentally, are labelled A, B, C and D.’

Fry, E. (1971) The Orangoutang score. Reading Teacher, 24, 360-2.

‘The formative assessment for Anne was not a supportive step toward summative assessment, but a significant hurdle in its own right; a moment of judgement of her aptitude for higher education and her identity. Therefore, for Anne, the formative process was one of anxious torment’ [pg515]

‘For our participants, as one assessment hurdle is jumped, another looms darkly in the distance.’ [pg517]

Cramp, A., Lamond, C., Coleyshaw, L & Beck, S. (2012). Empowering or disabling? Emotional reactions to assessment amongst part-time adult students. Teaching in Higher Education, 17(5), 509-521 [pg515]

‘Assessment is not working, or at least it is not working as it should. In our attempt to generate forms of assessment capable of addressing all the purposes for whish we use assessment, we have produced a Frankenstein that preys on the educational process, reducing large parts of teaching and learning to mindless mechanistic process whilst sapping the transformative power of education.’

Broadfoot, P. (2008) Assessment for learners: assessment literacy and the development of learning power. In Havnes, A. and McDowell, L. (eds).  Balancing dilemmas in assessment and learning in contemporary education. Routledge/Taylor and Francis. pp213-224.