Quote of the day

Posted on May 12, 2013 by Sally Jordan

I’m not sure about the detail of this one, but as a physicist I couldn’t resist the analogy between quantum mechanics and the testing effect!

‘The testing effect represents a conundrum, a small version of the Heisenberg uncertainty principle in psychology: Just as measuring the position of an electron changes that position, so the act of retrieving information from memory changes the mnemonic represention underlying retrieval – and enhances later retention of the tested information.’
Roediger, H. L., III, & Karpicke, J. D. (2006). The power of testing memory: Basic research and implications for educational practice. Perspectives on Psychological Science, 1, 181-210. pg 182.

Posted in quotes, testing effect | Tagged , | Leave a comment

Quote of the day

Posted on May 3, 2013 by Sally Jordan

‘Assessment is a moral activity. What we choose to assess and how shows quite starkly what we value.’

Knight, P. (ed) (1995) Assessment for learning in Higher Education. Kogan Page in association with SEDA. pg13 (introduction)

Posted in quotes | Tagged | 1 Comment

More on the automatic marking of essays

Posted on April 17, 2013 by Sally Jordan

Post now edited to include links…

Oh how much I missed in my last post!

Others, far better informed than me, have also reflected on the general principles of the use of the automated marking of essays. See, for example, Michael Feldstein, Audrey Watters and Justin Reich.

I think the interest was probably sparked by an ‘Automated Student Assessment Prize’ competition last year, funded by the William and Flora Hewlett Foundation. Phase 1 of this was to do with the automatic marking of essays and the results are here.

Things began to get interesting when the New York Times announced that edX were going to use automatic grading of essays. The report may or may not have been accurate, but concerns are being expressed about so-called ‘robo-marking’. See for example Les Pereman and the Professionals against Machine Grading of Essays group.

On Michael Feldstein’s blog ‘e-Literate’, Elijah Mayfield of LIghtSIDE Labs has now said more about why the NYT (and possibly the edX claim) is wrong. The LIghtSIDE labs approach seems very sensible – human marked essays are used as the basis for machine learning about good features of essays; this is used to provide feedback about the good and weak features of essays submitted by students and hence to peer review. Grading is still done by humans.

Some questions:
– I don’t know what edX are actually planning to do, in other words, how much can we believe the NYT report?
– I’d love to know more about the technology being used (by edX, LIGHTside, anyone) and whether they are marking essay content, style or both?

Posted in essay-marking software, essays | Tagged , , | Leave a comment

Automatic marking of essays

Posted on April 5, 2013 by Sally Jordan

I am grateful to Carol Bailey (see previous post) who, following a discussion over lunch, sent me a link to an extremely interesting paper:

Vojak, C., Kline, S., Cope, B., McCarthey, S. and Kalantzis, M. (2011) New Spaces and Old Places: An Analysis of Writing Assessment Software. Computers and Composition, 28, 97-111. Continue reading →

Posted in essay-marking software, essays, short-answer free text questions | Tagged , , | 1 Comment

Developing good academic practice

Posted on April 5, 2013 by Sally Jordan

A couple of weeks ago we had an internal conference at the Open University on ‘Developing good academic practice’. Day 1 was intended for our tutors and their line-managers (staff tutors); Day 2 for module team members and those with the assessment overview in faculties. That means I went both days! Each day, we had a ‘keynote’ presentation, then a series of workshops run by different ‘Faculty Academic Conduct Officers’. Continue reading →

Posted in plagiarism | Tagged , , | 1 Comment

Flagging questions as ‘unsound’

Posted on March 31, 2013 by Sally Jordan

I was talking at the eSTEeM Conference last week about the fact that, whilst our interactive computer-marked assessment (iCMA) questions are generally well liked by our students, occasional questions can cause problems (usually because we are not giving sufficiently targeted feedback, so students don’t understand why the answer they have given is wrong). Why not, someone said, have  an ‘unsound’ button? Well, we used to have such a function for our short-answer free text questions (shown below in use back in 2007) and it wasn’t my choice to stop using it. This post considers the pros and cons. Continue reading →

Posted in unsound questions | Tagged | Leave a comment

Quote of the day

Posted on March 30, 2013 by Sally Jordan

If learning that lasts is active and independent, integrative and experiential, assessment must judge performance in contexts related to life roles.

If learning that lasts is self-aware, reflective, self assessed and self-regarding, assessment must include explicitness of expected outcomes, public criteria and student self assessment.

If learning that lasts is developmental and individual, assessment must include multiplicity and be cumulative and expansive.

If learning that lasts is interactive and collaborative, assessment must include feedback and external perspectives as well as performance.

If learning that lasts is situated and transferable, assessment must be multiple in its mode and context.

Mentkowski, M. (2006) Accessible and adaptable elements of Alverno student assessment-as-learning: strategies and challenges for peer review. In Innovative assessment in Higher Education ed. Bryan, C & Clegg, K.V., London U.K., Routledge, pp48-63.

Posted in quotes | Tagged | Leave a comment

Quote of the day

Posted on March 16, 2013 by Sally Jordan

‘When the cook tastes the soup it is formative evaluation; when the dinner guest tastes the soup it is summative evaluation’

Harvey, J. (1998) Evaluation cookbook. Edinburgh: Learning Technology Dissemination Initiative. http://www.icbl.hw.ac.uk/ltdi/cookbook/ pg 7.

Note: this is about formative and summative evaluation, but gives some interesting ideas for assessment too.

Posted in quotes | Tagged | Leave a comment

Quotes of the day

Posted on March 15, 2013 by Sally Jordan

‘We are just beginning to have a small idea of the real extent of the possibilities for online assessment.’

Howell, S. C. and Hricko, M. (eds) (2006) Online assessment and measurement: case studies from higher education, K-12 and corporate. Information Science Publishing. pg xvii

‘[there] are tensions associated with e-assessment in which practices are driven by state-of-the-art technological know-how rather than pedagogy.’

Daly, C., Pachler,N., Mor, Y. & Mellar, H. (2010) Exploring formative e-assessment; using case stories and design patterns. Assessment & Evaluation in Higher Eduction, 35(5), 619-636.

Posted in e-assessment, quotes | Tagged , | Leave a comment

Three quotes on the links between assessment and learning

Posted on March 11, 2013 by Sally Jordan

‘Assessment is the ‘tail that wags the dog’…’

Dysthe, O. (2008) The challenges of assessment in a new learning culture. In Havnes, A. and McDowell, L. (eds).  Balancing dilemmas in assessment and learning in contemporary education. Routledge/Taylor and Francis. pp15-28.

‘Assessment does not stand outside teaching and learning but stands in dynamic interaction with it.’

Gipps, C.V. (1994) Beyond testing: towards a theory of educational assessment. London, Falmer. pg15.

‘..the interaction between assessment and learning was likened to a three legged race, in which neither partner can make progress without the others’ contribution.’

Harding, R. and Raikes, N. (2002) ICT in assessment and learning: the evolving role of an external examinations board. Maths CAA Series (http://ltsn.mathstore.ac.uk/articles/maths-caa-series/feb2002/index.shtml)

Posted in quotes | Tagged | Leave a comment