Monthly Archives: July 2011

Assessing Open University students – at residential schools and otherwise

I was due to be tutoring at the Open University residential school SXR103 Practising Science at the University of Sussex (shown left; the crane is a reminder of the huge amount of building work that is going on) for two … Continue reading

Posted in Open University, Residential schools | Tagged , | Leave a comment

Happy birthday blog!

It seems hard to believe that I’ve been blogging on assessment, especially e-assessment, and especially the impact of e-assessment on students, for a year now. Even more amazing is the fact that there is still so much I want to … Continue reading

Posted in Uncategorized | Leave a comment

Answer matching for short-answer questions: simple but not that simple

In describing our use of  (simple) PMatch for answer matching for short-answer free-text questions, I may have made it sound too simple. I’ll give two examples of the sorts of things you need to consider: Firstly, consider the question shown … Continue reading

Posted in short-answer free text questions | Tagged , , , | Leave a comment

Can online selected response questions really provide useful formative feedback?

The title of this post comes from the title of a thoughtful paper from John Dermo and Liz Carpenter at CAA 2011. In his presentation, John asked whether automated e-feedback can create ‘moments of contingency?’ (Black & Wiliam 2009). This is something I’ve … Continue reading

Posted in conferences, feedback | Tagged , , , , , | Leave a comment

Are you sure?

For various reasons I’ve been thinking a lot recently about confidence-based marking.  (Tony Gardner-Medwin, who does most of the work in this area also calls it ‘certainty-based marking’). The principle is that you get most marks for a correct response … Continue reading

Posted in confidence-based marking | Tagged , | 2 Comments

Let students not technology be the driver

Just home from CAA 2011 (the International Computer Assisted Assessment Conference in Southampton). The attendance was quite low ( probably a victim of the current economic climate) but the conference was good, with some very thoughtful presentations and extremely useful conversations. … Continue reading

Posted in conferences | Tagged , , , , | Leave a comment

Short-answer questions : how far can you go?

Finally for today, I’d like to talk about where I believe the limits currently sit in the use of short-answer free-text questions. I have written questions where the correct response requires three separate concepts. For example, I have written a … Continue reading

Posted in short-answer free text questions | Tagged | Leave a comment

Short-answer questions : when humans mark more accurately than computers

Hot on the heals of my previous post, I’d like to make it clear that human markers sometimes do better than computers in marking short-answer [less than 20 word] free-text questions.  I have found this to be the case in two situations … Continue reading

Posted in human marking, short-answer free text questions | Tagged , , , | Leave a comment

Short-answer questions : when computers mark more accurately than humans

Back to short-answer free-text questions. One of the startling findings of my work in this area was that computerised marking (whether provided by Intelligent Assessment Technologies’ FreeText Author or OpenMark PMatch) was consistently more accurate and reliable than human markers. At the time, … Continue reading

Posted in human marking, short-answer free text questions | Tagged , , , | Leave a comment