From 2006 - 2010 the Centre for Open Learning of Mathematics, Science, Computing and Technology (COLMSCT) ran a project that explored the marking of short textual responses; typically responses of a sentence or two.
Sally Jordan and Barbara Brocklebank used Free Text Author from Intelligent Assessment Technologies; Phil Butcher followed up with an equivalent study using OpenMark's internal response matching algorithm. Both ran trials using large numbers of OU student responses and both systems were found to provide similar, high, levels of marking accuracy.
The research team report on:
in Butcher, P. G. and Jordan, S. E. (2010) A Comparison Of Human And Computer Marking Of Short Free-Text Student Responses, Computers & Education, Volume 55, Issue 2, Pages 489-499 which may be accessed online at http://dx.doi.org/10.1016/j.compedu.2010.02.012
From autumn 2010 the OU will focus further work in this area on the OpenMark approach and here we provide a demonstration using the OpenMark response matching system.
In the demonstration you are asked to compose your response in the form of a sentence, or two. Your response is then compared against a variety of model-answers and is marked automatically by the computer.
There is a more extensive demonstration at Demonstration of free text response matching within OpenMark.