Thomas, P. G. Waugh, K. Smith, N (2007) Computer Assisted Assessment of Diagrams.

In this paper, we describe an approach to the grading (marking) of graph-based diagrams. The work is an application of a framework for the computer interpretation of imprecise diagrams (which students produce in response to assessment questions). We describe the interpretation framework and how it has been applied in the domain of entity-relationship diagrams used in data modeling. In our approach to grading, student diagrams are compared to specimen solutions representing correct solutions to an assessment question. The comparison between a student diagram and a specimen solution diagram results in a matching  of the two diagrams from which a grade is computed. The comparison matching is used to provide graphical feedback on the correctness of the student diagram. The approach has been applied to a test corpus of 394 diagrams produced in answer to an examination question with some good results: 91% of all automatically produced grades were within 0.5 of a mark of the human graders. However, when supertype-subtype associations are present, the performance drops to around 87%. The paper concludes with a discussion of some applications of the marker.