e-assessment (f)or learning

You may be relieved to hear that this will be my final posting (at least for a while) on our use of different variants of interactive computer-marked assignment (iCMA) questions. We know that, whilst the different variants of many questions are of equivalent difficulty, we can’t claim this for all our questions. How much does this matter? A typical iCMA might include 10 questions and contribute something between 2 and 5% to a student’s overall score. My Course Team colleagues and I have reassured ourselves that the effect of the different difficulty of some variants of some questions will, when averaged out over a large number of questions, have minimal impact on a student’s overall score. But is this true? (more…)

One glance at the figure from the previous post (reproduced to the right) makes it clear that whilst the variants of the question shown at the top are equivalent, those for the lower question are not.

Reasons why variants may be of differing difficulty include

• the variables selected may result in a more difficult mathematical task (e.g. rounding up instead of rounding down, understanding of negative exponents rather than positive ones)
• a graph to be interpreted may have a more awkward scale to read, or if different readings are to be taken from the same graph, some students may have to interpolate or extrapolate whilst others are taking readings where the graph crosses the grid-lines.
• the letters used may appear similar in lower and upper case and so be confused e.g. k and K are very similar whilst q is unlikely to be confused with Q.
• the words used in setting up the question may use language or describe a situation which is unfamiliar to the student. (more…)

So how can you make different variants of interactive computer-marked assignment questions?

Here are some strategies we’ve used:

• Use different numbers (so ‘Evaluate 3 + 7′ becomes’Evaluate  ’4 + 5′);
• Use different letters (so ‘Rearrange a=bc to make b the subject’ becomes ‘Rearrange b=cd to make c the subject’);
• Use different words (so ‘Find the area of the floorboard’ becomes ‘Find the area of the carpet’ or ‘Find the area of the runway’); (more…)

At the Open University we use different variants of our iCMA questions. So, to take a very simple example, when one student receives the question ‘Evaluate 3 + 7′, another might receive the question ‘Evaluate  ’4 + 5′. In summative use, different variants limit the opportunities for plagiarism. In formative-only use, different variants provide students with extra opportunities for practice. (more…)