e-assessment (f)or learning

Time to take a seasonal break from my rather tedious recent posts and to return to a reflection on feedback.

The column ‘Feedback’ (what else!) on the penultimate page of the Christmas and New Year New Scientist special (24/31 December 2011), includes the following:

‘John …recalls a senior manager urging staff to provide feedback for his latest project, ‘but it must be positive’…

John could not help but explain that negative feedback produced growth and stability and positive feedback produced burnout’

John is clearly a man after my own heart! Not only do scientists and senior managers use the adjectives ‘positive’ and ‘negative’ in this context to mean rather different things; those working in assessment are completely confused. The problem is that the impact of assessment feedback may be positive or negative (in both senses of both definitions) and the outcome is difficult to predict.

Happy Christmas and very best wishes for 2012.

As you’ll have realised, I’m currently analysing responses to questions on calculus. Maths for Science teaches just very basic differentiation and the position of the chapter on differentiation will be different in the revised edition of the course. In the new edition, the chapter will be where it belongs in a logical story line, immediately after the chapter on graphs and gradient. In the old edition, we thought differentiation might be a step too far for some students so offered Differentiation (Chapter 10) as a choice with Statistical Hypothesis Testing (Chapter 9). Add in the fact that we still allowed students to answer questions on both chapters if they wanted to, and the fact that many of the Chapter 10 questions are multiple choice, and it is really quite difficult to tell whether students are guessing answers to the Chapter 10 questions – even the summative ones. This is characteristically different from behaviour on earlier questions where it is very obvious that most students are not guessing.

An extension of this is that it is quite difficult to assess what the real difficulties are that students experience with the chapter on differentiation. We have some familiar stories e.g. answers where a function has been differentiated correctly and values subsituted correctly, but then answers have been given to an inappropriate number of significant figures. Hey ho – I think we get too hung up about this.

Now compare the two questions given below. (more…)

The screenshots below show two variants of the same question. I should start by emphasising that the question is only used formatively – it’s on the Maths for Science Practice Assignment.

The two variants behave very differently and the reason for this has very little to do with student’s understanding. (more…)

The common student error in the question below is somewhat predictable – but I’m not sure why the students make the error that they do.

Whilst 60.4% of responses are entirely correct, 21.7% select the three options that are actually the places where the function rather than its derivative is zero. In the existing version of the Maths for Science End of Module Assignment, this question is assessing the contents of a chapter that some students choose not to study – so it is more than usually  likely that a certain percentage of students are guessing, further encouraged by the fact that this is a multiple choice question.

So I’m not sure we can draw many conclusions from the errors that students make on this occasion. But if students really think that dy/dx is zero at the points where the graph crosses the horizontal axis, then they are mistaking the function and its gradient. This is a pretty basic mistake – perhaps a bit like mistaking x to the power of 3 with x times 3. That might be rather interesting.

As previously discussed, students aren’t great at giving correct units with their answers. However they have real probems with unit conversions!

The unit conversion shown on the left  is exceptionally badly answered, and a lot of students give up without even trying. The difficulties appear to stem from different aspects of the problem, as exemplified by the examples below.

First of all, let’s look at the problem students have in converting from one square (or cubic) unit to another, as on the right-hand side. Errors in this question fall into three basic categories:

(1) students who convert from km to m rather than from km^2 to m^2.

(2) students who do the converstion the wrong way round (i.e. convert from m^2 to km^2).

(3) students who do both of the above (i.e convert from m to km) – actually more common that just doing the conversion the wrong way round.

I’ve known for ages about the difficultly some people have in converting squared or cubic units. It fascinates me. (more…)