e-assessment (f)or learning

The second external meeting I attended last week, this time at the University of Warwick, was a meeting of the Institute of Physics Higher Education Group entitled ‘Conceptual understanding : beyond diagnostic testing’. The messages that I’ve come home with are that student misunderstandings may not be what we think they are – and that we need to find out more. Derek Raine’s talk ‘Metaphors and misunderstandings: addressing student misconceptions in physics’ started off with a (presumably apocryphol – and I’m sure I won’t do the story justice) tale of a famous actress being shown to a dressing room in a provincial theatre. Her hosts were embarassed about the poor standard of their facilities and apologised that the dressing room had no door. But, she said ‘if there’s no door, how do I get in?’ Yes, we really are sometimes that much at cross-purposes with our students.

In physics education research, much attention has been given over the years to the ‘Force Concept Inventory’ (FCI), where a series of questions is used to assess student understanding of the Newtonian concepts of force. At the meeting, Marion Birch described common trends in FCI results at the universities of Manchester, Hull and Edinburgh – two questions seem to cause particular problems wherever they are asked. More startling are the gender differences – women do less well than men and two questions (different from those that are poorly answered by all) have particularly large differences. What Marion was describing was inarguable (though some of the women at the meeting wanted to argue…) but I want to know what is causing the results! Is the difference at the level of conceptual (mis)understanding or is it something about these particular questions that is causing women more difficulty than men? This is just far to interesting to let it go – we must find out what is going on.

The final presentation of the day was from Paula Heron (by video link from the University of Washington) on ‘Using students’ spontaneous reasoning to guide the design of effective instructional strategies’. I think we do need to start observing our students carefully, and asking about their reasoning, rather than just assuming that they answer mutliple-choice questions in the way that they do because of a particular misconception.

Although I’m still trying to find time to rewrite S151 Maths for Science (hence the previous posts about students’ mathematical misconceptions) I am also incredibly busy with other things. Last week I attended two external meetings, both excellent, so I’ll take a bit of a break from the maths to talk about these. 

On Wednesday I was at a meeting of the JISC Learning and Teaching Practice Experts Group. I have only recently been invited to join this group, so I headed to Birmingham with some trepidation, but it was a superb meeting and I met some inspirational people (very few of whom I’d met previously) – and everyone was friendly. It’s difficult to pick the best from the excellent presentations and workshops I attended, but one that has already proved useful was a session led by Alan Masson of the University of Ulster, about the Viewpoints Project. From their own blurb…’the Viewpoints Project provides practitioners with a series of simple user-friendly tools that promote a creative and effective approach to the curriculum design process.’ Simple indeed – we spent some time discussing sets of cards on ‘Good assesment and feedback practice’ (based on the REAP principles) – discussing how we would use these in curriculum planning. But it was so effective (and has already led me to use cards in planning a curriculum change of our own). And so nice to see the encouragement to plan a curriculum holistically, based on sound principles (of assessment and feedback in this case, but there were other sets of cards too, which we’d have integrated if we’d had more time).

I’ve talked about students’ difficulties with units before – on 10th and 15th March 2011. In addition to the deeper problems that students encounter, they frequently give the incorrect case when writing abbreviations for units. When they write Kg instead of kg for kilograms in an answer to an iCMA question, I have a certain amount of sympathy (especially since K and k are completely impossible to distinguish in many people’s handwriting – including mine!). However Kg is just plain wrong, so I would be unhappy to give full credit for an answer that included this error. It is really important to give targeted feedback on errors of this type.

Students also get the case wrong when the correct case of a unit is more obvious e.g. they write j instead of J for joules. And they make similar errors in writing the case 0f variables e.g. in answer to a high-stakes question where the correct answer is nRT, more than 1% of responses were only incorret because of case, with many responses of nrt, NRT etc. I find it slightly disappointing that students don’t seem to appreciate that things like this matter (even though we have explained this in the teaching text).

Consider the simple question shown below:

This question is generally well answered, but when students make a mistake, what do they do wrong? (more…)