Correspondence tuition

Posted on November 14, 2010 by Sally Jordan

At the UK Open University, the distinction between the formative and summative functions of assessment has always been blurred. This is because our tutor-marked assignments (TMAs) and computer-marked assignments (old-fashioned CMAs or modern iCMAs) frequently count towards a student’s overall continuous assessment score. But yet tutors spend a huge amount of time writing comments on their students’ TMAs and we refer to the process as ‘correspondence tuition’ – a clear indication of the importance that we place on the formative potential of TMAs. In our distance-learning environment, where opportunities for face-to-face contact with students are extremely limited, the personalised teaching that takes place in correspondence tuition becomes extremely important. Continue reading →

Posted in correspondence tuition | Tagged , , , , | Leave a comment

What sorts of e-assessment questions do students do best at?

Posted on November 2, 2010 by Sally Jordan

What do you think?

Multiple choice? Short-answer free text?

Continue reading →

Posted in question difficulty | Tagged , , , , | 4 Comments

Peer assessment : is it better to give or to receive?

Posted on November 2, 2010 by Sally Jordan

Listening to Patricia Cartney from Middlesex University speak at the Centre for Distance Education Conference last week made me think about peer assessment. Apologies to those who have been working in this area for year, my thoughts are probably terribly naive.

Students comment on other students’ work and benefit is claimed both from the fact that students are receiving feedback from others and from the fact that they are giving feedback to others. I wonder which of these is more significant? I asked the question and Patricia said she thinks it is the latter; that seems likely, but is has deep implications. To my mind it is yet more evidence that we (teachers) are waiting a lot of time in giving feedback that students don’t even understand, let alone use. As one of Patricia’s students said, the peer assessment process ‘wasn’t just about giving feedback to other people it was also whilst I was giving the feedback I was questioning my own work and learning from other peoples’ styles’.

Posted in peer assessment | Tagged , , , | 1 Comment

Flattery or feedback?

Posted on October 28, 2010 by Sally Jordan

Reading this week’s New Scientist (23rd October 2010) on the train on my way to the Centre for Distance Education Conference in London yesterday, I found an interesting opinion piece from Clifford Nass ‘More than just a computer…’ The article talks about how people like to be flattered, even when the flattery is unjustified. This resonates with my own reaction to praise – I like to be told I’ve done well even when (and perhaps especially when) my own gut reaction is that I haven’t! Continue reading →

Posted in feedback | Tagged , , , , , | Leave a comment

Multiple-choice questions – love them or hate them

Posted on October 24, 2010 by Sally Jordan

I seem to have spent quite a lot of time muttering about multiple-choice questions. You might be wondering what I have against them. It’s partly that students don’t really have to engage with the assessment process in as deep a way if they are just picking an option rather than constructing a response for themselves, but also because it is possible to guess a correct answer – so when used summatively, if a student has got a multiple-choice question right, you don’t really know whether they understand a topic or have demonstrated a learning outcome. There are ways of dealing with this problem, primarily confidence-based marking.

Also, to be fair, I have seen some excellent and creative uses of multiple-choice questions. When the OpenCETL websites are mended I will give some examples. So if the e-assessment software you are using only allows multiple-choice questions, or if there isn’t another way of asking the question you want to ask, don’t despair.  Some multiple-choice questions are distinctly better than others!

Posted in multiple-choice questions | Tagged , | Leave a comment

Writing good interactive computer-marked assessment questions

Posted on October 21, 2010 by Sally Jordan

I run a lot of workshops trying to help colleagues to write good e-assessment questions. There are usually lots of brilliant ideas in the workshop, but somehow we end up slipping back into using lots of multiple choice questions because people think they are reliable.

I suppose it is true that the answer matching is easier to set up for multiple choice and multiple response questions, but beware – just because you (as author) can easily identify one response as ‘correct’ and others as ‘incorrect’ it doesn’t mean that the question is behaving in the way you expect. The question might be ambiguous or there might actually be more than one correct option. Or – the most common problem – whilst some options are definitely correct and others are definitely incorrect, there may also be options which could either be correct or incorrect, depending on your interpretation of them. Continue reading →

Posted in e-assessment, quality | Tagged , , | Leave a comment

Making independent learners

Posted on September 12, 2010 by Sally Jordan

The first keynote at the Physics Higher Education Conference was given by Professor Derek Raine of Leicester University, who is perhaps best known for using problem based learning to teach physics and interdisciplinary science. Thinking about Derek’s work leads me to the unsurprising conclusion that getting people to work things out for themselves leads to deep learning. Do we perhaps provide too much scaffolding?

Both Derek’s keynote and the second one (from Frederik Floether (just starting Part III at Cambridge and winner of the 2010 UK Physical Sciences Centre student essay competition) considered the future of physics education. Frederik described a situation in which students take learning into their own hands, using the internet to find things out for themselves. We need to encourage this sort of behaviour : ‘You can take learning into your hands and you can do it’. Frederik also described knowledge exchange in which ‘Everyone is a teacher and a learner’ – so Wikipedia has its uses!

Posted in independent learning | Tagged , , , , | Leave a comment

Extrinsic and intrinsic motivation

Posted on September 12, 2010 by Sally Jordan

I’m now home and conferenced out (not helped by problems on the East Coast Main Line on Friday evening). The Physics Higher Education Conference (PHEC) (in Glasgow this year) was considerably better than when I first attended the event. Some of the presentations, probably including mine, were a bit predictable and some (definitely not including mine!) were based on rather flimsy self-reported evidence of student behaviour. However there were also some excellent presentations, two excellent workshops and two excellent keynote talks.

A ‘byte-sized’ (5 minute) presentation from Vijay Tymms of Imperial College has really got me thinking. His title ‘Is repeated testing at school causing physics undergraduates to become more extrinsically motivated?’ really says it all. Students certainly seem to be becoming more extrinsically motivated in that they care more about their marks that about learning. But why is this? Vijay wonders if it might be the fault of all the testing that the National Curriculum has brought into schools? That’s certainly a possibility (as is ‘grade polishing’ by repeating AS and A-level modules). Both of these things could probably be investigated by a comparison with students from other countries. I also wonder if the increased emphasis that is placed on the importance of ‘good degree’ for employment might be significant.

Posted in motivation | Tagged , , , , | Leave a comment

Good posters at ALT-C

Posted on September 8, 2010 by Sally Jordan

I was bowled over by two posters at ALT-C: Matt Haigh’s ‘Changing the way we see test-items in a computer-based environment: screen design and question difficulty’ (session 096)and Silvester Draaijer’s ‘Design of a question design support tool’ (session 148).

Both Matt and Silvester have done things that others ought to have done years ago – but haven’t. Matt has looked at the way in which the appearance of a test items (e.g. does it include a picture) alters its apparent difficulty. Silvester has designed a tool to help academics write better e-assessment questions.

Posted in e-assessment | Tagged , , , , , | 1 Comment

Counting counts but syntax sucks

Posted on September 8, 2010 by Sally Jordan

The quote I’ve used as the title of this post has been attributed to the late Professor Roger Needham at the University of Cambridge.

I can’t believe I’ve been blogging for two months and am only now mentioning our work with short-answer free-text e-assessment questions for the first time. I talked about this  yesterday at ALT-C at Nottingham University (the photo shows the Millenium Garden), in a demonstration linked to the launch of the JISC guide to ‘Effective Assessment in a Digital Age’ (in which we feature as a case study). I’m talking about the work again on Friday, at the Physics Higher Education Conference in Glasgow.

In summary: we have written computer-marked e-assessment questions in which students give their answer as a short sentence (up to 20 words). We give students three attempts at each question, with increasing targeted feedback for incomplete and incorrect responses. We have used responses from students in developing our answer-matching – I think that’s key. We’ve used both linguistically-based software (Intelligent Assessment Technologies’ FreeText Author) and the OU’s own OpenMark PMatch, which is entirely based on keywords and word-order. Both have worked well, achieving better marking accuracy than human markers. I’m not advocating  doing away with human markers, however perhaps we could use this type of e-assessment to relieve them of some of the drudgery, leaving them free to mark more sophisticated work and to support students in other ways.

Posted in short-answer free text questions | Tagged , , , , , , , , | Leave a comment