I was due to be tutoring at the Open University residential school SXR103 Practising Science at the University of Sussex (shown left; the crane is a reminder of the huge amount of building work that is going on) for two weeks this summer. Unfortunately flu and laryngitis forced me to bail out after just one week. It was an excellent week, even if teaching is a bit challenging when you have no voice! – I’m extremely grateful to my colleagues, especially Keith, Chris and Anne-Marie, for helping to cover for me and to Dave for staying on an extra week in my place.
So now I’m home with unexpected time to catch up on some reading, writing and reflection (though my thinking may be even more befuddled than usual!). I’ve thought previously that the assessment-related issues confronting those of us who work at the UK Open University, with its ‘supported distance learning’, are not that dissimilar to those working at conventional ‘face to face’ universities. Now I’m beginning to wonder.
Let’s start by thinking about teaching and assessing introductory science at a distance (for example in the OU module S104 Exploring Science). We do everything we can to make the learning process active. Students are required to do experiments at home and there are other activities requiring them to watch videos and complete interactive activities. Each student has a tutor, and occasional tutorials (face to face or using ‘Elluminate’) are provided, but we can’t assume that our mostly part-time adult students will be able to attend these (for reasons of geography or other commitments at any specified time) so we can’t assess student contributions to them. However activities conducted within each tutor group’s online forum are sometimes assessed.
The module texts are peppered with in-text questions and self- assessment questions, and it is only a very small step from these to our formative-only interactive computer-marked questions (iCMAs). From here it is only a very small step to low-stakes summative use of iCMAs (with the weighting to act as an encouragement to students to attempt the questions in timely fashion). We know that students appreciate the instantaneous feedback on iCMA questions, and they also have a pacing and motivational function. Alongside iCMAs, students submit and receive feedback on regular tutor-marked assignments (TMAs) to their tutors, and the module ends with a synoptic end of module assessment (EMA).
All of these components have a useful function in assessing and teaching our students. Each has to be submitted by a particular date (with short extensions available on TMAs only), which helps to keep students up to date. However none of the components is timed and each is available to students for at least several weeks before the cut-off date. Everything that I’ve described so far is done for a good reason and practice has been developed and refined throughout the University’s 40-year history. However, the fact that students see their assessment questions substantially before each due-date means that assessment can drive learning to an unfortunate extent. Strategic students read the questions first and then only study relevant part of the module materials. This behaviour is prevented to a certain extent by exams on higher-level modules, but exams bring their own problems for many of our students and the learning outcomes being assessed in S104 were deemed to be more appropriately assessed by an end of module assessment (completed at home and with flexibility over when it is done and how much time is spent on it).
Compare this with SXR103 when suddenly our distance learners are being taught face-to-face, in the lab, field and on a project (requiring them to work as a member of a group). It’s pretty intense with lab work, field-trip or project from 9am-5.30pm each day and tutorials in the evening. The lab work is sometimes a bit recipe-driven (something that is being addressed as we alter the way in which we teach practical science) but I’d like to concentrate on the learning outcomes that are being assessed and the method of assessment. For the Earth science and physics activity I’ve been working on (‘Rocks and radioactivity’), the learning outcomes relate to working safely, observational skills, taking measurements, analysing results (including graph plotting and interpretation).
The key point is that both students and tutors work extremely hard to help students towards achieving these learning outcomes whilst they are with us in the lab. Assessment is by (1) satisfactory participation (not quality of results) and (2) an end of module assessment, requiring students to analyse and interpret results and to ‘write up’ one of the experiments they have completed during the week.
The significant point is that the EMA is not released until students have completed their week at residential school – so the sort of strategic behaviour I described above is not possible. This stops the residential school from being assessment-led, and does so most effectively. But we can’t currently do this on our conventional modules, because each student is working at a different pace and some require access to assessment materials before others do. For the future, a possible solution (being used on a few modules) is to only release assessment after a student has completed a particular online task, but for most modules the OU’s flexible online delivery definitely adds a problem that face-to-face teachers do not encounter.