Author Archives: admin

Scholarship of Teaching and Learning around the web

Arm holding globe against hip in field with sunset

image by Ben White on Unsplash

This is the first of an occasional series rounding up interesting SoTL-related stories from around the web.

All of us who write about SoTL will find food for thought in Making Sense of Writing about Learning and Teaching (Healey, Matthews, and Cook-Sather, 2020). The authors of this open access book argue that we need to learn new ways of writing when we write about scholarship, and at the same time unlearn some of our disciplinary norms where these create barriers to inclusive communication.

The last year has seen many institutions try to get to grips with ‘pandemic pedagogy’, and this article considers the importance of inclusive design to keep students engaged when interactions are on screen rather than in person.

Staying with the subject of student engagement, Dr Jenny Scoles writes frankly about an unsuccessful intervention aimed at addressing student dissatisfaction with assignment feedback, and how students and teachers have very different perceptions of the role of feedback.

Over the past few years many universities have invested heavily in the production of MOOCS (massive open online courses), and the authors of this article explore how MOOCS can be embedded into module content, and the challenges of integrating them into face-to-face teaching.

And finally, from The Open University’s own Faculty of Business and Law scholarship blog, academic and barrister Jessica Giles proposes that interdisciplinarity might offer some answers to the question many of us have been asking ourselves in these turbulent times: how do I solve the worlds problems from my desk?

If you’ve come across any interesting online articles or blogs about SoTL do please post the links in the comments section below.

Heather Richardson Senior Lecturer, Staff Tutor & Deputy Director FASSTEST

Making the most of monitoring

There exists in the Open University (OU) a large set of largely unheralded workers who put themselves forward for one of the most important roles – the role of monitoring, which involves looking at a sample of another colleague’s marking and assessing it for standardisation to the marking criteria and quality of feedback. This group of workers monitored 36,000 assignments across the OU in 2019, 6 percent of the total received, including 3,735 from the School of Psychology & Counselling and nearly 1000 from one module alone (DE300 ‘Investigating Psychology’). Monitoring of assessment, for standardisation of marking and quality of feedback, is an essential feature of maintaining academic standards in an era of scrutiny of higher education provision (Bloxham and Boyd, 2012). In a distance learning institution, some students’ experience of communication with their tutor may only exist through feedback provided on assignments despite tutors’ best efforts to engage (Tsagari, 2019), so the importance of this feedback could be seen as greater than at campus-based institutions. The subjective judgements made by markers (Brooks, 2012), despite module team provision of detailed marking criteria, can also lead to standardisation issues, with assignments marked leniently or harshly. The monitoring process within the OU is vital to ensure discrepancies in marking and variabilities in feedback are picked up and addressed. The role is fundamental to the university’s principles and purposes of assessment – they ensure quality of teaching, to contribute to the professional development of tutors, and to validate assessment scores.

Monitoring as a dialogue

Monitoring in the past, though, has sometimes been associated with questions about the difficulties of monitors providing what is essentially a peer review and judgements on their colleagues. Concerns about training for monitoring led to a previous scholarship project in Psychology and Counselling at the OU in 2016 on ‘Monitoring: Good Practice’, which resulted in improvements to training for monitors and a dedicated website. An important feature of monitoring for the university is that it should be a dialogic relationship with the monitor and the tutor further discussing the process, perhaps working together to think more about successful application of marking criteria and discussing effective feedback. However, further evidence was needed to critically evaluate the prevalence and effectiveness of the ‘dialogic process’ aspired to by The Open University. To address this gap, a small-scale scholarship project was carried out, supported by FASSTEST, between February and July 2020 to explore monitoring from the experience of both monitors (n=24) and tutors (n=27) in the School of Psychology & Counselling. Content analysis of monitoring reports (n=46) across a range of undergraduate and postgraduate modules also looked for examples of good and poor practice.

Encouraging outcomes

Outcomes of the project were encouraging, particularly around the impact of monitoring on professional development for both tutors and monitors – the majority of both reported that their own practice had improved from receiving monitoring reports (21/27) and being a monitor (24/24). Monitors had gained from seeing how other colleagues assessed, and often as a result adopted (or as some suggested ‘plagiarised’) examples of good practice. Tutors, particularly those who were new to the university or to the module being monitored, reported benefiting from the feedback particularly around knowing they were applying the marking criteria appropriately.

The dialogic process between monitors and tutors, however, was less evident, with only half of tutors having further contact with monitors, usually because they did not feel it beneficial to follow up. As one tutor put it, ‘if you receive a great report, there is little more to be said’. Sometimes the dialogue was more about challenging the report and explaining and justifying the assessment by referring back to knowledge a monitor may not have – the tutor’s relationship with the student, for example, which could mean less feedback because of the student feeling overwhelmed when receiving too much.  It is recognised that discussions between markers and assessors can be beneficial in improving standardisation (Grainger et al, 2008) so the OU expectation and encouragement for the dialogue between monitors and tutors could result in improved assessment practice. Monitors worked hard to encourage further dialogue with tutors; reports typically were friendly and encouraging, and most monitors included email contact information with a ‘please contact me to talk further’ message within the report. The decision then is left to the tutor whether further contact takes place, and that will be something that workload and time management may well also impact.

As with any peer review process, there were a small minority of experienced tutors who did not feel monitoring was particularly beneficial after teaching a module for several years and sometimes questioned the judgement of monitors who were less experienced than themselves. Receiving a good rather than an excellent from a monitor was occasionally met with resistance. However, within any peer review or assessment system, there will always be emotive responses alongside cognitive benefits particularly when perceptions of experience are involved (Cartney, 2010).

A shot of a table from above with a laptop, cup of coffee, pens, a mouse and magnifying glass displayed

Photo by Ian Dooley on Unsplash

Content of reports

The content analysis of reports showed that a majority demonstrated very good practice – supportive, friendly, accurate and full of appropriate advice. Sometimes there was evidence of cut and paste errors, though, where monitors had perhaps taken information from a previous report or another similar report and inadvertently not changed the name. Whilst these were very few overall, there were some instances were tutors felt these errors undermined the value of the reports. Tutors also felt that sometimes monitors were not able to acknowledge the personal relationships tutors had with their students, which could explain a lower level of feedback, for example, for students who a tutor might judge to manage a limited amount of information in each set of feedback better (Forsythe and Johnson, 2017). There was also doubt that monitors looked back to previous reports to acknowledge what had been said before.

Being honest in peer review

It can also be difficult for a monitor to be completely honest in a report. Nearly all monitors acknowledged they had monitored a friend or close colleague, although the majority did not consider this to be an issue. The ability, though, to provide an unbiased critical judgement of a friend’s or colleague’s assessment practice could well be affected (Abidin et al, 2018). On small modules avoiding monitoring close colleagues can be problematic – with only two monitors on a module, inevitably they will end up monitoring each other. However, on larger modules this can be avoided. Notably, in other faculties on larger modules monitors are asked in advance to check their list and remove any colleagues they are familiar with to avoid this.

Professionalising the monitor role

Perhaps one of the most important findings of this research, though, relates to the status of the role of monitor. Nearly all monitors considered their role as important and valued by the university and colleagues, but very few felt they were an important part of a team, and most reported they had very little communication about their work. Half of monitors reported feeling isolated in their role. The monitor role also lacks the professional status that other roles (for example, associated with academic conduct) command. A more recognised professional status of the monitor, with better integration into Module Teams, could go some way to making monitors and tutors see the role as more valued.

Finally, changes in the monitoring system were implemented during the period this research was being carried out. These changes affected monitoring levels, and the way in which the report form is completed, are significant improvements and early reports from monitors and tutors suggest they find the forms easier to use. It was encouraging that many of the findings of the research were being addressed by monitoring process changes that were taking place at the same time. This validates both the research findings and the decisions of the Open University’s Monitoring Implementation Group, which have designed a new website and handbook.

References

Abidin, A.N.Z., Masek, A, Mohd Faiz, N.S. & Sahdan, S. (2018), Exploring the elements of integrity in peer assessment, MATC Web of Conferences, 150, 05002. https://doi.org/10.1051/matecconf/201815005002

Bloxham, S. & Boyd, P. (2012) Accountability in grading student work: securing academic standards in a twenty-first century quality assurance context, British Educational Research Journal, 38, (4), pp. 615 – 634. https://doi.org/10.1080/01411926.2011.569007

Brooks, V. (2012), Marking as judgement, Research Papers in Education, 27, (1), pp. 63 – 80. https://doi.org/10.1080/02671520903331008

Cartney, P. (2010), Exploring the use of peer assessment as a vehicle for closing the gap between feedback given and feedback used, Assessment & Evaluation in Higher Education, 35, (5), pp. 551 – 564. https://doi.org/10.1080/02602931003632381

Forsythe, A. & Johnson, S. (2016), Thanks, but no thanks for the feedback, Assessment & Evaluation in Higher Education, 42, (6), pp. 850 – 859. https://doi.org/10.1080/0260938.2016.1202190

Grainger, P., Purnell, K. & Zipf, R. (2008), Judging quality through substantive conversations between markers, Assessment & Evaluation in Higher Education, 33, (2), pp. 133 – 142. https://doi.org/10.1080/02602930601125681

Tsagari, D. (2019), Interface between feedback, assessment and distance learning written assignments, Research Papers in Language Teaching and Learning, 10, (1), pp. 72 – 99. Available online at http://rplel.eap.gr

For further details please contact

Sue Nieland – Staff Tutor in Psychology and Counselling

Every picture tells a story: ‘Creative Interactions’

An experiment in creative learning

On a grey Saturday morning in March 2020, as the UK teetered on the brink of the first lockdown, a small group of students gathered at the Open University campus in Milton Keynes to take part in an experiment. Half of the group were studying Creative Writing, and the other half were students of Art History.

The experiment – a project called ‘Creative Interactions’ –   had been dreamt up by me (Dr Heather Richardson, Creative Writing) and Art Historian Dr Clare Taylor, ably assisted by two of our Associate Lecturer colleagues, Dr Helen Mosby and Dr Diana Newall. The plan was to find out what happens when the teaching techniques of the two disciplines are swapped around. Luckily for us the Open University has an extensive art collection, much of which is on display around the campus. This meant we were able to come face to face with artworks from all around the world over the course of the day.

Ekphrasis and visual analysis

Ekphrasis is a familiar phrase to many people studying Creative Writing and comes from the ancient Greek words for describing or speaking out. Put simply, ekphrasis is writing that describes a piece of visual art, and it’s cropped up in literature for hundreds of years – think of Keats’ ‘Ode on a Grecian Urn’, for example. Every Creative Writing tutor has some ekphrastic exercises in their toolbox, where we show the students an image and encourage them to let their imaginations off the leash. What we’re hoping for is an emotional response – a gut reaction – that will lead students to a creative response in the form of a story or poem inspired by what they see.

Art History tutors and students on the other hand engage with artworks in a very different way. Their first step when they encounter an artwork is to carry out a visual analysis, considering issues like composition, colour, and the materials used. Another important element is the historical or social context in which the work was produced. It’s all very different from the way Creative Writing students make use of visual art.

Of course, we wanted our student volunteers to get practical benefits from the workshop that would help them on their modules, so for the first part of the day they worked with the tutor from their own discipline and engaged with several pieces of art in the usual way for their subject. It was then that the experiment really began as the two groups swapped, with the Art History students working with the Creative Writing tutor, and vice-versa for the Creative Writing students.

Imaginative adventures

I sat in with the Art History students as Creative Writing tutor Helen Mosby led them through a series of activities while looking at David Tindle’s Mural Panel C. At first glance this looks to me like a lovely image of a summer day, viewed through partially open French windows. Helen drew our attention to the old-fashioned telephone, receiver off the hook, sitting outside the French windows, and we tried to imagine the circumstances behind it. Had someone been trying to make a secret phone call, out of earshot of anyone else in the house? Or had some emergency occurred which had led them to throw the receiver down and run off?

The Art History students pointed out the areas of shadow amongst the trees on the right-hand side of the painting, and the misty effect caused by the barely-visible voile curtains, and with our creative writing hats on we wondered if these suggested a darker or unsettling aspect to the stories we were inventing.

For the final part of the workshop we all came together to consider a triptych – an artwork made up of three pieces – with input and discussion from both disciplines, followed by a creative writing session.

Lined page filled with text

Participants response to one of the artworks

Insights

At the end of the workshop we asked the students what they had learnt. Unsurprisingly, both disciplines said they felt better able to carry out a visual and formal analysis of artworks. One Creative Writing student said, ‘Looking at artwork from an art historian’s perspective would lead to different stimuli’, while another said they had learnt ‘How to interpret art as a medium with a purpose’. The Art History students commented on the creative writing activities, with one saying, ‘Spontaneous creative writing is hard!’ while another acknowledged the benefits of writing without hesitation, stating emphatically, ‘I can write!’

The students reflected on how the workshop had enhanced their appreciation of both their ‘home’ and ‘other’ discipline, with Creative Writing students feeling that analysing the artworks in detail would help them with their descriptive writing, while the Art History students concluded that thinking of an artwork in terms of narrative or subject matter would add a new dimension to their interpretation.

What next?

After the workshop Clare, Diana, Helen and I agreed that the experiment had proved the value to students of a creative interaction between their disciplines. Our next step will be to roll out a virtual version of the workshop. Going online isn’t just about dealing with the ever-changing Covid-19 regulations, but also a way to ensure that more students get the opportunity to participate – not just those who live within travelling distance from Milton Keynes. And we plan to talk about ‘Creative Interactions’ at conferences and staff development events for Creative Writing and Art History tutors. Exciting things happen when two disciplines work together, and we’re going to make sure that story gets told.

For further details please contact:

Heather Richardson, Senior Lecturer, Staff Tutor & Deputy Director FASSTEST

The Battle for Latin: Reports from the front line of research into Latin teaching

Hiding behind any respectable project that seeks to improve the student experience is always an element of nerdiness – an obsessive love of stats and number-crunching – as well as a curtain-twitching urge to take a peek into the lives of others. What precisely do teachers and students get up to in their classrooms? How does it all go so right for students – and sometimes so wrong?

Well, that’s our experience anyway. We are James Robson (Professor of Classical Studies at the OU) and Dr Mair Lloyd (Associate Lecturer and former OU PhD student) and our story starts about seven years ago when we met up for the first time over multiple cups of tea, nursing the shared ambition of carrying out a bold project that hadn’t been attempted for a generation: a nuts-and-bolts survey of beginners’ Latin and Greek teaching in Classics departments across the UK. We carried out this survey in 2014 and as the data poured in and we feasted greedily on the diet of bar charts and pivot tables we were creating, one statistic kept jumping out at us: nearly one in four students who began studying Latin at university (23%) didn’t complete their module. Why was that? And how come the pass rates at different universities varied so greatly?

Image: Mair and James drinking tea in contrasting settings

That is how The Battle for Latin was born, a project dedicated to examining the factors driving student success, failure and withdrawal amongst beginners’ Latin students. More in hope than expectation, we put together a bid for the British Academy small grants scheme, drawing on James’ long experience of classical language teaching at the Open University and Mair’s expertise in Modern Foreign Language and Latin pedagogy. Importantly for the bid, we were able to cite the statistically-rich research that we had already published. And crucially, too, we had a burning question that, to us at least, seemed so vital to answer: why were so many aspiring Latinists in UK universities unable to stay the distance?

A few months later, we learnt that our bid had been successful and when our spontaneous whooping and partying eventually subsided, we set to work. As stats fiends, one thing we felt we needed was more targeted, up-to-date data, so we ran a new survey of UK Latin instructors, who between them kindly furnished us with data on 30 different Latin modules covering 888 students nationwide. As our graph below shows, the variation in completion rates was striking once again. Remarkably, too, we also learnt that, while a greater number of students were completing and passing our own Latin module at the OU, this was not a picture reflected across the sector: nationwide the completion rate for beginners’ Latin modules was stuck at 76%.

Image: One of Mair’s and James’ signature graphs: Beginners’ Latin modules in UK universities listed by anonymized alphanumeric code, showing percentages of those starting the module who passed, failed and withdrew.

So, what were the factors driving student success, withdrawal and failure? Our data allowed us to rule out elements such as module duration, credit value and even student contact hours to a large extent (the exception being a handful of particularly intensive modules which included five or more hours’ classroom time each week). Nor did the choice of textbook or assessment strategy appear to be determinative (although we did note a possible benefit of including substantial elements of assessed coursework). Ultimately, whatever hypothesis we investigated there always seemed to be modules that bucked the trend. Clearly, staring at the stats was only going to get us so far.

Fortunately, our project also built in human contact: a series of whole-class observations, interviews with instructors and students, and even an online student survey to allow us to understand better the obstacles to student success. We learnt a lot from our activities, not least the need to ask a small number of very focused questions if you hope to finish the interview on time! But we were also delighted to discover the warmth, dedication and reflectiveness not only of Latin instructors, but also of the students they teach, who were generous with their time and hugely thoughtful and thought-provoking in their responses to our questions.

So, is there a magic bullet for improving student success on beginners’ Latin modules? Well, maybe not, but our research nevertheless provides some useful trends and pointers, we hope. Plus, our forthcoming paper also distils some of the top tips that Latin students would pass on to new starters – all the more useful, we hope, for being direct quotations in the students’ own voices. If we have an overarching conclusion, however, it is perhaps that the magic lies somewhere in the dynamic interactions between the teacher, students, textbook, teaching methods and class as a whole. Of course, these are factors that are challenging to quantify and pin down – but this merely convinces restless enthusiasts like us that another research project is needed to scrutinize these more closely.

A paper with further details will be published shortly, we will post a link here when the article is live.

For further details please contact James and Mair:

James Robson, Professor of Classical Studies

Dr Mair Lloyd (Associate Lecturer and former OU PhD student)

Welcome to the FASSTEST blog!

FASSTEST (FASS Teaching Excellence and Scholarship of Teaching) is the Centre for Scholarship and Innovation at The Open University’s Faculty of Arts and Social Sciences. By ‘scholarship’, we mean the scholarship of teaching and learning. This means that FASSTEST supports the critical, scholarly evaluation of approaches to teaching and learning in the Arts and Social Sciences with the aim of providing an evidence base for good practice in higher education.

Our blog will include posts from practitioners and researchers discussing issues and research findings relevant to teaching and learning in the Arts and Social Sciences in HE. Here at The Open University we can draw on particular expertise in online and distance education, and we are especially inspired by The Open University’s mission to be open to people, places, methods and ideas. We are committed to placing innovation at the heart of teaching and learning and to exploring new and better ways to inspire and enable learning, opening up higher education to all, regardless of background or circumstances.

We welcome blog posts from anyone with an interest in the scholarship of teaching and learning, particularly in the Arts and Social Sciences. If you are interested in contributing please get in touch with FASS-Scholarship@open.ac.uk with a title and brief description of what you would like to write about.

Stefanie Sinclair
(FASSTEST Director)