Image by Tim Gouw on Unsplash

As an Open University tutor, I am very aware that some students find it hard to join online tutorials and others who do join may not participate. Although many participate by using the chat box or take part in other interactive activities such as answering polls or writing on the whiteboard, few students turn their microphones on to speak. Without visual clues, it is very hard to interpret how students are feeling when they take part in online tutorials, and in larger tutorials it is difficult to know when or why they might disengage from the session. As someone who experiences anxiety myself, particularly in relation to being in an online environment, I thought that it was likely that some students would similarly experience anxiety and I wanted to find out more about what the triggers for anxiety are in relation to the online tutorials that we offer as part of the teaching on an Open University module. I also wanted to explore ideas about what we could do as tutors and as an institution to help alleviate such anxiety which would in turn help students to develop their confidence both in joining and  participating in online tutorials to get the most out of their academic studies. 

My project was in two stages: a questionnaire sent out to students on three Social Science Level 1 modules followed by interviews a few months later with a small cohort of students who had completed the questionnaire. 

The questionnaire was completed by just over 600 students. It asked questions about anxiety in relation to joining and taking part in online tutorials. Students were asked to give their age band. The questionnaire was sent out when students were in their second month of study on their module, so they would have had several opportunities to join an online tutorial. It found that nearly 1 in 6 students had not attended an online tutorial because they did not feel comfortable in doing so and that nearly 3 in 4 students who have taken part in an online tutorial had experienced some level of anxiety in doing so. The aspects of online tutorials that caused the highest anxiety were being asked to speak and being asked to directly answer a question, with more than 3 in 4 students who have some anxiety about participating in online tutorials reporting significant or moderate anxiety about doing so. The percentages were higher amongst younger students, particularly those under 25. 

The interviews were conducted with a small number of students from different age bands registered to a variety of Social Science qualifications. All had taken part in online tutorials. They were asked questions in relation to the anxiety they experience when joining and taking part in these sessions and were invited to consider whether certain strategies would help them to manage their anxiety. During these more in-depth discussions, it became clear that there was a range of different interpretations of the concept of anxiety, with some students happy to talk about the anxiety that they experience whilst others preferred to talk about stress or apprehensiveness, although the triggers for these feelings were the same.  

The interviews found that the anxiety that students felt about joining or taking part in online tutorials at the start of the module fell into three key areas: 

1. Accessing the room/technical issues
2. What the tutorial would be like/the ‘abstractness’ of being in a virtual room
3. Fear of saying something ‘stupid’/not being able to keep up with the other students.

One of the key triggers for anxiety in relation to joining a first online tutorial was not knowing what it would be like, particularly whether students would be required to have cameras on. 

For the most part, interviewees felt that their anxiety diminished over time and with experience of how online tutorials are run and what their options are for taking part. It also helped to be familiar with the tutors running the sessions as well as names of the other students in the session. This was helpful in reducing anxiety related to feeling out of step with the group. 

All interviewees thought that being able to access a short video showing what the online tutorial platform looks like would have been very effective at helping to manage anxiety ahead of the first session. Interactive activities such as polling were popular as they allowed participation whilst remaining anonymous. Although the idea of breakout rooms was a trigger for anxiety amongst the interviewees, most said that they were more likely to speak in smaller groups and that this would help with getting to know other students. 

In summary, the questionnaires and interviews found that there were a number of triggers for anxiety amongst Level 1 students in relation to joining and participating in online tutorials, many of them related to the level of confidence of the student. A first step to helping to alleviate such anxiety will be to make a short video available that shows what an online room looks like as this will help students to know that they do not need to be on camera and can participate as much or as little as they want. In addition, it will be useful to think about how to develop confidence in participating, with well-managed small group activities that promote conversation and discussion, since familiarity in doing this should help alleviate anxiety in the longer term. 

Beyond this, it would be useful to think about what could be done to promote both student identity and a sense of student community since this will make it easier for new students to settle in to making academic study part of their lives and develop confidence in their skills. 

Janet Hunter, Lecturer in Social Sciences & Global Studies, Politics

Image by Jeanne Provost via Shutterstock

Admittedly, the Arts and Humanities are not traditionally known as the spiritual homeland of number crunching, but in 2019, colleagues and I from the departments of Classical Studies, History, English, and Creative Writing at the Open University broke with tradition and embarked on a quantitative analysis of student attendance patterns within our school.  

Three years before, the Open University had introduced a bold new framework for tuition, the Group Tuition Policy, that guaranteed students an online alternative for every face-to-face session on offer. The objective of the policy was to ensure students had a greater range of options (face to face or online/weekday evenings or Saturdays) when it came to choosing a tutorial to suit their schedule and needs.  

We reckoned it was time to take a good hard look at attendance in the wake of these changes.  What was working well? What wasn’t?  So much of our tuition strategy and implementation in the past had been based on gut instincts about what works well for students and tutor—but we wanted to establish a solid evidence-base for tuition moving into the future. With this in mind, we donned our data-wrangling chaps, and headed out into the wild frontiers of attendance statistics. 

Using data from our booking and registration system (LEM), we tracked student attendance from 2017-2018 (eventually adding in 2019-2020 in the second phase of our project) to help us get a broad overview of attendance across a range of Arts & Humanities modules, but also to give us a snapshot of student behaviour at a more granular level.    

We had all sorts of questions about attendance. What percentage of students actually attend learning events? What kind of events do students prefer to attend—online or face to face? Lectures or smaller tutorials? How does attendance relate to academic outcomes for students: do students who attend learning events do better than those who don’t? How has COVID impacted on student attendance? While the answers to these questions were interesting in themselves, what drove us was a desire to get it right for students in the future: how can we plan tuition so that it is attractive to students and helps them achieve their study goals?  How can we make the most of our present tuition resource so that our learning events benefit the maximum number of students? 

What we came up with, after many, many hours of wrangling data into submission, was a fascinating picture of student attendance behaviour within the school of Arts & Humanities.  It may not be the I Ching of attendance, but our report generated a number of valuable insights into the state of student attendance in our school. Here are a few tantalising snippets we thought you might like to know: 

Students want tuition.  There is clearly a strong demand for it. While attendance at face-to-face sessions remained fairly stable between 2017-19, attendance at online sessions seems to be growing, year on year, and we need to plan for that in terms of investment in our teaching platforms, staff development, and tuition offerings.   

Attendance has remained relatively stable, and even risen, under lockdown. Apart from a dip in mid-May 2020 when many end-of-module assessments were cancelled for students, attendance during 2019-2020 was generally higher than it was the previous year, and this trend appears to have continued on modules presenting from October 2020 as well.    

There is a strong link between attendance and attainment: students who got Pass 1 and 2s were more likely to have attended a learning event (and more of them) than those students with Pass 3 and 4 grades. 

Cancellation might be our best friend—when it comes to registering for online learning events at the beginning of a module, many students panic and book more events than they will realistically attend. Unfortunately, they often neglect to cancel bookings they don’t intend to use. Our study found that 1 in every 3 students who register for an event fails to show up! This means that a large number of the students on waitlists could have been accommodated at learning events if appropriate cancellations had been made.  Clearly, we’ve got a job to do creating a booking system that minimizes un-used bookings, but we also need to cultivate a culture amongst students of responsible registration. 

Some students REALLY like to attend. When we sampled student behaviour, it became clear that there was a wide range of engagement within our student cohort: some students didn’t attend at all, while others attended a great deal.  Of those who attended a great deal, we discovered the phenomenon of the ‘super-attender’, that species of student who not only books but also attends a massive amount of learning events on a single presentation. One student in our sample managed to attend a whopping 40+ sessions over the course of one module (and actually booked but did not attend a further 10). It’s not clear that attending this many sessions on a limited number of topics will actually help the student achieve their study objectives—in fact, it may rob them of time better spent on personal study and assignment writing.  While it’s great for students to have choice, booking into this many events means that one student’s choice may sometimes come at the cost of others who didn’t book early enough. Such students are relatively rare, but such a phenomenon does draw attention to some of the risks involved with having a booking system with no limits.  

The project has been challenging but hugely productive, laying the groundwork for a series of conversations with our colleagues in the school of Arts & Humanities who are responsible for designing and implementing tuition strategies on future presentations. Our findings have also been a springboard for broader conversations within the university community, with stakeholders ranging from tutors to university policymakers and system-commissioners.  All of us are keen to make the most of our resources to meet students’ needs in the coming years and we firmly believe that the best way to do that is by sharing our understanding of what is working—and what is not—in our present tuition and systems offering. 

Have you been involved in a project looking at student attendance, in another faculty of the Open University or at another institution? Have you found that these trends in student behaviour really resonate with you as a tutor, teacher, or tuition manager?  Or perhaps you’d like to read a more detailed account of our methodology, conclusions, and recommendations? We’d love to hear from you. Please do add a comment below or drop us a line at jennifer.shepherd@open.ac.uk or astrid.voigt@open.ac.uk. 

Jennifer Shepherd & Astrid Voigt 

on behalf of the ‘Investigating Tuition Attendance in Arts & Humanities’ Project Team (Robin Mackie, Steve Padley, Maddy Sharman, Jen Shepherd, Lee Simmonds, Astrid Voigt) 

Image by Jagrit Parajuli from Pixabay

During the Covid-19 pandemic we have seen an exponential growth in the number of online academic conferences. While the online medium has posed some challenges, it is increasingly becoming apparent that holding conferences online offers a lot of opportunities beyond the pandemic. Online conferences are more accessible for people with disabilities or caring responsibilities. They are a lot cheaper (given that there are no travel or venue costs and no catering or accommodation have to be provided for delegates) and environmentally friendly (again, because online conferences don’t require delegates to travel). Online conferences also open up a host of new opportunities for knowledge exchange and can be easily recorded and widely shared.   

However, organising and attending an online conference requires new thinking. It’s not just a question of the availability and technical knowledge of different tools and platforms, but of planning how to use these tools wisely and effectively to engage and energise participants, be inclusive, widen participation and deliver good content. At the Open University, we have a lot of experience of teaching online, and particularly with engaging people from a wide range of different backgrounds. How could this expertise be applied to running academic conferences and make them more engaging and inclusive? 

FASSTEST, the Centre for Scholarship and Innovation at the Open University’s Faculty of Arts and Social Sciences, has supported a scholarship project aiming to identify best practice in the organisation and delivery of online academic conferences in the Arts and Social Sciences. This involved a literature review (of scholarly literature on online engagement as well as of existing ‘how to’ guides on running conferences online) and semi-structured interviews with 20 colleagues at the Open University, who spoke about their experiences as organisers, attendees or support staff of online academic conferences. Based on this work, the project team have put together a guide with tips and tricks of how to run engaging and inclusive academic conferences online. This guide is now available and can be accessed here: Planning Online Conferences in the Arts and Social Sciences   

Even as we start to open up again, online conferences are here to stay – so let’s do our best to make them as useful as possible!  

Dr Stefanie Sinclair 

Dr Stefanie Sinclair | Faculty of Arts and Social Sciences (open.ac.uk) 

Image Hatice Yardim / Unsplash

Background

Between August 2011 and July 2019, I was the Lead Academic Conduct Officer for the Arts and Humanities at The Open University.  My role during this time primarily involved investigating cases of poor academic practice (aka plagiarism) which had been referred for disciplinary treatment.  As such, these cases required a formal approach, including sending students evidence of our concerns in the form of marked-up copies of their essays and asking students to explain in writing the reasons behind the lack of appropriate referencing in their assignments.  Students’ responses almost invariably brought up one or more of these issues:

• Lack of appreciation of when and why to acknowledge other people’s work,
• Panic and confusion around how to reference,
• Lack of confidence in their ability to write in their own words, or to paraphrase effectively,
• Special circumstances leading to difficulties finding time for or being able to concentrate on their studies.

The overwhelming reoccurrence of such responses soon led me to realise that plagiarism at The Open University truly was ‘the surface manifestation of complex learning difficulties’ (Angelil-Carter, 2000, p. 2).  Our open entry policy means that many of our students have limited prior educational experience and will often lack basic awareness of good academic practices.  I knew from this point that I needed to use whatever influence I had as an Academic Conduct Officer to move beyond ‘robust and transparent procedures for detecting and punishing plagiarism’ (Park, 2004, p. 294).

My early and relatively modest attempts to make a difference to students’ chances and experiences focused on ensuring that the disciplinary process was not just about issuing warnings or penalties but, also and crucially, more of ‘a place where learning can occur’ than had hitherto been the case (Carroll and Appleton, 2001, p. 30).  I complemented these reactive efforts with ongoing support and guidance to colleagues via regular staff development interventions as well as the production of a toolkit to assist tutors who are asked to deliver a study skills session.  Ultimately, my aspiration was to help introduce proactive ways of supporting students that were suitable to our part-time, distance learning model with relatively limited formal opportunities for students to meet tutors and fellow students and with formal teaching content delivered via our high-quality module materials.  And so, mindful also of the importance of relatable examples in helping students learn, I lobbied for study skills to become embedded into the teaching and assessment strategy of our modules (Lampert, 2008, p. 11)

From aspiration to reality: early indicators

The obvious place to introduce a new approach was our foundation Arts and Humanities module, which had an intake each academic year of around 4000 students, and an opportunity arose in 2015 as plans to revise our Level 1 teaching commenced.  Our new foundation module, Discovering the Arts and Humanities eventually launched in October 2019 and includes a number of innovations.  Amongst these is a unit on Academic Integrity, written by myself, which: contains practical advice and activities based on my experience of students’ struggles with good academic practices, draws directly from the module materials for examples, and is assessed via a short compulsory quiz.  The unit, whose primary concerns are with the when and why of referencing, with encouraging students to reflect on the impact of quotations, paraphrases, and summaries, and with providing some strategies to encourage them to use sources effectively, is supplemented by a larger suite of strategically placed Study Skills sessions with practical exercises on topics such as time management, note-taking, planning, and writing assignments, reading, and creating references.

Given that teaching good academic practices actively, extensively, and as part of a structured and integrated approach was new in the Arts and Humanities, I was keen to assess the impact of this innovative approach.  To this end, as part of a larger academic-conduct project I am currently undertaking, I am seeking to ascertain whether Discovering the Arts and Humanities is more effective in developing students’ skills than its predecessor module – The Arts Past and Present­ – was.  I am doing this by considering trends in poor academic practice referrals for the two modules as well as responses to a short questionnaire sent to students who have completed either module between October 2018 and September 2020 (so students from two presentations of each module).

It is admittedly difficult to link any change strictly to the modules’ handling of good academic practices.  For instance, although on average the profile of students undertaking the Arts and Humanities foundation modules since February 2018 is similar in terms of gender, age, disability, and ethnicity, it has been more likely for Discovering the Arts and Humanities students to be registered on more than 60 credits at any given time in the presentation than was the case with The Arts Past and Present­ (AA100) students (29.7% v. 19.54%). With regards to the questionnaires, also, the responses from AA100 students could well be influenced by these students’ experiences since completing the module, or by the time passed between finishing the module and responding to the questionnaire (about a year).  There is, additionally, the wider complicating factor of the Covid-19 pandemic, which will have affected all students who have enrolled on the current foundation module so far in different ways.

All of the above notwithstanding, preliminary analysis of both the quantitative and qualitative sets of data suggests that the new model is working.  On the one hand, academic conduct referral levels in the first three presentations of Discovering the Arts and Humanities (A111) are on average lower than those in the final three presentations of the previous foundation module (1.08% of all submissions were referred for poor academic practice v. 1.34%).  What is particularly noticeable, in fact, is the sharp decline in disciplinary referrals: these accounted for 16.9% of all A111 cases of concern between October 2019 and May 2021, whereas 32.3% of all referrals on The Arts Past and Present­ between February 2018 and September 2019 were for disciplinary treatment.  On the other hand, survey responses seem to indicate that the current module is providing students with more of what they need – at least judging by the number of comments (30 v. 8) picking up on the modules’ general effectiveness in developing students’ academic skills.  Appreciation for the new module’s focus on skills is repeatedly expressed in responses, often in connection to the advantage of study skills sessions being short, embedded in the module materials, and gradually released.  The Academic Integrity unit is also generally reviewed as a useful and insightful resource, helping build students’ confidence in when and why to reference.

Perhaps unsurprisingly, by far the most common response across both questionnaires when students were asked about areas their modules could have helped them more with is that referencing presents a real struggle and is something students identify as hard work.  There are some signs that this seems less acute for The Arts Past and Present­ students since they have had more practice in the area since completing this module.  Nonetheless, it is telling that referencing remains a source of concern after the foundation year, and that there are significant overlaps between the responses mentioning referencing across both questionnaires.  They highlight, for instance, that students particularly value referencing examples explicitly identified as such within the module materials (i.e., the Assessment Booklet for The Arts Past and Present­ or the ‘How to cite this unit’ pages within Discovering the Arts and Humanities).  Yet some students interestingly reflect that this promotes overreliance on these models rather than equipping students to understand how to reference, while some others in fact admit that they are still referring to the Assessment Booklet when referencing or that they felt lost once ‘How to cite this unit’ pages were no longer available towards the end of the presentation.  Respondents across both questionnaires also express their preference for more, more regular, or more structured guidance, and it is also apparent that at least some students would welcome an open discussion on referencing, tutor support being identified in various parts of the surveys as pivotal to students’ development of academic skills and, in particular, their understanding of referencing.

Next steps

Shortly after I finished writing the Academic Integrity unit for Discovering the Arts and Humanities (A111), I became the Deputy Chair for a module that in many ways follows on from it – Cultures – and was asked to take the lead developing its Study Skills strategy.  Eventually, this materialised as a suite of 8 sessions which appear at regular intervals throughout the presentation, use examples from the adjacent module materials, and build on the skills introduced on A111, with particular focus on working effectively with sources and academic writing development.  Responses to the questionnaires outlined above happily seem to confirm that these are two areas students would generally like more support with, and now that Cultures (A112) has presented twice (it launched in October 2020), we are now conducting a survey with current students as a way to test whether the study skills suite has worked in the way we anticipated.

I am also involved in Level 1-wide conversations about a number of follow-on initiatives, not least one which seeks to address two key findings from the questionnaires (the evident anxiety referencing still causes and the value students place on tutor advice and guidance) at the same time as upholding our firm belief that, at Level 1, the emphasis should be on the when and why and not so much on the how of referencing.

For further details please contact:  Encarna Trinidad, Lecturer and Staff Tutor in English

References

Angelil-Carter, S. (2000) Stolen Language?: Plagiarism in Writing. Harlow: Pearson Education.

Carroll, J. and Appleton, J. (2001) Plagiarism: A Good Practice Guide [Online]. Oxford: Oxford Brookes, Joint Information Systems Committee. Available at https://i.unisa.edu.au/siteassets/staff/tiu/documents/plagiarism—a-good-practice-guide-by-oxford-brookes-university.pdf (Accessed 9 June 2021).

Lampert, L. D. (2008) Combating student plagiarism: an academic librarian’s guide. Oxford: Chandos.

Park, C. (2004) ‘Rebels without a clause: towards an institutional framework for dealing with plagiarism by students,’ Journal of Further and Higher Education, vol. 28, no. 3, pp. 291-306.  Available at: https://doi-org.libezproxy.open.ac.uk/10.1080/0309877042000241760 (Accessed 9 June 2021).

Image by Fizkes on Shutterstock

Submitting an assignment is usually accompanied by a sense of relief but then there is the wait for the mark and the feedback which can cause anxiety amongst students.  This is particularly the case for the first assignment on a new module and happened with DD212 Understanding Criminology in its first year of presentation. 

At the Open University (OU) part of our teaching is through written assessment feedback: explaining how well students have met the learning outcomes and met the assignment requirements.   So aside from the mark, there are the inserted comments in the text plus detailed general advice on strengths and weaknesses at the end. 

Most OU tutors work part-time and many teach on several modules.  Each tutor has 10 working days after the cut-off date to complete all the marking of a particular module assignment, called tutor marked assignments or TMAs.  The assignments are returned via an electronic assignment portal – the eTMA system  

Whilst some tutors mark all assignments early on and might return after three or four working days; others mark towards the end of the period.  Some tutors mark a few and return as they go along.  Unlike the end of module results, where the results are all released electronically at the same time, the electronic TMA system does not ‘hold’ marked TMAs and release them all at the same time.   This means that there is no standard point at which students receive their TMAs.  A tutor can mark and then return all TMAs at the same time but the eTMA system does not do this returning in an automated way. 

As many students are now on social media – for example some students set up their own module Facebook group – they are readily commenting on getting their assignments back.  This can then cause concern and raise anxiety amongst those who haven’t yet had their assignment returned.  This quote from the module Facebook group is typical of the comments made by students: 

It’s just frustrating when there’s a considerable amount of people with their results back and I’m still over here waiting 

Sometimes this results in telephone enquiries from anxious students to the Student Support Team (SST) and occasionally complaints.    

In autumn 2018, the Module Team of the new level 2 Criminology module (DD212 Understanding Criminology) were alerted by the Student Support Team that some students were ringing in about not receiving their marked TMA01. The module Chair was a member of the DD212 Facebook group and was alerted to student grumbles about waiting for their mark and feedback. The Module Team decided to ask tutors to return TMAs at the same time, that is on the 10th working day after submission date.  This was particularly important for TMA02 as that was due in 17th December but the University closed on 20th December for two weeks over Christmas and New Year.  Tutors thus had until 8th January to return the TMAs.   Mindful of a potentially wide variance between tutors marking and returning before the Christmas break and those marking after the break – and potentially not returning the TMAs until 8th January – this seemed particularly important.   The module team decided to implement this new module policy as an experiment for the whole of the module, that is, to cover TMAs 2- 5. 

It mostly worked – a couple of tutors forgot to follow this new policy and had to be reminded.  For one tutor who was going away three days after TMA03 cut-off date, and planning to go abroad after marking promptly with no  ability to return the whole group whilst away, it was agreed he go ahead but alert his students to this exception. 

The evaluation of this experiment looked at student complaints, student comments on the student Facebook group and tutor feedback.  There was anecdotal evidence from the Student Support Team of a reduction in student phone calls about TMA turnaround but no mechanism for recording these on the OU management information system. 

However, there was evidence of reduced student anxiety about when they would get their TMA marks back on the student Facebook group and these postings stopped from TMA03 onwards after the module TMA turnaround policy had bedded down.    

In addition, the feedback from the tutors, whilst mostly positive about the idea of the policy, did point out that it relied on tutors remembering the correct return date and for tutors working on multiple modules in particular, this was felt to be onerous, as these two comments illustrate:- 

I much prefer this way of returning marks and feedback. I do hope this continues. It saves students worrying (or getting annoyed about) why some students have received their marks and others haven’t. 

On one hand I think this is a good idea. Everybody gets theirs back at the same time. On the other the stress of having to remember to send them back on a particular day and not on the day you marked the last one is difficult. 

This evaluation took the views of the students and tutors into account and found there is evidence of the practical benefits to students of returning all marked assignments at the same time and broad support from tutors to the idea of this policy. 

The evaluation of this module policy experiment recommended the following:- 

1. That the Student Support Team creates a category for formally recording informal complaints from students about TMA turnaround time  

1. That the FASS Teaching and Students Committee considers the required policy and technical changes need to implement a uniform 10 working-day TMA turnaround policy (excepting extensions), with a view to providing a recommendation for implementation of the new electronic TMA marking system, UNIWISE.  

Alison Penn – Senior Lecturer in Social Policy and Criminology

Dr Alison Penn | Faculty of Arts and Social Sciences (open.ac.uk)

image by Ben White on Unsplash

This is the first of an occasional series rounding up interesting SoTL-related stories from around the web.

All of us who write about SoTL will find food for thought in Making Sense of Writing about Learning and Teaching (Healey, Matthews, and Cook-Sather, 2020). The authors of this open access book argue that we need to learn new ways of writing when we write about scholarship, and at the same time unlearn some of our disciplinary norms where these create barriers to inclusive communication.

The last year has seen many institutions try to get to grips with ‘pandemic pedagogy’, and this article considers the importance of inclusive design to keep students engaged when interactions are on screen rather than in person.

Staying with the subject of student engagement, Dr Jenny Scoles writes frankly about an unsuccessful intervention aimed at addressing student dissatisfaction with assignment feedback, and how students and teachers have very different perceptions of the role of feedback.

Over the past few years many universities have invested heavily in the production of MOOCS (massive open online courses), and the authors of this article explore how MOOCS can be embedded into module content, and the challenges of integrating them into face-to-face teaching.

And finally, from The Open University’s own Faculty of Business and Law scholarship blog, academic and barrister Jessica Giles proposes that interdisciplinarity might offer some answers to the question many of us have been asking ourselves in these turbulent times: how do I solve the worlds problems from my desk?

If you’ve come across any interesting online articles or blogs about SoTL do please post the links in the comments section below.

Heather Richardson Senior Lecturer, Staff Tutor & Deputy Director FASSTEST

Image by StartupStockPhotos from Pixabay

There exists in the Open University (OU) a large set of largely unheralded workers who put themselves forward for one of the most important roles – the role of monitoring, which involves looking at a sample of another colleague’s marking and assessing it for standardisation to the marking criteria and quality of feedback. This group of workers monitored 36,000 assignments across the OU in 2019, 6 percent of the total received, including 3,735 from the School of Psychology & Counselling and nearly 1000 from one module alone (DE300 ‘Investigating Psychology’). Monitoring of assessment, for standardisation of marking and quality of feedback, is an essential feature of maintaining academic standards in an era of scrutiny of higher education provision (Bloxham and Boyd, 2012). In a distance learning institution, some students’ experience of communication with their tutor may only exist through feedback provided on assignments despite tutors’ best efforts to engage (Tsagari, 2019), so the importance of this feedback could be seen as greater than at campus-based institutions. The subjective judgements made by markers (Brooks, 2012), despite module team provision of detailed marking criteria, can also lead to standardisation issues, with assignments marked leniently or harshly. The monitoring process within the OU is vital to ensure discrepancies in marking and variabilities in feedback are picked up and addressed. The role is fundamental to the university’s principles and purposes of assessment – they ensure quality of teaching, to contribute to the professional development of tutors, and to validate assessment scores.

Monitoring as a dialogue

Monitoring in the past, though, has sometimes been associated with questions about the difficulties of monitors providing what is essentially a peer review and judgements on their colleagues. Concerns about training for monitoring led to a previous scholarship project in Psychology and Counselling at the OU in 2016 on ‘Monitoring: Good Practice’, which resulted in improvements to training for monitors and a dedicated website. An important feature of monitoring for the university is that it should be a dialogic relationship with the monitor and the tutor further discussing the process, perhaps working together to think more about successful application of marking criteria and discussing effective feedback. However, further evidence was needed to critically evaluate the prevalence and effectiveness of the ‘dialogic process’ aspired to by The Open University. To address this gap, a small-scale scholarship project was carried out, supported by FASSTEST, between February and July 2020 to explore monitoring from the experience of both monitors (n=24) and tutors (n=27) in the School of Psychology & Counselling. Content analysis of monitoring reports (n=46) across a range of undergraduate and postgraduate modules also looked for examples of good and poor practice.

Encouraging outcomes

Outcomes of the project were encouraging, particularly around the impact of monitoring on professional development for both tutors and monitors – the majority of both reported that their own practice had improved from receiving monitoring reports (21/27) and being a monitor (24/24). Monitors had gained from seeing how other colleagues assessed, and often as a result adopted (or as some suggested ‘plagiarised’) examples of good practice. Tutors, particularly those who were new to the university or to the module being monitored, reported benefiting from the feedback particularly around knowing they were applying the marking criteria appropriately.

The dialogic process between monitors and tutors, however, was less evident, with only half of tutors having further contact with monitors, usually because they did not feel it beneficial to follow up. As one tutor put it, ‘if you receive a great report, there is little more to be said’. Sometimes the dialogue was more about challenging the report and explaining and justifying the assessment by referring back to knowledge a monitor may not have – the tutor’s relationship with the student, for example, which could mean less feedback because of the student feeling overwhelmed when receiving too much.  It is recognised that discussions between markers and assessors can be beneficial in improving standardisation (Grainger et al, 2008) so the OU expectation and encouragement for the dialogue between monitors and tutors could result in improved assessment practice. Monitors worked hard to encourage further dialogue with tutors; reports typically were friendly and encouraging, and most monitors included email contact information with a ‘please contact me to talk further’ message within the report. The decision then is left to the tutor whether further contact takes place, and that will be something that workload and time management may well also impact.

As with any peer review process, there were a small minority of experienced tutors who did not feel monitoring was particularly beneficial after teaching a module for several years and sometimes questioned the judgement of monitors who were less experienced than themselves. Receiving a good rather than an excellent from a monitor was occasionally met with resistance. However, within any peer review or assessment system, there will always be emotive responses alongside cognitive benefits particularly when perceptions of experience are involved (Cartney, 2010).

Photo by Ian Dooley on Unsplash

Content of reports

The content analysis of reports showed that a majority demonstrated very good practice – supportive, friendly, accurate and full of appropriate advice. Sometimes there was evidence of cut and paste errors, though, where monitors had perhaps taken information from a previous report or another similar report and inadvertently not changed the name. Whilst these were very few overall, there were some instances were tutors felt these errors undermined the value of the reports. Tutors also felt that sometimes monitors were not able to acknowledge the personal relationships tutors had with their students, which could explain a lower level of feedback, for example, for students who a tutor might judge to manage a limited amount of information in each set of feedback better (Forsythe and Johnson, 2017). There was also doubt that monitors looked back to previous reports to acknowledge what had been said before.

Being honest in peer review

It can also be difficult for a monitor to be completely honest in a report. Nearly all monitors acknowledged they had monitored a friend or close colleague, although the majority did not consider this to be an issue. The ability, though, to provide an unbiased critical judgement of a friend’s or colleague’s assessment practice could well be affected (Abidin et al, 2018). On small modules avoiding monitoring close colleagues can be problematic – with only two monitors on a module, inevitably they will end up monitoring each other. However, on larger modules this can be avoided. Notably, in other faculties on larger modules monitors are asked in advance to check their list and remove any colleagues they are familiar with to avoid this.

Professionalising the monitor role

Perhaps one of the most important findings of this research, though, relates to the status of the role of monitor. Nearly all monitors considered their role as important and valued by the university and colleagues, but very few felt they were an important part of a team, and most reported they had very little communication about their work. Half of monitors reported feeling isolated in their role. The monitor role also lacks the professional status that other roles (for example, associated with academic conduct) command. A more recognised professional status of the monitor, with better integration into Module Teams, could go some way to making monitors and tutors see the role as more valued.

Finally, changes in the monitoring system were implemented during the period this research was being carried out. These changes affected monitoring levels, and the way in which the report form is completed, are significant improvements and early reports from monitors and tutors suggest they find the forms easier to use. It was encouraging that many of the findings of the research were being addressed by monitoring process changes that were taking place at the same time. This validates both the research findings and the decisions of the Open University’s Monitoring Implementation Group, which have designed a new website and handbook.

References

Abidin, A.N.Z., Masek, A, Mohd Faiz, N.S. & Sahdan, S. (2018), Exploring the elements of integrity in peer assessment, MATC Web of Conferences, 150, 05002. https://doi.org/10.1051/matecconf/201815005002

Bloxham, S. & Boyd, P. (2012) Accountability in grading student work: securing academic standards in a twenty-first century quality assurance context, British Educational Research Journal, 38, (4), pp. 615 – 634. https://doi.org/10.1080/01411926.2011.569007

Brooks, V. (2012), Marking as judgement, Research Papers in Education, 27, (1), pp. 63 – 80. https://doi.org/10.1080/02671520903331008

Cartney, P. (2010), Exploring the use of peer assessment as a vehicle for closing the gap between feedback given and feedback used, Assessment & Evaluation in Higher Education, 35, (5), pp. 551 – 564. https://doi.org/10.1080/02602931003632381

Forsythe, A. & Johnson, S. (2016), Thanks, but no thanks for the feedback, Assessment & Evaluation in Higher Education, 42, (6), pp. 850 – 859. https://doi.org/10.1080/0260938.2016.1202190

Grainger, P., Purnell, K. & Zipf, R. (2008), Judging quality through substantive conversations between markers, Assessment & Evaluation in Higher Education, 33, (2), pp. 133 – 142. https://doi.org/10.1080/02602930601125681

Tsagari, D. (2019), Interface between feedback, assessment and distance learning written assignments, Research Papers in Language Teaching and Learning, 10, (1), pp. 72 – 99. Available online at http://rplel.eap.gr

For further details please contact

Sue Nieland – Staff Tutor in Psychology and Counselling

An experiment in creative learning

Photo by Green Chameleon on Unsplash

On a grey Saturday morning in March 2020, as the UK teetered on the brink of the first lockdown, a small group of students gathered at the Open University campus in Milton Keynes to take part in an experiment. Half of the group were studying Creative Writing, and the other half were students of Art History.

The experiment – a project called ‘Creative Interactions’ –   had been dreamt up by me (Dr Heather Richardson, Creative Writing) and Art Historian Dr Clare Taylor, ably assisted by two of our Associate Lecturer colleagues, Dr Helen Mosby and Dr Diana Newall. The plan was to find out what happens when the teaching techniques of the two disciplines are swapped around. Luckily for us the Open University has an extensive art collection, much of which is on display around the campus. This meant we were able to come face to face with artworks from all around the world over the course of the day.

Ekphrasis and visual analysis

Ekphrasis is a familiar phrase to many people studying Creative Writing and comes from the ancient Greek words for describing or speaking out. Put simply, ekphrasis is writing that describes a piece of visual art, and it’s cropped up in literature for hundreds of years – think of Keats’ ‘Ode on a Grecian Urn’, for example. Every Creative Writing tutor has some ekphrastic exercises in their toolbox, where we show the students an image and encourage them to let their imaginations off the leash. What we’re hoping for is an emotional response – a gut reaction – that will lead students to a creative response in the form of a story or poem inspired by what they see.

Art History tutors and students on the other hand engage with artworks in a very different way. Their first step when they encounter an artwork is to carry out a visual analysis, considering issues like composition, colour, and the materials used. Another important element is the historical or social context in which the work was produced. It’s all very different from the way Creative Writing students make use of visual art.

Of course, we wanted our student volunteers to get practical benefits from the workshop that would help them on their modules, so for the first part of the day they worked with the tutor from their own discipline and engaged with several pieces of art in the usual way for their subject. It was then that the experiment really began as the two groups swapped, with the Art History students working with the Creative Writing tutor, and vice-versa for the Creative Writing students.

Imaginative adventures

I sat in with the Art History students as Creative Writing tutor Helen Mosby led them through a series of activities while looking at David Tindle’s Mural Panel C. At first glance this looks to me like a lovely image of a summer day, viewed through partially open French windows. Helen drew our attention to the old-fashioned telephone, receiver off the hook, sitting outside the French windows, and we tried to imagine the circumstances behind it. Had someone been trying to make a secret phone call, out of earshot of anyone else in the house? Or had some emergency occurred which had led them to throw the receiver down and run off?

The Art History students pointed out the areas of shadow amongst the trees on the right-hand side of the painting, and the misty effect caused by the barely-visible voile curtains, and with our creative writing hats on we wondered if these suggested a darker or unsettling aspect to the stories we were inventing.

For the final part of the workshop we all came together to consider a triptych – an artwork made up of three pieces – with input and discussion from both disciplines, followed by a creative writing session.

Participants response to one of the artworks

Insights

At the end of the workshop we asked the students what they had learnt. Unsurprisingly, both disciplines said they felt better able to carry out a visual and formal analysis of artworks. One Creative Writing student said, ‘Looking at artwork from an art historian’s perspective would lead to different stimuli’, while another said they had learnt ‘How to interpret art as a medium with a purpose’. The Art History students commented on the creative writing activities, with one saying, ‘Spontaneous creative writing is hard!’ while another acknowledged the benefits of writing without hesitation, stating emphatically, ‘I can write!’

The students reflected on how the workshop had enhanced their appreciation of both their ‘home’ and ‘other’ discipline, with Creative Writing students feeling that analysing the artworks in detail would help them with their descriptive writing, while the Art History students concluded that thinking of an artwork in terms of narrative or subject matter would add a new dimension to their interpretation.

What next?

After the workshop Clare, Diana, Helen and I agreed that the experiment had proved the value to students of a creative interaction between their disciplines. Our next step will be to roll out a virtual version of the workshop. Going online isn’t just about dealing with the ever-changing Covid-19 regulations, but also a way to ensure that more students get the opportunity to participate – not just those who live within travelling distance from Milton Keynes. And we plan to talk about ‘Creative Interactions’ at conferences and staff development events for Creative Writing and Art History tutors. Exciting things happen when two disciplines work together, and we’re going to make sure that story gets told.

For further details please contact:

Heather Richardson, Senior Lecturer, Staff Tutor & Deputy Director FASSTEST

Hiding behind any respectable project that seeks to improve the student experience is always an element of nerdiness – an obsessive love of stats and number-crunching – as well as a curtain-twitching urge to take a peek into the lives of others. What precisely do teachers and students get up to in their classrooms? How does it all go so right for students – and sometimes so wrong?

Well, that’s our experience anyway. We are James Robson (Professor of Classical Studies at the OU) and Dr Mair Lloyd (Associate Lecturer and former OU PhD student) and our story starts about seven years ago when we met up for the first time over multiple cups of tea, nursing the shared ambition of carrying out a bold project that hadn’t been attempted for a generation: a nuts-and-bolts survey of beginners’ Latin and Greek teaching in Classics departments across the UK. We carried out this survey in 2014 and as the data poured in and we feasted greedily on the diet of bar charts and pivot tables we were creating, one statistic kept jumping out at us: nearly one in four students who began studying Latin at university (23%) didn’t complete their module. Why was that? And how come the pass rates at different universities varied so greatly?

Image: Mair and James drinking tea in contrasting settings

That is how The Battle for Latin was born, a project dedicated to examining the factors driving student success, failure and withdrawal amongst beginners’ Latin students. More in hope than expectation, we put together a bid for the British Academy small grants scheme, drawing on James’ long experience of classical language teaching at the Open University and Mair’s expertise in Modern Foreign Language and Latin pedagogy. Importantly for the bid, we were able to cite the statistically-rich research that we had already published. And crucially, too, we had a burning question that, to us at least, seemed so vital to answer: why were so many aspiring Latinists in UK universities unable to stay the distance?

A few months later, we learnt that our bid had been successful and when our spontaneous whooping and partying eventually subsided, we set to work. As stats fiends, one thing we felt we needed was more targeted, up-to-date data, so we ran a new survey of UK Latin instructors, who between them kindly furnished us with data on 30 different Latin modules covering 888 students nationwide. As our graph below shows, the variation in completion rates was striking once again. Remarkably, too, we also learnt that, while a greater number of students were completing and passing our own Latin module at the OU, this was not a picture reflected across the sector: nationwide the completion rate for beginners’ Latin modules was stuck at 76%.

Image: One of Mair’s and James’ signature graphs: Beginners’ Latin modules in UK universities listed by anonymized alphanumeric code, showing percentages of those starting the module who passed, failed and withdrew.

So, what were the factors driving student success, withdrawal and failure? Our data allowed us to rule out elements such as module duration, credit value and even student contact hours to a large extent (the exception being a handful of particularly intensive modules which included five or more hours’ classroom time each week). Nor did the choice of textbook or assessment strategy appear to be determinative (although we did note a possible benefit of including substantial elements of assessed coursework). Ultimately, whatever hypothesis we investigated there always seemed to be modules that bucked the trend. Clearly, staring at the stats was only going to get us so far.

Fortunately, our project also built in human contact: a series of whole-class observations, interviews with instructors and students, and even an online student survey to allow us to understand better the obstacles to student success. We learnt a lot from our activities, not least the need to ask a small number of very focused questions if you hope to finish the interview on time! But we were also delighted to discover the warmth, dedication and reflectiveness not only of Latin instructors, but also of the students they teach, who were generous with their time and hugely thoughtful and thought-provoking in their responses to our questions.

So, is there a magic bullet for improving student success on beginners’ Latin modules? Well, maybe not, but our research nevertheless provides some useful trends and pointers, we hope. Plus, our forthcoming paper also distils some of the top tips that Latin students would pass on to new starters – all the more useful, we hope, for being direct quotations in the students’ own voices. If we have an overarching conclusion, however, it is perhaps that the magic lies somewhere in the dynamic interactions between the teacher, students, textbook, teaching methods and class as a whole. Of course, these are factors that are challenging to quantify and pin down – but this merely convinces restless enthusiasts like us that another research project is needed to scrutinize these more closely.

A paper with further details will be published shortly, we will post a link here when the article is live.

For further details please contact James and Mair:

James Robson, Professor of Classical Studies

Dr Mair Lloyd (Associate Lecturer and former OU PhD student)

FASSTEST (FASS Teaching Excellence and Scholarship of Teaching) is the Centre for Scholarship and Innovation at The Open University’s Faculty of Arts and Social Sciences. By ‘scholarship’, we mean the scholarship of teaching and learning. This means that FASSTEST supports the critical, scholarly evaluation of approaches to teaching and learning in the Arts and Social Sciences with the aim of providing an evidence base for good practice in higher education.

Our blog will include posts from practitioners and researchers discussing issues and research findings relevant to teaching and learning in the Arts and Social Sciences in HE. Here at The Open University we can draw on particular expertise in online and distance education, and we are especially inspired by The Open University’s mission to be open to people, places, methods and ideas. We are committed to placing innovation at the heart of teaching and learning and to exploring new and better ways to inspire and enable learning, opening up higher education to all, regardless of background or circumstances.

We welcome blog posts from anyone with an interest in the scholarship of teaching and learning, particularly in the Arts and Social Sciences. If you are interested in contributing please get in touch with FASS-Scholarship@open.ac.uk with a title and brief description of what you would like to write about.

Stefanie Sinclair
(FASSTEST Director)