Supporting minoritised student experience of Taught Postgraduate assessment: a co-creation project

This blog post is another in our series of scholarship-focused posts and summarises a report of this project, which investigates minoritised students’ experiences of assessment in higher education. The project lead was Dr Judy Chandler, and the co-researchers were Bob Hallawell and Rhiannon Moore. The project was funded by Praxis – WELS Centre for Scholarship and Innovation.

This blog post been edited by Dr Jane Cobb based on a Praxis project report by Dr Judy Chandler et al.

The Supporting Minoritised Students co-creation project found evidence of disparities in outcomes for minoritised PostGraduate Taught students and highlights the importance of students’ feedback literacy and existing capital.

“The general module assessment guidance… I seldom check that guidance because that’s too long and sometimes I think the information for me there is…is like too…too abstract, too general (interview participant).”

This study uses a change laboratory (a type of participatory, collaborative research within which ideas are tested) and case study methodology, in combination with analysis of quantitative data from outcomes of two Open University Master’s Programmes (in Education and in Childhood and Youth) and builds on existing literature.

The project reports the following key findings:

  • Many minoritised students do less well in qualifications than their counterparts. (The project defines minoritised students as: “students minoritised by language, social processes or institutions”.)  For example, Black, Asian and students who choose to share mental health conditions were awarded the lowest outcomes on certain courses.  Further, black students completed their taught postgraduate studies with the lowest course outcomes of these three minoritised demographic groups.
  • Application of feedback requires feedback literacy and is associated with students’ prior experiences and expectations. (I understand feedback literacy as being required to interpret the guidance and feedback around assessed work, which students have gained from previous study, and which would then feed forward to future assessed work.) The project finds that feedback literacy is associated with students’ prior (and cultural) experiences and expectations.
  • Existing assessment support contributes to the replication of patterns of existing capital and academic identity. Credit transfer students, those with prior experience of study or work within similar domains, and those who were already familiar with the Open University systems, are more able to navigate their assessment experience and emerge with positive course outcomes.  For students without pre-existing sources of this capital, strategies of navigation or resistance were more limited, as they were less able to draw on existing knowledge and experience to navigate the interpretation of assessment requirements, documentation, or feedback.

 

  • Data highlights the impact of negative feedback on a student’s confidence and view of their own ability. Further, a lack of clarity and specificity meant that feedback was sometimes difficult for students to apply, particularly for those for whom English is an additional language.

The project makes the following recommendations:

Academic teams should consider assessment data across courses, pathways and qualifications.

When designing assessment guidance and support, it is important to consider any assumptions made about students’ existing knowledge and skills and to ensure that support is in place for those students without this experience.

At a qualification level, endeavour to provide some consistency in tutor support.

Students declaring mental health conditions should receive continued support throughout their course, not just at its outset, in preparation for final assessment.

Create ways for all students to feel a sense of belonging during their course.

This project provides insight and empirical data which can be drawn upon by practitioners interested in improving student experience of assessment in higher education, particularly in a distance and online setting.

In so doing, it supports, for example, the ongoing work of the Anti-Racist and Inclusive Assessment.

The project team states that future iterations of this project will incorporate the experiences of course team members and tutors working on the Master’s in Education and Master’s in Childhood and Youth, alongside those of minoritised students. There will be a continued emphasis on the co-creation of inclusive and accessible assessment support, which builds upon the varied experiences of this diverse group of co-collaborators.

For further information on this project, please contact [email protected] or visit the Open University Scholarship Exchange 

 

“The proposal to develop this project to investigate the experience of all participants (tutors, course team and students), viewed as co-collaborators, seems an interesting way to strengthen and illuminate the data further.  Both the concept of feedback literacy itself and the fact that it is associated with prior experience resonate with me as an ex-OU tutor (of many years!).  Moreover, I have experienced that feedback can (unintentionally) affect a student’s confidence, and there is evidence to support this. Finally, the recommendation that a sense of belonging is created for students is something that, as tutors, we have been continually trying to achieve with varying degrees of success, such as by encouraging an active online student forum.” Dr Jane Cobb.

 

What are your thoughts on the findings and recommendations of this project?

 

What was the emotional impact of providing students with video feedback on their written assignments?

Photo by SHVETS production from Pexels: https://www.pexels.com/photo/smiling-woman-having-video-chat-via-laptop-in-library-7516347/

In this post, one of a series of Scholarship focussed blog posts, authors Ximena and Gerry talk about their recent research project exploring the emotional impact of  providing students with Video feedback on their written work. The OU – like so many other institutions – has its own quirky language and so I have editorialised to help someone who is unfamiliar with the OU to make sense of some of the highly specialised language OU colleagues use.

Written by:  Ximena Arias-Manzano [email protected]  & Gerry Howley [email protected]

Video feedback for TMAs : The emotional impact of hearing the tutor 

(TMA is an abbreviation for Tutor Marked Assessment, a formative assessment provided in response to students formal submission of course work, marked by their tutor rather than someone external to their course).

In the context of a distance learning University, this is feedback written by a tutor who had had little direct contact with a student on their course. 

The project focuses on tutors using video feedback for a range of assessments. Triangulation of quantitative and qualitative data are brought together to illustrate the beneficial effect that video feedback can have on accessibility, personalisation, engagement, and the tutor student relationship

 

Having this opportunity [to receive video feedback] brought a humanity to the feedback and made a huge difference in [my] circumstances. It did encourage me to ask questions more and definitely made me engage more with the module. I thoroughly enjoyed it all.

 

For the majority of students on the project, video feedback is shown to be a highly effective tool that improves accessibility, personalisation, engagement, and the tutor-student relationship. Most tutors found providing video feedback an efficient, enjoyable and beneficial alternative to written only feedback.

What was the project about?

This project was about providing an opportunity to a team of LAL   ALs (LAL refers to the school of Languages and Linguistics, ALs refers to Associate Lecturers) to experiment with innovative ways of giving feedback on their students formally submitted written work. Exploring the use of screencast technology, ALs were able to develop their own style of feedback creating an interactive approach. The production of video feedback in a guided environment where ALs had the opportunity to be trained and to receive comments from their peers to continue improving the use of screencasts in a more effective way when correcting assignments. The emotional impact of hearing the tutor and receiving more personalised feedback has been the focus of this scholarship project. The reflection and feedback collected from tutors and their students will form the basis of our final report.

What were the findings?

From focus group participants/mentors…

Student questionnaire:

    • Students overwhelmingly welcomed the new approach to feedback
    • 77% of 81 students who responded to the feedback questionnaire stated they preferred video feedback to text-only feedback.
    • 91% felt video feedback helped them understand their weaknesses and strengths more than text-only.
    • 91% said video feedback made them feel they had a closer relationship with their tutor.
    • However, positive outcomes are shown to be dependent on the approach of the tutor. So, adequate training and monitoring will be important moving forward.

A key topic that came out of both student and tutor feedback relates to the isolation of teaching/studying online. Using video feedback felt highly personalised and made them feel more connected to their tutor/students.

Student comments:

    • ‘It felt like I was sat in a one-to-one session with my tutor’.
    • ‘I felt a part of a community of students and a university. Not like studying alone far away from everyone else going through text on the screen’.
    • ‘It made me feel more encouraged and part of the community. Distance learning can be very isolating at times and this personalised visual feedback helped with that.’

Tutor testimonials:

    • “My marking is more enjoyable now (less isolated) and I feel I am having a conversation with students”.
    • “All in all, the project really (re) opened up my eyes to what can be achieved online without being a tech expert”.

Tutors reported on their feelings when preparing the feedback and appearing on a webcam, but also the positive feelings of easing their isolation.

Tutors found it a positive learning experience to develop alternative and innovative ways of giving feedback. After the initial hurdle with technology and trial and error period, they felt more confident, and some decided to use this way of giving feedback in all the modules they teach.

We learned that…

    • Tutors have seen the potential of video feedback for positive outcomes for their students, as well as helping them to feel closer to their students. This is something that has been missing over recent years due to organisational changes within the OU teaching structure as well as the disruption caused during COVID.
    • Students’ reactions to video feedback also demonstrate its effectiveness in terms of closing the gap in the student tutor relationship, which in turn can increase engagement and accessibility.
    • The enthusiasm we have seen from the ALs who took part in the project as well as module teams and tutors who have not yet been involved is testament to the potential of video feedback being used much more widely in the future. This is the primary focus of our next steps.

What next?

The project’s next steps are to create a group of champions from those who took part in the project. This team of champions will be trained as trainers to cascade down their knowledge and expertise to Associate Lecturers in LAL from 23J onwards.  (23J refers to  September 2023) We have already received interest from two Module Chairs re. using video feedback on their modules and we will be rolling out the training to their ALs (AL = Associate Lecturers) in time for 23J TMAs.

A long-term future plan would be to also expand the delivery of SD sessions to ALs university-wide.  The team of champions will also have the role of creating a “Guide to good practice” to be discriminated across the OU to form an integral part of the training for the use of video feedback in TMAs.