What research questions the project addresses, aims & themes
Technology has a role to play in enhancing the assessment feedback cycle but only if it has been designed to improve the effectiveness of the assessment from a learner's point of view. The call for a pedagogically driven model for e-assessment was acknowledged as part of a vision for teaching and learning in 2014 by Whitelock and Brasher (2006). Experts believe that such a model will allow students in Higher Education to take more control of their learning and hence become more reflective but how can this be supported with tutor feedback? One of the problems with tutor feedback to students is that a balanced combination of socio-emotive and cognitive support is required from the teaching staff. One approach adopted by Whitelock et al (2004) to solving this problem was to build an electronic tool to support tutors with the feedback process. This tool, known as OpenMentor, analyses and displays the different types of comments provided by the tutor as feedback to the students. It then provides reflective comments to the tutors about their feedback practice.
How the research questions are addressed by the project (methodology and activity/environment)
This tool was designed by and used within the Open University. However, there is interest in improving the feedback given to students in HE throughout the UK. This interest was prompted by the annual Student Survey and has led to an awareness of how OpenMentor might assist other institutions in supporting tutors with their feedback to students.
A project was set up to transfer the OpenMentor technology from the Open University to two external institutions, the University of Southampton and King's College London. This paper reports the OpenMentor Technology Transfer (OMtetra) project's progress to date and addresses the following research questions:
- Would tutors who were not trained by the Open University accept the comments given to them about their feedback to students given by OpenMentor?
- What changes would be needed to facilitate cross institutional use of OpenMentor?
- Could OpenMentor be used for training purposes?
The findings should assist the project in producing an open source tool to enable the software to be freely used without licensing costs.
Findings and outputs
The pilot testing has revealed that tried and tested pedagogical strategies in a number of disciplines can be enhanced by the use of automated feedback. The main findings from the pilot study addressed the research questions as follows:
- Open Mentor's theoretical framework was robust enough to facilitate and encourage dialogue and reflective activities for the participating tutors. Tutors from the partner institutions were positive about the system's functions to support provision of feedback.
- The changes needed to facilitate cross institutional use of Open Mentor included the development of a module for user authentication and management; enhancement of the user interface to allow some level of customisation to the look of the system; and most importantly, the development of OM reports to help tutors to progress towards the ideal 'state' of feedback provided.
- There was agreement that Open Mentor could be used for training purposes as an academic development tool. The project needs to explore ways of how to support this type of activity.
There is a growing consensus in the field of assessment that times are changing and that assessment needs to become more embedded in the teaching learning cycle. However the provision of feedback that students will actually respond to and is timely and pertinent is indeed a challenge. This project provides another phase in this type of research where the balance of socio emotive content contained in feedback cannot be ignored (Draper, 2009). Feedback that encourages the student to actively change their ideas and ways of organising their answers and discourse within a given subject domain is what is required and advocated by Whitelock (2011) as "advice for action". There is still much work to be done but if the tools are under-pinned with appropriate learning and assessment strategies we are on the way.
Whitelock, Denise M., Gilbert, Lester, Hatzipanagos, Stylianos, Watt, Stuart, Zhang, Pei, Gillary, Paul and Recio, Alejandra (2012) Addressing the Challenges of Assessment and Feedback in Higher Education: A collaborative effort across three
UK Universities. In Proceedings INTED 2012, Valencia, Spain. ISBN: 978-84-615-5563-5
Whitelock, D., Gilbert, L., Hatzipanagos, S., Watt, S., Zhang, P., Gillary, P. Recio, A., (2012). Assessment for Learning: Supporting Tutors with their Feedback using an electronic system that can be used across the Higher Education sector, In Proceedings 10th International Conference on Computer Based Learning in Science, CBLIS 2012, 26-29 June, Barcelona, Spain (accepted)
Whitelock, D., Gilbert, L., Hatzipanagos, S., Watt, S., Zhang, P., Gillary, P. Recio, A., (2012). Improving a tutor’s feedback assessment tool: transforming Open Mentor following two recent deployments. Computer Assisted Assessment, 10th & 11th July 2012, Southampton (accepted)
Dr Denise Whitelock (The Open University)
Lester Gilbert (Southampton)
Stylianos Hatzipanagos (Kings College)
Stuart Watt (OpenMentor designer)
Pei Zhang (Southampton)
Paul Gillary (Kings College)
Alejandra Recio Saucedo (Southampton)
Project partners and links
Start Date and duration
Original dates: September 2011 to July 2012
Revised dates: September 2011 to December 2012