How should formative assessments be assessed? A study of S217 online quizzes

Project leader(s): Jonathan NylkAndy Diament
Theme: Innovative assessment
Faculty: STEM
Status: Archived
Dates: November 2022 to August 2023

The aim of this project is to evaluate the level of student engagement with the online quizzes on S217 Physics: from Classical to Quantum, comparing the current model of assessment with a formative thresholded assessment model which has been shown to have a marked positive impact on student engagement (Jordan, 2014), and other relevant models identified in the literature. The results of this study will be used to inform the future assessment strategy for S217, and potentially other modules across the School of Physical Sciences (SPS) and the STEM faculty.

S217 features online quizzes that are designed to help reinforce learning and pace students’ study, but only feature indirectly in the assessment strategy of the module. Instead of direct credit, students are encouraged to engage with the online quizzes by items in formative tutor marked assignments (TMAs) that ask students for self-reflection on their performance in the quizzes.

This project will utilise three approaches:

The collection of learning analytics data to determine the level of student engagement with online quizzes
A survey of students to determine their motivations for engaging with the online quizzes
The use of published literature and data to compare engagement and motivations on S217 to other assessment strategies

Learning analytics data will be used to determine the level of student engagement with these activities (such as when students access the quiz, and for how long, what mark students achieve, and how many times students repeat the quiz) and correlating this with the grade awarded to students in their reflective TMA item.

A student survey will be conducted to investigate students’ motivations for engaging with the online quizzes in comparison with other module activities. The survey will feature free-text responses to allow for qualitative analysis (e.g. thematic analysis) of responses.

The level of engagement will be compared with a formative thresholded assessment strategy which can be done in two ways: firstly, by comparing a student’s mark on a quiz with the mark awarded on the reflective TMA item, and secondly, comparison with similar modules that have already implemented a formative thresholded assessment strategy through a detailed literature search.

References:

S. Jordan and P. Butcher (2010), “Using e-assessment to support distance learners of science”, in The GIREP-EPIC & PHEC 2009 International Conference, 17-21 Aug 2009, Leicester, UK, http://oro.open.ac.uk/24619/1/GIREP2009_Jordan%26Butcher.pdf

S. Jordan (2014), “Thresholded assessment: does it work?”, eSTEeM project, https://www.open.ac.uk/scholarship-and-innovation/esteem/projects/themes/innovative-assessment/thresholded-assessment-does-it-work

S. Jordan and J. Bolton (unpublished work), “Student engagement with a novel online assessment strategy: Lessons beyond Covid-19”

A. Norton (2016), “Assessment analytics of student engagement with, and performance on, S217 online quizzes”, eSTEeM project, https://www.open.ac.uk/scholarship-and-innovation/esteem/projects/themes/innovative-assessment/assessment-analytics-student-engagement-and-performance-s217

C. Rust, B. O’Donovan and M. Price (2005), “A social constructivist assessment process model: how the research literature shows us this could be best practice”, Assessment and Evaluation in Higher Education, 30(3), 231-240, DOI: 10.1080/02602930500063819