by Ale Okada   

I had a great meeting with Prof Jane Seale and Dr Lauren OHagan about Understanding 4 REF Impact in Education: Reach and Significance Matter Most*

I am attending the   Global Education Meetign led by UNESCO in Brazil Fortaleza

As we prepare for the next Research Excellence Framework (REF) submission, it’s important to demystify the gold standard of REF impact case studies—those rated 4*. Unlike research outputs, there’s no explicit document that defines what makes an impact case “four-star.”  Jane clarifies that there are two core criteria : Reach and Significance.

🔹 Reach is not just about numbers or geography. It refers to the extent and diversity of beneficiaries—how many different types of individuals, groups, or organisations your work has affected. It’s about how far your research has travelled across sectors or communities, and how many kinds of people or systems it has touched. It’s not “where” or “how big,” but how broad and inclusive.

🔹 Significance is about depth and value of the change. Has your work changed someone’s understanding, influenced a policy, transformed a teaching practice, or enhanced wellbeing? Think of it as the difference your research made—how it enriched, enabled, or shifted something that matters to the people or systems it reached.

Together, Reach and Significance must be evidenced—not assumed. Strong case studies show how research translated into real-world benefits, supported by credible, independent, and varied forms of proof.

So, what does a 4* impact case study look like in education? It’s one where the research catalysed change that was both wide in influence and deep in effect—with the evidence to back it up.

Let’s keep asking: who did our work really reach—and what difference did it truly make?

Source: https://2021.ref.ac.uk/media/1450/ref-2019_02-panel-criteria-and-working-methods.pdf

This conversation lead to more questions:

What does it mean reach? What means difference?  Is there levels of variation and how those levels are ranked in terms of reach and difference?

In terms of reach, I am now thinking  as the breadth of your research’s footprint—not just how many, but how varied.

• Did the impact reach different types of people? (e.g. policymakers, teachers, students, NGOs, communities)

• Did it influence systems at different levels? (e.g. school practices, local policy, national curriculum)

• Did it cross sectoral boundaries? (e.g. education, health, technology, social care)

  Regarding significance,  I am now considering

• How important was the change to the beneficiaries?

• Was it a transformational shift or an incremental improvement?

• Did it enable new thinking, change practice, improve wellbeing, or shape policy?

  • I am now at the airport going to Brazil and I start to think about What types of evidence should be co-identified with, and for, beneficiaries to demonstrate Reach and Significance in an REF impact case study in Education?

  To robustly demonstrate Reach and Significance, evidence should ideally be:

  1. Credible – independently verifiable and attributable to the research.

  2. Co-identified – meaning it is not only gathered about the beneficiaries but also with them, reflecting their voices, experiences, and perspectives.

  3. Purposeful – aligned with the specific change you are claiming (e.g., shifts in practice, policy, awareness, or wellbeing).

  4. Type of Evidence	Reach	Significance	Co-Identified With	Usefulness For
     🗣️ Testimonials (e.g. from teachers, learners, policymakers)	✔️ diversity of audience	✔️ depth of impact	Direct voices of beneficiaries	Humanising the change
     📄 Letters of support or impact statements from organisations or community leaders	✔️ sectoral or institutional reach	✔️ confirmed influence	Partner organisations	Validates influence on policy/practice
     📊 Surveys or evaluations with beneficiaries	✔️ sample size & diversity	✔️ perceived benefit or change	Designed with or adapted for end users	Empirical feedback
     📈 Before-and-after data (e.g. attendance, learning gains, access to resources)	✔️ group-level reach	✔️ measurable outcomes	Collected in collaboration with stakeholders	Demonstrates tangible change
     🎯 Policy citations, curriculum changes, or programme uptake	✔️ systemic or policy-level reach	✔️ institutional significance	Documented shifts influenced by research	Shows real-world uptake
     📢 Media coverage highlighting community outcomes or educational transformation	✔️ public reach	✔️ awareness & societal discourse	Reflects public reception	Visibility and narrative framing
     🧾 Funding or adoption by others (e.g. NGOs, ministries, schools)	✔️ institutional buy-in	✔️ practical endorsement of value	Often emerges via partnerships	Tangible evidence of worth

  5. In summary:

     For a 4* impact case in education, your evidence should not just “show” change—it should be anchored in the realities and reflections of those affected. Ideally, this evidence is co-developed or co-validated through:

     • Collaborative instruments (e.g. joint evaluation tools)

     • Participatory methods (e.g. reflective workshops, co-produced reports)

     • Feedback loops (e.g. testimonials or surveys that were returned to stakeholders for review)

     This helps ensure that the evidence is not only relevant and meaningful, and not only trustworthy in the eyes of the REF panel, but ultimately reflects research that is worthwhile and makes a real difference!