{"id":851,"date":"2024-10-15T10:31:53","date_gmt":"2024-10-15T09:31:53","guid":{"rendered":"https:\/\/www.open.ac.uk\/blogs\/rumpus\/?p=851"},"modified":"2025-04-25T13:58:10","modified_gmt":"2025-04-25T12:58:10","slug":"851","status":"publish","type":"post","link":"https:\/\/www.open.ac.uk\/blogs\/rumpus\/index.php\/2024\/10\/15\/851\/","title":{"rendered":"REF*  Impact in Education &#8211; Case Reflections&#8230;"},"content":{"rendered":"<p class=\"\" data-start=\"148\" data-end=\"228\"><em data-start=\"148\" data-end=\"227\">by Ale Okada\u00a0 \u00a0<\/em><\/p>\n<p class=\"\" data-start=\"148\" data-end=\"228\"><em data-start=\"148\" data-end=\"227\">I had a great meeting with Prof Jane Seale and Dr Lauren OHagan about Understanding 4 REF Impact in Education: Reach and Significance Matter Most<\/em>*<\/p>\n<p data-start=\"148\" data-end=\"228\">I am attending the\u00a0 \u00a0Global Education Meetign led by UNESCO in Brazil Fortaleza<\/p>\n<p data-start=\"148\" data-end=\"228\"><iframe loading=\"lazy\" title=\"YouTube video player\" src=\"https:\/\/www.youtube-nocookie.com\/embed\/bZs-eUalfyw?si=q-FWWRheINR9XMZO\" width=\"560\" height=\"315\" frameborder=\"0\" allowfullscreen=\"allowfullscreen\"><\/iframe><\/p>\n<p class=\"\" data-start=\"230\" data-end=\"571\">As we prepare for the next Research Excellence Framework (REF) submission, it\u2019s important to demystify the gold standard of REF impact case studies\u2014those rated <strong data-start=\"390\" data-end=\"395\">4<\/strong>*. Unlike research outputs, there\u2019s no explicit document that defines what makes an impact case \u201cfour-star.\u201d\u00a0 Jane clarifies that there are two core criteria : <strong data-start=\"540\" data-end=\"549\">Reach<\/strong> and <strong data-start=\"554\" data-end=\"570\">Significance<\/strong>.<\/p>\n<p class=\"\" data-start=\"573\" data-end=\"968\">&#x1f539; <strong data-start=\"576\" data-end=\"585\">Reach<\/strong> is not just about numbers or geography. It refers to <em data-start=\"639\" data-end=\"682\">the extent and diversity of beneficiaries<\/em>\u2014how many different types of individuals, groups, or organisations your work has affected. It\u2019s about how far your research has travelled across sectors or communities, and how many kinds of people or systems it has touched. It\u2019s not \u201cwhere\u201d or \u201chow big,\u201d but <em data-start=\"942\" data-end=\"967\">how broad and inclusive<\/em>.<\/p>\n<p class=\"\" data-start=\"970\" data-end=\"1301\">&#x1f539; <strong data-start=\"973\" data-end=\"989\">Significance<\/strong> is about <em data-start=\"999\" data-end=\"1030\">depth and value of the change<\/em>. Has your work changed someone\u2019s understanding, influenced a policy, transformed a teaching practice, or enhanced wellbeing? Think of it as the difference your research made\u2014how it enriched, enabled, or shifted something that matters to the people or systems it reached.<\/p>\n<p class=\"\" data-start=\"1303\" data-end=\"1512\">Together, <strong data-start=\"1313\" data-end=\"1322\">Reach<\/strong> and <strong data-start=\"1327\" data-end=\"1343\">Significance<\/strong> must be evidenced\u2014not assumed. Strong case studies show how research translated into real-world benefits, supported by credible, independent, and varied forms of proof.<\/p>\n<p class=\"\" data-start=\"1514\" data-end=\"1707\">So, what does a 4* impact case study look like in education? It\u2019s one where the research catalysed change that was <em data-start=\"1629\" data-end=\"1653\">both wide in influence<\/em> and <em data-start=\"1658\" data-end=\"1674\">deep in effect<\/em>\u2014with the evidence to back it up.<\/p>\n<p class=\"\" data-start=\"1709\" data-end=\"1796\">Let\u2019s keep asking: who did our work really reach\u2014and what difference did it truly make?<\/p>\n<p data-start=\"1709\" data-end=\"1796\">Source: <a href=\"https:\/\/2021.ref.ac.uk\/media\/1450\/ref-2019_02-panel-criteria-and-working-methods.pdf\" >https:\/\/2021.ref.ac.uk\/media\/1450\/ref-2019_02-panel-criteria-and-working-methods.pdf<\/a><\/p>\n<p>This conversation lead to more questions:<\/p>\n<p>What does it mean reach? What means difference?\u00a0 Is there levels of variation and how those levels are ranked in terms of reach and difference?<\/p>\n<p>In terms of reach, I am now thinking\u00a0 as the <em data-start=\"851\" data-end=\"860\">breadth<\/em> of your research&#8217;s footprint\u2014not just <em data-start=\"899\" data-end=\"909\">how many<\/em>, but <em data-start=\"915\" data-end=\"927\">how varied<\/em>.<\/p>\n<ul>\n<li class=\"\" data-start=\"512\" data-end=\"624\">\n<p class=\"\" data-start=\"514\" data-end=\"624\">Did the impact reach <strong data-start=\"535\" data-end=\"554\">different types<\/strong> of people? (e.g. policymakers, teachers, students, NGOs, communities)<\/p>\n<\/li>\n<li class=\"\" data-start=\"625\" data-end=\"735\">\n<p class=\"\" data-start=\"627\" data-end=\"735\">Did it influence <strong data-start=\"644\" data-end=\"675\">systems at different levels<\/strong>? (e.g. school practices, local policy, national curriculum)<\/p>\n<\/li>\n<li class=\"\" data-start=\"736\" data-end=\"825\">\n<p class=\"\" data-start=\"738\" data-end=\"825\">Did it cross <strong data-start=\"751\" data-end=\"774\">sectoral boundaries<\/strong>? (e.g. education, health, technology, social care)<\/p>\n<p>Regarding significance,\u00a0 I am now considering<\/p>\n<\/li>\n<li class=\"\" data-start=\"736\" data-end=\"825\">\n<p class=\"\" data-start=\"738\" data-end=\"825\">How <em data-start=\"1092\" data-end=\"1103\">important<\/em> was the change to the beneficiaries?<\/p>\n<\/li>\n<li class=\"\" data-start=\"736\" data-end=\"825\">\n<p class=\"\" data-start=\"738\" data-end=\"825\">Was it a <em data-start=\"1152\" data-end=\"1176\">transformational shift<\/em> or an <em data-start=\"1183\" data-end=\"1208\">incremental improvement<\/em>?<\/p>\n<\/li>\n<li class=\"\" data-start=\"736\" data-end=\"825\">\n<p class=\"\" data-start=\"738\" data-end=\"825\">Did it <em data-start=\"1219\" data-end=\"1240\">enable new thinking<\/em>, change practice, improve wellbeing, or shape policy?<\/p>\n<ul data-start=\"1086\" data-end=\"1294\">\n<li style=\"list-style-type: none;\">\n<ul data-start=\"1086\" data-end=\"1294\">I am now at the airport going to Brazil and I start to think about What types of evidence should be co-identified with, and for, beneficiaries to demonstrate Reach and Significance in an REF impact case study in Education?<\/ul>\n<\/li>\n<\/ul>\n<blockquote data-start=\"1296\" data-end=\"1390\">\n<p class=\"\" data-start=\"307\" data-end=\"390\">\nTo robustly demonstrate <strong data-start=\"331\" data-end=\"340\">Reach<\/strong> and <strong data-start=\"345\" data-end=\"361\">Significance<\/strong>, evidence should ideally be:<\/p>\n<ol data-start=\"392\" data-end=\"758\">\n<li class=\"\" data-start=\"392\" data-end=\"468\">\n<p class=\"\" data-start=\"395\" data-end=\"468\"><strong data-start=\"395\" data-end=\"407\">Credible<\/strong> \u2013 independently verifiable and attributable to the research.<\/p>\n<\/li>\n<li class=\"\" data-start=\"469\" data-end=\"627\">\n<p class=\"\" data-start=\"472\" data-end=\"627\"><strong data-start=\"472\" data-end=\"489\">Co-identified<\/strong> \u2013 meaning it is not only gathered <em data-start=\"524\" data-end=\"531\">about<\/em> the beneficiaries but also <em data-start=\"559\" data-end=\"565\">with<\/em> them, reflecting their voices, experiences, and perspectives.<\/p>\n<\/li>\n<li class=\"\" data-start=\"628\" data-end=\"758\">\n<p class=\"\" data-start=\"631\" data-end=\"758\"><strong data-start=\"631\" data-end=\"645\">Purposeful<\/strong> \u2013 aligned with the specific change you are claiming (e.g., shifts in practice, policy, awareness, or wellbeing).<\/p>\n<\/li>\n<li data-start=\"628\" data-end=\"758\">\n<table class=\"min-w-full\" style=\"height: 1560px;\" width=\"635\" data-start=\"837\" data-end=\"2439\">\n<thead data-start=\"837\" data-end=\"938\">\n<tr data-start=\"837\" data-end=\"938\">\n<th data-start=\"837\" data-end=\"860\"><strong data-start=\"839\" data-end=\"859\">Type of Evidence<\/strong><\/th>\n<th data-start=\"860\" data-end=\"872\"><strong data-start=\"862\" data-end=\"871\">Reach<\/strong><\/th>\n<th data-start=\"872\" data-end=\"891\"><strong data-start=\"874\" data-end=\"890\">Significance<\/strong><\/th>\n<th data-start=\"891\" data-end=\"916\"><strong data-start=\"893\" data-end=\"915\">Co-Identified With<\/strong><\/th>\n<th data-start=\"916\" data-end=\"938\"><strong data-start=\"918\" data-end=\"936\">Usefulness For<\/strong><\/th>\n<\/tr>\n<\/thead>\n<tbody data-start=\"1043\" data-end=\"2439\">\n<tr data-start=\"1043\" data-end=\"1217\">\n<td class=\"max-w-[calc(var(--thread-content-max-width)*2\/3)] min-w-[calc(var(--thread-content-max-width)\/3)]\" data-start=\"1043\" data-end=\"1111\">&#x1f5e3;&#xfe0f; <strong data-start=\"1049\" data-end=\"1065\">Testimonials<\/strong> (e.g. from teachers, learners, policymakers)<\/td>\n<td class=\"max-w-[calc(var(--thread-content-max-width)*2\/3)]\" data-start=\"1111\" data-end=\"1138\">&#x2714;&#xfe0f; diversity of audience<\/td>\n<td class=\"max-w-[calc(var(--thread-content-max-width)*2\/3)]\" data-start=\"1138\" data-end=\"1159\">&#x2714;&#xfe0f; depth of impact<\/td>\n<td class=\"max-w-[calc(var(--thread-content-max-width)*2\/3)]\" data-start=\"1159\" data-end=\"1192\">Direct voices of beneficiaries<\/td>\n<td class=\"max-w-[calc(var(--thread-content-max-width)*2\/3)]\" data-start=\"1192\" data-end=\"1217\">Humanising the change<\/td>\n<\/tr>\n<tr data-start=\"1218\" data-end=\"1439\">\n<td class=\"max-w-[calc(var(--thread-content-max-width)*2\/3)] min-w-[calc(var(--thread-content-max-width)\/2)]\" data-start=\"1218\" data-end=\"1311\">&#x1f4c4; <strong data-start=\"1223\" data-end=\"1245\">Letters of support<\/strong> or <strong data-start=\"1249\" data-end=\"1270\">impact statements<\/strong> from organisations or community leaders<\/td>\n<td class=\"max-w-[calc(var(--thread-content-max-width)*2\/3)]\" data-start=\"1311\" data-end=\"1348\">&#x2714;&#xfe0f; sectoral or institutional reach<\/td>\n<td class=\"max-w-[calc(var(--thread-content-max-width)*2\/3)]\" data-start=\"1348\" data-end=\"1373\">&#x2714;&#xfe0f; confirmed influence<\/td>\n<td class=\"max-w-[calc(var(--thread-content-max-width)*2\/3)]\" data-start=\"1373\" data-end=\"1397\">Partner organisations<\/td>\n<td class=\"max-w-[calc(var(--thread-content-max-width)*2\/3)]\" data-start=\"1397\" data-end=\"1439\">Validates influence on policy\/practice<\/td>\n<\/tr>\n<tr data-start=\"1440\" data-end=\"1616\">\n<td class=\"max-w-[calc(var(--thread-content-max-width)*2\/3)] min-w-[calc(var(--thread-content-max-width)\/3)]\" data-start=\"1440\" data-end=\"1491\">&#x1f4ca; <strong data-start=\"1445\" data-end=\"1471\">Surveys or evaluations<\/strong> with beneficiaries<\/td>\n<td class=\"max-w-[calc(var(--thread-content-max-width)*2\/3)]\" data-start=\"1491\" data-end=\"1520\">&#x2714;&#xfe0f; sample size &amp; diversity<\/td>\n<td class=\"max-w-[calc(var(--thread-content-max-width)*2\/3)]\" data-start=\"1520\" data-end=\"1553\">&#x2714;&#xfe0f; perceived benefit or change<\/td>\n<td class=\"max-w-[calc(var(--thread-content-max-width)*2\/3)]\" data-start=\"1553\" data-end=\"1594\">Designed with or adapted for end users<\/td>\n<td class=\"max-w-[calc(var(--thread-content-max-width)*2\/3)]\" data-start=\"1594\" data-end=\"1616\">Empirical feedback<\/td>\n<\/tr>\n<tr data-start=\"1617\" data-end=\"1830\">\n<td class=\"max-w-[calc(var(--thread-content-max-width)*2\/3)] min-w-[calc(var(--thread-content-max-width)\/3)]\" data-start=\"1617\" data-end=\"1703\">&#x1f4c8; <strong data-start=\"1622\" data-end=\"1647\">Before-and-after data<\/strong> (e.g. attendance, learning gains, access to resources)<\/td>\n<td class=\"max-w-[calc(var(--thread-content-max-width)*2\/3)]\" data-start=\"1703\" data-end=\"1726\">&#x2714;&#xfe0f; group-level reach<\/td>\n<td class=\"max-w-[calc(var(--thread-content-max-width)*2\/3)]\" data-start=\"1726\" data-end=\"1751\">&#x2714;&#xfe0f; measurable outcomes<\/td>\n<td class=\"max-w-[calc(var(--thread-content-max-width)*2\/3)] min-w-[calc(var(--thread-content-max-width)\/3)]\" data-start=\"1751\" data-end=\"1798\">Collected in collaboration with stakeholders<\/td>\n<td class=\"max-w-[calc(var(--thread-content-max-width)*2\/3)]\" data-start=\"1798\" data-end=\"1830\">Demonstrates tangible change<\/td>\n<\/tr>\n<tr data-start=\"1831\" data-end=\"2036\">\n<td class=\"max-w-[calc(var(--thread-content-max-width)*2\/3)] min-w-[calc(var(--thread-content-max-width)\/3)]\" data-start=\"1831\" data-end=\"1898\">&#x1f3af; <strong data-start=\"1836\" data-end=\"1856\">Policy citations<\/strong>, curriculum changes, or programme uptake<\/td>\n<td class=\"max-w-[calc(var(--thread-content-max-width)*2\/3)]\" data-start=\"1898\" data-end=\"1934\">&#x2714;&#xfe0f; systemic or policy-level reach<\/td>\n<td class=\"max-w-[calc(var(--thread-content-max-width)*2\/3)]\" data-start=\"1934\" data-end=\"1966\">&#x2714;&#xfe0f; institutional significance<\/td>\n<td class=\"max-w-[calc(var(--thread-content-max-width)*2\/3)]\" data-start=\"1966\" data-end=\"2009\">Documented shifts influenced by research<\/td>\n<td class=\"max-w-[calc(var(--thread-content-max-width)*2\/3)]\" data-start=\"2009\" data-end=\"2036\">Shows real-world uptake<\/td>\n<\/tr>\n<tr data-start=\"2037\" data-end=\"2241\">\n<td class=\"max-w-[calc(var(--thread-content-max-width)*2\/3)] min-w-[calc(var(--thread-content-max-width)\/3)]\" data-start=\"2037\" data-end=\"2123\">&#x1f4e2; <strong data-start=\"2042\" data-end=\"2060\">Media coverage<\/strong> highlighting community outcomes or educational transformation<\/td>\n<td class=\"max-w-[calc(var(--thread-content-max-width)*2\/3)]\" data-start=\"2123\" data-end=\"2141\">&#x2714;&#xfe0f; public reach<\/td>\n<td class=\"max-w-[calc(var(--thread-content-max-width)*2\/3)]\" data-start=\"2141\" data-end=\"2177\">&#x2714;&#xfe0f; awareness &amp; societal discourse<\/td>\n<td class=\"max-w-[calc(var(--thread-content-max-width)*2\/3)]\" data-start=\"2177\" data-end=\"2205\">Reflects public reception<\/td>\n<td class=\"max-w-[calc(var(--thread-content-max-width)*2\/3)]\" data-start=\"2205\" data-end=\"2241\">Visibility and narrative framing<\/td>\n<\/tr>\n<tr data-start=\"2242\" data-end=\"2439\">\n<td class=\"max-w-[calc(var(--thread-content-max-width)*2\/3)] min-w-[calc(var(--thread-content-max-width)\/3)]\" data-start=\"2242\" data-end=\"2314\">&#x1f9fe; <strong data-start=\"2247\" data-end=\"2280\">Funding or adoption by others<\/strong> (e.g. NGOs, ministries, schools)<\/td>\n<td class=\"max-w-[calc(var(--thread-content-max-width)*2\/3)]\" data-start=\"2314\" data-end=\"2340\">&#x2714;&#xfe0f; institutional buy-in<\/td>\n<td class=\"max-w-[calc(var(--thread-content-max-width)*2\/3)]\" data-start=\"2340\" data-end=\"2376\">&#x2714;&#xfe0f; practical endorsement of value<\/td>\n<td class=\"max-w-[calc(var(--thread-content-max-width)*2\/3)]\" data-start=\"2376\" data-end=\"2409\">Often emerges via partnerships<\/td>\n<td class=\"max-w-[calc(var(--thread-content-max-width)*2\/3)]\" data-start=\"2409\" data-end=\"2439\">Tangible evidence of worth<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/li>\n<li data-start=\"628\" data-end=\"758\">\n<h3 class=\"\" data-start=\"2446\" data-end=\"2464\">In summary:<\/h3>\n<p class=\"\" data-start=\"2466\" data-end=\"2686\">For a 4* impact case in education, your evidence should not just \u201cshow\u201d change\u2014it should be <strong data-start=\"2558\" data-end=\"2621\">anchored in the realities and reflections of those affected<\/strong>. Ideally, this evidence is co-developed or co-validated through:<\/p>\n<ul data-start=\"2688\" data-end=\"2924\">\n<li class=\"\" data-start=\"2688\" data-end=\"2749\">\n<p class=\"\" data-start=\"2690\" data-end=\"2749\"><strong data-start=\"2690\" data-end=\"2719\">Collaborative instruments<\/strong> (e.g. joint evaluation tools)<\/p>\n<\/li>\n<li class=\"\" data-start=\"2750\" data-end=\"2826\">\n<p class=\"\" data-start=\"2752\" data-end=\"2826\"><strong data-start=\"2752\" data-end=\"2777\">Participatory methods<\/strong> (e.g. reflective workshops, co-produced reports)<\/p>\n<\/li>\n<li class=\"\" data-start=\"2827\" data-end=\"2924\">\n<p class=\"\" data-start=\"2829\" data-end=\"2924\"><strong data-start=\"2829\" data-end=\"2847\">Feedback loops<\/strong> (e.g. testimonials or surveys that were returned to stakeholders for review)<\/p>\n<\/li>\n<\/ul>\n<p class=\"\" data-start=\"2926\" data-end=\"3048\">This helps ensure that the evidence is not only <strong data-start=\"129\" data-end=\"156\">relevant and meaningful<\/strong>, and not only <strong data-start=\"171\" data-end=\"186\">trustworthy<\/strong> in the eyes of the REF panel, but ultimately reflects <strong data-start=\"241\" data-end=\"300\">research that is worthwhile and makes a real difference!<\/strong><\/p>\n<\/li>\n<\/ol>\n<\/blockquote>\n<\/li>\n<\/ul>\n","protected":false},"excerpt":{"rendered":"<p>by Ale Okada\u00a0 \u00a0 I had a great meeting with Prof Jane Seale and Dr Lauren OHagan about Understanding 4 REF Impact in Education: Reach and Significance Matter Most* I am attending the\u00a0 \u00a0Global Education Meetign led by UNESCO in Brazil Fortaleza As we prepare for the next Research Excellence Framework (REF) submission, it\u2019s important &hellip; <\/p>\n<p class=\"link-more\"><a href=\"https:\/\/www.open.ac.uk\/blogs\/rumpus\/index.php\/2024\/10\/15\/851\/\" class=\"more-link\" >Continue reading<span class=\"screen-reader-text\"> &#8220;REF*  Impact in Education &#8211; Case Reflections&#8230;&#8221;<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[15,1,9],"tags":[],"class_list":["post-851","post","type-post","status-publish","format-standard","hentry","category-articles","category-fun","category-news"],"_links":{"self":[{"href":"https:\/\/www.open.ac.uk\/blogs\/rumpus\/index.php\/wp-json\/wp\/v2\/posts\/851","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.open.ac.uk\/blogs\/rumpus\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.open.ac.uk\/blogs\/rumpus\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.open.ac.uk\/blogs\/rumpus\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.open.ac.uk\/blogs\/rumpus\/index.php\/wp-json\/wp\/v2\/comments?post=851"}],"version-history":[{"count":3,"href":"https:\/\/www.open.ac.uk\/blogs\/rumpus\/index.php\/wp-json\/wp\/v2\/posts\/851\/revisions"}],"predecessor-version":[{"id":977,"href":"https:\/\/www.open.ac.uk\/blogs\/rumpus\/index.php\/wp-json\/wp\/v2\/posts\/851\/revisions\/977"}],"wp:attachment":[{"href":"https:\/\/www.open.ac.uk\/blogs\/rumpus\/index.php\/wp-json\/wp\/v2\/media?parent=851"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.open.ac.uk\/blogs\/rumpus\/index.php\/wp-json\/wp\/v2\/categories?post=851"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.open.ac.uk\/blogs\/rumpus\/index.php\/wp-json\/wp\/v2\/tags?post=851"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}