Professor Richard Holliman, The Open University.

Over recent years, STFC’s Public Engagement Team has undertaken a series of strategic reviews to better understand and refine the systems used to create, support, and evaluate high-quality public engagement in the STFC community. (STFC is the Science and Technology Facilities Council.) The most recent of these is called Pathways to Excellence in Public Engagement.

A Catalyst for PEACE
Given my previous role as the operational lead on the OU’s Public Engagement with Research (PER) Catalyst, An Open Research University, I was delighted to be involved in two of these important reviews.

In the first instance, I was a member of the review team that produced the PEACE Report (Public Engagement: Attitudes, Culture and Ethos).

Public Engagement: Attitudes, Culture and Ethos (STFC, 2016).

This review uncovered STFC researchers’ perceptions of the challenges and opportunities they faced in the course of balancing engagement activities alongside their scientific and engineering careers (see ‘Give PEACE a chance’).

Together with the findings from the PER Catalyst programme (NCCPE, 2016), the PEACE report answered some significant questions in relation to how researchers are adapting to the routine requirements to plan for, enact and evidence impacts derived from publicly-funded research.

Collectively, these pieces of work also raised other important questions, not least in relation to whether Pathways to Impact Planning in its current form was consistently delivering rigorous, well-resourced programmes of impact-generating activity.

Pathways to Excellence in Public Engagement

It is in this context that I was asked by the STFC Public Engagement Team to lead a review to explore these important issues. The recently published PEPE Report (Pathways to Excellence in Public Engagement) Report is the result of this review.

PEPE explores how STFC researchers are responding to the requirements to plan for, assess, monitor and report impacts. The scope of the review was defined according to the stages of the peer review system – planning, assessment, monitoring, and reporting. We collected and analysed quantitative and qualitative evidence from the STFC community and the outputs of their research programmes as reported to STFC.

We have found that the current system of Pathways to Impact Planning is struggling to consistently deliver rigorous, well-resourced programmes of impact-generating activity, including public engagement.

Pathways to Excellence in Public Engagement (STFC, 2018).

Planning
In relation to planning, we identified confusion within the STFC research community about: the distinction between academic and non-academic impact; how to deliver high-quality, upstream Pathways to Impact Planning; whether high-quality Pathways to Impact Planning is an essential requirement for funding; and from which schemes it is appropriate to seek funding.

Further, we found that the tailoring and prioritisation of Pathways to Impact Planning is often linked with issues of under-resourcing. In effect, high-quality, upstream Pathways to Impact Planning is less likely to be prioritised when it is not valued in monetary terms.

Reviewing
The current system for assessing Pathways to Impact is inconsistent, unclear and lacks rigour in places. There is a lack of clarity about what constitutes ‘acceptability’ in relation to Pathways to Impact.

Funding for such planning is rarely requested; priority is given by panels to what are considered to be bone fide research activities. Applicants are adapting their Pathways to Impact Planning processes accordingly, e.g. by not requesting funding for these activities.

Variability in the assessment of Pathways to Impact is a key issue that requires attention from STFC. Applicants should have confidence that the system is rigorous, fair, consistent and transparent.

Monitoring
In relation to the monitoring of Pathways to Impact Planning, we have identified that planning and assessment of Pathways to Impact is failing to routinely deliver excellence in the ways impact-generating activities are delivered and evidenced.

Established ‘transmission’ practices are the norm for the vast majority of STFC researchers to the detriment of cooperative and collaborative forms of engagement. There is little evidence of active management of Pathways to Impact Planning in terms of progress or performance.

Reporting
The measuring and reporting of Pathways to Impact outcomes is a work in progress. Whilst we found that greater numbers of activities are being recorded, there is still some way to go to ensure that the impact of these activities is rigorously measured and reported consistently.

Recommendations and Actions
The findings documented in the PEPE Report show that, to some degree, all aspects of Pathways to Impact – planning, assessment, monitoring and reporting – could be improved.

Comprehensive, upstream, tailored planning should become an essential requirement for Pathways to Impact. Clear and consistent advice and guidance for researchers will be required to improve the planning phase. This needs to be matched by a rigorous, consistent and transparent process of assessment and feedback. Panels require routine access to relevant expertise to make high-quality assessments and offer constructive feedback.

Given the challenges identified in the PEPE Report it is important to note that each of the issues can be resolved. To this end, nine recommendations are listed in the report for STFC to consider. Further, an initial action plan has been drafted, discussed and refined following deliberations with STFC Grant Managers and the Advisory Panel for Public Engagement.

The PEPE Report is another step along the pathway to delivering excellence in public engagement with STFC research. In the coming months and years, STFC (as part of UKRI) is likely to require the support of researchers and impact experts as they seek to implement change.

If we continue to embrace the best principles of engagement, together we can deliver a publicly-funded system where excellence is at the heart of all aspects of research. Given the origins of this work, such a result could be considered a ‘noble PEACE prize’.

Acknowledgements
PEPE was a team effort. As Head of Skills and Engagement at STFC, Dr Derek Gillespie commissioned the review and supported the process throughout. He convened a Working Group to explore these issues and invited me to act as the Chair. In this role, I worked closely with Dr Neville Hollingworth (STFC), with support from Katherine Leech (University of York) and Dr June McCombie (University of Nottingham).

As a sub-group of the wider STFC Advisory Panel for Public Engagement, we were joined by Ian Fuller (STFC) and Dr Ruth Townsley (an independent consultant).