As we reported last summer, The Metric Tide is an independent report commissioned by HEFCE that was published in July 2015. It looks at the current and future roles of quantitative evaluation of research (i.e. bibliometrics, altmetrics and the like) and has a particular focus on the use of these measures in relation to the next REF.

Nearly a year on, we will look at its recommendations and reception further, especially in the light of other documents regarding research assessment.

The Metric Tide recommends that metrics are used responsibly and provides a definition of responsible use:

“Robustness: basing metrics on the best possible data in terms of accuracy and scope

Humility: recognising that quantitative evaluation should support – but not supplant – qualitative, expert assessment

Transparency: keeping data collection and analytical processes open and transparent, so that those being evaluated can test and verify the results

Diversity: accounting for variation by field, and using a range of indicators to reflect and support a plurality of research and researcher career paths across the system

Reflexivity: recognising and anticipating the systemic and potential effects of indicators, and updating them in response.”

The report also recommends that metrics are used to play a supporting role alongside expert qualitative evaluation (i.e. peer review etc.) rather than replace it. As such, it can be seen to be broadly in line with the San Francisco Declaration on Research Assessment (DORA) and the Leiden Manifesto for Research Metrics.  To many, these documents represent moves towards a new and different balance to research assessment.

The recommendations in The Metric Tide could be adopted as part of the next REF but this is not guaranteed. Indeed, it has been observed that developments in the UK higher education and research environment could affect if and how the report’s recommendations are implemented.  Since those observations were made there have been many further developments and publication of Lord Stern’s review of the REF, which could have a particular impact, is still awaited.

The chair of the review wrote that the report has increased healthy debate and provides evidence to help the UK higher education community shape research assessment. However, even amongst those who agree with the report’s method and findings, it has been argued that the environment surrounding evaluation of research in UK higher education makes implementing metrics responsibly very difficult.