REF 2021 – Assessing Value for Money

This microsite presents the results of a value for money exercise based on an analysis of public domain data from the UK REF 2021 Research Assessment Exercise.  Note this analysis focuses solely on Unit of Assessment 11 – Computer Science and Informatics, but this analysis could be repeated for other areas.

Perhaps unsurprisingly, ‘league tables’ sorted by value for money are markedly different from those sorted by raw score.  This casts doubt on the prudence of allocation of research funding based largely on research output volume.  It also suggests we need a better understanding of the circumstances that lead to better research outcomes.

Each year the UK governments invests around two billion pounds in general underpinning funding (QR) for university research with around nine billion pounds distributed to university and industry projects through UKRI.  It is critical that this money is spent well, and the REF exercise was created explicitly to inform the first of theses, QR (quality related) funding.

REF evaluates the overall research quality and volume of research in individual units of assessments (roughly a university department or subject areas) and these research league tables are also a critical part of university esteem and attracting students.

However, on its own this does not tell us where the best value for money.  In terms of business return on investment (RoI), where does a million pounds of research spend create the greatest amount of high quality research?

I was part of the UoA 11 panel during the 2014 REF exercise and after it was finished I analysed the public domain data revealing a shockingly high apparent bias within the subject against certain kinds of institution and subject areas.  In that exercise the financial data to make a RoI calculation was not available in the public domain data.

The corresponding information for bias analysis was not available in the public domain REF 2021, data, but earlier this year I realised that the financials are included.  This has enabled a fairly straightforward value for money calculations.  The report and this microsite presents the results and methods of that analysis.