This report outlines an initial exercise using public domain REF 2021 data to perform a value-for-money assessment of UoA11 (Computing and Informatics) submissions.

REF created results as a series of quality profiles (4* to 1*) against three areas: Outputs, Impact and Environment.  There are various ways one can combine these to produce a single quality metric and three are presented here: overall GPA (weighting 4*,3*,2*,1* by 4,3,2,1 respectively), overall ‘QR formula’ (4* and 3* weighted 4:1 with 2* and below zero weighted [RE23, p.15]), and 4* outputs only (largely published papers, which some academics would regard as the gold standard).  In all cases the league tables are partially or completely reversed with far higher value-for-money evident in what would normally be regarded as lower-ranking research institutions.

The analysis could be performed on any UoA, and computing was chosen for this initial analysis in large part because it is the author’s discipline.  However, it is probably also a good general indicator of trends as computing research is often money intensive, so one expects funding to lead to outputs, but largely in terms of people rather than large equipment (as in Physics), so less variable across different sub-areas of the discipline.

This report first describes some of the background that led to the analysis (section 2); followed by details on how the data was obtained and processed (section 3); the results (section 4); and final discussion (section 5).

It should be noted that the various ‘league tables’ produced by this should not be taken as an assessment of any individual institution as there are individual factors that affect the precise mapping between income and outputs that can skew results.  However, the overall trends do seem robust and mirror analysis of NSERC in Canada, which found diminishing returns of funding [FC13].