![\"\"](\"https:\/\/alandix.com\/ref2021\/files\/2025\/12\/F4-GPA-raw-order-top-and-bottom-scaled.jpg\")
<\/a><\/p>\nFig 4. Ordered by GPA value (column AF)<\/p>\n
However, if we reorder by the value for money criterion (Fig. 5), a completely different picture emerges.\u00a0 \u00a0Now Liverpool Hope and UCLAN lead the table as giving the best GPA per pound of research investment.\u00a0 Notably, while some are based on small submissions, so could be argued to be subject to noise, others, notably Northumbria, have a substantial submission.\u00a0 Birkbeck is the only pre-1992 institution in the top ten (Loughborough is next at slot 13).<\/p>\n
Fig 5. Ordered by GPA value for money (column AP)<\/p>\n The bottom of the table is now almost a complete reverse of the raw GPA ordering with Imperial, Manchester, Edinburgh and Oxford at the tail.\u00a0 Wrexham is an outlier as it recorded negative research income for the period, so while it is included in Figure 5 for completeness and transparency, it should be disregarded for most purposes.<\/p>\n Note too that the value for money ratio between the top and low end of the table (ignoring Liverpool Hope as it is another outlier) is 30:1, that is it costs twenty to thirty times more to get a similar \u2018GPA worth\u2019 of output at universities such as Imperial or Manchester compared to those such as UCLAN or Northumbria.<\/p>\n The exact figures for each individual institution should not be read too precisely as there are many local factors that need to be taken into account.\u00a0 However, the overall pattern is clear with institutions that would normally (and largely rightly) be regarded as the front runners in terms of overall research quality faring far less well in terms of value for money compared with those that would be regarded as far lower in terms of research esteem.<\/p>\n The results for using the QR-formula follow a roughly similar pattern with expected institutions (Imperial, Oxford, Birmingham) leading the raw value table (Fig. 6), and post-1992 institutions (London Met, Wolverhampton, Wrexham) at the bottom.<\/p>\n Fig 6. Ordered by overall QR\u2013formula weighting (column AF)<\/p>\n However, just as with the GPA-based measures, the value-for-money ordering (Fig. 7)\u00a0 radically changes this picture.\u00a0 The top is dominated by post-1992 institutions: Liverpool Hope, Northumbria, UCLAN, etc.) with Birkbeck again a notable exception and Loughborough also in the top ten.<\/p>\n Fig 7. Ordered by QR\u2013formula-weighting value for money (column AP)<\/p>\n The bottom of the table is a little more mixed, but the lower part is still dominated by research-intensive universities and only a few institutions bucking the trend.\u00a0 Of the Russell Group, only two get into the top quarter of the value for money table: Durham (slot 17) and Warwick (slot 23).<\/p>\n As expected, if we focus entirely on 4* outputs, the same institutions dominate the top of the league table (Imperials, Oxford, Warwick), with post-1992 institutions at the bottom.<\/p>\n Fig 8. Ordered by 4* output volume (column AG)<\/p>\n Note that three, Wrexham, Sunderland and Robert Gordon have no 4* outputs in UoA11, which lead to anomalous results (Excel \u201c#DIV\/0!\u201d error) in the value for money spreadsheet (Fig. 9).\u00a0 Ignoring these, post-1992 institutions again rank high (Liverpool Hope, Northumbria, etc.), with Birkbeck as an exception and this time other pre-1992 institutions in the top 10 with Universities of Liverpool, Kent and Warwick at positions 8, 9 and 10.<\/p>\n Fig 9. Ordered by 4* output volume value for money (column AQ)<\/p>\n The bottom of the table is also a little more mixed with Imperial managing about a quarter of the way up from the bottom and Oxford at around the half way point.\u00a0 Also, the overall ratio of value-for-money varies a little less across the table.<\/p>\n In each case the results follow very similar trends.\u00a0 If one ranks by raw quality metrics then the well-known leaders are at the top, mostly Russell Group, with mainly post-a992 institutions towards the end if the table.<\/p>\n However, when we apply value-for-money measures the opposite trend is seen, with post-1992 institutions largely dominating the top of the value-for-money tables.\u00a0 One could argue that REF is about research excellence<\/em>, and therefore metrics should focus most highly towards the 4* and 3* and the QR-formula weighted metric reflects this.\u00a0 Others would focus entirely on publications and impact within<\/em> the research community (of course, this this is not the view of government, nor REF as a whole), and so the 4* outputs only measure focuses entirely on this.\u00a0 However, while this extreme picture is a little more mixed, even using this more heavily 4* and publication weighted measure, the overall value for money trends are still apparent.<\/p>\n It should be noted also that in my analysis of the REF2014 outcomes [Dx15b], it was clear that in UoA11, we had very strong implicit bias towards the more obvious institutions, tending on average to under-rate post-1992 institutions and over-rate Russell Group with respect to internationally normalised metrics.\u00a0 If the above analysis were repeated with, say, Scopus field-weighted citation data the results would be likely to be more extreme.<\/p>\n","protected":false},"excerpt":{"rendered":" 4.1\u00a0 Overall GPA based orderings Figure 4 shows the top and bottom of the table of institutions ordered by UoA11 overall GPA.\u00a0 The top few institutions are exactly as one would expect: Imperial, Oxford, and other Russell group members.\u00a0 Similarly … Continue reading <\/a><\/p>\n4.2\u00a0 Overall QR-formula based orderings<\/h2>\n
<\/a><\/p>\n<\/a><\/p>\n4.3\u00a0 Four star output based orderings<\/h2>\n
<\/a><\/p>\n<\/a><\/p>\n4.4\u00a0 Result summary<\/h2>\n