{"id":47,"date":"2025-12-07T14:13:31","date_gmt":"2025-12-07T14:13:31","guid":{"rendered":"https:\/\/alandix.com\/ref2021\/?page_id=47"},"modified":"2025-12-07T14:15:04","modified_gmt":"2025-12-07T14:15:04","slug":"3-procedure","status":"publish","type":"page","link":"https:\/\/alandix.com\/ref2021\/report\/3-procedure\/","title":{"rendered":"3.\u00a0 Procedure for Analysing REF Value for Money"},"content":{"rendered":"<p>All raw data was obtained from the REF2021 results and submissions database for UoA11, Computer Science and Informatics [REF21a] and the REF2021 \u2013 Results and submissions \u2013 Environment submissions database [REF21b]. Full details of this procedure can be found in Appendix A.<\/p>\n<p>Two of the downloadable CSV files were used. One has four rows for each institution listing a profile of 4*,3*, 2*, 1* and unclassified results for Overall, Outputs, Impact and Environments (Fig. 1). The other has fifteen rows for each institution listing various types of income source (excluding QR) and an overall total over different year ranges (Fig. 2).\u00a0 There is also an income-in-kind spreadsheet, but this appears very partial and has not been used for this analysis.<\/p>\n<p style=\"text-align: center;\"><a href=\"https:\/\/alandix.com\/ref2021\/files\/2025\/12\/F1-results-spreadsheet-detail.jpg\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-large wp-image-48\" src=\"https:\/\/alandix.com\/ref2021\/files\/2025\/12\/F1-results-spreadsheet-detail-1024x325.jpg\" alt=\"\" width=\"584\" height=\"185\" srcset=\"https:\/\/alandix.com\/ref2021\/files\/2025\/12\/F1-results-spreadsheet-detail-1024x325.jpg 1024w, https:\/\/alandix.com\/ref2021\/files\/2025\/12\/F1-results-spreadsheet-detail-300x95.jpg 300w, https:\/\/alandix.com\/ref2021\/files\/2025\/12\/F1-results-spreadsheet-detail-768x244.jpg 768w, https:\/\/alandix.com\/ref2021\/files\/2025\/12\/F1-results-spreadsheet-detail-500x159.jpg 500w, https:\/\/alandix.com\/ref2021\/files\/2025\/12\/F1-results-spreadsheet-detail.jpg 1084w\" sizes=\"auto, (max-width: 584px) 100vw, 584px\" \/><\/a><\/p>\n<p style=\"text-align: center;\">Fig 1. Results spreadsheet detail (example entry, University of Aberdeen)<\/p>\n<p style=\"text-align: center;\"><a href=\"https:\/\/alandix.com\/ref2021\/files\/2025\/12\/F2-income-spreadsheet-detail.jpg\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-large wp-image-49\" src=\"https:\/\/alandix.com\/ref2021\/files\/2025\/12\/F2-income-spreadsheet-detail-1024x448.jpg\" alt=\"\" width=\"584\" height=\"256\" srcset=\"https:\/\/alandix.com\/ref2021\/files\/2025\/12\/F2-income-spreadsheet-detail-1024x448.jpg 1024w, https:\/\/alandix.com\/ref2021\/files\/2025\/12\/F2-income-spreadsheet-detail-300x131.jpg 300w, https:\/\/alandix.com\/ref2021\/files\/2025\/12\/F2-income-spreadsheet-detail-768x336.jpg 768w, https:\/\/alandix.com\/ref2021\/files\/2025\/12\/F2-income-spreadsheet-detail-500x219.jpg 500w, https:\/\/alandix.com\/ref2021\/files\/2025\/12\/F2-income-spreadsheet-detail.jpg 1368w\" sizes=\"auto, (max-width: 584px) 100vw, 584px\" \/><\/a><\/p>\n<p style=\"text-align: center;\">Fig 2. Income spreadsheet detail (example entry, University of Aberdeen)<\/p>\n<p>These CSV files were combined using bespoke code to create a single JSON file and then an extract of this used to create a summary spreadsheet (Fig. 3), which was used for subsequent analysis.\u00a0 The summary spreadsheet has one row per institution including all of the profiles and three of the income figures.\u00a0 This analysis only used the total income from all sources as there were so many subcategories.\u00a0 It would be possible to repeat the analysis focusing on specific categories, for example UKRI income.\u00a0 The summary spreadsheet includes the figures for \u2018average income for academic years 2015-16 to 2019-20\u2019, \u2018average income for academic years 2013-14 to 2019-20\u2019 and \u2018total income for academic years 2013-14 to 2019-20\u2019, but only the last was used in the value for money calculations as the three figures correlate strongly.<\/p>\n<p><a href=\"https:\/\/alandix.com\/ref2021\/files\/2025\/12\/F3-combined-spreadsheet-detail-scaled.jpg\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-large wp-image-50\" src=\"https:\/\/alandix.com\/ref2021\/files\/2025\/12\/F3-combined-spreadsheet-detail-1024x266.jpg\" alt=\"\" width=\"584\" height=\"152\" srcset=\"https:\/\/alandix.com\/ref2021\/files\/2025\/12\/F3-combined-spreadsheet-detail-1024x266.jpg 1024w, https:\/\/alandix.com\/ref2021\/files\/2025\/12\/F3-combined-spreadsheet-detail-300x78.jpg 300w, https:\/\/alandix.com\/ref2021\/files\/2025\/12\/F3-combined-spreadsheet-detail-768x199.jpg 768w, https:\/\/alandix.com\/ref2021\/files\/2025\/12\/F3-combined-spreadsheet-detail-1536x399.jpg 1536w, https:\/\/alandix.com\/ref2021\/files\/2025\/12\/F3-combined-spreadsheet-detail-2048x532.jpg 2048w, https:\/\/alandix.com\/ref2021\/files\/2025\/12\/F3-combined-spreadsheet-detail-500x130.jpg 500w\" sizes=\"auto, (max-width: 584px) 100vw, 584px\" \/><\/a><\/p>\n<p>Fig 3. Combined spreadsheet detail<\/p>\n<p>This data was then used to look at three overall quality metrics and corresponding value-for-money measures.\u00a0 The quality metrics used were:<\/p>\n<ul>\n<li><em>overall GPA<\/em> \u2013 \u00a0This uses the overall profile (itself a 60%:25%:15% weighted sum of the output, impact and environment profiles) and then weights the 4*, 3*, 2* and 1* scores by 4,3,2,1 respectively.\u00a0 This form of GPA is widely used on university websites.<\/li>\n<li><em>overall \u2018QR\u2019 formula<\/em> \u2013 \u00a0This uses the same output profile, but weights 4*, 3*, 2* and 1* scores by 4,1,0,0 respectively, following the Research England funding model [RE23, p.15].<\/li>\n<li><em>4* outputs only<\/em> \u2013 \u00a0This just counted the 4* rated outputs.\u00a0 Some academics would regard this as the gold standard for research discounting the value of impact.\u00a0 While this is a diminishing viewpoint in modern academia, it allows an extreme point to be measured.<\/li>\n<\/ul>\n<p>To turn this into a quality for money measure the raw score was multiplied by the volume of the submission (FTE submitted column) to give a quality volume.\u00a0 This is because the actual number of outputs and impact case studies is roughly proportionate to the submission size.\u00a0 For environment, the multiplier is less clear, however the quality of the environment affects more staff in a bigger submission and many costs (travel, etc.) are proportionate to the staff base, so multiplying the raw environment score by volume also seems appropriate.\u00a0 The quotient of this quality volume and the total income then gives a cost per unit of quality and value for money metric.<\/p>\n<p>&nbsp;<\/p>\n","protected":false},"excerpt":{"rendered":"<p>All raw data was obtained from the REF2021 results and submissions database for UoA11, Computer Science and Informatics [REF21a] and the REF2021 \u2013 Results and submissions \u2013 Environment submissions database [REF21b]. Full details of this procedure can be found in &hellip; <a href=\"https:\/\/alandix.com\/ref2021\/report\/3-procedure\/\">Continue reading <span class=\"meta-nav\">&rarr;<\/span><\/a><\/p>\n","protected":false},"author":2,"featured_media":0,"parent":37,"menu_order":3,"comment_status":"closed","ping_status":"closed","template":"","meta":{"_themeisle_gutenberg_block_has_review":false,"footnotes":""},"class_list":["post-47","page","type-page","status-publish","hentry"],"_links":{"self":[{"href":"https:\/\/alandix.com\/ref2021\/wp-json\/wp\/v2\/pages\/47","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/alandix.com\/ref2021\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/alandix.com\/ref2021\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/alandix.com\/ref2021\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/alandix.com\/ref2021\/wp-json\/wp\/v2\/comments?post=47"}],"version-history":[{"count":1,"href":"https:\/\/alandix.com\/ref2021\/wp-json\/wp\/v2\/pages\/47\/revisions"}],"predecessor-version":[{"id":51,"href":"https:\/\/alandix.com\/ref2021\/wp-json\/wp\/v2\/pages\/47\/revisions\/51"}],"up":[{"embeddable":true,"href":"https:\/\/alandix.com\/ref2021\/wp-json\/wp\/v2\/pages\/37"}],"wp:attachment":[{"href":"https:\/\/alandix.com\/ref2021\/wp-json\/wp\/v2\/media?parent=47"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}