Metrics are rubbish, but ...

Alan Dix

HCI Centre, University of Birmingham, UK

Presentation at "Doing metrics responsibly: how far have we come and what are the next steps", ARMA Metrics Special Interest Group, Association of Research Managers and Administrators Annual Conference, Liverpool, 7th June 2017

Download position paper (PDF, 74K)

Panel presentation based on my experince as a member of the computer science and informatics subpanel at REF2014, and subsequent metrics-based analysis of public-domain REF data, which revealed order of magnitude biases.

My final conclusion is:

metrics are rubbush
but ...
people are far worse

Keywords: research assessment, REF, metrics, citations.

Metrics are rubbish, but ... from Alan Dix


Alan Dix is a Professor in the HCI Centre at the University of Birmingham and Senior Researcher at Talis. He has worked in human–computer interaction research since the mid 1980s, and is the author of one of the major international textbooks on HCI and over 400 research publications. In 2013 he produced an HCI MOOC, which is now hosted at, and in the same year he walked 1000 miles round the coast of Wales. The data from the latter is available in the public domain as an ‘open science’ resource. He organises a twice yearly workshop, Tiree Tech Wave, on the small Scottish island where he lives, and where he has been engaged in a number of projects relating to heritage , communications, energy use and open data. His Talis research is focused on aspects of education including learning analytics and flipped class teaching, often using HCI educational resources as exemplars. He was a member of sub-panel 11 in REF2014, and has since analysed the public domain REF data using citation data to uncover unintentional bias. The experience has converted him from being a metrics sceptic, to a vocal proponent of greater use of metrics in research assessment..

Alan Dix 8/6/2017