Panel presentation based on my experince as a member of the computer science and informatics subpanel at REF2014, and subsequent metrics-based analysis of public-domain REF data, which revealed order of magnitude biases.
My final conclusion is:
metrics are rubbush
but ...
people are far worse
Keywords: research assessment, REF, metrics, citations.
Bio
Alan Dix is a Professor in the HCI Centre at the University of Birmingham and Senior Researcher at Talis. He has worked in human–computer interaction research since the mid 1980s, and is the author of one of the major international textbooks on HCI and over 400 research publications. In 2013 he produced an HCI MOOC, which is now hosted at InteractionDesign.org, and in the same year he walked 1000 miles round the coast of Wales. The data from the latter is available in the public domain as an ‘open science’ resource. He organises a twice yearly workshop, Tiree Tech Wave, on the small Scottish island where he lives, and where he has been engaged in a number of projects relating to heritage , communications, energy use and open data. His Talis research is focused on aspects of education including learning analytics and flipped class teaching, often using HCI educational resources as exemplars. He was a member of sub-panel 11 in REF2014, and has since analysed the public domain REF data using citation data to uncover unintentional bias. The experience has converted him from being a metrics sceptic, to a vocal proponent of greater use of metrics in research assessment..
|
