HARK No More: On the Preregistration of CHI Experiments

Andy Cockburn1, Carl Gutwin2, Alan Dix3

1 University of Canterbury, Christchurch, New Zealand
2 University of Saskatchewan, Saskatoon, Canada
3 HCI Centre, University of Birmingham, UK and Talis, Birmingham, UK

Paper presented at CHI 2018, Monteal, Canada (CHI Best Paper Award)


Experimental preregistration is required for publication in many scientific disciplines and venues. When experimental intentions are preregistered, reviewers and readers can be confident that experimental evidence in support of reported hypotheses is not the result of HARKing, which stands for Hypothesising After the Results are Known. We review the motivation and outcomes of experimental preregistration across a variety of disciplines, as well as previous work commenting on the role of evaluation in HCI research. We then discuss how experimental preregistration could be adapted to the distinctive characteristics of Human-Computer Interaction empirical research, to the betterment of the discipline.

Keywords: Experimental preregistration; HARKing, p-fishing; NHST controversy; file drawer effect; replication.

 

 

References

Andy Cockburn presenting the paper at CHI 2018.

 

 

 

 

 


http://www.hcibook.com/alan/papers/chi2018-hark-no-more/

Alan Dix 28/6/2020