Handouts: the plains of Nali  |  two horse races   |  a little experiment

Many researchers find statistics confusing.  The aim of this tutorial is to help attendees understand what the statistics they see around them actually mean. It fills the gap between the ‘how to’ knowledge in basic statistics courses and deep understanding, from traditional hypothesis testing to Bayesian techniques.  This tutorial builds on previous tutorials and master classes, and his recently published book “Statistics for HCI: Making Sense of Quantitative Data” and also extensive freely available online materials including videos and glossary.

Many of the issues discussed will not be found in a traditional textbook or statistics course, so there will be things to learn for those already using statistics in their work.  However, the material will be expressed in ways that do not assume any existing statistical expertise.  It includes aspects of statistical `craft’ skill that you will not find in conventional material.

Intended Audience

This tutorial is intended for both experienced researchers and those who have already engaged, or intend to engage, in quantitative analysis of empirical data or other forms of statistical analysis. It will also be of value to practitioners using quantitative evaluation. There will be occasional formulae, but the focus is on conceptual understanding, not mathematical skills.

Main Topics

As there are extensive online materials available and the book itself, the tutorial can follow the T-model of teaching, combining a broad roadmap of the area, with depth in a few topics.  If they can feel mastery of the material and know what is available, they can fill in gaps in a ‘just in time’ manner when they wish.

Roadmap topics (likely foci in bold)

Understanding randomness

• Helping the attendees to get a gut feel for the behaviour of random phenomenon and in particular how easy it is to see structure when there is none.
•  Learning how probability can help us by modelling random phenomena and filling in the gaps where our intuition fails.
• The reason for the ubiquity of the Normal distribution and why the statistics based on the Normal distribution cannot be applied without great care to power-law data from social networks and similar phenomena.

Doing it (if not p then what?)

• traditional (p-testing) and Bayesian statistics – what they mean and what they don’t
• philosophical differences between the two
• common issues including the role of the researcher vs the experiment

Design and implementation

• gaining power – how to ensure studies and experiments reveal real effects including the noise–effect–number triangle
• interpreting and visualising results

The highlighted aspects are likely areas to drop into depth as they are least common in standard ‘how to do it’ treatments.  However, I’d also like to offer plenty of opportunities for participants to ask questions, so will weight up presented content vs. discursive elements.  The ultimate aim is not to cover loads of content – the online materials provide that anyway, but to give the participants the sense that this is something they can engage with themselves.