Contents
- 10.1 If there is something there, make sure you find it
- 10.2 The noise–effect–number triangle
- 10.2.1 General strategies
- 10.3 Subjects
- 10.3.1 More subjects or trials (increase number)
- 10.3.2 Within-subjects/within-groups studies (reduce noise)
- 10.3.3 Matched users (reduce noise)
- 10.3.4 Targeted user group (increase effect)
- 10.4 Tasks
- 10.4.1 Distractor tasks (increase effect)
- 10.4.2 Targeted tasks (increase effect)
- 10.4.3 Demonic interventions! (increase effect)
- 10.4.4 Restricted tasks (reduce noise)
Glossary items referenced in this chapter
A–B test, alternative hypothesis, balance orders, Bayesian statistics, between-subjects design, cognitive load, control conditions, controlled environments, controlled experiment, controlled tasks, coping mechanisms, demonic intervention, distractor task, ecological validity, effect size, elderly users, error bars, expert slip, false baseline, false negative, false positive, Fitts' Law, five users (the myth), formative evaluation, Human–Computer Interaction, HCI studies, hypothesis testing, image recognition, in-the-wild, increase effect, increase number, individual differences, iterative development, learning effects, manipulate the sensitivity, matching subjects, mathematics, mean (μ), menu navigation, Müller–Lyer illusion, narrow user group, noise–effect–number triangle, nonlinearity, not significant, null hypothesis, order effects, pair subjects, parameters, power-law distribution, presidential inauguration, random noise, randomise orders, real world, reduce noise, restricted device, restricted system, restricted tasks, sample, size of the `noise', size of the study, social network data, sonic enhancement, standard deviation (s.d., σ), standard error (s.e.), statistical analysis, statistical power, statistically significant, steering task, subjects, summative evaluation, targeted tasks, targeted user group, task completion time, tasks, the job of statistics, uncontrolled factors, uncontrolled variation, user experience, user experience studies, user studies, variability, WEIRD people, within-groups studies, within-subjects experiment, Wizard of Oz prototyping