Contents
- 8.1 Cherry picking
- 8.1.1 Multiple tests
- 8.1.2 Multiple statistics
- 8.1.3 Outliers
- 8.1.4 Post-hoc hypothesis
- 8.1.5 The file drawer effect
- 8.2 Inter-related factors
- 8.2.1 Non-independently controllable factors
- 8.2.2 Correlated features
- 8.3 Everything is random
- 8.4 The same or worse
- 8.4.1 Everything is unlikely
- 8.4.2 Numeric data
- 8.4.3 More complex ‘or worse’
- 8.4.4 Post-hoc corrections
- 8.5 What you can say – phenomena and statisticians
Glossary items referenced in this chapter
Bayesian reasoning, Bayesian statistics, Bayesian test, bias, Bonferroni correction, cherry picking, coin tossing, confidence interval, confirmation bias, continuous distribution, emergent patterns, estimated standard deviation, extreme values, factorial, fair coin, false negative, file drawer effect, fishing expedition, hypothesis testing, inter-related factors, likelihood, machine learning, mean (μ), multiple statistics, multiple tests, multiplicity control, negative results, network data, non-independence, non-significant result, nonparametric statistics, Normal distribution, null hypothesis, numeric data, odds ratio, one-tailed test, outliers, p-curve, p-hacking, p-value, positive results, post-hoc hypothesis, posterior distribution, pre-registration, prior distribution, probability distribution, R, response time, sample, sample variance, selective publishing, significance level, significance test, significant range, significant result, SPSS, standard deviation (s.d., σ), starting hypothesis, statistical power, statistical significance, Student's t distribution, Student's t-test, systematic effects, task completion time, traditional statistics, two-tailed test, Type II error, uniform prior, user experience, user interface, user interface properties, user studies, variance