Contents
- 8.1 Cherry picking
- 8.1.1 Multiple tests
- 8.1.2 Multiple statistics
- 8.1.3 Outliers
- 8.1.4 Post-hoc hypothesis
- 8.1.5 The file drawer effect
- 8.2 Inter-related factors
- 8.2.1 Non-independently controllable factors
- 8.2.2 Correlated features
- 8.3 Everything is random
- 8.4 The same or worse
- 8.4.1 Everything is unlikely
- 8.4.2 Numeric data
- 8.4.3 More complex ‘or worse’
- 8.4.4 Post-hoc corrections
- 8.5 Simulation and empirical methods
- 8.6 What you can say—phenomena and statisticians
Glossary items referenced in this chapter
Bayesian reasoning, Bayesian statistics, Bayesian test, bias, Bonferroni correction, cherry picking, closed formula, coin tossing, computer simulation, confidence interval, confirmation bias, continuous distribution, emergent patterns, empirical data, empirical methods, estimated standard deviation, extreme statistics, extreme values, factorial, fair coin, false negative, file drawer effect, fishing expedition, hypothesis testing, inter-related factors, likelihood, machine learning, mathematics, mean (μ), multiple statistics, multiple tests, multiplicity control, negative results, network data, non-significant result, nonparametric statistics, Normal distribution, not independent, null hypothesis, numeric data, odds ratio, one-tailed test, outliers, p-curve, p-hacking, p-value, positive results, post-hoc hypothesis, posterior distribution, power-law distribution, pre-registration, prior distribution, probability distribution, R, removing outliers, response time, sample, sample variance, selective publishing, significance, significance level, significance test, significant range, significant result, simulation methods, social network data, SPSS, standard deviation (s.d., σ), starting hypothesis, statistical power, statistical significance, statistically significant, statistically significant effect, Student's t distribution, Student's t-test, systematic effects, task completion time, theoretical distribution, traditional statistics, two-tailed test, Type II error, uniform prior, user experience, user interface, user interface properties, user test, variance