Type I error

Terms from Statistics for HCI: Making Sense of Quantitative Data

This is a term from hypothesis testing where you wrongly believe there to be an effect when there is none; that is a false positive result. The significance level you choose helps to control for this. There is only a 1 in 20 chance that you hit a p-value of 5% by chance when there is no effect; that is, if you choose a significance level of 5%, the probability of a Type I error is 0.05 (1 in 20). Similarly if you choose a 1% significance level then the probability of the Type I error is only 0.01 (in in a hundred).

Used on page 62