This is a term from hypothesis testing where you wrongly believe there to be an effect when there is none; that is a false positive result. The significance level you choose helps to control for this. There is only a 1 in 20 chance that you hit a p-value of 5% by chance when there is no effect; that is, if you choose a significance level of 5%, the probability of a Type I error is 0.05 (1 in 20). Similarly if you choose a 1% significance level then the probability of the Type I error is only 0.01 (in in a hundred).
Also used in hcistats2e: Chap. 6: page 67
Used in glossary entries: false positive, hypothesis testing, p-value, significance level, Type I error