publication bias

Terms from Statistics for HCI: Making Sense of Quantitative Data

If only positive results are publshied, then it may appaer that they are more significant (in all senses of the word) than they really are. This is becasue of you do enough experiments or studies, then by raodm chance one of them will come up 5% significant about 1 time in 20. Publication bias occurs artly because researchers self-select to report things that are most impressive, but also because journal and conference reviewers may not accept papers with negative results. In qualitative studies, there can be similar effects, if you look at enough people, or enough events you will see some interesting things, and may tend to report these leaing to a sense that they are common, whereas they may not be at all typical.