On the number of significant findings to be expected by chance

When multiple significance tests are computed, a certain number of significant findings will emerge simply because of chance fluctuations. In the present paper, some factors affecting the number of nominally significant results are elaborated and a general method is suggested which permits unbiased inference as to the significance of a set of findings,as a set. The method advocated employs a high speed computer to generate empirically a sampling distribution tailormade to a particular data matrix. The method is illustrated in the case of dichotomous response to inventory items, where it is found that the statistical model still often used as a basis for estimation is overly conservative. Some problems in the application of the method are discussed.This investigation, part of a larger research project, was supported primarily by research grant M-1078 from the National Institute of Mental Health of the United States Public Health Service. Gratitude is also extended to the National Science Foundation for making available research time on the IBM Model 701 computer. The present paper has benefited greatly from the comments on an earlier version offered by John Tukey and an editor ofPsychometrika.