VALIDITY GENERALIZATION RESULTS FOR LAW ENFORCEMENT OCCUPATIONS

The Schmidt-Hunter interactive validity generalization procedure was applied to validity data for cognitive abilities tests for law enforcement occupations. Both assumed artifact distributions, and distributions of artifacts constructed from information contained in the current sample of studies were used to test the hypothesis of situational specificity and to estimate validity generalizability. Results for studies using a criterion of performance in training programs showed that validities ranged from .41 to .71, and for four test types the hypothesis of situational specificity could be rejected using the 75% decision rule. For the remaining test types, validity was generalizable, based on 90% credibility values ranging from .37 to .71. Results for studies using a criterion of performance on the job indicated that the hypothesis of situational specificity was not tenable for three test types, which had validities between .17 and .31. For the remaining test types, estimated mean true validities ranged from .10 to .26 and were generalizable to a majority of situations. Results for both groups of studies were essentially identical for the two types of artifact distribution. Possible reasons for the apparently lower validities and lesser generalizability for job performance criteria are discussed, including possible low validity of the criterion (due to lack of opportunity by supervisors to observe behavior) and the potential role of noncognitive factors in the determination of law enforcement job success. Suggestions for specifically targeted additional research are made.     

Technology and guidance: a balance

The following symposium discusses the place of technology in guidance and some of the concerns involved. Tiedeman, a leading authority on technological applications in guidance, takes the position that guidance technology is an important new development. Schmidt, who has written widely in the areas of psychology and counseling, questions the place of technology in guidance and society and underlines some of the concerns related to the adoption of technology. The authors' discussions of “one-way” versus “two-way” thinking and the problem of control over the technology should be of interest to readers concerned with applying technology in guidance.     

MODERATOR RESEARCH AND THE LAW OF SMALL NUMBERS1

The thesis of this paper is that many proposed moderators in personnel psychology are probably illusory, having been created solely by belief in the law of small numbers. Evidence is presented that race as a moderator of test validity is one such illusory moderator. In addition, a model for validity generalization is described which, in addition to eliminating the need for criterion-related validity studies under certain circumstances, strongly calls into question the idea that situations moderate test validity, i.e., the traditional doctrine of situational specificity of test validities. Calculations are presented which show that adequate statistical power in moderator research requires much larger sample sizes than have typically been employed. This requirement is illustrated empirically using validity data for the Army Classification Battery for 35 jobs and 21,000 individuals. These analyses show that (1) even when a moderator is generally assumed to be large, large samples are required to gauge its effect reliably and (2) large sample research may show that moderators that appear plausible and important a priori are nonexistent or trivial in magnitude. The practice of pooling across numerous small sample studies to obtain statistical power equivalent to that of large sample studies is recommended. In light of the evidence that many proposed moderators may not exist, the authors hypothesize that the true structure of underlying relationships in personnel psychology is considerably simpler than personnel psychologists have generally imagined it to be.     

THE VALIDITY and FAIRNESS OF EMPLOYMENT and EDUCATIONAL TESTS FOR HISPANIC AMERICANS: A REVIEW and ANALYSIS

This study examined the questions of whether employment and educational tests demonstrate either differential validity or test unfairness to Hispanic Americans relative to the majority group. Relevant data from 19 published and unpublished studies formed the basis of the review and analysis. Results for employment tests, based on a very large amount of data, indicated that differential validity occurs no more frequently than would be expected on the basis of chance plus the operation of various statistical artifacts. The frequencies of slope and intercept differences between Hispanic and majority regressions were also within the chance range. A similar pattern of results obtained for the more limited amount of data available on educational tests. Taken together, the results provide strong evidence that tests are neither differentially valid for, nor unfair to, Hispanics. These findings are consistent with the results of previous studies of test validity and fairness for blacks.