MODERATOR RESEARCH AND THE LAW OF SMALL NUMBERS1

The thesis of this paper is that many proposed moderators in personnel psychology are probably illusory, having been created solely by belief in the law of small numbers. Evidence is presented that race as a moderator of test validity is one such illusory moderator. In addition, a model for validity generalization is described which, in addition to eliminating the need for criterion-related validity studies under certain circumstances, strongly calls into question the idea that situations moderate test validity, i.e., the traditional doctrine of situational specificity of test validities. Calculations are presented which show that adequate statistical power in moderator research requires much larger sample sizes than have typically been employed. This requirement is illustrated empirically using validity data for the Army Classification Battery for 35 jobs and 21,000 individuals. These analyses show that (1) even when a moderator is generally assumed to be large, large samples are required to gauge its effect reliably and (2) large sample research may show that moderators that appear plausible and important a priori are nonexistent or trivial in magnitude. The practice of pooling across numerous small sample studies to obtain statistical power equivalent to that of large sample studies is recommended. In light of the evidence that many proposed moderators may not exist, the authors hypothesize that the true structure of underlying relationships in personnel psychology is considerably simpler than personnel psychologists have generally imagined it to be.     

THE ROLE OF COGNITIVE ABILITY IN THE SUBGROUP DIFFERENCES AND INCREMENTAL VALIDITY OF ASSESSMENT CENTER EXERCISES

This study investigates the degree to which subgroup (Black-White) mean differences on various assessment center exercises (e.g., in-basket, role play) may be a function of the type of exercise employed; and furthermore, begins to explore why these different types of exercises result in subgroup differences. The sample consisted of 633 participants who completed a managerial assessment center that evaluated them on 14 ability dimensions across 7 different types of assessment exercises. In addition, each participant completed a cognitive ability measure. The results suggest that subgroup differences varied by type of assessment exercise; and furthermore that the subgroup difference appeared to be a function of the cognitive component of the exercise. Lastly, preliminary support is found that the validity of some of the assessment center exercises in predicting supervisor ratings of job performance is based, in part, on their cognitive component; however, evidence of incremental validity does exist.     

The Nature of Preference: A Response to Grantham and Gordon

Grantham and Gordon (1986) contended that preference was inadequately understood as a construct and that a more incisive conceptualization was needed to promote research and theoretical considerations. We contend that their offering adds to the terminological confusion. A brief critique of Grantham and Gordon's notions and an alternative conceptualization and schema are offered.     

THE ECONOMIC IMPACT OF JOB SELECTION METHODS ON SIZE, PRODUCTIVITY, AND PAYROLL COSTS OF THE FEDERAL WORK FORCE: AN EMPIRICALLY BASED DEMONSTRATION

In this study, job performance increases resulting from improved selection validity were measured empirically rather than estimated from the standard linear regression utility equations. Selection utility analyses based on these empirical measurements were carried out for most white-collar jobs in the federal government. Results indicate that selection of a one-year cohort based on valid measures of cognitive ability, rather than on non-test procedures (mostly evaluations of education and experience), produces increases in output worth up to $600 million for each year that the new employees remain employed by the government. Newly hired federal employees remain with the government an average of approximately 13 years, resulting in a total gain in output of almost $8 billion over this period. This gain represents a 9.7% increase in output among new hires. If total output is held constant rather than increased, new hiring can be reduced by up to 20,044 per year (a 9% decrease), resulting in payroll savings of $272 million for every year the new cohort of employees remains on the job. The percentage of new hires in the bottom decile of the non-test-selected job performance distribution     

Note on the Perceived Occupational Future of Physically Disabled College Students

How do physically disabled college students perceive their occupational future? The findings indicate that although the vast majority of students held high career aspirations, many sensed that their career goals may be unattainable. The findings also show that nearly three out of four respondents preferred a career that involved working with people. At the same time, most wanted to work for either a large or small business rather than an educational institution or state or federal governmental agency.     

THE EFFECT OF QUALITY CIRCLE INITIATION ON MOTIVATION TO ATTEND QUALITY CIRCLE MEETINGS AND ON TASK PERFORMANCE

The motivation to attend quality circle (QC) meetings and the task performance of 47 quality circles over a three-year period were examined as a function of quality circle initiation (self- vs. management-initiated) in a field study. Motivation to attend meetings was operationally defined as the number of members per QC (QC size) and the attendance rate at QC meetings. Task performance reflected the number of projects completed and the amount of time required to complete the project. The results of a MANOVA suggested that QC initiation had significant impacts on motivation to attend meetings in that self-initiated QCs had more QC members than had management-initiated QCs. Further, management-initiated QCs solved more work-related problems and solved their problems significantly faster than did self-initiated QCs. The effect of QC initiation on the number of projects completed failed to reach significance, however, when QC size was controlled as a covariate. Further, a high attendance rate at QC meetings was associated with a low quantity of projects completed in self-initiated QCs and a slow speed of problem solving in management-initiated QCs. Some implications related to the effectiveness of QCs are discussed.     

ASSESSING THE ECONOMIC IMPACT OF PERSONNEL PROGRAMS ON WORKFORCE PRODUCTIVITY

This article describes and illustrates the adaptation of the linear-regression-based decision-theoretic equations used to estimate the dollar impact of valid selection procedures on workforce productivity to the evaluation of intervention programs designed to improve job performance. The appropriate equations are derived and explained, methods for estimating equation parameters are discussed, and the use of these equations is illustrated by means of a hypothetical example. It is concluded that in the future these methods and equations will allow psychologists to make more accurate assessments of the impact of intervention programs on workforce productivity than has heretofore been the case.     

black concern with behavior modification

In this article two black psychologists and a white educator explain why the black community should be concerned with and actively involved in behavior modification practices. It is pointed out by use of the Tuskegee study how blacks were used as subjects in a behavior modification study long before these practices became popularized. The concerns specified were that (a) blacks have been and are being used as subjects in behavior modification studies with and without their consent and (b) social value judgments are being made concerning behaviors of blacks without their input. Alternatives are offered for increasing the number of black professionals in the field of behavior modification to help preserve the integrity of the black community.