THE DIVERSITY–VALIDITY DILEMMA: OVERVIEW AND LEGAL CONTEXT

Due to racioethnic and sex subgroup differences on predictor scores in many selection procedures, it is difficult for organizations to simultaneously maximize the validity of their selection procedures and hire a diverse workforce. One response to this diversity–validity dilemma is to revise the selection procedures, an approach developed by Ployhart and Holtz (this issue, 2008) . A second possible response is to use affirmative action to increase workforce diversity, an approach developed by Kravitz (this issue, 2008) . This paper briefly presents the legal context that motivates and constrains these approaches. We begin by defining key terms, describing adverse impact, and outlining the burden of proof in adverse impact cases. We then turn to the use of racioethnic minority and female preferences, summarizing some key court decisions and the conditions under which private and public employers may use preferences.     

THE DIVERSITY–VALIDITY DILEMMA: BEYOND SELECTION—THE ROLE OF AFFIRMATIVE ACTION

Several of the most valid predictors used to make employment decisions create a diversity–validity dilemma ( Pyburn, Ployhart, & Kravitz, this issue, 2008 ). This diversity–validity dilemma can be resolved by (a) reducing adverse impact through a variety of technical steps ( Ployhart & Holtz, this issue, 2008 ) or (b) using affirmative action to increase representation of the disadvantaged groups. This paper focuses on the second approach. The paper begins with a very brief review of the legal bases of affirmative action and a summary of the research on affirmative action attitudes. This is followed with reviews of research on the ongoing existence of workplace discrimination, the economic impact of affirmative action on target groups and organizations, and stigmatization of target group members by others and by target group members themselves. Most problems with affirmative action apply only or primarily to preference-based forms, so nonpreferential approaches to affirmative action are recommended to increase the attraction, selection, inclusion, and retention of underrepresented group members.     

Combining predictors to achieve optimal trade-offs between selection quality and adverse impact

The authors propose a procedure to determine (a) predictor composites that result in a Pareto-optimal trade-off between the often competing goals in personnel selection of quality and adverse impact and (b) the relative importance of the quality and impact objectives that correspond to each of these trade-offs. They also investigated whether the obtained Pareto-optimal composites continue to perform well under variability of the selection parameters that characterize the intended selection decision. The results of this investigation indicate that this is indeed the case. The authors suggest that the procedure be used as one of a number of potential strategies for addressing the quality-adverse impact problem in settings where estimates of the selection parameters (e.g., validity estimates, predictor intercorrelations, subgroup mean differences on the predictors and criteria) are available from either a local validation study or meta-analytic research.     

Hiring on vocational interests to simultaneously improve validity and organizational diversity

We discuss how using vocational interests in the selection process can help address the diversity-validity dilemma. First, we point out how incorporating vocational interests as predictors in selection could help to reduce adverse impact. We further suggest that by using optimal predictor weights, one could simultaneously improve validity while enhancing organisational diversity. Finally, the predictive validity of vocational interests arises from their ability to capture the congruence between individuals and occupations, which is a cross-level phenomenon. Thus, when gathering validity evidence for vocational interests, multi-occupation samples should be incorporated into validation efforts.     

UNDERSTANDING THE IMPACT OF TEST VALIDITY AND BIAS ON SELECTION ERRORS AND ADVERSE IMPACT IN HUMAN RESOURCE SELECTION

We propose an integrative framework for understanding the relationship among 4 closely related issues in human resource (HR) selection: test validity, test bias, selection errors, and adverse impact. One byproduct of our integrative approach is the concept of a previously undocumented source of selection errors we call bias‐based selection errors (i.e., errors that arise from using a biased test as if it were unbiased). Our integrative framework provides researchers and practitioners with a unique tool that generates numerical answers to questions such as the following: What are the anticipated consequences for bias‐based selection errors of various degrees of test validity and test bias? What are the anticipated consequences for adverse impact of various degrees of test validity and test bias? From a theory point of view, our framework provides a more complete picture of the selection process by integrating 4 key concepts that have not been examined simultaneously thus far. From a practical point of view, our framework provides test developers, employers, and policy makers a broader perspective and new insights regarding practical consequences associated with various selection systems that vary on their degree of validity and bias. We present a computer program available online to perform all needed calculations.     

Ethnic and gender subgroup differences in assessment center ratings: a meta-analysis

Assessment centers are widely believed to have relatively small standardized subgroup differences (d). However, no meta-analytic review to date has examined ds for assessment centers. The authors conducted a meta-analysis of available data and found an overall Black-White d of 0.52, an overall Hispanic-White d of 0.28, and an overall male-female d of -0.19. Consistent with our expectations, results suggest that Black-White ds in assessment center data may be larger than was previously thought. Hispanic-White comparisons were smaller than were Black-White comparisons. Females, on average, scored higher than did males in assessment centers. As such, assessment centers may be associated with more adverse impact against Blacks than is portrayed in the literature, but the predictor may have less adverse impact and be more "diversity friendly" for Hispanics and females.     

Pareto‐Optimal Predictor Composite Formation: A complementary approach to alleviating the selection quality/adverse impact dilemma

In our rejoinder to the comments of Kehoe (this issue) and Potosky, Bobko and Roth (this issue) we emphasize that our proposal on Pareto‐optimal predictor composite formation is a complementary and not a competitive alternative for reducing the tension between selection quality and adverse impact. Our work addresses the decisions to be made once one has decided to use a predictor composite. We also further clarify the basic features of Pareto‐optimal tradeoffs and Pareto‐optimal composites within the context of personnel selection. In particular, we indicate that Pareto‐optimal tradeoffs between validity and adverse impact emerge because these goals are different and not because of any dualism between them.     

Alternative Predictors for Dealing with the Diversity–Validity Dilemma in Personnel Selection: The constructed response multimedia test

In the context of the diversity–validity dilemma in personnel selection, the present field study compared ethnic subgroup differences on an innovative constructed response multimedia test to other commonly used selection instruments. Applicants (N = 245, 27% ethnic minorities) for entry‐level police jobs completed a constructed response multimedia test, cognitive ability test, language proficiency test, personality inventory, structured interview, and role play. Results demonstrated minor ethnic subgroup differences on constructed response multimedia test scores as compared to other instruments. Constructed response multimedia test scores were related to the selection decision, and no evidence for predictive bias was found. Subgroup differences were also examined on the dimensional level, with cognitively loaded dimension scores displaying larger differences.     

Selecting Predictor Subsets: Considering validity and adverse impact

The paper proposes a procedure for designing Pareto‐optimal selection systems considering validity, adverse impact and constraints on the number of predictors from a larger subset that can be included in an operational selection system. The procedure determines Pareto‐optimal composites of a given maximum size thereby solving the dual task of identifying the predictors that will be included in the reduced set and determining the weights with which the retained predictors will be combined to the composite predictor. Compared with earlier proposals, the simultaneous consideration of both tasks makes it possible to combine several strategies for reducing adverse impact in a single procedure. In particular, the present approach allows integrating (a) investigating a large number of possible predictors (such as multitest battery of ability tests, or a collection of ability and nonability measures); (b) explicit predictor weighting within feasible test procedures of a given limited size.     

Forming Composites of Cognitive Ability and Alternative Measures to Predict Job Performance and Reduce Adverse Impact: Corrected Estimates and Realistic Expectations

Although there has been empirical attention paid to the criterion‐related validity of predictor composites, there has been much less attention paid to the standardized ethnic group differences associated with these composites. One important area of inquiry in predictor composite research is the influence of adding predictors to a test of general mental ability. The limited empirical literature on this practice is mixed, but the prevailing expectation is that there is likely to be higher validity and less adverse impact. Unfortunately, much of the previous work is limited by the presence of inaccurate validity and standardized ethnic group difference values. In this analysis we formed meta‐analytic matrices to more accurately estimate the validity and standardized ethnic group differences of several composites that combine a measure of cognitive ability with measures of conscientiousness, a structured interview, or biodata. While results were somewhat complex, we found that adding alternative predictors does not result in a situation in which validity automatically goes up and adverse impact potential automatically goes down. In fact, the reductions in adverse impact (if any) from adding “non‐cognitive” predictors were more modest than much of the literature suggests.     

Linkages Between Racioethnicity,Appraisal Reactions,and Employee Engagement

Because diversity is vital to many businesses, it is important to understand prospective racioethnic differences in employee engagement. Using survey data collected from a large (N = 5,537), diverse sample of retail employees, we found that more favorable appraisal reactions corresponded with more favorable psychological diversity climate perceptions; thus, higher levels of engagement. This indirect relationship was significantly stronger for ethnic minority employees (Blacks and Hispanics) than for White employees, indicating that members of traditionally disadvantaged groups respond differently to perceptions of appraisal systems. Finally, an exploratory assessment found that the hypothesized effects for racioethnicity do not generalize to sex, as the indirect effect of appraisal reactions on engagement was slightly, but not significantly stronger for women than for men.     

SYNTHETIC VALIDITY: PAST, PRESENT, AND FUTURE

Practitioners in the area of personnel selection can be confronted with situations in which the use of traditional validation strategies is not feasible. This situation can result from small numbers of incumbents, rapidly changing jobs, insufficient resources, or the lack of high-quality criterion data. Although traditional strategies are not possible, a set of techniques falling under the rubric of synthetic validity can be used to estimate predictor validity in these situations. Synthetic validity is a logical process of inferring validity on the basis of the relationships between components of a job and tests of the attributes that are needed to perform the job components. Unfortunately, synthetic validity is infrequently used and not well understood. This article provides an overview and conceptualization of synthetic validity, reviews the historical approaches to synthetic validity, describes the recent developments and trends, and suggests future directions for synthetic validity application and research. The relationship of synthetic validity with other validity concepts, the advantages, the limitations, and the legal issues are considered throughout.     

Modeling the Practical Effects of Applicant Reactions: Subgroup Differences in Test‐Taking Motivation,Test Performance,and Selection Rates

Research suggests that Black–White differences in test‐taking motivation may be related to subgroup test score differences, but this research has not shown the extent to which minimizing subgroup motivation differences will reduce subgroup differences in selection rates and adverse impact. This Monte Carlo study examined how enhancing Blacks' test‐taking motivation for cognitive ability tests might reduce adverse impact across a range of (a) subgroup test differences, (b) selection ratios, (c) subgroup differences in test‐taking motivation, and (d) relationships between motivation and test scores. The results suggest that although enhancing test‐taking motivation will consistently reduce subgroup differences in test performance and adverse impact, the effect is often small and will not eliminate adverse impact for any condition we examine. However, under some conditions the reduction may be important, and the discussion considers conditions where even these minimal reductions may be practically helpful.     

Subgroup differences in situational judgment test scores: Evidence from large applicant samples

To promote diversity in organizations it is important to have accurate knowledge about subgroup differences associated with selection procedures. However, current estimates of subgroup differences in situational judgment tests (SJTs) are overwhelmingly based on range‐restricted incumbent samples that are downwardly biased. This study provides much‐needed applicant level estimates of SJT subgroup differences (N = 37,530). As a key finding, Black‐White differences (d = 0.66) were higher than in incumbent samples (d = 0.38). Overall, sex differences were small. Females scored higher for management jobs (d = ?0.13) and males scored higher for administrative jobs (d = 0.15). By analyzing applicant samples that do not suffer from range restriction, this study adds knowledge about subgroup differences in SJTs.     

Designing Pareto-optimal selection systems: formalizing the decisions required for selection system development

The article presents an analytic method for designing Pareto-optimal selection systems where the applicants belong to a mixture of candidate populations. The method is useful in both applied and research settings. In an applied context, the present method is the first to assist the selection practitioner when deciding on 6 major selection design issues: (1) the predictor subset, (2) the selection rule, (3) the selection staging, (4) the predictor sequencing, (5) the predictor weighting, and (6) the stage retention decision issue. From a research perspective, the method offers a unique opportunity for studying the impact and relative importance of different strategies for reducing adverse impact.     

TRAINING PROGRAMS FOR OBSERVERS OF BEHAVIOR: A REVIEW1

This review covers the past 25 years of research literature on training observers of behavior, specifically in the areas of interviewing, reducing rater bias, interpersonal perception and observation as a research tool. The focus is on training procedure (i.e., the various training designs and their components). An attempt is made to organize and systematize the research and to answer two important questions. Which approach(es) used to train observers of behavior has (have) been most successful? What are the theoretical or empirical bases for the development of those training programs?     

THE IMPACT OF JOB ANALYSIS ON EMPLOYMENT TEST VALIDATION FOR MINORITY AND NONMINORITY ACCOUNTING PERSONNEL

The purposes of the present study were (a) to examine the comparative validity of a written job knowledge test constructed on the basis of a systematic job analysis with that of a commercial employment test selected in the absence of a prior job analysis for accounting positions and (b) to determine the fairness of each test for minority and nonminority job applicants. Results indicated that the job knowledge test was a valid and unbiased predictor of relevant criteria of job performance while the commercial employment test produced adverse impact and lacked validity. Implications of the results for future research studies and test validation efforts involving differential prediction are discussed.     

HONESTY TESTING FOR PERSONNEL SELECTION: A REVIEW AND CRITIQUE

Paper and pencil predictors of employee theft are described and studies of validity, reliability, and adverse impact of these tests are examined. Validity studies for 10 tests were grouped into 5 categories: comparisons with polygraph examination results, correlations with admissions of past theft, predictive studies using future job behaviors as criteria, comparisons of shrinkage rates before and after the introduction of a testing program, and comparisons of test scores of groups known to be dishonest with groups representing the general population. While positive correlations were consistently found, a variety of methodological differences between studies were identified which make the direct comparison of test validities suspect. High reliabilities are consistently reported, and test score comparisons by race and sex generally report no differences. Ethical issues in honesty test usage are considered and future research needs are identified.     

Diversity and General Student Scholarship Recipient Essays: 2010 National Society of Genetic Counselors Membership Committee

In an effort to increase the diversity of the membership of the National Society of Genetic Counselors (NSGC), the Membership Committee provided two $500 scholarships to genetic counseling students planning to attend the NSGC AEC meeting in Dallas, Texas in October 2010. Requirements for applicants of both scholarships included enrollment in the fall of 2010, good standing at an accredited genetic counseling training program, and NSGC membership or plans to join in 2011. Students who are from communities underrepresented in the NSGC, including, but not limited to, those of minority cultural/ethnic backgrounds and those with disabilities were eligible to apply for the “Diversity” scholarship. Students from all backgrounds who have an interest in diversity issues were eligible to apply for the “General” scholarship. Applicants wrote essays 1000 words or less answering the following questions: How has your identity as a member of a group underrepresented in the genetic counseling profession affected your pursuit of this career? What do you feel is lacking in genetic counseling to address the issues of underrepresented groups? What strategies do you recommend for addressing these issues and/or increasing diversity? Why do you think diversity is an important issue for the field of genetic counseling? What strategies do you recommend to attract and retain students, especially those from underrepresented populations, into the field of genetic counseling? How do you envision contributing to these strategies? The essays by the award recipients elucidated interesting perspectives and ideas for increasing diversity in the genetic counseling profession.     

DERIVATION AND IMPLICATIONS OF A META-ANALYTIC MATRIX INCORPORATING COGNITIVE ABILITY, ALTERNATIVE PREDICTORS, AND JOB PERFORMANCE

A variety of recent articles in the personnel selection literature have used analyses of meta-analytically derived matrices to draw general conclusions for the field. The purpose of this article is to construct a matrix that incorporates as complete information as possible on the relationships among cognitive ability measures, three sets of alternative predictors, and job performance, We build upon a starting matrix used by Schmitt, Rodgers, Chan, Sheppard, and Jennings (1997). Mean differences, by race, for each of the measures and the potential for adverse impact of predictor composites are also considered. We demonstrate that the use of alternative predictors alone to predict job performance (in the absence of cognitive ability) lowers the potential for adverse impact. However, in contrast to recent claims, adverse impact continues to occur at many commonly used selection ratios. Future researchers are encouraged to use our matrix and to expand upon it as new primary research becomes available. We also report and reaffirm many methodological lessons along the way, including the many judgment calls that appear in an effort of this magnitude and a reminder that the field could benefit from even greater conceptual care regarding what is labeled an "alternative predictor." Directions for future meta-analyses and for future primary research activities are also derived.