Work Sample Testing

The general evidence supporting the use of work sample tests (good criterion — related validity, positive applicant reaction and job preview capability) is reviewed, limitations are also noted. The extent to which work sample tests may be used to limit adverse impact in selection is reviewed and the extent to which work sample tests may be used to complement other selection instruments, in particular, cognitive ability tests, is discussed. Work sample tests are based on attempts to embody key tasks within the job in a selection instrument. Five distinctive (though not mutually exclusive) types of work sample tests are identified and validity evidence is reviewed. Key dimensions (bandwidth, fidelity, task specificity, necessary experience, type of tasks and mode of presentation and response) are identified and used to compare different types of work sample tests.     

Applicant Reactions to Selection Methods in China

This study examines applicant reactions to ten popular selection methods in China. Using a sample of 294 graduates we found that Chinese applicants’ reactions were highly favorable for work sample tests, interviews, and written ability tests, whereas Guanxi (i.e., relying on personal contacts when applying for a job) and graphology were perceived as the least favorable selection methods. Guanxi was also perceived as significantly less fair method compared with all others on all seven procedural dimensions studied. These findings suggest that Guanxi as an informal selection channel might threaten the fairness of personnel selection in China. Implications for the design of selection systems in Chinese companies are discussed, and ramifications for future research into applicant reactions are considered.     

Applicant Reactions in Saudi Arabia: Organizational attractiveness and core‐self evaluation

This paper reports findings from a survey into applicant reactions of working adults in Saudi Arabia. A sample of 193 participants from four job functions was obtained, with measures of organizational attractiveness, core‐self evaluation, and applicant reactions to four popular selection methods in the country – interviews, résumés, work sample tests, and references – being included. Findings indicate a notably similar pattern of preference reactions to previous studies in other (Western) countries, affirming arguments for so‐called reaction generalizability. Work sample tests were rated the most favorably followed by interviews, résumés, and references. For specific procedural dimensions, résumés were perceived as the most favorable, followed by work sample tests, interviews, and references. Several significant differences were found across job functions, mostly for interviews and résumés. Significant effects were found between reactions and organizational attractiveness, and between reactions and core‐self evaluation, including some interaction effects. Implications for future research and for practice in employee selection are considered in the conclusion.     

Work Sample Exams and Gender Adverse Impact Potential: The influence of self‐concept,social skills,and written skills

We clarify the existing literature on gender differences in work sample exams by disentangling such differences on work sample exams from other predictors (e.g., situational judgment tests). In fact, we note that there are only two articles specifically related to gender differences in work sample exams. Based on theory and literature from social psychology, neuropsychology, and applied psychology, we propose that the three constructs of self‐concept, social skills, and writing skills are likely to influence work sample gender differences. We tested our hypotheses on two samples of managers. We found in one instance that males scored higher on a technical exercise, but there was stronger support for females, on average, scoring higher on exercises that involved social skills and on exercises that involved writing skills. Work samples that targeted a broad array of knowledge, skills, and abilities were associated with higher overall scores for females (ds of ?.37 and ?.34) and, thus, were unlikely to be associated with gender‐based adverse impact against females.     

The Role of Person Characteristics in Perceptions of the Validity of Cognitive Ability Testing

Individuals vary in how they perceive cognitive ability tests; thus, it is useful for organizations to consider how individual differences influence applicant perceptions of selection tools. The present study examined the influence of implicit theories of ability and locus of control on perceptions of face validity and predictive validity for two cognitive ability tests. Relationships between perceptions and test experience, job‐relevant experience, and job familiarity were also examined. Interactions between implicit theories and self‐assessed performance in predicting perceptions were found, although not of the form hypothesized. Furthermore, job familiarity and prior success in selection contexts were related to perceptions. Finally, sample type interacted with test type to influence perceptions. Implications for selection system design and research on applicant perceptions are discussed.     

PERSONALITY TESTS AT THE CROSSROADS: A RESPONSE TO MORGESON, CAMPION, DIPBOYE, HOLLENBECK, MURPHY, AND SCHMITT (2007)

Despite widespread and growing acceptance that published personality tests are valid predictors of job performance, Morgeson et al. (2007) propose they be abandoned in personnel selection because average validity estimates are low. Our review of the literature shows that Morgeson et al.'s skepticism is unfounded. Meta-analyses have demonstrated that published personality tests, in fact, yield useful validity estimates when validation is based on confirmatory research using job analysis and taking into account the bidirectionality of trait–performance linkages. Further gains are likely by use of narrow over broad measures, multivariate prediction, and theory attuned to the complexities of trait expression and evaluation at work. Morgeson et al. also suggest that faking has little, if any, impact on personality test validity and that it may even contribute positively to job performance. Job applicant research suggests that faking under true hiring conditions attenuates personality test validity but that validity is still sufficiently strong to warrant personality test use in hiring. Contrary to Morgeson et al., we argue that the full value of published personality tests in organizations has yet to be realized, calling for programmatic theory-driven research.     

Applicant Faking,Social Desirability,and the Prediction of Counterproductive Work Behaviors

Recent studies have pointed to within-subjects designs as an especially effective tool for gauging the occurrence of faking behavior in applicant samples. The current study utilized a within-subjects design and data from a sample of job applicants to compare estimates of faking via within-subjects score change to estimates based on a social desirability scale. In addition, we examined the impact of faking on the relationship between Conscientiousness and counterproductive work behaviors (CWBs), as well as the direct linkage between faking and CWBs. Our results suggest that social desirability scales are poor indicators of within-subjects score change, and applicant faking is both related to CWBs and has a negative impact on the criterion-related validity of Conscientiousness as a predictor of CWBs.     

WORK SAMPLE TESTS IN PERSONNEL SELECTION: A META‐ANALYSIS OF BLACK–WHITE DIFFERENCES IN OVERALL AND EXERCISE SCORES

Work sample exams are generally thought to have either low or comparatively low levels of ethnic group differences when used for personnel selection. Such exams are sometimes called “simulation exercises” and involve having applicants perform a set of tasks that are similar to those performed on the job. The nearly ubiquitous meta‐analytic value of Black–White subgroup differences in the literature is d= .38. Unfortunately, this estimate is plagued by a variety of problems (e.g., range restriction, inclusion of nonwork sample tests). Further, there are virtually no analyses that examine how the saturation of different constructs influence work sample tests. We gathered available data for Black–White ethnic group differences and found that overall work sample differences were markedly larger for samples of job applicants (d= .73) than previously thought. We also examined how different exercises and saturation of different constructs influenced work sample ds. For example, work sample test ratings of cognitive and job knowledge skills were associated with a mean observed d= .80, whereas ratings of various social skills were associated with mean observed ds that varied from .21 to .27. We urge scientists and practitioners to consider both the method and the constructs that are targeted when forecasting predictor ds.     

Subgroup differences in situational judgment test scores: Evidence from large applicant samples

To promote diversity in organizations it is important to have accurate knowledge about subgroup differences associated with selection procedures. However, current estimates of subgroup differences in situational judgment tests (SJTs) are overwhelmingly based on range‐restricted incumbent samples that are downwardly biased. This study provides much‐needed applicant level estimates of SJT subgroup differences (N = 37,530). As a key finding, Black‐White differences (d = 0.66) were higher than in incumbent samples (d = 0.38). Overall, sex differences were small. Females scored higher for management jobs (d = ?0.13) and males scored higher for administrative jobs (d = 0.15). By analyzing applicant samples that do not suffer from range restriction, this study adds knowledge about subgroup differences in SJTs.     

Applicant Reactions to Selection Procedures: An Updated Model and Meta-Analysis

An updated theoretical model of applicant reactions to selection procedures is proposed and tested using meta-analysis. Results from 86 independent samples  ( N = 48,750)  indicated that applicants who hold positive perceptions about selection are more likely to view the organization favorably and report stronger intentions to accept job offers and recommend the employer to others. Applicant perceptions were positively correlated with actual and perceived performance on selection tools and with self-perceptions. The average correlation between applicant perceptions and gender, age, and ethnic background was near zero. Face validity and perceived predictive validity were strong predictors of many applicant perceptions including procedural justice, distributive justice, attitudes towards tests, and attitudes towards selection. Interviews and work samples were perceived more favorably than cognitive ability tests, which were perceived more favorably than personality inventories, honesty tests, biodata, and graphology. The discussion identifies remaining theoretical and methodological issues as well as directions for future research.     

Applicant Reactions in Selection: Comprehensive meta‐analysis into reaction generalization versus situational specificity

This paper reports a comprehensive quantitative summary into applicant reactions to popular methods of employee selection. A detailed search of published and unpublished studies resulted in a final sample (k) of 38 independent samples covering 10 popular candidate assessment methods. Applicant reactions primary studies included samples from 17 countries internationally. Counter to previous suggestions for the situational specificity of applicant reactions, findings showed considerable similarity supporting the reaction generalizability hypothesis. Reaction favorability was structurally similar across countries and revealed a three‐tier clustering of overall favorability perceptions –most preferred (work samples, interviews), favorably evaluated (resumes, cognitive tests, references, biodata, personality inventories), and least preferred (honesty tests, personal contacts, graphology). Some differences in applicant reactions to dimension‐specific perceptions were found, however. Further analyses revealed strong positive correlations between favorability ratings and their validity and international usage. Implications for future research and ramifications for practice are considered in conclusion.     

Reviewing,Categorizing, and Analyzing the Literature on Black–White Mean Differences for Predictors of Job Performance: Verifying Some perceptions and Updating/Correcting Others

In both theoretical and applied literatures, there is confusion regarding accurate values for expected Black–White subgroup differences in personnel selection test scores. Much confusion arises because empirical estimates of standardized subgroup differences (d) are subject to many of the same biasing factors associated with validity coefficients (i.e., d is functionally related to a point‐biserial r). To address such issues, we review/cumulate, categorize, and analyze a systematic set of many predictor‐specific meta‐analyses in the literature. We focus on confounds due to general use of concurrent, versus applicant, samples in the literature on Black–White d. We also focus on potential confusion due to different constructs being assessed within the same selection test method, as well as the influence of those constructs on d. It is shown that many types of predictors (such as biodata inventories or assessment centers) can have magnitudes of d that are much larger than previously thought. Indeed, some predictors (such as work samples) can have ds similar to that associated with paper‐and‐pencil tests of cognitive ability. We present more realistic values of d for both researcher and practitioner use. Implications for practice and future research are noted.     

The Influence of Perceived Interviewer and Job and Organizational Characteristics on Applicant Attraction and Job Choice Intentions: The role of applicant anxiety

This study examined whether interviewer characteristics have (a) a direct influence on applicant attraction and job choice intentions, (b) an indirect influence via job and organizational characteristics, and (c) direct influence on applicant anxiety. A sample of graduate applicants (N=450) was surveyed before a selection interview (Time 1) and after the employment interview (Time 2). Structural equation modeling was used to examine the hypothesized model. The results showed that interviewer characteristics (warmth, unfriendliness, job knowledge, general competence and humor) had both a direct and indirect effect on applicant attraction and job choice intentions. In addition, interviewer characteristics had a significant positive impact on applicant anxiety. Theoretical and practical implications of the findings are discussed.     

GROUP DIFFERENCES IN PERSONALITY: META‐ANALYSES COMPARING FIVE U.S. RACIAL GROUPS

This research poses 2 applied questions: How large are racial group differences on personality scales and are these differences likely to cause adverse impact in personnel selection? We examined the extent to which racial groups differ across Big 5 personality factors and facets. Large‐scale, quantitative estimates based on over 700 effect sizes were meta‐analytically summarized. Multiple personality instruments and understudied racial groups, particularly Asian Americans and American Indians, were included in the meta‐analyses. Most group comparisons and personality scales yielded negligible differences and are not likely to cause adverse impact in selection. However, facet‐level analyses produced different d‐values, with some group comparisons showing moderate differences, suggesting that the use of personality measures in selection does not uniformly circumvent adverse impact concerns. The veracity of this conclusion depends on (a) the particular trait in question, (b) the composition of the applicant pool (i.e., which groups are to be compared with one another), (c) the effect size (i.e., d‐value), and (d) the selection ratio. For practitioners, we present a quantitatively informed summary of when and where to expect adverse impact to result from personality measures used in personnel selection.     

Beyond Impression Management: Evaluating three measures of response distortion and their relationship to job performance

Three measures of response distortion (i.e., social desirability, covariance index, and implausible answers) were examined in both applicant and incumbent samples. Performance data, including supervisor ratings of task and contextual performance as well as objective performance criteria such as tardiness, work‐related accidents, and a customized work simulation, were obtained for the incumbent sample. Results provided further support for the existence of applicant faking behavior and shed light into the relationship between faking and job performance, largely depending on how one defines and measures faking as well as the performance criteria evaluated. Implications for future research and practice in personality assessment for selection purposes were discussed.     

EFFECTS OF JOB ATTRIBUTES AND RECRUITING PRACTICES ON APPLICANT DECISIONS: A COMPARISON1

Previous research on the factors which affect applicant decisions concerning jobs has focused on the effects of either job attributes or recruiting practices. The present study examined the simultaneous impact of job attributes and recruiting practices on the likelihood of job acceptance by actual job applicants. Path analysis revealed that job attributes but not recruiting practices influenced the likelihood of job acceptance by graduating college students (N= 200). Given that college students might be expected to be the job applicants most swayed by recruiting practices because of their lack of work experience, the results suggest that the emphasis placed on recruiting practices as determinants of applicant decisions may be overstated in the literature.     

Educational attainment as a proxy for cognitive ability in selection: effects on levels of cognitive ability and adverse impact

The authors examined the differences in mean level of cognitive ability and adverse impact that can be expected when selecting employees solely on educational attainment as a proxy for cognitive ability versus selecting employees directly on cognitive ability. Selection using cognitive ability worked as a more efficient cognitive screen. Imposing an educational attainment standard of at least 1 year of college, though, did result in noticeably higher levels of cognitive ability in potential applicant pools than did random selection, meaning that educational attainment does work as a cognitive screen. These results held not only in a nationally representative sample but also within and across 6 different occupational groups. Finally, adverse impact is examined for selection using educational attainment, compared with selection on the basis of cognitive ability.     

Range Shrinkage of Cognitive Ability Test Scores in Applicant Pools for German Governmental Jobs: Implications for range restriction corrections

Range restriction corrections require the predictor standard deviation in the applicant pool of interest. Unfortunately, this information is frequently not available in applied contexts. The common strategy in this type of situations is to use national‐norm standard deviation estimates. This study used data from 8,276 applicants applying to nine jobs in German governmental organizations to compare applicant pool standard deviations for two cognitive ability tests with national‐norm standard deviation estimates, and standard deviations for the total group of governmental applicants. Results revealed that job‐ and organizational context‐specific applicant pool standard deviations were on average about 10–12% smaller than estimates from national norms, and about 4–6% smaller than standard deviations for the total group of governmental applicants.     

The Employment Interview: The Role of Sex Stereotypes in the Evaluation of Male and Female Job Applicants in the Netherlands1

This field study focused on the influence of sex stereotypes in the evaluation of male (N=38) and female (N=21) job applicants in the Netherlands. The employee selection process for higher-level technical and academic jobs in real life situations was studied, with special attention to the assessment of applicants by members of selection committees. It was demonstrated that, according to the job interviewers, the ideal applicant had more masculine than feminine traits. Males and females were regarded as having the same qualifications for the job, but because male applicants were assessed as having more masculine characteristics and female applicants more feminine characteristics, the male applicants were accepted more often. The job interviewers acted according to a fit model: The applicant most similar in traits to the ideal applicant was hired for each job.     

DERIVATION AND IMPLICATIONS OF A META-ANALYTIC MATRIX INCORPORATING COGNITIVE ABILITY, ALTERNATIVE PREDICTORS, AND JOB PERFORMANCE

A variety of recent articles in the personnel selection literature have used analyses of meta-analytically derived matrices to draw general conclusions for the field. The purpose of this article is to construct a matrix that incorporates as complete information as possible on the relationships among cognitive ability measures, three sets of alternative predictors, and job performance, We build upon a starting matrix used by Schmitt, Rodgers, Chan, Sheppard, and Jennings (1997). Mean differences, by race, for each of the measures and the potential for adverse impact of predictor composites are also considered. We demonstrate that the use of alternative predictors alone to predict job performance (in the absence of cognitive ability) lowers the potential for adverse impact. However, in contrast to recent claims, adverse impact continues to occur at many commonly used selection ratios. Future researchers are encouraged to use our matrix and to expand upon it as new primary research becomes available. We also report and reaffirm many methodological lessons along the way, including the many judgment calls that appear in an effort of this magnitude and a reminder that the field could benefit from even greater conceptual care regarding what is labeled an "alternative predictor." Directions for future meta-analyses and for future primary research activities are also derived.