THE VALIDITY OF A MEASURE OF JOB COMPATIBILITY IN THE PREDICTION OF JOB PERFORMANCE AND TURNOVER OF MOTION PICTURE THEATER PERSONNEL

The predictive validity of a measure of job compatibility was studied for theater personnel. Scores on a forced-choice instrument, developed from the Job Compatibility Questionnaire (JCQ), predicted employee performance (r = .22, p< .05), turnover (r = -.35, p< .01), and scores on a "value composite" (reflecting a combination of job performance and employee retention criteria) as defined by the research sponsor (P = .41, p< .01). Furthermore, job compatibility scores explained a statistically significant increment in turnover and value composite score variance when analyzed concomitantly with verbal and numerical ability test scores. Finally, job compatibility scores were shown to be nonredundant with hiring decisions based on an application review, reference check, and interview, whereas the cognitive ability test scores shared considerable redundancy with hiring decisions based on the current selection system.     

人事决策信息利用和效能预测模型

本研究以 1 2 2名人事管理人员为被试 ,采用问卷测量法 ,分析了企业人事决策信息利用的现状 ,信息利用指标和效能的关系。结果发现 :1企业人事决策中比较重视能力要求信息、组织环境信息和个人档案材料 ,忽略心理特点信息的利用 ;2非程序信息、人职匹配信息对效能指标的直接预测力较强 ,其他指标的预测力较弱 ;3能力要求、考核成绩、个人档案材料等信息通过人职匹配信息的利用对效能指标产生影响。文章还构建了人事决策的效能预测模型 ,并讨论了研究结果对企业人事决策诊断和发展的理论和实际意义。     

TELL ME SOME MORE: EXPLORING HOW VERBAL ABILITY AND ITEM VERIFIABILITY INFLUENCE RESPONSES TO BIODATA QUESTIONS IN A HIGH‐STAKES SELECTION CONTEXT

Although self‐rated or self‐scored selection measures are commonly used in selection contexts, they are potentially susceptible to applicant response distortion or faking. The response elaboration technique (RET), which requires job applicants to provide supporting information to justify their responses, has been identified as a potential way to minimize applicant response distortion. In a large‐scale, high‐stakes selection context (N= 16,304), we investigate the extent to which RET affects responding on a biodata test as well as the underlying reasons for any potential effect. We find that asking job applicants to elaborate their responses leads to overall lower scores on a biodata test. Item verifiability affects the extent to which RET decreases faking, which we suggest is due to increased accountability. In addition, verbal ability was more strongly related to biodata item scores when items require elaboration, although the effect of verbal ability was small. The implications of these findings for reducing faking in personnel selection are delineated.     

PREDICTING LONG-TERM FIREFIGHTER PERFORMANCE FROM COGNITIVE AND PHYSICAL ABILITY MEASURES

Firefighters from 1 academy training class were observed for 23 years, beginning with their selection test consisting of a g-saturated written exam (GCA) and firefighting simulations loaded on a strength/endurance (SE) factor. Operational validity coefficients for both GCA and SE were high for training success and remained consistently high for job performance ratings throughout the study. The operational validity for combined GCA and SE predictors was .86 for a composite job rating measure covering 21 years of service. A structural model produced similar results for more broadly defined GCA and SE latent variables. Both analyses suggested approximately equal weighting for GCA and SE for a fire service selection test. Results indicate considerable latitude in choosing cognitive and physical predictors for firefighter screening if the predictors are highly loaded on GCA and SE.     

Understanding subgroup differences with general mental ability tests in employment selection: Exploring socio-cultural factors across inter-generational groups

In employment selection, general mental ability (GMA) tests predict training and job performance but also lead to subgroup differences which in turn can produce adverse impact against minority groups. Although researchers have explored genetic, developmental, and environmental explanations for ethnic group differences, few studies have explored socio-cultural factors comparing immigrant and non-immigrant job applicants. Given that many ethnic job applicants may also be immigrants, understanding these factors can provide insight into GMA test score differences. The purpose of this paper is to explore the impact of individual and socio-cultural factors on GMA test scores with immigrant and non-immigrant bus driver applicants. This is the first study of its kind to our knowledge that has attempted to disentangle the effects of socio-cultural factors from race/ethnicity in the study of subgroup differences. Incorporating these variables between non-visible minority and minority groups accounted for considerable variance in GMA test scores across groups. The implications of focusing on socio-cultural variables to enhance our understanding of subgroup differences are discussed. Our results specifically suggest that practitioners attend to the issue of the intersecting grounds of potential discrimination when using GMA tests in personnel selection.     

Validity of Selection/ Classification Variables for Performance in Three Navy Ratings

This study examined speed of advancement and score on a job knowledge test as job performance criteria in the validation of selection and classification variables for three Navy ratings. Longitudinal samples spanning a 4-year period were drawn for men in radioman, machinist's mate, and hospital corpsman ratings who, within their first enlistment, advanced to E-4 by means of the Navy competitive advancement system. Validity coefficients for Navy enlisted selection and classification variables (Armed Services Vocational Aptitude Battery [ASVAB] and education) were computed for personnel in each rating, subgrouped by entry pay grade. Validity coefficients for the job knowledge test were high by historical standards, and the effects were highest for the subgroups with the highest selection standards. Validity coefficients for the advancement criterion were much lower. Advancement, the criterion that appeared to have the greater face validity, was not as good a measure of job performance as the job knowledge test. The criteria were predicted much better by mental ability than by level of education variables. Personnel who enlisted at E-2 and E-3 pay grades spent substantially more time working at the journeyman performance level during their first enlistment than did personnel who enlisted at E-1.     

Revisiting predictive bias of cognitive ability tests against Hispanic American job applicants

Previous studies have concluded that cognitive ability tests are not predictively biased against Hispanic American job applicants because test scores generally overpredict, rather than underpredict, their job performance. However, we highlight two important shortcomings of these past studies and use meta-analytic and computation modeling techniques to address these two shortcomings. In Study 1, an updated meta-analysis of the Hispanic–White mean difference (d-value) on job performance was carried out. In Study 2, computation modeling was used to correct the Study 1 d-values for indirect range restriction and combine them with other meta-analytic parameters relevant to predictive bias to determine how often cognitive ability test scores underpredict Hispanic applicants’ job performance. Hispanic applicants’ job performance was underpredicted by a small to moderate amount in most conditions of the computation model. In contrast to previous studies, this suggests cognitive ability tests can be expected to exhibit predictive bias against Hispanic applicants much of the time. However, some conditions did not exhibit underprediction, highlighting that predictive bias depends on various selection system parameters, such as the criterion-related validity of cognitive ability tests and other predictors used in selection. Regardless, our results challenge “lack of predictive bias” as a rationale for supporting test use.     

Stereotype Threat in Men on a Test of Social Sensitivity

This study provides evidence of stereotype threat in men on a test of a feminine ability called social sensitivity, that is, the ability to decode nonverbal cues. Men who were told that the test assessed social sensitivity and produced better scores for women than men performed worse on the test than did men who were told that the test assessed information processing. Because social sensitivity can be an automatic skill and stereotype threat uses mental capacity, this effect was moderated by self-reported strategy usage. Mens performance worsened in the threat condition only when they reported more deliberative and less intuitive strategies for decoding nonverbal cues.     

Doing It All At Once: Multitasking as a predictor of call center agents’ performance and performance‐based dismissal

Multitasking has become an important skill in many jobs. Still, the predictive validity of job‐applicants’ multitasking abilities has rarely been tested empirically. The current study focuses on the multitasking ability of call center applicants. Results from a Dutch call center show that applicants’ multitasking ability as assessed during personnel selection indeed predicts their later job performance as call center agents as well as their likelihood of losing their job for poor performance. While some of these relationships could be explained via applicants’ fluid intelligence, results also support the usefulness of including measures of multitasking ability in the current selection procedure.     

Updating the trainability tests literature on Black-White subgroup differences and reconsidering criterion-related validity

A number of applied psychologists have suggested that trainability test Black–White ethnic group differences are low or relatively low (e.g., Siegel & Bergman, 1975), though data are scarce. Likewise, there are relatively few estimates of criterion-related validity for trainability tests predicting job performance (cf. Robertson & Downs, 1989). We review and clarify the existing (and limited) literature on Black–White group differences on trainability tests, provide new trainability test data from a recent video-based trainability exam, and present archival data about how trainability test scores relate to cognitive ability, Black–White differences, and job performance. Consistent with hypotheses, our results suggest large correlations of trainability tests with cognitive ability (e.g., .80) and larger standardized ethnic group differences than previously thought (ds of 0.86, 1.10, and 1.21 for 3 samples). Results also suggest that trainability tests have higher validity than previously thought. Overall, our analysis provides a substantial amount of data to update our understanding of the use of trainability tests in personnel selection.     

Emotional Intelligence in Selection Contexts: Measurement method,criterion‐related validity,and vulnerability to response distortion

Two studies examined aspects of the validity of self‐report and performance‐based measures of emotional intelligence (EI) relevant to their use in personnel selection. In Study 1, structural equation modeling indicated that a two‐factor model with separate factors for the two types of EI measures fit better than a one‐factor solution. The performance‐based EI factor was more related to cognitive ability (R=.38) than personality (R=.26), whereas the self‐report EI factor was more related to personality (R=.85) than cognitive ability (R=.09). Although the performance‐based EI factor correlated more strongly with job performance (ρ=.24) than did that of the self‐report (ρ=.05), it provided little incremental validity beyond cognitive ability and conscientiousness. In Study 2, participants were asked to complete the measures as if applying for a job, and mean scores were then compared with those of Study 1. Results indicated that self‐report EI measures were more vulnerable to distortion than were the performance‐based measures. Implications for the assessment of EI in personnel selection contexts are discussed.     

人事测评中的概化理论应用

概化理论是现代心理与教育测量理论之一,可应用在各种人事测评中,如表现性评价、多源评估、心理测验、结构化面试、水平测试、工作分析、评价中心等.与经典测量理论相比,概化理论应用于人事测评,表现出较强的优势,能够同时考察多种因素、确定多个维度权重等,其应用对象主要包括两大类,即企业和机构.概化理论应用于人事测评,存在应用领域、样本数据、测评效度和微观分析等问题.     

The Role of Person Characteristics in Perceptions of the Validity of Cognitive Ability Testing

Individuals vary in how they perceive cognitive ability tests; thus, it is useful for organizations to consider how individual differences influence applicant perceptions of selection tools. The present study examined the influence of implicit theories of ability and locus of control on perceptions of face validity and predictive validity for two cognitive ability tests. Relationships between perceptions and test experience, job‐relevant experience, and job familiarity were also examined. Interactions between implicit theories and self‐assessed performance in predicting perceptions were found, although not of the form hypothesized. Furthermore, job familiarity and prior success in selection contexts were related to perceptions. Finally, sample type interacted with test type to influence perceptions. Implications for selection system design and research on applicant perceptions are discussed.     

The Sensitivity of the CPI Good Impression Scale for Detecting ‘Faking Good’ among Norwegian Students and Job Applicants

This study assessed the sensitivity of the California Psychological Inventory's Good Impression scale in the detection of ‘faking good’ in Norway. The original development of the scale was replicated using Norwegian students (N = 248). Paired sample T‐tests revealed significant differences for 31 of the 40 items when scores from normal test conditions were contrasted with scores obtained under `fake good' instructions. Further, job applicants (N = 494) scored significantly higher on the Good Impression scale than students. Comparisons of mean scale scores between Norwegian and US student samples suggested that the Norwegian students displayed a more modest self‐presentation. The Commonality Scale of the CPI was employed to investigate differences in scale usage between these two cultures. The analyses suggested that the differences in the Good Impression scale observed between these two cultures could not be explained by differences in acquiescence tendencies. Results suggest that the Good Impression scale is able to capture response distortion in Norwegian personnel selection situations, but modification of some items is needed.     

Introduction to the Special Issue: Personality and the Prediction of Job Performance: More Than the Big Five

The effects of motivated distortion on forced-choice (FC) and normative inventories were examined in three studies. Study 1 examined the effects of distortion on the construct validity of the two item formats in terms of convergent and discriminant validity. The results showed that both types of measures were susceptible to motivated distortion, however the FC items were better indicators of personality and less related to socially desirable responding when participants were asked to respond as if applying for a job. Study 2 considered the criterion-related validity of the inventories in terms of predicting supervisors' ratings of job performance, finding that distortion had a more deleterious effect on the validity of the normative inventory with some enhancement of the validity of the FC inventory being observed. Study 3 investigated whether additional constructs are introduced into the measurement process when motivated respondents attempt to increase scores on FC items. Results of Study 3 indicated that individuals higher in cognitive ability tend to have more accurate theories about which traits are job-related and therefore are more successful at improving scores on FC inventories. Implications for using personality inventories in personnel selection are discussed.     

The Role of General Cognitive Ability and Job Performance: Why There Cannot Be a Debate

Given the overwhelming research evidence showing the strong link between general cognitive ability (GCA) and job performance, it is not logically possible for industrial -organizational (I/O) psychologists to have a serious debate over whether GCA is important for job performance. However, even if none of this evidence existed in I/O psychology, research findings in differential psychology on the nature and correlates of GCA provide a sufficient basis for the conclusion that GCA is strongly related to job performance. In I/O psychology, the theoretical basis for the empirical evidence linking GCA and job performance is rarely presented, but is critical to understanding and acceptance of these findings. The theory explains the why behind the empirical findings. From the viewpoint of the kind of world we would like to live in-and would like to believe we live in-the research findings on GCA are not what most people would hope for and are not welcome. However, if we want to remain a science-based field, we cannot reject what we know to be true in favor of what we would like to be true.     

Fakability of an empirically derived selection instrument

Fakability of an empirically derived selection instrument developed by criterion-keying for a specific occupation and organization was investigated. Ninety-four subjects completed the test under both Honest and Fake instructions. Instructions to simulate a highly motivated job applicant produced significant score increases on 7 of 10 scales and significant changes in narrative statements of predicted job performance based on these scores. Theoretical and practical implications are discussed. The caveat to use tests only when the subject is motivated to answer honestly is repeated.     

PERSONALITY TESTS AT THE CROSSROADS: A RESPONSE TO MORGESON, CAMPION, DIPBOYE, HOLLENBECK, MURPHY, AND SCHMITT (2007)

Despite widespread and growing acceptance that published personality tests are valid predictors of job performance, Morgeson et al. (2007) propose they be abandoned in personnel selection because average validity estimates are low. Our review of the literature shows that Morgeson et al.'s skepticism is unfounded. Meta-analyses have demonstrated that published personality tests, in fact, yield useful validity estimates when validation is based on confirmatory research using job analysis and taking into account the bidirectionality of trait–performance linkages. Further gains are likely by use of narrow over broad measures, multivariate prediction, and theory attuned to the complexities of trait expression and evaluation at work. Morgeson et al. also suggest that faking has little, if any, impact on personality test validity and that it may even contribute positively to job performance. Job applicant research suggests that faking under true hiring conditions attenuates personality test validity but that validity is still sufficiently strong to warrant personality test use in hiring. Contrary to Morgeson et al., we argue that the full value of published personality tests in organizations has yet to be realized, calling for programmatic theory-driven research.     

COMBINING BIODATA TEST AND INTERVIEW INFORMATION: PREDICTING DECISIONS AND PERFORMANCE CRITERIA

An important question in real world selection situations is: How do interviewers integrate employment test scores and interview information in making selection decisions? The present study expands on previous research by examining how valid biodata test scores and interview information interact in the prediction of actual interviewer selection decisions and actual job performance criteria for hired candidates. The sample for the study was 577 actual candidates for the position of life insurance sales agent evaluated by insurance agency managers from a single company. The results indicated the presence of interaction effects in predicting both the interviewers' decision and 12-month survival. The interview information predicted the decision and survival best for candidates with low passing scores on the biodata test and poorest for candidates with high passing scores. The results clarify the conditions under which interview information is most predictive of job performance criteria.     

A Research Note on the Incremental Validity of Job Knowledge and Integrity Tests for Predicting Maximal Performance

In a sample of industrial job applicants, relationships among scores on an integrity test, a job knowledge measure, and maximal performance as assessed by a work sample measure were investigated. The observed correlation between the personality-based integrity test and maximal performance was .27, indicating that integrity tests can be predictive of maximal performance. Furthermore, integrity test scores correlated .14 with job knowledge and job knowledge scores had a validity of .36 for maximal performance. Theoretical implications for the maximal/typical performance distinction are discussed.