Incremental Validity of Situational Judgment Tests for Task and Contextual Job Performance

This paper has three goals. First, it responds to calls for additional research on subgroup differences in situational judgment tests. Second, it expands the cumulative knowledge on the incremental validity of situational judgment tests beyond cognitive ability and personality. Third, it examines the validity and incremental validity of various predictors for both task and contextual performance.     

Integrity Testing and Job Performance Within High Performance Work Teams: A Short Note

Integrity tests have exhibited robust validity for predicting performance among traditional jobs. However, some organizations have shifted to self-managed work teams and the relationship between integrity and performance within these jobs is untested. One hundred fourteen high performance team members within a single department completed the Hogan Personality Inventory (HPI). The HPI Reliability scale was used as the integrity measure. The subjects' job performance was evaluated by their team leaders. Performance ratings were transformed to equalize ratings across the eight teams. Results indicate that integrity was related to transformed performance ratings (r = .25) within a high performance team environment. Limitations and implications are discussed.     

Examining the Incremental Validity and Adverse Impact of Cognitive Ability and Conscientiousness on Job Performance

Using archival data from a large home improvement organization, the current study examined the incremental validity of conscientiousness over cognitive ability in predicting different performance criteria. Also, the study examined the effects of conscientiousness and cognitive ability on adverse impact. Results indicated that conscientiousness provided incremental validity over cognitive ability in the prediction of several performance criteria. Specifically, conscientiousness had the highest increases in validity over cognitive ability in the prediction of contextual performance factors (e.g., contextual performance, customer service orientation, dealing with others). However, the addition of conscientiousness failed to ameliorate the adverse impact associated with cognitive ability. In addition, limitations of the study and future directions of research are addressed.     

Effects of Nonlinearity and Heteroscedasticity on the Validity of Conscientiousness in Predicting Overall Job Performance

Most research on the relationship between personality and overall job performance assumes linearity and homoscedasticity. This study investigated the prevalence and nature of nonlinearities and heteroscedasticies in relationships between conscientiousness and supervisory ratings of overall job performance across five independent samples using both concurrent (k = 4) and predictive (k = 1) designs. Hierarchical polynomial regression analyses found evidence of robust linear effects but no evidence of statistically significant quadratic or cubic effects. A statistically and practically significant heteroscedastic effect was found in only one sample such that lower errors of prediction were evidenced in the ends in comparison to the middle of the bivariate distribution. Implications for the use of conscientiousness in personnel selection are suggested. Limitations of the current study and directions for future research are noted.     

More than g: Evidence for the Incremental Validity of Performance-Based Assessments for Predicting Training Performance

Past research has consistently shown that tests measuring specific cognitive abilities provide little if any incremental validity over tests of general mental ability when predicting performance on the job. In this study, we suggest that the seeming lack of incremental validity may have been due to the type of content that has traditionally been assessed. Therefore, we hypothesised that incremental validity can be obtained using specific cognitive abilities that are less highly correlated with g and are matched to the tasks performed on the job. To test this, we examined a recently developed performance-based measure that assesses a number of cognitive abilities related to training performance. In a sample of 310 US Navy student pilots, results indicated that performance-based scores added sizeable incremental validity to a measure of g. The significant increases in R² ranged from .08 to .10 across criteria. Similar results were obtained after correcting correlations for range restriction, though the magnitude of incremental validity was slightly smaller (ΔR² ranged from .05 to .07).     

Cognitive Ability Tests and the Situational Interview: A Test of Incremental Validity

This study investigated the validity and incremental validity of a situational interview beyond that of a composite measure of cognitive ability. Forty-seven factory service technicians underwent an interview and took four cognitive ability tests. Supervisors rated the performance of these subjects in a concurrent validation study. The interview was found to be a valid predictor of a supervisor rating of performance (r = 0.32, p < 0.05 uncorrected), however, was unable to show incremental validity over ability tests (Incremental R²= 0.05, n.s.). Limitations of the present study and directions for future research are discussed.     

The Validity of Conscientiousness for Predicting Job Performance: A meta‐analytic test of two hypotheses

This study examined two hypotheses regarding the moderating effects of job characteristics on the validity of personality. Using meta‐analytic techniques, the authors explored the extent to which the structural characteristics and cognitive ability requirements of jobs influence the role of conscientiousness in predicting performance. The results suggest that conscientiousness is a stronger predictor of performance in jobs that are highly routinized, and a weaker predictor of performance in jobs with high levels of cognitive ability requirements. Implications for theory and future research are discussed.     

On the Construct Validity of Integrity Tests: Individual and Situational Factors as Predictors of Test Performance

Although integrity tests are widely applied in screening job applicants, there is a need for research for examining the construct validity of these tests. In the present study, a theoretical model examining the causes of destructive behavior in organizational settings was used to develop background data measures of individual and situational variables that might be related to integrity test scores. Subsequently, 692 undergraduates were asked to complete these background data scales along with (a) two overt integrity tests – the Reid Report and the Personnel Selection Inventory, and (b) two personality‐based measures – the delinquency and socialization scales of the California Psychological Inventory. When scores of these measures were correlated with and regressed on the background data scales, it was found that relevant individual variables, such as narcissism and power motives, and relevant situational variables, such as alienation and exposure to negative peer groups, were related to scores on both types of integrity tests. However, a stronger pattern of validity evidence was obtained for the personality‐based measures and, in all cases, situational variables were found to be better predictors than individual variables. The implications of these findings for the validity of inferences drawn from overt and personality‐based integrity tests are discussed.     

A Comparative Study of Integrity Tests: The Criterion-Related Validity of Personality-Based and Overt Measures of Integrity

In the field of personality assessment, one construct or set of constructs that is emerging as important for both publishers and users of psychological testing has been labelled variously as integrity, conscientiousness, reliability, delinquency, or responsibility. The associated organizational outcome has been broadly labelled counterproductivity. In the present study, four integrity scales, selected scales from mainstream personality inventories, derived molar integrity factors and optimal scale linear combinations were correlated with admissions of counterproductivity. It was found that (a) all four integrity scales were significantly correlated with the counterproductivity criterion, (b) individual personality scales from normal personality inventories, as well as unit-weighted linear combinations of them, were approximately as highly correlated with counterproductivity as were the integrity tests, and (c) personality traits conceptually distinct from the domain of integrity were also related to counterproductivity.     

Predicting Training and Job Performance for Transit Operators

Although considerable meta‐analytic research has validated the use of cognitive ability tests, structured interviews, and personality tests with training and job performance criteria, few studies have investigated the validity of these measures with transit operators. There are the only two single studies of concurrent validation research specifically with transit operators. This article presents the results of a predictive validation study conducted with transit operator applicants for a large urban transit authority in Canada. Key knowledge, skills, and abilities were determined for the role and used as a basis for the design and choice of predictors and criteria. Four predictors were used in the study: education, cognitive ability, personality assessment, and structured interview. Criteria included training performance (formative and summative), probationary performance, preventable accidents, and lost time injuries. Validation results supported cognitive ability, structured interview, and several personality factors as predictors of training performance, but less so for job performance. The use of formative training ratings greatly augmented the evidence supporting the predictors beyond typical organizational criteria.     

The Influence of Work Personality on Job Satisfaction: Incremental Validity and Mediation Effects

ABSTRACT Drawing from recent developments regarding the contextual nature of personality (e.g., D. Wood & B. W. Roberts, 2006 ), we conducted 2 studies (1 cross-sectional and 1 longitudinal over 1 year) to examine the validity of work personality in predicting job satisfaction and its mediation of the effect of global personality on job satisfaction. Study 1 showed that (a) individuals vary systematically in their personality between roles— they were significantly more conscientious and open to experience and less extraverted at work compared to at home; (b) work personality was a better predictor of job satisfaction than both global personality and home personality; and (c) work personality demonstrated incremental validity above and beyond the other two personality measures. Study 2 further showed that each of the work personality dimensions fully mediated the association between its corresponding global personality trait and job satisfaction. Evidence for the discriminant validity of the findings is also presented.     

Incremental Validity of Time Urgency and Other Type A Subcomponents in Predicting Behavioral and Health Criteria1

Although previous studies have revealed relationships between Type A behavior pattern (TABP) measures and behavioral indexes of time urgency, such studies utilized global TABP measures that have been criticized for their lack of construct validity. The present study linked recently developed measures of time urgency and other TABP subcomponents (e. g., impatience‐irritability) to observable temporal behaviors and health outcomes. In a sample of 194 undergraduate students, the results indicated that the TABP subcomponents provided incremental validity above the contribution of the global TABP measure in predicting behavioral and health criteria. Applied implications and future directions for research on time urgency and other TABP subcomponents are discussed.     

Incremental Validity of Assessment Center Ratings Over Cognitive Ability Tests: A Study at the Executive Management Level

Both tests of cognitive ability and assessment center (AC) ratings of various performance attributes have proven useful in personnel selection and promotion contexts. To be of theoretical or practical value, however, the AC method must show incremental predictive accuracy over cognitive ability tests given the cost disparities between the two predictors. In the present study, we investigated this issue in the context of promotion of managers in German police departments into a training academy for high‐level executive positions. Candidates completed a set of cognitive ability tests and a 2‐day AC. The criterion measure was the final grade at the police academy. Results indicated that AC ratings of managerial abilities were important predictors of training success, even after accounting for cognitive ability test scores. These results confirm that AC ratings provide unique contribution to the understanding and prediction of training performance of high‐level executive positions beyond cognitive ability tests.     

Effects of Organizationally Endorsed Coaching on Performance and Validity of Situational Judgment Tests

There is growing interest in organizationally provided or organizationally endorsed coaching. However, little is known about the effects of such coaching on test scores in operational settings. This study reports on an examination of such a program in the context of the use of a situational judgment test (SJT) for medical school admissions. We examine the effects of multiple types of coaching methods on SJT scores and on their construct‐related and predictive validities. Results suggest that (1) commercial coaching techniques may not be as effective as previously thought, whereas organizationally provided methods may be more effective, and that (2) the criterion‐related validity of the SJT scores is not degraded by the availability of coaching. Generally, this study illustrates that concerns about potential unfairness of coaching can be countered by making effective coaching available to all examinees, in the form of organizationally endorsed coaching.     

工作绩效预测研究述评

文章介绍了国内外有关人格、能力与工作绩效关系研究和工作绩效预测模型研究的最新成果与进展。研究表明,个体的能力、人格等个性特征影响其工作绩效,合适的能力、人格测验能较好的预测其工作绩效,并受到一些中介变量和调节变量的影响。文章提出应从工作绩效结构、预测模型和跨文化研究三个方面加强工作绩效的预测研究     

Generalizability Theory as Construct-Related Evidence of the Validity of Job Performance Ratings

Investigations of the construct-related evidence of the validity of performance ratings have been rare, perhaps because researchers are dissuaded by the con- siderable amount of evidence needed to show construct validity (Landy, 1986). It is argued that generalizability (G) theory (Cronbach, Gleser, Nanda, & Rajaratnam, 1972) is well-suited to investigations of construct-related evi- dence of validity because a single generalizability investigation may provide multiple inferences of validity. G theory permits the researcher to partition observed score variance into universe (true) score variance and multiple, distinct estimates of error variance. G theory was illustrated through the anal- ysis of proficiency ratings of 256 Air Force jet engine mechanics. Mechanics were rated on three different rating forms by themselves, peers, and supervi- sors. Interpretation of G study variance components revealed suitable evi- dence of construct validity. Ratings within sources were reliable. Proficiency ratings showed strong convergence over rating forms, though not over rating sources. Raters showed adequate discriminant validity across rating dimen- sions. The expectation of convergence over sources was further questioned.     

Predicting Supervisor Ratings of Job Performance Using the NEO Personality Inventory

The resurging interest in identifying personality predictors of job performance has led researchers to use the five-factor model of personality as an important conceptual tool. Two recent meta-analyses illustrated the potential predictive power of the model within an employment context. In this study, we evaluated the relations between a well-validated self-report measure of the five-factor model and supervisor ratings of performance. Fifty-two women and 159 men completed the NEO Personality Inventory (Costa & McCrae, 1985) and were independently rated on various performance dimensions by their supervisors. Scores on the Conscientiousness scales correlated with all the performance ratings over this diverse number of occupational groups. Low Neuroticism and high Extraversion scores also predicted high performance. Low straightforwardness (a facet of agreeableness) was associated with ability to accomplish work-related goals and to adapt to changing work conditions. The results support the utility of the five-factor model as a predictor of job performance.