Situational Judgment and Job Performance

Data from 160 civil service employees demonstrate the validity of a situational judgment test in predicting overall job performance as well as three performance dimensions: task performance (core technical proficiency), motivational contextual performance (job dedication), and interpersonal contextual performance (interpersonal facilitation). Situational judgment also provided incremental validity over the prediction provided jointly by cognitive ability, the Big Five personality traits, and job experience. These findings extended the work of Clevenger, Pereira, Wiechmann, Schmitt, and Harvey (2001) on the incremental validity of situational judgment tests as well as the meta-analytic results reported by McDaniel, Morgeson, Finnegan, Campion, and Braverman (2001). Implications are discussed in terms of research on the prediction and understanding of job performance.     

Incremental validity of person-organization fit over the Big Five personality measures

Few studies have provided the validity evidence of a measure of objective person-organization fit (P-O fit) as a selection tool. The present study used a concurrent validation design to examine the criterion-related validity and the incremental validity of a P-O fit measure beyond the validity of the Big Five personality test for predicting job performance (task performance and organizational citizenship behavior) and employee commitment (organizational commitment and supervisory commitment) for a group of high-tech professional employees in Taiwan. Results showed that P-O fit predicted the contextual component of overall job performance and was significantly related to two types of employee commitment. Moreover, P-O fit had an incremental validity beyond that of the personality measures for predicting some of our outcome variables.     

EMPLOYEES' CHALLENGING JOB EXPERIENCES AND SUPERVISORS' EVALUATIONS OF PROMOTABILITY

Promotability evaluations are important for individuals' career development and organizations' human resource management practices. Nevertheless, little empirical research has addressed predictors of promotability evaluations, and the studies that have, have often focused on current job performance and fixed, nonbehavioral predictors. This study takes a more behavioral approach, and investigates whether besides how one performs (i.e., job performance) what one performs also serves as an indicator of promotability. Specifically, we examine the relationship between employees' challenging job experiences and supervisors' evaluations of employees' promotability over and above employees' current job performance. Results from 3 field studies, sampling different types of employees via different measures, consistently showed that challenging job experiences explained incremental variance in supervisory and organizational evaluations of promotability over and above current job performance and job tenure.     

The validity and incremental validity of knowledge tests, low-fidelity simulations, and high-fidelity simulations for predicting job performance in advanced-level high-stakes selection

In high-stakes selection among candidates with considerable domain-specific knowledge and experience, investigations of whether high-fidelity simulations (assessment centers; ACs) have incremental validity over low-fidelity simulations (situational judgment tests; SJTs) are lacking. Therefore, this article integrates research on the validity of knowledge tests, low-fidelity simulations, and high-fidelity simulations in advanced-level high-stakes settings. A model and hypotheses of how these 3 predictors work in combination to predict job performance were developed. In a sample of 196 applicants, all 3 predictors were significantly related to job performance. Both the SJT and the AC had incremental validity over the knowledge test. Moreover, the AC had incremental validity over the SJT. Model tests showed that the SJT fully mediated the effects of declarative knowledge on job performance, whereas the AC partially mediated the effects of the SJT.     

More than g: Evidence for the Incremental Validity of Performance-Based Assessments for Predicting Training Performance

Past research has consistently shown that tests measuring specific cognitive abilities provide little if any incremental validity over tests of general mental ability when predicting performance on the job. In this study, we suggest that the seeming lack of incremental validity may have been due to the type of content that has traditionally been assessed. Therefore, we hypothesised that incremental validity can be obtained using specific cognitive abilities that are less highly correlated with g and are matched to the tasks performed on the job. To test this, we examined a recently developed performance-based measure that assesses a number of cognitive abilities related to training performance. In a sample of 310 US Navy student pilots, results indicated that performance-based scores added sizeable incremental validity to a measure of g. The significant increases in R² ranged from .08 to .10 across criteria. Similar results were obtained after correcting correlations for range restriction, though the magnitude of incremental validity was slightly smaller (ΔR² ranged from .05 to .07).     

Cognitive Misfit of Problem-Solving Style at Work: A Facet of Person-Organization Fit

Based on Kirton's Adaption-Innovation Theory (1976, 1977) of problem-solving style at work, the construct of cognitive misfit was developed and proposed as one viable facet of Person-Organization fit. Cognitive misfit refers to the degree of mismatch between an individual's cognitive style of problem solving and the style demands of the work context. Relationships among cognitive misfit, job performance, and actual individual turnover after 3 years were examined in a sample of 253 engineers in either a Staff engineering function or a Research and Development (R&D) engineering function. The Staff function and the R&D function corresponded to a work context predominant in adaptive-style demands and a work context predominant in innovative-style demands, respectively. Results from logistic regression analyses showed that cognitive misfit was uncorrelated with job performance, but provided significant and substantial incremental validity in predicting actual turnover over the predictability provided by performance. Implications, limitations, and directions for future research were discussed in terms of levels of analysis issues and the current focus on Person-Organization fit in selection and organizational behavior research.     

Predicting Performance with Contextualized Inventories,No Frame‐of‐reference Effect?

A recent meta‐analysis showed that contextualized personality inventories have incremental predictive validity over generic personality inventories when predicting job performance. This study aimed to investigate the differences between two types of contextualization of items: Adding an ‘at work’ tag versus completely modifying items. One hundred thirty‐nine pharmacy assistants from 29 pharmacies filled out a generic, a tagged, and a completely modified personality inventory. The assistants also provided participant reactions for each of the personality inventories. Performance ratings were collected from the supervising pharmacists. We expected to find incremental criterion validity for both the tagged inventory and the completely modified inventory for predicting job performance. However, the results showed an unexpected decrease in predictive validity for the contextualized inventories. Contextualized inventories were liked less than the generic inventory, but were considered somewhat more face valid and predictive by the participants.     

Validation of intelligence and mechanical aptitude tests in selecting employees for manufacturing jobs

One hundred ninety-three manufacturing employees who produce electro-mechanical components participated in a concurrent criterion-related validity study. The employees were administered three tests: The Bennett Mechanical Comprehension Test (Form S); The Flanagan Aptitude Classification Test-Mechanics; and the Thurstone Test of Mental Alertness (Form A). Job performance was measured by a supervisor rating of fifteen job dimensions, assessed at two points in time separated by 60 days. Correlational and multiple regression analyses were used to assess the relationship between test scores and job performance ratings. The results revealed that the Bennett Mechanical Comprehension test was the best single predictor of job performance (uncorrectedr =.38), and the incremental gain in predictability from additional tests was not significant. The results were discussed in the context of the changing nature of manufacturing jobs and the inadequacy of conventional mechanical aptitude tests to be sensitive to these changes.     

Socioanalytic theory and work behavior: Roles of work values and political skill in job performance and promotability assessment

Socioanalytic theory postulates that job performance ratings are predicted by basic social motives moderated by social competency. The two motives are the motive to get along with others and the motive to achieve status and power. The present two-study investigation assessed these motives as work values and collected supervisors' job performance and promotability assessments. Social competency was assessed as political skill at work. The results provided strong and consistent support for the hypotheses, thus providing a more direct test of socioanalytic theory and extending it to demonstrate effects beyond overall job performance ratings on contextual performance and promotability assessments. Contributions and implications of these results, strengths and limitations, directions for future research, and practical implications are discussed.     

Validity of observer ratings of the five-factor model of personality traits: a meta-analysis

Conclusions reached in previous research about the magnitude and nature of personality-performance linkages have been based almost exclusively on self-report measures of personality. The purpose of this study is to address this void in the literature by conducting a meta-analysis of the relationship between observer ratings of the five-factor model (FFM) personality traits and overall job performance. Our results show that the operational validities of FFM traits based on observer ratings are higher than those based on self-report ratings. In addition, the results show that when based on observer ratings, all FFM traits are significant predictors of overall performance. Further, observer ratings of FFM traits show meaningful incremental validity over self-reports of corresponding FFM traits in predicting overall performance, but the reverse is not true. We conclude that the validity of FFM traits in predicting overall performance is higher than previously believed, and our results underscore the importance of disentangling the validity of personality traits from the method of measurement of the traits.     

Which Aspects of a Semistructured Interview,Besides Cognitive Ability Tests,Contribute Incrementally to Predicting the Training Success of Air Traffic Controller Trainees?

This study examined whether a semistructured interview is valid in predicting training performance in theoretical and simulation examinations and overall training success; which aspects of the interview show incremental validity over cognitive ability tests; and whether additional variables enhance the prediction from the interview. The sample involved data from 337 trainees of the German Air Navigation Service Provider selected by the German Aerospace Center and 657 (interview) and 6,885 (cognitive ability) candidates being the corresponding reference samples. Findings indicate incremental validity of the interview over the cognitive ability tests. Additional variables contributed only to the prediction of theoretical examination performance. Attention, general motivation, and social competence predicted the criteria best, particularly the theoretical examination performance and overall training success.     

Incremental Validity of Locus of Control After Controlling for Cognitive Ability and Conscientiousness

This research examined the criterion-related validity of work-specific locus of control in predicting job performance, including incremental validity after controlling for cognitive ability and conscientiousness. Data from a student sample and from a large employee sample were used to evaluate the scale properties of measures of locus of control, conscientiousness, and cognitive ability. Two concurrent criterion-related validation studies were then conducted to evaluate the incremental validity of locus of control. In both validation studies, locus of control demonstrated overall and incremental relationships with performance after controlling for ability and conscientiousness, such that employees with higher internal locus of control performed more effectively than externals.     

The Incremental Validity of Honesty–Humility Over Cognitive Ability and the Big Five Personality Traits

The present study examines the incremental validity of Honesty–Humility (H-H), a measure of the tendency to be fair and genuine in dealing with others, for supervisory ratings of job performance (including both task and contextual performance) over cognitive ability and the Big Five personality traits. Specifically, we explore the incremental validity of H-H in predicting contextual performance. Results based on 217 South Korean military officer candidates are consistent with previous conclusions that Conscientiousness is the strongest predictor of contextual performance and that cognitive ability is the strongest predictor of task performance. More important, our results reveal that H-H offers moderate incremental validity for contextual performance but no incremental validity for task performance over the effects of cognitive ability and the Big Five personality traits.     

Webcam testing: Validation of an innovative open-ended multimedia test

A modern test that takes advantage of the opportunities provided by advancements in computer technology is the multimedia test. The purpose of this study was to investigate the criterion-related validity of a specific open-ended multimedia test, namely a webcam test, by means of a concurrent validity study. In a webcam test a number of work-related situations are presented and participants have to respond as if these were real work situations. The responses are recorded with a webcam. The aim of the webcam test which we investigated is to measure the effectiveness of social work behaviour. This first field study on a webcam test was conducted in an employment agency in The Netherlands. The sample consisted of 188 consultants who participated in a certification process. For the webcam test, good interrater reliabilities and internal consistencies were found. The results showed the webcam test to be significantly correlated with job placement success. The webcam test scores were also found to be related to job knowledge. Hierarchical regression analysis demonstrated that the webcam test has incremental validity up to and beyond job knowledge in predicting job placement success. The webcam test, therefore, seems a promising type of instrument for personnel selection.     

EXPLORING OPTIONS FOR SUPPORTING TEST USE IN SITUATIONS PRECLUDING LOCAL VALIDATION

This paper examines some of the options available to practitioners interested in supporting the use of selection measures in an organization, including test transportability, validity generalization (VG), and synthetic validation, reviewing some of the advantages, disadvantages and requirements of each approach. Results of four unpublished, proprietary validation studies are reported which compare validity estimates provided by the job component validation (JCV; a type of synthetic validation) routine inherent in the Position Analysis Questionnaire (PAQ) with observed validity coefficients for a variety of criterion measures. We then examine the accuracy of the JCV procedure in predicting validity coefficients for 51 clerical classifications extracted from an existing PAQ job evaluation database of a large utility company. Predicted JCVs are compared to mean observed validity coefficients for five DOT clerical categories provided by Pearlman, Schmidt, and Hunter (1980). The VG and JCV methods provided highly similar and converging estimates of the validity of cognitive ability tests for predicting performance in clerical occupations. Implications for practice are discussed, particularly the need to use multiple, converging lines of evidence to support test use.     

The incremental validity of organizational commitment,organizational trust,and organizational identification

Organizational commitment (OC), organizational trust (OT), and organizational identification (OI) are three types of psychological attachment to an organization. Each of these three variables captures an organization-targeted attitude toward an employment relationship, but it is unclear whether they have incremental validity over each other. To address this question, this study examined the incremental validity of each variable in predicting job involvement, job satisfaction, turnover intentions, and non-self-report measures of task performance and citizenship behavior. It also examined whether perceived organizational support and psychological contract breach, two other organization-targeted attitudinal variables, were related to OC, OT, and OI when the latter were considered jointly. Meta-analytical evidence suggests that OC, OT, and OI have incremental validity over and above one another in their relationships with some, but not all, of the above correlates. This highlights the need for future research to distinguish these three types of psychological attachment to an organization.     

A Longitudinal Investigation of Task and Contextual Performance Influences on Promotability Judgments

Promotions in organizations traditionally have represented the principal measure of career success, and they tend to be based on evaluations or judgments of employees' promotability made by supervisors. Yet theory and research on the antecedents of promotability judgments have presented an inconsistent and ambiguous picture of just what factors are best predictive of such evaluations. In the present investigation, longitudinal data obtained from supervisors of professional employees were used to rigorously test the relative influence of task and contextual performance on judgments of promotability. Results indicate that task and contextual performance not only explain unique variance in judgments of promotability but also interact, such that subordinates who excel in task performance and in the job dedication aspect of contextual performance are judged more suitable for promotion than subordinates who excel in one but not in the other. Implications of results are discussed and future research directions are offered.     

The Effects of Social Desirability and Faking on Personality and Integrity Assessment for Personnel Selection

A review of the extant literature and new empirical research suggests that social desirability is not much of a concern in personality and integrity testing for personnel selection. In particular, based on meta-analytically derived evidence, it appears that social desirability influences do not destroy the convergent and discriminant validity of the Big Five dimensions of personality (Emotional Stability, Extraversion, Openness to Experience, Agreeableness, and Conscientiousness). We also present new empirical evidence regarding gender and age differences in socially desirable re- sponding. Although social desirability predicts a number of important work variables such as job satisfaction, organizational commitment, and supervisor ratings of training success, social desirability does not seem to be a predictor of overall job performance and is only very weakly related to specific dimensions of job performance such as technical proficiency (r = -.07) and personal discipline ( r = .05). Large sample investigations of the moderating influences of social desirability in actual work settings indicate that social desirability does not moderate the criterion-related validities of personality variables or integrity tests. The criterion-related validity of integrity tests for overall job performance with applicant samples in predictive studies is .41. Controlling for social desirability in integrity or personality test scores leaves the operational validities intact, thereby suggesting that social desirability functions neither as a mediator nor as a suppressor variable in personality-performance.     

IMPLICATIONS OF THE MULTIDIMENSIONAL NATURE OF JOB PERFORMANCE FOR THE VALIDITY OF SELECTION TESTS: MULTIVARIATE FRAMEWORKS FOR STUDYING TEST VALIDITY

Although most studies of criterion-related validity focus on univariate relationships, the complex and multidimensional nature of the performance construct and the widespread use of multiple selection devices argue in favor of multivariate frameworks for evaluating validity. Using a Monte Carlo simulation we estimated the validity of general cognitive ability tests and personality tests in predicting "job performance," where performance is conceptualized as a composite of multiple performance measures (i.e., individual job task performance and organizational citizenship behaviors). The validity of a selection battery varies substantially as a function of the relative weight given to both predictors and criteria; the 95% confidence interval for validities ranged from .20 to .78. The effective weights given to performance dimensions accounted for 34% of the variance in selection battery validities; depending on precisely how "performance" is defined, the same test battery can have relatively high or relatively low levels of validity. Our model suggests that the way an organization defines job performance is a source of true and important variability in validities, and that the validity of selection tests for predicting complex performance criteria may show considerably less generalizability that current metaanalysis of univariate validities would suggest.     

The criterion-related validity of integrity tests: an updated meta-analysis

Integrity tests have become a prominent predictor within the selection literature over the past few decades. However, some researchers have expressed concerns about the criterion-related validity evidence for such tests because of a perceived lack of methodological rigor within this literature, as well as a heavy reliance on unpublished data from test publishers. In response to these concerns, we meta-analyzed 104 studies (representing 134 independent samples), which were authored by a similar proportion of test publishers and non-publishers, whose conduct was consistent with professional standards for test validation, and whose results were relevant to the validity of integrity-specific scales for predicting individual work behavior. Overall mean observed validity estimates and validity estimates corrected for unreliability in the criterion (respectively) were .12 and .15 for job performance, .13 and .16 for training performance, .26 and .32 for counterproductive work behavior, and .07 and .09 for turnover. Although data on restriction of range were sparse, illustrative corrections for indirect range restriction did increase validities slightly (e.g., from .15 to .18 for job performance). Several variables appeared to moderate relations between integrity tests and the criteria. For example, corrected validities for job performance criteria were larger when based on studies authored by integrity test publishers (.27) than when based on studies from non-publishers (.12). In addition, corrected validities for counterproductive work behavior criteria were larger when based on self-reports (.42) than when based on other-reports (.11) or employee records (.15).