A Research Note on the Incremental Validity of Job Knowledge and Integrity Tests for Predicting Maximal Performance

In a sample of industrial job applicants, relationships among scores on an integrity test, a job knowledge measure, and maximal performance as assessed by a work sample measure were investigated. The observed correlation between the personality-based integrity test and maximal performance was .27, indicating that integrity tests can be predictive of maximal performance. Furthermore, integrity test scores correlated .14 with job knowledge and job knowledge scores had a validity of .36 for maximal performance. Theoretical implications for the maximal/typical performance distinction are discussed.     

The Incremental Validity of Self-Report and Performance-Based Methods for Assessing Hostility to Predict Cardiovascular Disease in Physicians

We evaluated the utility of an integrative, multimethod approach for assessing hostility-related constructs to predict premature cardiovascular disease (CVD) and premature coronary heart disease (CHD) using participants from the Johns Hopkins Precursors Study, which was designed to identify risk factors for heart disease. Participants were assessed at baseline while in medical school from 1946 to 1962 (M age = 24.6) and have been followed annually since then. Baseline assessment included individually administered Rorschach protocols (N = 416) scored for aggressive imagery (i.e., Aggressive Content, Aggressive Past) and self-reports of 3 possible anger responses to stress. Cox regression analyses predicting morbidity or mortality by age 55 revealed a significant interaction effect; high levels of Aggressive Content with high self-reported hostility predicted an increased rate of premature CVD and CHD, and incrementally predicted the rate of these events after controlling for the significant covariates of smoking (CVD and CHD) and cholesterol (CHD) that were also assessed at baseline. The hostility and anger measures, as well as other baseline covariates, were not predictors of CVD risk factors assessed at midlife during follow-up. Overall, this integrative model of hostility illustrates the potential value of multimethod assessment to areas of health psychology and preventive medicine.     

Validity of USES Aptitude Test Batteries for Predicting MDTA Training Success

As a result of the continuing Federal-State cooperative test research program in the Employment Service, aptitude test batteries for a large number of specific occupations have been developed. These test batteries are in daily use by Employment Service counselors of candidates for training programs established under the Manpower Development and Training Act (MDTA) of 1962. To what extent are these test batteries predictive of success of enrollees in MDTA courses? This article summarizes the available evidence on this question, based on data for 875 MDTA trainees in 12 courses. In 10 of the 12 samples the operational test norms showed significant cross validity.     

Incremental Validity of Situational Judgment Tests for Task and Contextual Job Performance

This paper has three goals. First, it responds to calls for additional research on subgroup differences in situational judgment tests. Second, it expands the cumulative knowledge on the incremental validity of situational judgment tests beyond cognitive ability and personality. Third, it examines the validity and incremental validity of various predictors for both task and contextual performance.     

The Predictive Validity of Injury Proxies: Predicting Early Adolescent Injuries with Assessments of Minor Injuries

This study offers an initial step in establishing the predictive validity of injury proxies. Proxies are utilized frequently by injury researchers to overcome various methodological and ethical issues inherent in injury research, although psychometric data on proxies are limited. Using data from the NICHD Study of Early Child Care and Youth Development we found that minor injuries predicted adolescent injuries longitudinally, even in the context of well-established predictors of injury risk. This study is the first to demonstrate the predictive validity of minor injuries, a common proxy of pediatric injury risk. Findings are discussed with respect to implications for conceptual understanding of injury risk, future research, and prevention efforts.     

Multitasking as a Predictor of Pilot Performance: Validity Beyond Serial Single-Task Assessments

Although multitasking has been identified as an important competency for many jobs, standardized preemployment tests of multitasking are uncommon, and their validity beyond separate single-task assessments is unclear. To address this gap, this study compared the validity of a preemployment multitasking assessment (with math, memorization, and monitoring tasks presented concurrently) to an assessment in which the same constituent tasks were assessed separately. Among military pilot applicants, concurrent multitasking predicted both flying and academic performance, whereas serial single-task scores and their combination predicted only academic performance. Larger multitasking decrements (i.e., operationalized as the difference between single-task and multiple-task performance of the same constituent tasks) were associated with poorer flying performance. Results suggest the potential utility of multitasking assessments in selection for military jobs requiring competing, concurrent task demands.     

Predicting Training and Job Performance for Transit Operators

Although considerable meta‐analytic research has validated the use of cognitive ability tests, structured interviews, and personality tests with training and job performance criteria, few studies have investigated the validity of these measures with transit operators. There are the only two single studies of concurrent validation research specifically with transit operators. This article presents the results of a predictive validation study conducted with transit operator applicants for a large urban transit authority in Canada. Key knowledge, skills, and abilities were determined for the role and used as a basis for the design and choice of predictors and criteria. Four predictors were used in the study: education, cognitive ability, personality assessment, and structured interview. Criteria included training performance (formative and summative), probationary performance, preventable accidents, and lost time injuries. Validation results supported cognitive ability, structured interview, and several personality factors as predictors of training performance, but less so for job performance. The use of formative training ratings greatly augmented the evidence supporting the predictors beyond typical organizational criteria.     

History Assessments of Thinking: A Validity Study

This article reports a validity study of History Assessments of Thinking (HATs), which are short, constructed-response assessments of historical thinking. In particular, this study focuses on aspects of cognitive validity, which is an examination of whether assessments tap the intended constructs. Think-aloud interviews with 26 high school students were used to examine the thinking elicited by 8 HATs and multiple-choice versions of these tasks. Results showed that although both HATs and multiple-choice items tapped historical thinking processes, HATs better reflected student proficiency in historical thinking than their multiple-choice counterparts. Item format also influenced the thinking elicited, with multiple-choice items eliciting more instances of construct-irrelevant reasoning than the constructed-response versions. Implications for history assessment are discussed.     

Research Validity Scales for the NEO-PI-R: Additional Evidence for Reliability and Validity

We examined the reliability and validity of the research validity scales (Schinka, Kinder, & Kremer, 1997) for the NEO-Personality Inventory-Revised (NEO-PI-R) in a clinical sample. The Negative Presentation Management (NPM) and Positive Presentation Management (PPM) scales were found to have satisfactory internal consistency reliability. Support for the validity of these scales was provided by the pattern of convergent and discriminant correlations with respective Personality Assessment Inventory (PAI) validity scales. Finally, PAI profiles of individuals with invalid NPM scores were found to differ significantly from those with valid NPM scores. Comparisons of the invalid profiles with profiles from other clinical samples provided additional support for the use of the NPM scale as a measure of negative impression management.     

Evidence of Predictive and Incremental Validity of Personality Factors,Vocal Attractiveness and the Situational Interview

This study evaluates the impact of the situational interview, personality measures, vocal attractiveness, and interactions between personality and vocal attractiveness on job performance. In a concurrent validity study with 154 customer service employees, a structured, situational interview was conducted and three of the Big Five personality factors (agreeableness, conscientiousness, and extraversion) are measured with two different frames‐of‐reference: at‐work and typical. Among the results are the validity of the situational interview and at‐work personality measures is established; vocal attractiveness correlates with both interview ratings and job performance; and finally, agreeableness and conscientiousness are found to vary in a pattern related to levels of vocal attractiveness such that both personality factors predict performance more strongly for people with more attractive voices.     

Predicting DSM-IV Cluster B Personality Disorder Criteria From MMPI-2 and Rorschach Data: A Test of Incremental Validity

Despite their frequent conjoint clinical use, the incremental validity of Rorschach (Rorschach, 1921/1942) and MMPI (Hathaway & McKinley, 1943) data has not been adequately established, nor has any study to date explored the incremental validity of these tests for predicting Diagnostic and Statistical Manual of Mental Disorders (4th ed. [DSM-IV]; American Psychiatric Association, 1994) personality disorders (PDs). In a reanalysis of existing data, we used select Rorschach variables and the MMPI PD scales to predict DSM-IV antisocial, borderline, histrionic, and narcissistic PD criteria in a sample of treatment-seeking outpatients. The correlational findings revealed a limited relation between Rorschach and MMPI-2 (Butcher, Dahlstrom, Graham, Tellegen, & Kaemmer, 1989) variables, with only 5 of 30 correlations reaching significance (p < .05). Hierarchical regression analyses showed that both the MMPI and Rorschach data add incrementally in the prediction of DSM-IV borderline and narcissistic PD total criteria scores. The findings were less clear for the incremental value of Rorschach and MMPI-2 data in predicting the total number of DSM-IV histrionic PD criteria, which were best predicted by Rorschach data, and antisocial PD criteria, which were best predicted by MMPI-2 data. In addition to providing evidence of the incremental validity of Rorschach data, these findings also shed light on the psychological characteristics of the DSM-IV Cluster B PDs.     

Examining the Incremental Validity and Adverse Impact of Cognitive Ability and Conscientiousness on Job Performance

Using archival data from a large home improvement organization, the current study examined the incremental validity of conscientiousness over cognitive ability in predicting different performance criteria. Also, the study examined the effects of conscientiousness and cognitive ability on adverse impact. Results indicated that conscientiousness provided incremental validity over cognitive ability in the prediction of several performance criteria. Specifically, conscientiousness had the highest increases in validity over cognitive ability in the prediction of contextual performance factors (e.g., contextual performance, customer service orientation, dealing with others). However, the addition of conscientiousness failed to ameliorate the adverse impact associated with cognitive ability. In addition, limitations of the study and future directions of research are addressed.     

Generalizability Theory as Construct-Related Evidence of the Validity of Job Performance Ratings

Investigations of the construct-related evidence of the validity of performance ratings have been rare, perhaps because researchers are dissuaded by the con- siderable amount of evidence needed to show construct validity (Landy, 1986). It is argued that generalizability (G) theory (Cronbach, Gleser, Nanda, & Rajaratnam, 1972) is well-suited to investigations of construct-related evi- dence of validity because a single generalizability investigation may provide multiple inferences of validity. G theory permits the researcher to partition observed score variance into universe (true) score variance and multiple, distinct estimates of error variance. G theory was illustrated through the anal- ysis of proficiency ratings of 256 Air Force jet engine mechanics. Mechanics were rated on three different rating forms by themselves, peers, and supervi- sors. Interpretation of G study variance components revealed suitable evi- dence of construct validity. Ratings within sources were reliable. Proficiency ratings showed strong convergence over rating forms, though not over rating sources. Raters showed adequate discriminant validity across rating dimen- sions. The expectation of convergence over sources was further questioned.     

Incremental Validity of Time Urgency and Other Type A Subcomponents in Predicting Behavioral and Health Criteria1

Although previous studies have revealed relationships between Type A behavior pattern (TABP) measures and behavioral indexes of time urgency, such studies utilized global TABP measures that have been criticized for their lack of construct validity. The present study linked recently developed measures of time urgency and other TABP subcomponents (e. g., impatience‐irritability) to observable temporal behaviors and health outcomes. In a sample of 194 undergraduate students, the results indicated that the TABP subcomponents provided incremental validity above the contribution of the global TABP measure in predicting behavioral and health criteria. Applied implications and future directions for research on time urgency and other TABP subcomponents are discussed.     

The Validity of Conscientiousness for Predicting Job Performance: A meta‐analytic test of two hypotheses

This study examined two hypotheses regarding the moderating effects of job characteristics on the validity of personality. Using meta‐analytic techniques, the authors explored the extent to which the structural characteristics and cognitive ability requirements of jobs influence the role of conscientiousness in predicting performance. The results suggest that conscientiousness is a stronger predictor of performance in jobs that are highly routinized, and a weaker predictor of performance in jobs with high levels of cognitive ability requirements. Implications for theory and future research are discussed.     

Predicting your own and others' thoughts and feelings: More like a stranger than a friend

Subjects made predictions about the likely cognitions or affective reactions that they, a well-known other, and a prototypic other, would have to ink-blot and pictorial stimuli. Their responses were categorized using Karniol's (1986) 10 transformation rules. No differences between the targets of prediction were found on any of the measures when ink-blot stimuli were used. For pictorial stimuli, a greater variety of transformation rules was used to make predictions about a well-known other than about self, and a greater variety of transformation rules was used for self than for a prototypic other. Again, for pictorial stimuli, more predictions about a well-known other were made using associations, transformation rules that represent personalized knowledge than about self, and more predictions using associations were made about self than about a prototypic other. Finally, the specific transformation rules used to make predictions about self and a prototypic other were more often the same than were the rules used to make predictions about self and a well-known other. The implications of the findings for the way procedural and declarative knowledge are represented are discussed within the context of the transformation rule model. © 1997 by John Wiley & Sons, Ltd.     

Incremental Validity of Leaderless Group Discussion Ratings Over and Above General Mental Ability and Personality in Predicting Promotion

Leaderless group discussions (LGDs) constitute one of the oldest assessment center exercises. In recent times, their added value has sometimes been questioned in light of trends to streamline assessment centers. The purpose of the present study is to examine the incremental validity of LGD ratings over cognitive ability scores and personality ratings for the prediction of extrinsic career success (i.e., promotion speed and number of promotions). We investigated this issue in the context of the promotion of French naval officers (N = 93) in an academy for high‐level executive positions over a 10‐year period. Results indicated that LGD ratings accounted for incremental variance in the prediction of promotion criterion measures, beyond cognitive ability and personality test scores. These results confirm that LGD ratings provide a unique contribution to the prediction of extrinsic career success in high‐level executive positions.     

Effects of Nonlinearity and Heteroscedasticity on the Validity of Conscientiousness in Predicting Overall Job Performance

Most research on the relationship between personality and overall job performance assumes linearity and homoscedasticity. This study investigated the prevalence and nature of nonlinearities and heteroscedasticies in relationships between conscientiousness and supervisory ratings of overall job performance across five independent samples using both concurrent (k = 4) and predictive (k = 1) designs. Hierarchical polynomial regression analyses found evidence of robust linear effects but no evidence of statistically significant quadratic or cubic effects. A statistically and practically significant heteroscedastic effect was found in only one sample such that lower errors of prediction were evidenced in the ends in comparison to the middle of the bivariate distribution. Implications for the use of conscientiousness in personnel selection are suggested. Limitations of the current study and directions for future research are noted.