Comparing the validity of structured interviews for managerial‐level employees: Should we look to the past or focus on the future?

The current research investigated questions that persist regarding the criterion‐related and construct validity of situational (SI) versus past‐behaviour (PBI) structured interview formats in predicting the job performance of managers. Analyses of data collected from 157 applicants to managerial positions showed that the PBI format significantly predicted job performance ratings (r = .32, p <.01), whereas the SI format did not (r = .09, ns). Investigation of potential construct differences between the SI and PBI formats showed that the PBI was more highly related to manager‐relevant cognitive ability measures, assessment centre exercises and personality traits, as compared with the SI. Such differences help to explain the predictive validity differences between the SI and PBI observed in current and previous research.     

Criterion‐Related Validity of Dutch Police‐Selection Measures and Differences Between Ethnic Groups

This study investigated the criterion‐related validity of cognitive ability as well as non‐cognitive ability measures and differences between ethnic majority (N=2365) and minority applicants (N=682) in Dutch police officer selection. Findings confirmed the relatively low predictive validity of cognitive ability generally found for police jobs. Previous research reported no differential prediction. The present study, however, found small but systematic evidence for differences in validity for the ethnic majority and minority group of both cognitive and non‐cognitive measures. For the minority group, training performance appeared to be mainly predicted by the cognitive ability test. For the majority group, cognitive ability showed very little predictive power. Non‐cognitive ability variables appeared to be somewhat more predictive in this group.     

General and Specific Cognitive and Psychomotor Abilities in Personnel Selection: The Prediction of Training and Job Performance

Use of ability tests in personnel selection is addressed beginning with methodological issues. Studies are reviewed that show that almost all the validity of cognitive tests comes from general cognitive ability, g. Psychomotor ability is reviewed and found to have both higher- and lower-order factors, contrary to long-held beliefs. It was found that the higher-order general psychomotor factor was one source of validity of psychomotor tests. Additionally, psychomotor tests were shown to contain measures of g and to increment the validity of g-based measures very little.     

Cognitive Ability Tests and the Situational Interview: A Test of Incremental Validity

This study investigated the validity and incremental validity of a situational interview beyond that of a composite measure of cognitive ability. Forty-seven factory service technicians underwent an interview and took four cognitive ability tests. Supervisors rated the performance of these subjects in a concurrent validation study. The interview was found to be a valid predictor of a supervisor rating of performance (r = 0.32, p < 0.05 uncorrected), however, was unable to show incremental validity over ability tests (Incremental R²= 0.05, n.s.). Limitations of the present study and directions for future research are discussed.     

Emotional Intelligence in Selection Contexts: Measurement method,criterion‐related validity,and vulnerability to response distortion

Two studies examined aspects of the validity of self‐report and performance‐based measures of emotional intelligence (EI) relevant to their use in personnel selection. In Study 1, structural equation modeling indicated that a two‐factor model with separate factors for the two types of EI measures fit better than a one‐factor solution. The performance‐based EI factor was more related to cognitive ability (R=.38) than personality (R=.26), whereas the self‐report EI factor was more related to personality (R=.85) than cognitive ability (R=.09). Although the performance‐based EI factor correlated more strongly with job performance (ρ=.24) than did that of the self‐report (ρ=.05), it provided little incremental validity beyond cognitive ability and conscientiousness. In Study 2, participants were asked to complete the measures as if applying for a job, and mean scores were then compared with those of Study 1. Results indicated that self‐report EI measures were more vulnerable to distortion than were the performance‐based measures. Implications for the assessment of EI in personnel selection contexts are discussed.     

Assessment Center Dimensions: Individual differences correlates and meta‐analytic incremental validity

This study provides an investigation of the nomological net for the seven primary assessment center (AC) dimensions identified by Arthur, Day, McNelly, and Eden (Personnel Psychology, 56, 125–154, 2003). In doing so, the authors provide the first robust estimates of the relationships between all primary AC dimensions with cognitive ability and the Big 5 factors of personality. Additionally, intercorrelations between AC dimensions based on sample sizes much larger than those previously available in the meta‐analytic literature are presented. Data were obtained from two large managerial samples (total N=4985). Primary data on AC dimensions, personality, and cognitive ability interrelationships were subsequently integrated with meta‐analytic data to estimate incremental validity for optimally and unit‐weighted AC dimension composites as well as overall AC ratings over psychometric tests of personality and cognitive ability. Results show that unit‐ and optimally weighted composites of construct‐based AC dimensions add incremental validity over tests of personality and cognitive ability, while overall AC ratings (including those obtained using subjective methods of data combination) do not.     

Examining the Criterion Related Validity of the General Ability Measure for Adults: A two sample investigation

Across two employee samples (N=262 and 265) the authors examined the criterion‐related validity of the General Ability Measure for Adults (GAMA), a brief nonverbal measure of cognitive ability. Results suggested that GAMA predicted job performance in both samples (uncorrected rs ranged from .27 to .50). No evidence was found for differential prediction across gender and age. These results highlight the usefulness of nonverbal cognitive ability tests in selection contexts.     

A measure of inspection time in 4‐year‐old children: The Benny Bee IT task

Inspection time (IT) measures speed of information processing without the confounding influence of motor speed. While IT has been found to relate to cognitive abilities in adults and older children, no measure of IT has been validated for use with children younger than 6 years. This study examined the validity of a new measure of IT for preschool children. N=71 4‐year‐old children completed the new IT task and standardized measures of fluid ability, visuospatial ability, and speed of processing. N=50 adults completed the same tasks and, additionally, a standard IT task. Results showed that the new IT task is a stable, reliable measure of IT in 4‐year‐old children. The new task had reasonable concurrent validity with the standard IT task in adults and the relationships between cognitive abilities, particularly general cognitive ability, and IT are sufficiently similar in young children and adults to suggest that the new IT task may be a useful tool for research in populations where IT was previously not measurable.     

More than g: Evidence for the Incremental Validity of Performance-Based Assessments for Predicting Training Performance

Past research has consistently shown that tests measuring specific cognitive abilities provide little if any incremental validity over tests of general mental ability when predicting performance on the job. In this study, we suggest that the seeming lack of incremental validity may have been due to the type of content that has traditionally been assessed. Therefore, we hypothesised that incremental validity can be obtained using specific cognitive abilities that are less highly correlated with g and are matched to the tasks performed on the job. To test this, we examined a recently developed performance-based measure that assesses a number of cognitive abilities related to training performance. In a sample of 310 US Navy student pilots, results indicated that performance-based scores added sizeable incremental validity to a measure of g. The significant increases in R² ranged from .08 to .10 across criteria. Similar results were obtained after correcting correlations for range restriction, though the magnitude of incremental validity was slightly smaller (ΔR² ranged from .05 to .07).     

Cognitive Tests,Constructs, and Content Validity: A commentary on Schmidt ()

This commentary discusses a number of issues that build on Schmidt's (International Journal of Selection and Assessment, 20, 1‐13 (2012)) perspective on content validity and cognitive tests. First, it elaborates on the relationship between the treatment of content validity in various professional standards and government guidelines. Second, it offers a differing perspective on the definition of ‘construct’ than that taken by Schmidt. Third, it elaborates on the settings in which content validity can and cannot be used to support the use of a given test in the cognitive ability domain.     

Expanding the Nexus of Cognitive and Psychomotor Abilities

A study was conducted to expand the nexus of cognitive and psychomotor abilities. A cognitive aptitude battery and a psychomotor battery were administered to 429 military recruits. A confirmatory factor analysis yielded higher-order factors of general cognitive ability (g) and psychomotor/technical knowledge (PM/TK). PM/TK was interpreted as Vernon's (1969) practical factor (k:m). In the joint analysis of these batteries, g and PM/TK each accounted for about 31% of the common variance. No residualized lower-order factor accounted for more than 7% PM/TK influenced a broad range of lower-order psychomotor factors. The first practical implication of these findings is that psychomotor tests are expected to be at least generally interchangeable. A second implication is that the incremental validity of psychomotor tests beyond cognitive tests is expected to be small. These findings should help guide test developers and inform personnel selecting agencies regarding the expected utility of psychomotor tests.     

Individual Differences in Social Dominance Orientation Predict Support for the Use of Cognitive Ability Tests

This study investigates the personality processes involved in the debate surrounding the use of cognitive ability tests in college admissions. In Study 1, 108 undergraduates (M_age = 18.88 years, 60 women, 80 Whites) completed measures of social dominance orientation (SDO), testing self‐efficacy, and attitudes regarding the use of cognitive ability tests in college admissions; SAT/ACT scores were collected from the registrar. Sixty‐seven undergraduates (M_age = 19.06 years, 39 women, 49 Whites) completed the same measures in Study 2, along with measures of endorsement of commonly presented arguments about test use. In Study 3, 321 American adults (M_age = 35.58 years, 180 women, 251 Whites) completed the same measures used in Study 2; half were provided with facts about race and validity issues surrounding cognitive ability tests. Individual differences in SDO significantly predicted support for the use of cognitive ability tests in all samples, after controlling for SAT/ACT scores and test self‐efficacy and also among participants who read facts about cognitive ability tests. Moreover, arguments for and against test use mediated this effect. The present study sheds new light on an old debate by demonstrating that individual differences in beliefs about hierarchy play a key role in attitudes toward cognitive ability test use.     

THE INCREMENTAL VALIDITY OF INTERVIEW SCORES OVER AND ABOVE COGNITIVE ABILITY AND CONSCIENTIOUSNESS SCORES

Recent research has suggested that scores on measures of cognitive ability, measures of Conscientiousness, and interview scores are positively correlated with job performance. There remains, however, a question of incremental validity: To what extent do interviews predict above and beyond cognitive ability and Conscientiousness? This question was addressed in this paper by (a) conducting meta‐analyses of the relationships among cognitive ability, Conscientiousness, and interviews, (b) combining these results with predictive validity results from previous meta‐analyses to form a “meta‐correlation matrix” representing the relationships among cognitive ability, Conscientiousness, interviews, and job performance, and (c) performing 9 hierarchical regressions to examine the incremental validity of 3 levels of structured interviews in best, actual, and worst case scenarios for prediction. Results suggested that interview scores contribute to the prediction of job performance over and above cognitive ability and Conscientiousness to the extent that they are structured, with scores from highly structured interviews contributing substantially to prediction. Directions for future research are discussed.     

Assessing Workplace Emotional Intelligence: Development and Validation of an Ability-based Measure

Existing measures of Emotional Intelligence (EI), defined as the ability to perceive, understand, and manage emotions for productive purposes, have displayed limitations in predicting workplace outcomes, likely in part because they do not target this context. Such considerations led to the development of an ability EI measure with work-related scenarios in which respondents infer the likely emotions (perception) and combinations of emotion (understanding) that would occur to protagonists while rating the effectiveness of ways of responding (management). Study 1 (n = 290 undergraduates) used item-total correlations to select scenarios from a larger pool and Study 2 (n = 578) reduced the measure—termed the NEAT—to 30 scenarios on the basis of structural equation modeling. Study 3 (n = 96) then showed that the NEAT had expected correlations with personality and cognitive ability and Study 4 (n = 85) demonstrated convergent validity with other ability EI measures. Last, study 5 (n = 91) established that the NEAT had predictive validity with respect to job satisfaction, job stress, and job performance. The findings affirm the importance of EI in the workplace in the context of a valid new instrument for assessing relevant skills.     

The combined effects of goal type and cognitive ability on performance

We tested the combined effects of goal type and cognitive ability on task performance using a moderately complex task. Business students (N = 105) worked on a 24 min class scheduling task. The results showed that participants with higher cognitive ability benefited more from the setting of a performance goal as opposed to a learning goal. The reverse pattern was true for participants with lower cognitive ability. Performance goals were more effective for participants with higher cognitive ability vis-à-vis those with lower cognitive ability. The correlation between goal commitment and performance was positive and significant as was the correlation between cognitive ability and performance.     

The moderation of conscientiousness by cognitive ability when predicting workplace safety behavior

Research and industry practice emphasize the usefulness of personality-based assessment, particularly measures of conscientiousness, for predicting workplace rule compliance and safety behavior. However, recent research suggests that it may also be valuable to consider potential moderators of the personality – safety relationship. Accordingly, this study uses a field sample (N = 219) to examine the degree to which cognitive ability moderates conscientiousness when predicting workplace safety behavior. As hypothesized, we found that those individuals with higher levels of cognitive ability were more likely to demonstrate higher safety behavior regardless of level of conscientiousness. In contrast, conscientiousness was a stronger predictor of safety behavior for individuals with lower levels of cognitive ability. Implications for understanding the way cognitive ability and conscientiousness interact are discussed.