The predictive validity of cognitive ability tests: A UK meta‐analysis

A meta‐analysis on the validity of tests of general mental ability (GMA) and specific cognitive abilities for predicting job performance and training success in the UK was conducted. An extensive literature search resulted in a database of 283 independent samples with job performance as the criterion (N=13,262), and 223 with training success as the criterion (N=75,311). Primary studies were also coded by occupational group, resulting in seven main groups (clerical, engineer, professional, driver, operator, manager, and sales), and by type of specific ability test (verbal, numerical, perceptual, and spatial). Results indicate that GMA and specific ability tests are valid predictors of both job performance and training success, with operational validities in the magnitude of .5–.6. Minor differences between these UK findings and previous US meta‐analyses are reported. As expected, operational validities were moderated by occupational group, with occupational families possessing greater job complexity demonstrating higher operational validities between cognitive tests and job performance and training success. Implications for the practical use of tests of GMA and specific cognitive abilities in the context of UK selection practices are discussed in conclusion.     

Central role of g in military pilot selection

The role of general cognitive ability (g) in the selection of military pilots is discussed. Four seminal issues that threaten the interpretation of the results of ability studies are introduced and examined. A brief history of the use of g in pilot selection is presented, going back to the World War I era. At that time, many countries used tests such as perception and reaction time, later shown to be mostly measures of g. The World War II era brought the age of the multiple aptitude batteries, and with it, the theory of differential abilities. However, most militaries still used highly g-saturated measures. More recently, an awareness of the prominence of g in job performance has led to a series of studies that showed the central role of g in predicting pilot success. In comparative analyses, g was found to be a better predictor of pilot criteria than specific abilities. However, some specific abilities or measures of job knowledge were found to increment the predictiveness of g. Several selection variables that appeared to measure characteristics other than g were found to measure, at least in some part, g. These include psychomotor tests and structured interviews. Finally, speculation on the future of the measurement of g is presented.     

Validity of General Mental Ability for the Prediction of Job Performance and Training Success in Germany: A meta‐analysis1

A meta‐analysis into the operational validity of general mental ability (GMA) measures in Germany is presented. The meta‐analysis addresses the question whether findings of US and European meta‐analyses are generalizable to Germany given the differences in the education systems of these countries. The high level of differentiation in the German educational system is expected to enhance the homogeneity of applicant pools resulting in a low level of variability in predictor scores which reduces the observed GMA–performance relationships. Our analysis is based on 54 independent German articles and unpublished reports. Results indicated an operational validity of ρ=.467 for training success (k=90; N=11,969) and ρ=.534 for job performance (k=9; N=746). Moderator analyses showed that job complexity and the year of publication are relevant moderator variables, with lower job complexity levels and older studies being associated with higher operational validities. Findings suggest that overall German operational validities are comparable with findings in the United States or other European countries. However, for training success operational GMA validities are slightly lower in Germany compared with US or European meta‐analyses.     

Predicting Training and Job Performance for Transit Operators

Although considerable meta‐analytic research has validated the use of cognitive ability tests, structured interviews, and personality tests with training and job performance criteria, few studies have investigated the validity of these measures with transit operators. There are the only two single studies of concurrent validation research specifically with transit operators. This article presents the results of a predictive validation study conducted with transit operator applicants for a large urban transit authority in Canada. Key knowledge, skills, and abilities were determined for the role and used as a basis for the design and choice of predictors and criteria. Four predictors were used in the study: education, cognitive ability, personality assessment, and structured interview. Criteria included training performance (formative and summative), probationary performance, preventable accidents, and lost time injuries. Validation results supported cognitive ability, structured interview, and several personality factors as predictors of training performance, but less so for job performance. The use of formative training ratings greatly augmented the evidence supporting the predictors beyond typical organizational criteria.     

Higher-Order Structure-Of-Intellect Abilities

Using data available from the published reports of factor analyses performed by the Aptitudes Research Project at the University of Southern California, investigations were made into the possibility of demonstrating higher-order factorial abilities latent in those data. Since the policy in those analyses had been to rotate axes orthogonally rather than obliquely, there was no direct information regarding intercorrelations among first-order or basic factors. By combining salient tests for each basic factor, and using their loadings on the obtained factors, correlations were estimated among representatives of those factors. Several methods were devised for factoranalyzing those intercorrelations, applying experimental controls that seemed to be needed, in view of the three-faceted nature of each basic ability. Evidence was found for a large number of second-order abilities and a smaller number of third-order factors, connected with many parts of the structure-of-intellect model, within the limits of available data. In general, the results provide excellent support for the model and for its oblique nature, and for relative discriminability of its categories. Implications for higher-order factorial procedures, for the testing of intelligence, for research, both bivariate and multivariate, and for general psychological theory are suggested.     

Assessment Center Dimensions: Individual differences correlates and meta‐analytic incremental validity

This study provides an investigation of the nomological net for the seven primary assessment center (AC) dimensions identified by Arthur, Day, McNelly, and Eden (Personnel Psychology, 56, 125–154, 2003). In doing so, the authors provide the first robust estimates of the relationships between all primary AC dimensions with cognitive ability and the Big 5 factors of personality. Additionally, intercorrelations between AC dimensions based on sample sizes much larger than those previously available in the meta‐analytic literature are presented. Data were obtained from two large managerial samples (total N=4985). Primary data on AC dimensions, personality, and cognitive ability interrelationships were subsequently integrated with meta‐analytic data to estimate incremental validity for optimally and unit‐weighted AC dimension composites as well as overall AC ratings over psychometric tests of personality and cognitive ability. Results show that unit‐ and optimally weighted composites of construct‐based AC dimensions add incremental validity over tests of personality and cognitive ability, while overall AC ratings (including those obtained using subjective methods of data combination) do not.     

“A joke a day keeps the doctor away?” Meta‐analytical evidence of differential associations of habitual humor styles with mental health

Humor and mental health are interconnected as is evidenced by a large number of studies. However, associations are only small and inconsistent as the operationalization of humor poses a methodological challenge. The Humor Styles Questionnaire (HSQ ) differentiates four humor styles that might be beneficial or harmful to mental health. The aim of the present study was to meta‐analytically aggregate studies using the HSQ to assess the associations of different humor styles with four areas of mental health (self‐esteem, life satisfaction, optimism, depression). An extensive electronic database literature search identified 37 studies that reported correlations between the HSQ scales and the four areas of mental health in 45 independent samples (total N = 12,734). In total, 16 meta‐analyses were conducted. Moderating effects of participant age, sex, and geographic region were examined via subgroup analyses and meta‐regression. Humor styles differed in terms of their associations with mental health. Health‐promoting humor styles were overall positively correlated with mental health (small‐to‐medium effect sizes). Self‐defeating humor was overall negatively correlated with mental health. Aggressive humor was overall unrelated with mental health. Moderator analyses suggested geographic differences (Eastern vs. Western samples) and sex differences for some of these associations. Fostering specific humor styles may be beneficial for mental health. In addition, observing the habitual use of humor styles might help therapists to develop a better understanding of their clients. Differences in the utilization and the correlates of humor styles in Eastern and Western societies, and sex differences, need to be addressed in future research.     

The Association of Cognitive Ability with Right‐wing Ideological Attitudes and Prejudice: A Meta‐analytic Review

The cognitive functioning of individuals with stronger endorsement of right‐wing and prejudiced attitudes has elicited much scholarly interest. Whereas many studies investigated cognitive styles, less attention has been directed towards cognitive ability. Studies investigating the latter topic generally reveal lower cognitive ability to be associated with stronger endorsement of right‐wing ideological attitudes and greater prejudice. However, this relationship has remained widely unrecognized in literature. The present meta‐analyses revealed an average effect size of r = ?.20 [95% confidence interval (95% CI) [?0.23, ?0.17]; based on 67 studies, N = 84 017] for the relationship between cognitive ability and right‐wing ideological attitudes and an average effect size of r = ?.19 (95% CI [?0.23, ?0.16]; based on 23 studies, N = 27 011) for the relationship between cognitive ability and prejudice. Effect sizes did not vary significantly across different cognitive abilities and sample characteristics. The effect strongly depended on the measure used for ideological attitudes and prejudice, with the strongest effect sizes for authoritarianism and ethnocentrism. We conclude that cognitive ability is an important factor in the genesis of ideological attitudes and prejudice and thus should become more central in theorizing and model building. Copyright © 2015 European Association of Personality Psychology     

Executive Attention: An Alternative Perspective on General Mental Ability,Performance, and Subgroup Differences

The validity‐adverse impact tradeoff associated with the relationships among general mental ability (GMA), ethnicity, and employee performance represents one of the most pressing concerns in organizational staffing. We conducted 4 studies with 273 bank employees and 197 university students designed to assess the extent to which executive attention (EA) and GMA predict simulation performance and supervisory ratings of performance. We also assess the extent to which measures of EA and GMA are associated with subgroup differences. Results indicate that, like GMA, EA positively predicts managerial simulation and supervisory ratings of performance. In addition, although reaching statistical significance in only 1 of our 4 studies, EA was generally associated with smaller subgroup differences than GMA, and meta‐analysis across our samples supports this reduced subgroup difference. Moreover, advantages of EA tend to increase as studies move from the laboratory with undergraduate students to a concurrent validation organizational setting with employees. We discuss implications for a theory‐based view of cognitive ability in employee selection and implications for managerial practice.     

Political Skill and Work Outcomes: A Theoretical Extension,Meta‐Analytic Investigation,and Agenda for the Future

This quantitative review explored the political skill construct and its predictive ability across a number of organizational outcomes. First, we extended the Ferris et al. meta‐theoretical framework of political skill. Next, incorporating meta‐analysis, we found political skill is positively related to self‐efficacy, job satisfaction, organizational commitment, work productivity, organizational citizenship behavior (OCB), career success, and personal reputation, and negatively related to physiological strain. Political skill was not significantly related to psychological strain or perceptions of organizational politics. Using meta‐regression and dominance analyses, political skill predicted task performance after controlling for the Big Five personality characteristics and general mental ability. In a test of indirect relationships, our results suggest that personal reputation and self‐efficacy partially mediate the political skill–task performance relationship. Finally, in a post hoc test of political skill dimensions, we found that networking ability, interpersonal influence, and apparent sincerity (but not social astuteness) predicted task performance. Our findings provide a comprehensive assessment of theory and research to date on political skill and extend theoretical foundations to stimulate new inquiry into the operation of this important construct.     

Domain specificity of spatial expertise: the case of video game players

Two experiments examined whether video game expertise transfers to performance on measures of spatial ability. In Experiment 1, skilled Tetris players outperformed non‐Tetris players on mental rotation of shapes that were either identical to or very similar to Tetris shapes, but not on other tests of spatial ability. The pattern of performance on those mental rotation tasks revealed that skilled Tetris Players used the same mental rotation procedures as non‐Tetris players, but when Tetris shapes were used, they executed them more quickly. In Experiment 2, non‐Tetris players who received 12 hours of Tetris‐playing experience did not differ from matched control students in pretest‐to‐posttest gains on tests of spatial ability. However, Tetris‐experienced participants were more likely to use an alternative type of mental rotation for Tetris shapes than were Tetris‐inexperienced participants. The results suggest that spatial expertise is highly domain‐specific and does not transfer broadly to other domains. Copyright © 2002 John Wiley & Sons, Ltd.     

Simultaneous and successive syntheses and intelligence

Recent studies on information processing have developed an alternative model of mental abilities which constitutes an integration of Russian clinical research and North American factor analytic research. This model proposes that information may be processed in a simultaneous synthesis or a successive synthesis. The present study examines variations in these syntheses in three intelligence groups in order to identify the types of information processing characteristic of different intelligence groups. A stratified sampling of verbal and nonverbal IQ was used to select 60 boys of age 9–10 in each of three verbal IQ ranges: 71–90, 91–110, and 111–130. A battery of tests used in previous research was administered to all subjects. Multivariate analyses of variance and discriminant analyses indicated that the IQ groups differed markedly on tests used to identify simultaneous synthesis, most notably Raven's Progressive Matrices, and differed slightly less on tests of successive synthesis. Factor analyses of the test intercorrelations for each group revealed more factorial invariance across the IQ groups than had been suggested by a previous exploratory study. It was concluded that strong individual differences exist in simultaneous and successive syntheses over the major portion of the standardized intelligence range, and no high degree of specialization of information processing was characteristic of an intelligence group.     

Assessing the validity of inferences from scores on the cognitive reflection test

Decision‐making researchers purport that a novel cognitive ability construct, cognitive reflection, explains variance in intuitive thinking processes that traditional mental ability constructs do not. However, researchers have questioned the validity of the primary measure because of poor construct conceptualization and lack of validity studies. Prior studies have not adequately aligned the analytical techniques with the theoretical basis of the construct, dual‐processing theory of reasoning. The present study assessed the validity of inferences drawn from the cognitive reflection test (CRT) scores. We analyzed response processes with an item response tree model, a method that aligns with the dual‐processing theory in order to interpret CRT scores. Findings indicate that the intuitive and reflective factors that the test purportedly measures were indistinguishable. Exploratory, post hoc analyses demonstrate that CRT scores are most likely capturing mental abilities. We suggest that future researchers recognize and distinguish between individual differences in cognitive abilities and cognitive processes.     

Using General Mental Ability and Personality Traits to Predict Job Performance in Three Chilean Organizations

Although the field of personnel selection has amounted around 100 years of research, there has been an overrepresentation of American and Western European samples in these studies. In particular, samples from Latin America have been almost entirely absent from industrial and organizational psychology journals. Thus, it is unknown whether well‐documented findings, such as the prediction of job performance based on general mental ability and conscientiousness, replicate in this region. This research intended to address this gap in the literature with three studies conducted in Chilean organizations, using different research designs, and different operationalizations of predictors and criteria. Results are generally consistent with previous studies, showing that conscientiousness and general mental ability significantly predict job performance in these Chilean samples.     

THE INCREMENTAL VALIDITY OF INTERVIEW SCORES OVER AND ABOVE COGNITIVE ABILITY AND CONSCIENTIOUSNESS SCORES

Recent research has suggested that scores on measures of cognitive ability, measures of Conscientiousness, and interview scores are positively correlated with job performance. There remains, however, a question of incremental validity: To what extent do interviews predict above and beyond cognitive ability and Conscientiousness? This question was addressed in this paper by (a) conducting meta‐analyses of the relationships among cognitive ability, Conscientiousness, and interviews, (b) combining these results with predictive validity results from previous meta‐analyses to form a “meta‐correlation matrix” representing the relationships among cognitive ability, Conscientiousness, interviews, and job performance, and (c) performing 9 hierarchical regressions to examine the incremental validity of 3 levels of structured interviews in best, actual, and worst case scenarios for prediction. Results suggested that interview scores contribute to the prediction of job performance over and above cognitive ability and Conscientiousness to the extent that they are structured, with scores from highly structured interviews contributing substantially to prediction. Directions for future research are discussed.     

Neuropsychological Assessment of Mental Capacity

The assessment of mental capacity to assist legal determinations of competency is potentially a growth area for neuropsychology, although to date neuropsychologists have published relatively little in this area. In this paper a systematic review of methods used to assess capacity is presented, including coverage of specialized tests and interviews used for this purpose. A neuropsychological model for conducting capacity assessments is proposed. This model involves comprehensive assessment of a wide range of cognitive abilities as well as assessment of specific skills and knowledge related to the type of capacity being assessed. The purpose of proposing this model is to stimulate further discussion and debate about the contribution neuropsychologists might make in this area.     

Specific Aptitude Theory Revisited: Is There Incremental Validity for Training Performance?

There has been controversy over the years about whether specific mental abilities increment validity for predicting performance above and beyond the validity for general mental ability (GMA). Despite its appeal, specific aptitude theory has received only sporadic empirical support. Using more exact statistical and measurement methods and a larger data set than previous studies, this study provides further evidence that specific aptitude theory is not tenable with regard to training performance. Across 10 jobs, differential weighting of specific aptitudes and specific aptitude tests were found not to improve the prediction of training performance over the validity of GMA. Implications of this finding for training research and practice are discussed.     

The Relationship between Online Game Experience and Multitasking Ability in a Virtual Environment

Online game playing has become popular entertainment, yet its relationship with individuals' multitasking ability was inconsistent. Types of online game genre so far have not been compared and may be associated with multitasking abilities. This study proposed to explore the relationships between types of online game playing and multitasking ability, using Edinburgh Virtual Errands Test (EVET). One hundred and sixteen participants playing different online game genres, including multiplayer online battle arena (MOBA), other online game playing, and no‐online game playing were compared. Each participant was required to fill in Chen's Internet Addiction Scale and the Internet Usage Questionnaire and perform EVET and working memory tests. The results showed a positive correlation between multitasking ability and working memory. In addition, a positive association was found between MOBA‐type gaming and multitasking abilities measured by EVET. In conclusion, MOBA‐type gaming compared with other game playing is associated with better multitasking abilities in a virtual environment.Copyright © 2017 John Wiley & Sons, Ltd.     

Placing inspection time, reaction time, and perceptual speed in the broader context of cognitive ability: The VPR model in the Lothian Birth Cohort 1936

The idea that information processing speed is related to cognitive ability has a long history. Much evidence has been amassed in its support, with respect to both individual differences in general intelligence and developmental trajectories. Two so-called elementary cognitive tasks, reaction time and inspection time, have been used to compile this evidence, but most studies have used either one or the other. Relations between speed and fluid intelligence have tended to be stronger than those between speed and crystallized intelligence, but studies testing this have confounded verbal abilities with crystallized intelligence and spatial/perceptual abilities with fluid intelligence. Questions have also been raised regarding whether speed contributes directly to general intelligence or to more specific cognitive abilities to which general intelligence also contributes. We used 18 ability and speed measures in the Lothian Birth Cohort 1936, assessed at approximately age 70, to construct alternative versions of the Verbal-Perceptual-Image Rotation (Johnson & Bouchard, 2005a) model of cognitive ability to test different hypotheses regarding these issues. Though differences in the extents to which our models fit the data were relatively small, they suggested that reaction and inspection time tasks were comparable indicators of information processing speed with respect to general intelligence, that verbal and spatial abilities were similarly related to information processing speed, and that spatial, verbal, and perceptual speed abilities were more directly related to information processing speed than was general intelligence. We discuss the theoretical implications of these results.