Cognitive and GMA Testing in the European Community: Issues and Evidence

This article reviews the evidence on General Mental Ability (GMA) and cognitive ability tests in connection with employment in the European Community (EC). Five themes are reviewed: prevalence, applicant reactions, testing standards, criterion validity, and recent advances. The first section shows that GMA and cognitive ability tests are used more often in Europe than in America. The second section, regarding applicant reactions, shows that the studies carried out in the EC have shown that there are no differences between Europe and America. The third section shows that there is an initiative to harmonize the standards used across the European countries. In the fourth section, we report on a validity generalization study carried out using the primary studies conducted in Great Britain and Spain. The results showed that GMA and cognitive tests are valid predictors of job performance and training success in both countries and they generalize validity across occupations. Furthermore, the size of the observed validity estimates is similar to those found in the American meta-analyses. Finally, two recent advances are mentioned: the British "Project B" and the use of virtual reality technology for predicting job performance.     

The predictive validity of cognitive ability tests: A UK meta‐analysis

A meta‐analysis on the validity of tests of general mental ability (GMA) and specific cognitive abilities for predicting job performance and training success in the UK was conducted. An extensive literature search resulted in a database of 283 independent samples with job performance as the criterion (N=13,262), and 223 with training success as the criterion (N=75,311). Primary studies were also coded by occupational group, resulting in seven main groups (clerical, engineer, professional, driver, operator, manager, and sales), and by type of specific ability test (verbal, numerical, perceptual, and spatial). Results indicate that GMA and specific ability tests are valid predictors of both job performance and training success, with operational validities in the magnitude of .5–.6. Minor differences between these UK findings and previous US meta‐analyses are reported. As expected, operational validities were moderated by occupational group, with occupational families possessing greater job complexity demonstrating higher operational validities between cognitive tests and job performance and training success. Implications for the practical use of tests of GMA and specific cognitive abilities in the context of UK selection practices are discussed in conclusion.     

A meta-analytic study of general mental ability validity for different occupations in the European community

A comprehensive meta-analysis of the validity of general mental ability (GMA) measures across 12 occupational categories in the European Community (EC) is presented. GMA measures showed that there is validity generalization and large operational validities for job performance and training success in 11 occupational groups. Results also showed that job complexity moderated the magnitude of the operational validity of GMA tests across three levels of job complexity: low, medium, and high. In general, results were similar to those found in the United States, although the European findings showed a slightly larger magnitude of operational validity in some cases. Theoretical and practical implications of these findings for personnel selection are discussed. ((c) 2003 APA, all rights reserved)     

Validity of General Mental Ability for the Prediction of Job Performance and Training Success in Germany: A meta‐analysis1

A meta‐analysis into the operational validity of general mental ability (GMA) measures in Germany is presented. The meta‐analysis addresses the question whether findings of US and European meta‐analyses are generalizable to Germany given the differences in the education systems of these countries. The high level of differentiation in the German educational system is expected to enhance the homogeneity of applicant pools resulting in a low level of variability in predictor scores which reduces the observed GMA–performance relationships. Our analysis is based on 54 independent German articles and unpublished reports. Results indicated an operational validity of ρ=.467 for training success (k=90; N=11,969) and ρ=.534 for job performance (k=9; N=746). Moderator analyses showed that job complexity and the year of publication are relevant moderator variables, with lower job complexity levels and older studies being associated with higher operational validities. Findings suggest that overall German operational validities are comparable with findings in the United States or other European countries. However, for training success operational GMA validities are slightly lower in Germany compared with US or European meta‐analyses.     

Specific Aptitude Theory Revisited: Is There Incremental Validity for Training Performance?

There has been controversy over the years about whether specific mental abilities increment validity for predicting performance above and beyond the validity for general mental ability (GMA). Despite its appeal, specific aptitude theory has received only sporadic empirical support. Using more exact statistical and measurement methods and a larger data set than previous studies, this study provides further evidence that specific aptitude theory is not tenable with regard to training performance. Across 10 jobs, differential weighting of specific aptitudes and specific aptitude tests were found not to improve the prediction of training performance over the validity of GMA. Implications of this finding for training research and practice are discussed.     

More than g: Evidence for the Incremental Validity of Performance-Based Assessments for Predicting Training Performance

Past research has consistently shown that tests measuring specific cognitive abilities provide little if any incremental validity over tests of general mental ability when predicting performance on the job. In this study, we suggest that the seeming lack of incremental validity may have been due to the type of content that has traditionally been assessed. Therefore, we hypothesised that incremental validity can be obtained using specific cognitive abilities that are less highly correlated with g and are matched to the tasks performed on the job. To test this, we examined a recently developed performance-based measure that assesses a number of cognitive abilities related to training performance. In a sample of 310 US Navy student pilots, results indicated that performance-based scores added sizeable incremental validity to a measure of g. The significant increases in R² ranged from .08 to .10 across criteria. Similar results were obtained after correcting correlations for range restriction, though the magnitude of incremental validity was slightly smaller (ΔR² ranged from .05 to .07).     

The Validity of Job Knowledge Measures

The validity of written job knowledge tests as predictors of job performance and training success was examined. Based on a sample of 363,528 persons and 502 coefficients, meta-analyses were performed to investigate the extent of validity generalization and the effects of two hypothesized moderator variables: content similarity and job complexity. Corrected mean validities were 0.45 for studies predicting job performance and 0.47 for studies of training success. Support was found for both moderators since validities were higher for high complexity jobs and when job-test content similarity was high. The implications of this study for personnel selection and research in the area of work performance are discussed.     

GENERAL MENTAL ABILITY,NARROWER COGNITIVE ABILITIES,AND JOB PERFORMANCE: THE PERSPECTIVE OF THE NESTED-FACTORS MODEL OF COGNITIVE ABILITIES

The nested-factors model is a well-established structural model of cognitive abilities in cognitive ability research but has not yet been used to investigate the role of cognitive abilities in job performance. Core assumptions of the nested-factors model are that a broad general mental ability (GMA) exists besides narrower abilities and that this GMA differs from the narrower cognitive abilities in breadth but not in subordination. The authors of this article propose that a recently emerging statistical technique—relative importance analysis—corresponds to the assumptions of the nested-factors model. To empirically study the implications of using the nested-factors model, the authors applied relative importance analysis to a meta-analytic matrix linking measures of 7 narrower cognitive abilities from an established ability taxonomy (Thurstone's primary mental abilities), GMA, and job performance. Results revealed that GMA accounted for 10.9% to 28.6% of the total variance explained in job performance and that GMA was not consistently the most important predictor. The discussion focuses on potential theoretical, methodological, and practical implications of the nested-factors model for personnel psychology.     

Which Aspects of a Semistructured Interview,Besides Cognitive Ability Tests,Contribute Incrementally to Predicting the Training Success of Air Traffic Controller Trainees?

This study examined whether a semistructured interview is valid in predicting training performance in theoretical and simulation examinations and overall training success; which aspects of the interview show incremental validity over cognitive ability tests; and whether additional variables enhance the prediction from the interview. The sample involved data from 337 trainees of the German Air Navigation Service Provider selected by the German Aerospace Center and 657 (interview) and 6,885 (cognitive ability) candidates being the corresponding reference samples. Findings indicate incremental validity of the interview over the cognitive ability tests. Additional variables contributed only to the prediction of theoretical examination performance. Attention, general motivation, and social competence predicted the criteria best, particularly the theoretical examination performance and overall training success.     

Predicting Training and Job Performance for Transit Operators

Although considerable meta‐analytic research has validated the use of cognitive ability tests, structured interviews, and personality tests with training and job performance criteria, few studies have investigated the validity of these measures with transit operators. There are the only two single studies of concurrent validation research specifically with transit operators. This article presents the results of a predictive validation study conducted with transit operator applicants for a large urban transit authority in Canada. Key knowledge, skills, and abilities were determined for the role and used as a basis for the design and choice of predictors and criteria. Four predictors were used in the study: education, cognitive ability, personality assessment, and structured interview. Criteria included training performance (formative and summative), probationary performance, preventable accidents, and lost time injuries. Validation results supported cognitive ability, structured interview, and several personality factors as predictors of training performance, but less so for job performance. The use of formative training ratings greatly augmented the evidence supporting the predictors beyond typical organizational criteria.     

Generalization of Cognitive and Noncognitive Validities across Personality‐based Job Families

The positive relationship between complexity of work and the validity of general mental ability (GMA) measures across a variety of occupations is well supported by research and provides important practical and theoretical support for cognitive ability measures. However, there is currently no research demonstrating a systematic relationship between the size of the validities of any personality measure and the personality requirements of jobs, thus leaving open to question the predictive and construct validity of personality measures for applicant selection. We compared the validities of two biodata measures – one scored to measure social competence and one to measure GMA – across six job families that varied in social requirements. The validity of the GMA measure generalized at approximately the same level across the job families while the validity of the social competence measure decreased as social requirements diminished.     

Using individual differences to predict job performance: correcting for direct and indirect restriction of range

The present study investigates the relationship between individual differences, indicated by personality (FFM) and general mental ability (GMA), and job performance applying two different methods of correction for range restriction. The results, derived by analyzing meta-analytic correlations, show that the more accurate method of correcting for indirect range restriction increased the operational validity of individual differences in predicting job performance and that this increase primarily was due to general mental ability being a stronger predictor than any of the personality traits. The estimates for single traits can be applied in practice to maximize prediction of job performance. Further, differences in the relative importance of general mental ability in relation to overall personality assessment methods was substantive and the estimates provided enables practitioners to perform a correct utility analysis of their overall selection procedure.     

The Role of Cognitive Ability Tests in Employment Selection

Cognitive ability tests correlate with measures of job performance across many jobs. However, cognitive ability tests produce racial differences that are 3 to 5 times larger than other predictors-such as biodata, personality inventories, and the structured interview-that are valid predictors of job performance. Given that (a) cognitive ability tests can be combined with other predictors such that adverse impact is reduced while overall validity is increased, and (b) alternative predictors with less adverse impact can produce validity coefficients comparable to those obtained with cognitive ability tests alone, sole reliance on cognitive ability tests when alternatives are available is unwarranted.     

Cognitive ability,cognitive aptitudes,job knowledge,and job performance

This paper reviews the hundreds of studies showing that general cognitive ability predicts job performance in all jobs. The first section shows that general cognitive ability predicts supervisor ratings and training success. The second section shows that general cognitive ability predicts objective, rigorously content valid work sample performance with even higher validity. Path analysis shows that much of this predictive power stems from the fact that general cognitive ability predicts job knowledge (r = .80 for civilian jobs) and job knowledge predicts job performance (r = .80). However, cognitive ability predicts performance beyond this value (r = .75 versus r = [.80][.80] = .64) verifying job analyses showing that most major cognitive skills are used in everyday work. The third section of the paper briefly reviews evidence showing that it is general cognitive ability and not specific cognitive aptitudes that predict performance.     

Big Five Personality Dimensions and Job Performance in Army and Civil Occupations: A European Perspective

This article reports three meta-analyses on the relation between the Big Five personality dimensions and job performance using exclusively European samples. Also, the incremental validity of Conscientiousness and Emotional Stability over General Mental Ability is reported. The results show that Conscientiousness (p = .23) and Emotional Stability (p = .23) generalized validity across criteria and civil and military occupations. The results also showed that Conscientiousness and Emotional Stability had incremental validity and added to total validity a percentage around 10% to 11 %. Implications for the international generalizability of validity and the practice of personnel selection are commented on.     

Central role of g in military pilot selection

The role of general cognitive ability (g) in the selection of military pilots is discussed. Four seminal issues that threaten the interpretation of the results of ability studies are introduced and examined. A brief history of the use of g in pilot selection is presented, going back to the World War I era. At that time, many countries used tests such as perception and reaction time, later shown to be mostly measures of g. The World War II era brought the age of the multiple aptitude batteries, and with it, the theory of differential abilities. However, most militaries still used highly g-saturated measures. More recently, an awareness of the prominence of g in job performance has led to a series of studies that showed the central role of g in predicting pilot success. In comparative analyses, g was found to be a better predictor of pilot criteria than specific abilities. However, some specific abilities or measures of job knowledge were found to increment the predictiveness of g. Several selection variables that appeared to measure characteristics other than g were found to measure, at least in some part, g. These include psychomotor tests and structured interviews. Finally, speculation on the future of the measurement of g is presented.     

Understanding subgroup differences with general mental ability tests in employment selection: Exploring socio-cultural factors across inter-generational groups

In employment selection, general mental ability (GMA) tests predict training and job performance but also lead to subgroup differences which in turn can produce adverse impact against minority groups. Although researchers have explored genetic, developmental, and environmental explanations for ethnic group differences, few studies have explored socio-cultural factors comparing immigrant and non-immigrant job applicants. Given that many ethnic job applicants may also be immigrants, understanding these factors can provide insight into GMA test score differences. The purpose of this paper is to explore the impact of individual and socio-cultural factors on GMA test scores with immigrant and non-immigrant bus driver applicants. This is the first study of its kind to our knowledge that has attempted to disentangle the effects of socio-cultural factors from race/ethnicity in the study of subgroup differences. Incorporating these variables between non-visible minority and minority groups accounted for considerable variance in GMA test scores across groups. The implications of focusing on socio-cultural variables to enhance our understanding of subgroup differences are discussed. Our results specifically suggest that practitioners attend to the issue of the intersecting grounds of potential discrimination when using GMA tests in personnel selection.     

The Role of Person Characteristics in Perceptions of the Validity of Cognitive Ability Testing

Individuals vary in how they perceive cognitive ability tests; thus, it is useful for organizations to consider how individual differences influence applicant perceptions of selection tools. The present study examined the influence of implicit theories of ability and locus of control on perceptions of face validity and predictive validity for two cognitive ability tests. Relationships between perceptions and test experience, job‐relevant experience, and job familiarity were also examined. Interactions between implicit theories and self‐assessed performance in predicting perceptions were found, although not of the form hypothesized. Furthermore, job familiarity and prior success in selection contexts were related to perceptions. Finally, sample type interacted with test type to influence perceptions. Implications for selection system design and research on applicant perceptions are discussed.     

Academic performance, career potential, creativity, and job performance: can one construct predict them all?

This meta-analysis addresses the question of whether 1 general cognitive ability measure developed for predicting academic performance is valid for predicting performance in both educational and work domains. The validity of the Miller Analogies Test (MAT; W. S. Miller, 1960) for predicting 18 academic and work-related criteria was examined. MAT correlations with other cognitive tests (e.g., Raven's Matrices [J. C. Raven, 1965]; Graduate Record Examinations) also were meta-analyzed. The results indicate that the abilities measured by the MAT are shared with other cognitive ability instruments and that these abilities are generalizably valid predictors of academic and vocational criteria, as well as evaluations of career potential and creativity. These findings contradict the notion that intelligence at work is wholly different from intelligence at school, extending the voluminous literature that supports the broad importance of general cognitive ability (g).     

A counterintuitive hypothesis about employment interview validity and some supporting evidence

This study found mixed support for the hypothesis that the difference in criterion-related validity between unstructured and structured employment interviews is due solely to the greater reliability of structured interviews. Using data from prior meta-analyses, this hypothesis was tested in 4 data sets by using standard psychometric procedures to remove the effects of measurement error in interview scores from correlations with rated job performance and training performance. In the 1st data set. support was found for this hypothesis. However, in a 2nd data set structured interviews had higher true score correlations with performance ratings, and in 2 other data sets unstructured interviews had higher true score correlations. We also found that averaging across 3 to 4 independent unstructured interviews provides the same level of validity for predicting job performance as a structured interview administered by a single interviewer. Practical and theoretical implications are discussed.