Central role of g in military pilot selection

The role of general cognitive ability (g) in the selection of military pilots is discussed. Four seminal issues that threaten the interpretation of the results of ability studies are introduced and examined. A brief history of the use of g in pilot selection is presented, going back to the World War I era. At that time, many countries used tests such as perception and reaction time, later shown to be mostly measures of g. The World War II era brought the age of the multiple aptitude batteries, and with it, the theory of differential abilities. However, most militaries still used highly g-saturated measures. More recently, an awareness of the prominence of g in job performance has led to a series of studies that showed the central role of g in predicting pilot success. In comparative analyses, g was found to be a better predictor of pilot criteria than specific abilities. However, some specific abilities or measures of job knowledge were found to increment the predictiveness of g. Several selection variables that appeared to measure characteristics other than g were found to measure, at least in some part, g. These include psychomotor tests and structured interviews. Finally, speculation on the future of the measurement of g is presented.     

Air force officer qualifying test validity for predicting pilot training performance

The AFOQT was validated for the prediction of pilot training criteria. Subjects were 7,563 men and women selected for pilot training on the basis of educational attainment and AFOQT scores. Criterion variables included daily flight training grades, check flight grades in subsonic and transonic aircraft, and overall academic performance in the 53 week pilot training course. Test validities were presented as observed, corrected for multivariate range restriction, and corrected for multivariate range restriction and unreliability. The Aviation Information and Instrument Comprehension tests, measures of job knowledge, were most predictive of daily and check flights in the initial subsonic jet aircraft. This reflects the relative greater importance of prior job knowledge early in training. The Scale Reading test, a measure of perceptual speed, was most predictive for daily and check flights in the advanced transonic training aircraft. The Arithmetic Reasoning test, a good measure of general cognitive ability, was most predictive of aeronautics in ground school. The development of an improved pilot selection composite is suggested by the results of the validity analyses.The views expressed are those of the authors and not necessarily those of the Department of the Air Force, Department of Defense, or the Government of the United States.     

Large sample validations of three general predictors of pilot training success

This article addresses one of the perennial problems of pilot selection research: obtaining an adequate sample size for reliable statistical analysis of predictive validity. Results from three studies involving the same computerized tests of instrument comprehension and psychomotor ability were combined in a meta-analysis to determine whether the validities of these tests generalized across three contexts. These were Royal Air Force and Turkish Air Force fixed-wing pilot training and British Army Air Corps rotary-wing pilot training. In this article, we discuss the adequacy of samples for estimating the validity of the tests, and the persistence of predictive validity to later stages of training as shown by British Army Air Corps data. Reference is also made to data from a fourth independent study of Qantas pilot training.     

The Correlation of General Cognitive Ability and Psychomotor Tracking Tests

A study was conducted to investigate the nexus of cognitive and psychomotor tests as might be used for personnel selection and assessment. These two domains are frequently seen as independent. A multiple aptitude cognitive test battery and a psychomotor test battery were administered to 354 United States Air Force recruits. The average multiple correlation of the cognitive tests and each psychomotor score as a criterion was 0.34, corrected for range restriction. Confirmatory factor analyses disclosed general cognitive and general psychomotor factors, three lower-order psychomotor factors, and two lower-order cognitive factors. The general cognitive factor accounted for 39% of the variance and the general psychomotor factor accounted for 29% of the variance. Residualized, the lower-order factors accounted for between 10% and 3% of the variance. The average g saturations (loadings) of the cognitive and psychomotor tests were 0.82 and 0.34 respectively. An implication for personnel selection is that the incremental validity of psychomotor tracking tests beyond the validity of cognitive tests will be small due to the commonality of measurement. A further implication of findings is the need to study the validity of the general and specific psychomotor factors.     

Predicting Military Flight Training Success by a Compensatory Tracking Task

This study evaluated the validity of a compensatory tracking task for pre- dicting success in naval flight training. Its purpose was to determine if psychomotor test scores would add unique variance to the current aviation selection model. Aviation candidates (N = 187) admitted on the basis of current selection test scores were administered a 20-min computer-based tracking task before beginning flight training. Performance on the task was compared with flight training outcomes as measured by passlfailure and cu- mulative flight grade. The results indicated that the tracking task scores differentiated candidates who completed training from those who failed. Multiple regression analyses confirmed that the tracking task scores added unique variance to the current selection model.     

Organization and components of psychomotor ability

We investigated the organization and components of psychomotor abilities by administering a diverse set of cognitive and psychomotor tasks to a group of recent high school graduates (N = 161). Confirmatory factor analyses identified two psychomotor factors: a general factor associated with all psychomotor tests and an orthogonal psychomotor learning factor associated exclusively with practiced psychomotor tests. Path analyses suggested that the general psychomotor factor could be largely accounted for by two cognitive factors, general working-memory capacity (r = .67) and an orthogonal time estimation factor (r = .32). Most of the psychomotor learning factor variance was unique, but psychomotor learning was somewhat related to processing speed (r = .49). We conclude that initial psychomotor performance is constrained by working-memory limits and the ability to keep track of time. Practiced psychomotor skill is additionally limited by processing speed, consistent with the literature on the development of process automaticity. The Discussion addresses (a) the small dimensionality of the psychomotor abilities space, (b) our discovery of the importance of time estimation and other cognitive factors in psychomotor learning, and (c) the changing nature of psychomotor skill with practice.     

Specificity in interest measurement: Basic Interest Scales and major field of study

The Basic Interest Scales (BISs) of the Strong Interest Inventory (SII; Harmon, Hansen, Borgen, & Hammer, 1994) have a 35-year history. The BISs are the specific content scales of the SII, as opposed to the SII general content scales, the General Occupational Themes (GOTs), which measure the six Holland (1997) RIASEC themes. Using 17,074 participants from the 1994 SII national norm group who majored in 24 different areas, this study examined the incremental value of the specific BISs over the general GOTs in explaining major field of education or training. Multivariately, discriminant function analyses were used to compare the ability of the set of the six GOTs and the set of the 25 BISs to predict the 24 major fields. Results showed substantial increases in explanatory power when the BISs were added to the GOTs. For 22 of the 24 fields, the addition of BISs to the GOTs added significantly to prediction of the major. These results showed that specific interest scales, such as the BISs, are highly useful in explaining specific criteria, such as choice of major field of education or training.     

DYNAMIC CRITERIA REVISITED: A LONGITUDINAL STUDY OF PERFORMANCE STABILITY AND PREDICTIVE VALIDITY

The concept of dynamic criteria has been the subject of a recent debate regarding both the definition and prevalence of the phenomenon (Austin, Humphreys, & Hulin, 1989; Barrett & Alexander, 1989; Barrett, Caldwell, & Alexander, 1985). The present paper questions the adequacy of the conceptual framework underlying the debate and provides data supporting a refined concept of dynamic criteria. The incidence and possible causes of change in relative performance were investigated using weekly performance data from 509 sewing machine operators. Analyses were conducted to determine the degree of performance consistency, potential moderators of consistency, and the stability of predictor-criteria relationships using multiple predictors and criteria. Results revealed a steady decline in performance stability coefficients as the interval between measures increased. This decay was evident regardless of employees' prior job experience, cognitive ability, or psychomotor ability. Analyses of predictive validity coefficients revealed temporal changes in validity for both objective and subjective criteria, but not in the expected direction. The validity of cognitive ability increased, the validity of psychomotor ability was stable, and that of prior job experience decreased over time. Implications for theory and research are discussed.     

General and Specific Cognitive and Psychomotor Abilities in Personnel Selection: The Prediction of Training and Job Performance

Use of ability tests in personnel selection is addressed beginning with methodological issues. Studies are reviewed that show that almost all the validity of cognitive tests comes from general cognitive ability, g. Psychomotor ability is reviewed and found to have both higher- and lower-order factors, contrary to long-held beliefs. It was found that the higher-order general psychomotor factor was one source of validity of psychomotor tests. Additionally, psychomotor tests were shown to contain measures of g and to increment the validity of g-based measures very little.     

Psychological measures as predictors of pilot performance: a meta-analysis

The purposes of this study were to review the validity evidence for psychological measures used in pilot selection and to detect possible moderators for the relationship between predictors and pilot performance. A total of 66 independent samples from 50 studies were located and included in the meta-analysis. The best predictors of pilot performance were previous training experience (.30) and combined indexes, a combination of several cognitive and/or psychomotor tests (.37). The next best predictors were tests measuring cognitive (.24) and psychomotor/information-processing abilities (.24), as well as aviation information (.24) and biographical inventories (.23). The personality, intelligence, and academic tests yielded lowest mean validities (.14, .16, and .15, respectively). The analysis indicated that moderators might be operating for all test categories except academics, and the effect of several moderators were examined.     

Review of psychomotor skills in pilot selection research of the U.S. military services

This review provides an historical perspective of the use of psychomotor, perceptual--cognitive paper-and-pencil, and automated tests for the selection of pilot trainees by the U.S. military services. Automated versions of vintage psychomotor tests (developed in the 1930s and 1940s) seem to be as predictive of military pilot/aviator performance today as in the past. The psychomotor tests receiving the most attention today are the Complex Coordination and Two-Hand Coordination tests originally developed by Mashburn and colleagues [correction of colleges] before World War II (Mashburn, 1934). These tests were significant predictors of Air Force and Navy pass--fail criteria in the past, and automated versions are similarly predictive today. The U.S. Army and Air Force are now using a combination of paper-and-pencil and automated psychomotor--cognitive tests for initial selection (Air Force) or helicopter assignment (Army). It appears that the Navy is considering the use of automated cognitive and psychomotor tests in a selection battery of the future.     

Learner-centered versus automatic adaptive motor skill training

Three types of training (fixed-difficulty, automatic-adaptive, and learner-centered) were used to teach 18 male and 18 female students a two-dimensional pursuit-tracking task. A 7-min tracking session, in which task difficulty shifted each minute, was used to measure transfer. Although training type did not result in differences in training time, students trained under learner-centered procedures had less tracking error during transfer. Females required on the average twice as much training as mates. During transfer no sex differences were noted. The differences in training time for males and females may reflect previous experience with similar motor-control tasks.     

More than g: Evidence for the Incremental Validity of Performance-Based Assessments for Predicting Training Performance

Past research has consistently shown that tests measuring specific cognitive abilities provide little if any incremental validity over tests of general mental ability when predicting performance on the job. In this study, we suggest that the seeming lack of incremental validity may have been due to the type of content that has traditionally been assessed. Therefore, we hypothesised that incremental validity can be obtained using specific cognitive abilities that are less highly correlated with g and are matched to the tasks performed on the job. To test this, we examined a recently developed performance-based measure that assesses a number of cognitive abilities related to training performance. In a sample of 310 US Navy student pilots, results indicated that performance-based scores added sizeable incremental validity to a measure of g. The significant increases in R² ranged from .08 to .10 across criteria. Similar results were obtained after correcting correlations for range restriction, though the magnitude of incremental validity was slightly smaller (ΔR² ranged from .05 to .07).     

Cross Validation of Experimental USAF Pilot Training Performance Models

In a recent study, several experimental computerized tests demonstrated utility for supporting United States Air Force (USAF) pilot candidate selection de- cisions and adding to the predictive validity of the Air Force Officer Qualifying Test (AFOQT), the ability measure currently included in the USAF pilot candidate selection process (Carretta, 1989). Pilot candidates with good hand-eye coordination, who make quick and accurate decisions, and who exhibited a cautious risk-taking style were more likely to complete training successfully. The purpose of this study was to cross validate these results to determine their stability. To perform the cross validation, 885 USAF Undergraduate Pilot Training (UPT) students were assigned randomly to two groups (478 of these students also were used in Carretta, 1989). Pilot candidate selection models were developed independently for each group. The "best fitting" regression weights from each group then were applied to subjects in the other group to determine the stability of the regression solutions. In the model development phase, subjects in both groups who had good hand-eye coordination and who made quick and accurate decisions were more likely to complete pilot training successfully. Although there was some reduc- tion in the validity coefficients in the cross-validation phase, the selection models were related significantly to final training outcome in both groups. These results suggest that the experimental pilot candidate selection models are robust enough to be used as adjuncts to operational USAF pilot candidate selection procedures.     

A speeded coding task using a computer-based mouse response

This study assessed whether a speeded coding task that used a computer-based mouse response (CBMR) format was a measure of general processing speed (Gs). By analyzing the task within a network of tasks representing both traditional Gs tests and reaction time tasks, it was shown that a CBMR test can be used to measure the same construct as traditional paper-and-pencil (PP) tests and that this response format does not introduce variance associated with psychomotor performance. Differences between PP and CBMR formats were observed, and it is argued that these may provide information on individual differences in performance not available from traditional coding tests.     

Menstrual cycle effects on task performance examined in the context of stress research

The study examined the effects of degree of paramenstrual distress, signal frequency, task load and time on task on psychomotor performance across the menstrual cycle. Simple reaction time, choice reaction time and pursuit tracking were the tasks employed, and performance was assessed using the metrics of mean level, variability, and errors of omission and commission. The results showed performance as a function of menstrual cycle phase and paramenstrual distress to be highly task and metric specific, a finding typical of stressor variables: pursuit tracking was affected by phase, with mean performance at its worst premenstrually; choice reaction time was dependent on a complex interaction of experimental variables and measurement metrics; simple reaction time showed no significant effects. Patterns of attention allocation in a time-sharing situation combining both pursuit tracking and choice reaction time components were also investigated. Trends here indicated relatively better performance both menstrually and intermenstrually as components increased in signal/response frequency, with a corresponding deterioration premenstrually. Overall performance capacity appeared to be greatest at menstruation, and the degree of attentional selectivity was least. High distress women performed better on low frequency components, whereas asymptomatic women did better at higher signal rates.     

Degradation of validity over time: a test and extension of Ackerman's model

The dynamic nature of the predictor-criterion relationship has long been a concern in psychology, especially with regard to the deterioration of validity over time. The authors examine P. L. Ackerman's ( 1987, 1988) hypothesized relationships between different types of predictors and criteria over time using data from previous longitudinal studies. Expert ratings categorized predictors and criteria according to P. L. Ackerman's model. Regression results support the predicted negative curvilinear relationship between cognitive ability and consistent and inconsistent task performance but do not support the predicted relationships between perceptual speed ability and psychomotor ability and consistent and inconsistent task performance. Deterioration of validity was more ubiquitous than has been suggested previously, and the pervasive form of deterioration was cubic with a negative trend. Findings are discussed in the context of catastrophe-chaos models.     

Development of a portable psychometric testing device for use in the field: an alcohol investigation

Cognitive and psychomotor performance have traditionally been assessed in the laboratory. There is a need for an objective portable assessment tool to assess cognitive and psychomotor performance. This study investigated the viability of a portable psychometric test battery, in a controlled laboratory environment, possibly leading to use in the field. A randomised, double-blind placebo controlled, three-way crossover design was employed. 16 subjects received 50 mg/100 ml and 80 mg/100 ml of alcohol and alcohol placebo. Performance was assessed with a tracking task, and an attention task presented on a small ruggedised handheld computer. The attention task showed no significant training effects; however, an element of the tracking task did. Statistical significance, effect size, and test-retest reliability analyses are presented indicating sensitivity of the portable psychometric test battery to the impairing effects of two separate doses of alcohol. Ability to undertake wide-scale impairment testing in the field with meaningful results in the absence of baseline data collection may have wide reaching implications, particularly in relation to the assessment of drivers impaired by drug use.