Large sample validations of three general predictors of pilot training success

This article addresses one of the perennial problems of pilot selection research: obtaining an adequate sample size for reliable statistical analysis of predictive validity. Results from three studies involving the same computerized tests of instrument comprehension and psychomotor ability were combined in a meta-analysis to determine whether the validities of these tests generalized across three contexts. These were Royal Air Force and Turkish Air Force fixed-wing pilot training and British Army Air Corps rotary-wing pilot training. In this article, we discuss the adequacy of samples for estimating the validity of the tests, and the persistence of predictive validity to later stages of training as shown by British Army Air Corps data. Reference is also made to data from a fourth independent study of Qantas pilot training.     

Unmanned Aircraft Systems Selection Practices: Current Research and Future Directions

This review examines evolving personnel selection practices for unmanned aircraft systems/remotely piloted aircraft (UAS/RPA) across the U.S. Air Force, U.S. Naval Services, and U.S. Army. Findings across services revealed several themes, including strengths (e.g., strong predictive validities, consistency in operator profiles), weaknesses (e.g., small number of predictive validation studies, small sample sizes), and gaps (e.g., need for further investigation of noncognitive predictors). Ongoing and proposed research, including development and implementation of new instruments and methodologies, are discussed, followed by suggestions to facilitate enhanced UAS/RPA selection practices across the services.     

A Study of Active Duty Military Personnel With the MMPI-2

The Minnesota Multiphasic Personality Inventory (MMPI) has a long history of application in military settings. Our investigation is an exploration of the relevance and appropriateness of the revised version of this instrument, the MMPI-2, in military settings. The question of whether special norms are needed for use in the military was addressed. Results showed that military personnel (N = 1,156 men from the Army, Navy, Air Force, and Marine Corps) responded, as a group, in a manner similar to the MMPI-2 restandardized sample. Similar mean scores and factor structures were obtained. Age effects were also studied. Consistent with past research, there were clear age differences on the MMPI-2 with younger subjects producing more scale elevation, particularly on the Psychopathic Deviate (Pd), Psychastenia (Pt), Schizophrenia (Sc), and Hypomania (Ma) scales. The results of the study show that special norms for military settings are not needed.     

A stochastic model for free recall

A statistical model for verbal learning is presented and tested against experimental data. The model describes a Markov process with a realizable absorbing state, allowing complete learning on some finite trial as well as imperfect retention prior to this trial.This work was carried out while the author was at Lincoln Laboratory, Massachusetts Institute of Technology.Operated with support from the U. S. Army, Navy, and Air Force.     

Multivariate information transmission

A multivariate analysis based on transmitted information is presented. It is shown that sample transmitted information provides a simple method for measuring and testing association in multi-dimensional contingency tables. Relations with analysis of variance are pointed out, and statistical tests are described.This work was supported in part by the Air Force Human Factors Operations Research Laboratories, and in part jointly by the Army, Navy, and Air Force under contract with the Massachusetts Institute of Technology.Several of the indices and tests discussed in this paper have been developed independently by J. E. Keith Smith (11) at the University of Michigan, and by W. R. Garner at Johns Hopkins University.     

Validation of the Micropat Battery

Micropat is a battery of computer-based aptitude tests designed to assess aspects of psychomotor coordination and information management. Initial research and development work on Micropat started in 1980 with the Army Air Corps and the tests were successfully validated against rotary wing pilot training outcome. In 1985, funding for the project was taken over by the Royal Navy (RN) with a view to improving selection for both RN Pilots and RN Observers. Validation of the main battery was carried out on Pilot and Observer trainees at Britannia Royal Naval College (BRNC), Dartmouth, UK. The paper outlines the development of the Micropat programme over the past decade, focusing on a number of key practical methodological issues. These include discussion of the task-based approach to test construction, the use made of the potential afforded by computer-based assessment and the problem of developing robust composite predictors from small samples. In particular, the relative merits of unit-weighted, rationally weighted and empirically weighted composite predictors are examined. Evidence for the validity of the tests is summarized. It is concluded that we are unlikely to improve substantially on the levels of prediction which were being obtained in the 1950s. Continual development and improvement in selection testing is needed simply to maintain levels of prediction as the demands of flying change. The new forms of test made possible by computer-based assessment technology provide the means of maintaining useful levels of prediction as flying training courses become longer and more complex.     

Computerized Testing in the United States Air Force

Computerized testing in the United States Air Force finds its historical beginnings in apparatus tests developed during World War II. The advent of fast, small, and inexpensive computers led to the establishment of a research program to use computers in pilot selection. Later, computerized tests were demonstrated for more familiar aptitude measures such as verbal and quantitative ability. Several large-scale efforts to implement computerized testing have proved informative. Many Air Force tests are routinely administered by computer. Continued advances in computer technology promise to make computerized testing more common and more commonplace. Computer use in measuring physiological correlates of human ability is discussed.     

Cross Validation of Experimental USAF Pilot Training Performance Models

In a recent study, several experimental computerized tests demonstrated utility for supporting United States Air Force (USAF) pilot candidate selection de- cisions and adding to the predictive validity of the Air Force Officer Qualifying Test (AFOQT), the ability measure currently included in the USAF pilot candidate selection process (Carretta, 1989). Pilot candidates with good hand-eye coordination, who make quick and accurate decisions, and who exhibited a cautious risk-taking style were more likely to complete training successfully. The purpose of this study was to cross validate these results to determine their stability. To perform the cross validation, 885 USAF Undergraduate Pilot Training (UPT) students were assigned randomly to two groups (478 of these students also were used in Carretta, 1989). Pilot candidate selection models were developed independently for each group. The "best fitting" regression weights from each group then were applied to subjects in the other group to determine the stability of the regression solutions. In the model development phase, subjects in both groups who had good hand-eye coordination and who made quick and accurate decisions were more likely to complete pilot training successfully. Although there was some reduc- tion in the validity coefficients in the cross-validation phase, the selection models were related significantly to final training outcome in both groups. These results suggest that the experimental pilot candidate selection models are robust enough to be used as adjuncts to operational USAF pilot candidate selection procedures.     

A revised law of comparative judgment

In contrast to Thurstone's Law of Comparative Judgment, a model in which a comparison pair and its complement are assumed to give rise to two different distributions of differences is considered. The consequences of this revised model on scaling problems is developed.The research in this article was supported jointly by the Army, Navy, and Air Force under contract with the Massachusetts Institute of Technology.     

Prediction of situational awareness in F-15 pilots

Situational awareness (SA) is a skill often deemed essential to pilot performance in both combat and noncombat flying. A study was conducted to determine if SA in U.S. Air Force F-15 pilots could be predicted. The participants were 171 active duty F-15 A/C pilots who completed a test battery representative of various psychological constructs proposed or demonstrated to be valid for the prediction of performance in a wide variety of military and civilian jobs. These predictors encompassed measures of cognitive ability, psychomotor ability, and personality. Supervisor and peer ratings of SA were collected. Supervisors and peers showed substantial agreement on the SA ratings of the pilots. The first unrotated principle component extracted from the supervisor and peer ratings accounted for 92.5% of the variability of ratings. The unrotated first principal component served as the SA criterion. Flying experience measured in number of F-15 hours was the best predictor of SA. After controlling for the effects of F-15 flying hours, the measures of general cognitive ability based on working memory, spatial reasoning, and divided attention were found to be predictive of SA. Psychomotor and personality measures were not predictive. With additional F-15 flying hours it is expected that pilots would improve their ratings of SA.     

Suicides in active-duty enlisted Navy personnel.

Cases of completed suicide among U.S. Navy enlisted personnel occurring from 1974 through 1985 were identified and compared with similar rates in the U.S. general population and in the U.S. Army. Although the Navy suicide rate was the lowest of the three groups, this rate increased between 1976 and 1983. This was in contrast to national and Army trends during the same period. Young white males in the apprentice/recruit and blue-collar occupations had the highest rates of completed suicide in the Navy. Recommendations for prevention and treatment are discussed.     

Chaplains' Engagement with Suicidality among Their Service Users: Findings from the VA/DoD Integrated Mental Health Strategy

Chaplains play an important role in supporting the mental health of current and former military personnel; in this study, the engagement of Department of Veterans Affairs (VA), Army, Navy, and Air Force chaplains with suicidality among their service users were examined. An online survey was used to collect data from 440 VA and 1,723 Department of Defense (DoD) chaplains as part of the VA/DoD Integrated Mental Health Strategy. Differences were noted for demographics, work setting characteristics, encountering suicidality, and self‐perceived preparation for dealing with suicidality. Compared to DoD chaplains, VA chaplains encounter more at‐risk service users, yet feel less prepared for dealing with suicidality.     

Intelligence,Dominance, Masculinity,and Self-Monitoring: Predicting Leadership Emergence in a Military Setting

Organizations today place great emphasis on the selection of their managerial and supervisory personnel. Consequently, the need to develop strong predictors of leadership for use in selection tests is immense. Theoretically and historically, intelligence, dominance, masculinity-femininity, and self-monitoring have been strong indicators of leadership. Accordingly, biographical questionnaires and personality inventories were administered to 1137 Air Force officers attending Squadron Officer School to determine their levels of intelligence, dominance, femininity, masculinity, and self-monitoring. These traits and individual difference variables were then used in a discriminant analysis to predict the subject's classification into one of four leadership categories. Analyses showed differences between leaders and nonleaders for intelligence, dominance, and femininity, but not for self-monitoring. However, a discriminant analysis and associated cross-validation resulted in no predictive capability. A possible reason for the lack of classification capability was the uniqueness of the military sample. Since military individuals seem to share very common experiences and beliefs, the sample appeared to be quite homogeneous, making differences between leaders and nonleaders extremely difficult to detect. Future studies should address this potential problem.     

Suicide in the U.S. Army: epidemiological and periodic aspects

An overview of suicide in the U.S. Army is presented in two sections: (1) the epidemiology of U.S. Army suicides, based on biennium reports, and (2) the temporal aspects of those suicides compared with the data for the United States as a whole. A brief historical review documents some of the changes in contemporary military suicide rates compared to those of the past century. The cycles in the number of suicides by day of the week, day of the month, and the month of the year for the U.S. Army are computed and contrasted with those reported for the U.S. resident population.     

Complex analyses of variance: General problems

Problems in applying the analysis of variance are discussed. Emphasis is placed on using the technique to understand the data. The scale of the dependent variable is important for the analysis. Crossed and nested categories must be recognized. The error terms in the analysis depend on whether the classes of each independent variable are (1) all out of a few or (2) a few out of many. To simplify the analysis, mean squares should be aggregated with their error term when they are less than twice its size. An illustrative example is discussed in detail.Prepared in connection with research sponsored in part by the Office of Naval Research, in part by the Educational Testing Service, and in part by Lincoln Laboratory, a center for research operated by Massachusetts Institute of Technology with the joint support of the U. S. Army, Navy, and Air Force.     

Employment test usage as a predictor of gross domestic product

The relationship between the usage of employment tests—particularly ability tests-and job performance criteria has been extensively studied. The usage of testing was reported to have declined after the landmark personnel selection caseGriggs (1971). Models of selection utility were reviewed, and an ecological systems theory of test incidence and the American economy was developed. For both ability tests and non-personality employment tests, a positive relationship was hypothesized between the historical incidence of firms using tests and American Gross Domestic Product. Using historical survey and economic data, results were consistent with the hypothesis involving ability testing incidence.The views expressed are those of the author and not necessarily those of the U.S. Air Force, the Department of Defense, or the U.S. government.     

On the Treatment of Outliners in Cognitive and Psychomotor Test Data

Authors of many statistical texts and review articles have pointed to the possi- ble adverse effects that outliers can have on the calculation of sample statistics and have suggested several methods for detecting and treating outliers. We investigated two different methods-data censoring and transformation-for treating outliers in aptitude test data at the item level and total-score level and their effects on the internal consistency and predictive validity of six computer- ized tests being evaluated by the U.S. Air Force. Results from our sample of more than 2,000 pilot training candidates indicated that neither outlier treat- ment method at either level of analysis had significant effects on the tests' internal consistencies or predictive validities. Possible reasons for these findings include the frequency with which outliers occur and the robustness of linear modeling methods.     

Mechanisms of Risk and Resilience in Military Families: Theoretical and Empirical Basis of a Family-Focused Resilience Enhancement Program

Recent studies have confirmed that repeated wartime deployment of a parent exacts a toll on military children and families and that the quality and functionality of familial relations is linked to force preservation and readiness. As a result, family-centered care has increasingly become a priority across the military health system. FOCUS (Families OverComing Under Stress), a family-centered, resilience-enhancing program developed by a team at UCLA and Harvard Schools of Medicine, is a primary initiative in this movement. In a large-scale implementation project initiated by the Bureau of Navy Medicine, FOCUS has been delivered to thousands of Navy, Marine, Navy Special Warfare, Army, and Air Force families since 2008. This article describes the theoretical and empirical foundation and rationale for FOCUS, which is rooted in a broad conception of family resilience. We review the literature on family resilience, noting that an important next step in building a clinically useful theory of family resilience is to move beyond developing broad “shopping lists” of risk indicators by proposing specific mechanisms of risk and resilience. Based on the literature, we propose five primary risk mechanisms for military families and common negative “chain reaction” pathways through which they undermine the resilience of families contending with wartime deployments and parental injury. In addition, we propose specific mechanisms that mobilize and enhance resilience in military families and that comprise central features of the FOCUS Program. We describe these resilience-enhancing mechanisms in detail, followed by a discussion of the ways in which evaluation data from the program’s first 2 years of operation supports the proposed model and the specified mechanisms of action.     

Pilot selection in the Norwegian Air Force: a validation and meta-analysis of the test battery

The purpose of this study was to evaluate the test battery currently used for pilot selection to the Norwegian Air Force. Selection is currently based on a standard battery of 20 different psychological tests as well as on medical tests and on an interview by a licensed psychologist. First, two-factor analyses were conducted to examine the relation between the tests in the battery. Then, a correlation study was conducted to evaluate the predictive validity of the tests against two criteria of pilot performance collected during the basic training period. Finally, a small-scale meta-analysis of previous validation studies in Norway was conducted. me best predictors of success in training, based on the meta-analysis, were Instrument Comprehension (mean r = .29), Mechanical Principles (mean r = .23), and Aviation Information (mean r = .22)