IDENTIFYING OPTIMAL PREDICTOR COMPOSITES AND TESTING FOR GENERALIZABILITY ACROSS JOBS AND PERFORMANCE FACTORS

The initial examination of validity generalization in the Army Selection and Classification Project used data from a concurrent validation sample of 4,039 job incumbents drawn from a representative sample of nine jobs. The available data consisted of 24 predictor scores and five job performance factor scores on each individual. The major objectives were to determine (a) the degree of validity generalization across the major components of performance, with the job held constant, and (b) the degree of validity generalization across jobs within each major performance factor. After reducing the predictor set by eliminating variables that added no information, a modified confirmatory analysis was used to test the hypotheses that one equation would fit the data from all performance components and that one equation would fit the data from all jobs, given a particular performance component. The major findings were that different predictor equations were needed for each of the five criterion factors. For generalization across jobs, within each criterion factor, one equation fit the data for four of the five performance components. Different prediction equations were required for the component that reflects proficiency on the technical tasks specific to each job.     

PROJECT A: SPECIFICATION OF THE PREDICTOR DOMAIN AND DEVELOPMENT OF NEW SELECTION/ CLASSIFICATION TESTS

A major goal of the Army Selection and Classification Project was to develop an experimental predictor battery that would best supplement the Armed Forces Vocational Aptitude Battery for making selection and classification decisions for entry-level enlisted personnel. That is, what predictor measures would best serve the needs of all the jobs in an entire selection/classification system? This paper describes the characteristics of the new test battery and the procedures that were used to develop it. The major steps in the procedure were a structured literature search using a standard protocol, an extensive expert judgment study of expected true validities for a population of predictor variables against a population of performance components, fabrication of modularized software and a special response pedestal for computerized measurement of perceptual and psychomotor abilities, evaluations of experimental measures in three iterative pilot tests and one major field test, and a series of reviews by a panel of scientific advisers. The test battery that resulted from this 2 1/2-year development effort is described. The basic psychometric properties of each measure, as determined in a large concurrent validation sample, are also described.     

AN OVERVIEW OF THE ARMY SELECTION AND CLASSIFICATION PROJECT (PROJECT A)

This paper serves as an introduction and background for the remaining papers in this issue, which describe in somewhat more detail the major pieces in the Army's Selection and Classification Project (Project A). The principal objectives, the methods being used, and the basic research design of Project A are outlined. Project A was a 7-year effort conducted by the Army Research Institute and a consortium of three research firms. The overall objective was to generate the instrumentation and validity information data base that are necessary for modeling and developing an organization-wide selection and classification system. Both the scientific and practical implications of the project are noted.     

Does adolescent heavier alcohol use predict young adult aggression and delinquency? Parallel analyses from four Australasian cohort studies

While the association between heavy alcohol consumption and aggression has been well documented, the causal direction of this association, particularly at a population level, is disputed. A number of causal sequences have been proposed. First, that aggression leads to heavy alcohol use. Second, that heavy alcohol use leads to aggression. Third, that the association between alcohol use and aggression is due to confounding by (a) sociodemographic variables or (b) delinquency. We report here data from four Australasian prospective longitudinal studies of adolescents, to assess the temporal sequence of heavy drinking and aggression over the period from adolescence to young adulthood. The four cohort studies provide a total sample of 6,706 persons (Australian Temperament Project, n = 1701; Christchurch Health and Development Study, n = 931; Mater-University of Queensland Study of Pregnancy, n = 2437; Victorian Adolescent Health Cohort Study, n = 1637). We use multinomial logistic regression to determine whether early adolescent aggression predicts subsequent age of onset of heavy episodic drinking (HED), after adjustment for concurrent sociodemographic factors and delinquency. We then consider whether HED predicts subsequent aggression, after adjusting for past aggression, concurrent delinquency, and a range of confounders. There are broadly consistent findings across the four cohort studies. Early aggression strongly predicts subsequent HED. HED predicts later aggression after adjustment for prior aggression and other confounders. Policies that alter population levels of alcohol consumption are likely to impact on levels of aggression in societies where HED linked to aggression is more common.     

ProjectA对我军士兵心理选拔研究的启示

美国军队选拔与分类项目（The Army’s Selection And Classification Project,Project A）是美军20世纪80年代为改进军队人员的选拔、分类和使用,加强军队质量建设而进行的一项大规模的研究工作,其目的是建立一套完整、科学、符合军队建设需要的军队人员选拔与分类方法,形成一整套用于军队人员军队人员选拔与分类的预测工具。本文简要介绍了Project A产生的背景、研究目的、基本实验设计、绩效评价系统建立、预测工具以及预测效度,并对我军开展士兵心理选拔分类以及安置的必要性和可行性做了简要分析,提出了基本构想     

The Variability of Criterion‐Related Validity Estimates Among Interviewers and Interview Panels

The authors examined differences in criterion‐related validity estimates among ratings from individual interviewers and interview panels within a structured interview. Senior non‐commissioned officers (NCOs) in the U.S. Army (N=64) conducted panel interviews with 944 junior NCOs during a concurrent validation project. Analysis of the data revealed considerable variation in interviewer validity coefficients in relation to multiple performance criteria. Results also indicated the importance of adopting a multivariate perspective when evaluating interviewer validity differences in that the amount of variation in validity coefficients differed both by interview dimension and criterion. A similar pattern of findings emerged when analyses were performed on ratings averaged within interview panel. Nonetheless, when meta‐analysis was used to estimate the amount of true variance in interviewer‐ and panel‐level validity coefficients, most or all of the variance for some interview‐criterion combinations appeared to be due to statistical artifacts.     

General Mental Ability and Selection in Private Sector Organizations: A Commentary

This article reviews a number of construct validation issues within the frarne- work of B i g and Barrett's (1989) Inference 8, which states that "the crite- rion measure is an adequate sample from the performance domain" (p. 480). Emphasis is placed on managerial selection, behavior, and criteria, but illus- trative data from Project A are included. The argument is made that more re- search on the construct validity of our criteria must be done if we are to understand what we are predicting in our selection research. Instead of con- tinuing to rely on ratings, we must understand what behaviors lead to what results, and how those behaviors and results relate to our predictors.     

A new approach to assessing optimism: the development of a german version of the positive and negative expectancies questionnaire (paneq-g)

This study describes the construction and validation of a German version of the Positive and Negative Expectancies Scale, PANEQ-G, which is based on the original English version of the PANEQ. The German version showed the same factor structure as the original, comprising the three subscales of Negative Affectivity/Pessimism, Positive Affectivity/Optimism, and Fighting Spirit. The reliability data for the new scale were acceptable, and the factor structure was confirmed by means of a confirmatory factor analysis in an independent sample. As expected, concurrent validation uncovered substantial correlations of the three established scales. Negative Affectivity/Pessimism was related to Neuroticism and Positive Affectivity/Optimism was related to Extraversion. Fighting Spirit, the third scale, was related to Self-Efficacy and Conscientiousness. The PANEQ-G provides a reliable and valid instrument that can be used in conjunction with the original PANEQ for cross-cultural studies on Optimism and Pessimism. It also offers a new third component, Fighting Spirit, which can enrich research in this area.     

A new approach to assessing optimism: the development of a german version of the positive and negative expectancies questionnaire (paneq-g)

This study describes the construction and validation of a German version of the Positive and Negative Expectancies Scale, PANEQ-G, which is based on the original English version of the PANEQ. The German version showed the same factor structure as the original, comprising the three subscales of Negative Affectivity/Pessimism, Positive Affectivity/Optimism, and Fighting Spirit. The reliability data for the new scale were acceptable, and the factor structure was confirmed by means of a confirmatory factor analysis in an independent sample. As expected, concurrent validation uncovered substantial correlations of the three established scales. Negative Affectivity/Pessimism was related to Neuroticism and Positive Affectivity/Optimism was related to Extraversion. Fighting Spirit, the third scale, was related to Self-Efficacy and Conscientiousness. The PANEQ-G provides a reliable and valid instrument that can be used in conjunction with the original PANEQ for cross-cultural studies on Optimism and Pessimism. It also offers a new third component, Fighting Spirit, which can enrich research in this area.     

MODELING JOB PERFORMANCE IN A POPULATION OF JOBS

The Army Selection and Classification Project has produced a comprehensive examination of job performance in 19 entry-level Army jobs (Military Occupational Specialties) sampled from the existing population of entry-level positions. Multiple methods of job analysis and criterion measurement were utilized in a subsample of nine jobs to generate over 200 performance indicators, which were then used to assess performance in a combined sample of 9,430 job incumbents. An iterative procedure involving a series of content analyses and principal components analyses was used to develop a basic array of up to 32 criterion scores for each job. This basic set of scores formed the starting point of an attempt to model the latent structure of performance in this population of jobs. After alternative models were proposed for the latent structure, the models were submitted to a goodness-of-fit test via LIS-REL VI. After accounting for two components of method variance, a five-factor solution was judged as the best fit. The implications of the results and the modeling procedure for future personnel research are discussed.     

Incremental Validity of Locus of Control After Controlling for Cognitive Ability and Conscientiousness

This research examined the criterion-related validity of work-specific locus of control in predicting job performance, including incremental validity after controlling for cognitive ability and conscientiousness. Data from a student sample and from a large employee sample were used to evaluate the scale properties of measures of locus of control, conscientiousness, and cognitive ability. Two concurrent criterion-related validation studies were then conducted to evaluate the incremental validity of locus of control. In both validation studies, locus of control demonstrated overall and incremental relationships with performance after controlling for ability and conscientiousness, such that employees with higher internal locus of control performed more effectively than externals.     

Change in goals and values of men and women from early to mature adulthood

Changes in goals and values between early and middle adulthood were investigated using a longitudinal sample of women and their male partners. Measures designed to assess personality changes as described by Gutmann and Jung were coded from open-ended responses to questions about middle-aged subjects' concurrent and recollected goals and values. The findings indicated that both women and men declined in gender-traditional goals and values and increased in individual goals and values. Archival data for women showed that recollections were accurate and that both cohort and personality factors were related to the extent to which individuals' goals and values changed over time.     

Unmanned Aircraft Systems Selection Practices: Current Research and Future Directions

This review examines evolving personnel selection practices for unmanned aircraft systems/remotely piloted aircraft (UAS/RPA) across the U.S. Air Force, U.S. Naval Services, and U.S. Army. Findings across services revealed several themes, including strengths (e.g., strong predictive validities, consistency in operator profiles), weaknesses (e.g., small number of predictive validation studies, small sample sizes), and gaps (e.g., need for further investigation of noncognitive predictors). Ongoing and proposed research, including development and implementation of new instruments and methodologies, are discussed, followed by suggestions to facilitate enhanced UAS/RPA selection practices across the services.     

Development and validation of the Illness Behavior Inventory

The development and validation of a measure of illness behavior are presented. From a large domain sample of illness-related behaviors, a 20-item self-report measure was developed, the Illness Behavior Inventory (IBI). Validation data are presented from a series of independent studies with healthy and chronically ill individuals. Results indicate that the IBI possesses acceptable levels of internal consistency, test-retest reliability, structural reliability, discriminant validity, concurrent validity, and predictive validity. The applicability of the IBI to various areas of inquiry in behavioral medicine research and practice is discussed and directions for future research are outlined.     

DEVELOPMENT OF MULTIPLE JOB PERFORMANCE MEASURES IN A REPRESENTATIVE SAMPLE OF JOBS

The goal of criterion development in Project A was to construct multiple measures of the major components of job performance such that the total performance domain for a representative sample of the population of entry-level enlisted positions in the U.S. Army was covered. These measures were to be used as criteria against which to validate both experimental and existing predictors of job performance. The initial model specified that performance is multidimensional within two major categories of dimensions designated as organization-wide and job specific. The development strategy involved describing the total domain of job content via extensive task analyses and critical incident analyses, generating the critical performance dimensions that constitute it, constructing measures for each dimension, and evaluating each measure using expert judgment and field test data. The specific measures developed consisted of rating scales, tests of job knowledge, hands-on job samples, and archival records. The major steps in the job analyses, content sampling, instrument construction, and instrument evaluation are described, and the final array of criterion measures is presented.     

Conjoint developmental trajectories of young adult alcohol and tobacco use

Developmental and etiological advances have set the stage for considering trajectories of problem behavior across the life course, but little work thus far addresses co-occurring problem behavior trajectories. Although recent work characterizes drinking and smoking trajectories, none has explored the course of concurrent drinking and smoking. Using panel data from the Monitoring the Future Project (N=32,087), the authors applied growth mixture modeling to 4 waves of heavy drinking and smoking in a young-adult sample. The authors extracted a single latent group membership factor from heavy drinking and smoking. Associations between trajectory classes and risk factors were relatively unique to the substance being predicted. The association of smoking with alcohol expectancies and delinquency appeared to exist by virtue of smoking's comorbidity with drinking.     

VALIDATING SYNTHETIC VALIDATION: COMPARING TRADITIONAL AND SYNTHETIC VALIDITY COEFFICIENTS

We describe a unique application of a synthetic validation technique to a selection system development project in a large organization. Job analysis data were collected from 4,725 job incumbents and 619 supervisors, and were used to identify 11 job families and 27 job components. We developed 12 tests to predict performance on these job components and conducted a concurrent validation study to collect test and job component data for 1,926 incumbents. We created a test composite for each job component and then chose a test battery for each job family based on its relevant job components. Synthetic validity coefficients were computed for each test battery and compared to traditional validity coefficients that were computed within job families with large sample sizes. The synthetic validity coefficient was very close to the within-family validity coefficient for most job families and was within the bounds of sampling error for all job families. Validities tended to be highest when predictors were weighted according to the number of job components to which they were relevant and job component criterion measures were unit weighted.     

GROUP HOMOGENEITY, RANGE RESTRICTION, AND RANGE ENHANCEMENT EFFECTS ON CORRELATIONS

Recently, the issue of range enhancement effects on correlations has been raised as a potential criticism of correction for range restriction in validity generalization. While adding to or deleting from a sample can affect both the variance and correlation in the sample, increases or decreases in individual differences in an intact sample resulting from such factors as history, maturation, training, and experience will not necessarily affect correlations. If such changes in individual differences do occur in concurrent validation studies, the most likely result will be an underestimate of the unrestricted validity.     

Lifetime Suicidal Behaviors and Career Characteristics Among U.S. Army Soldiers: Results from the Army Study to Assess Risk and Resilience in Servicemembers (Army STARRS)

The current report presents data on lifetime prevalence of suicide ideation and nonfatal attempts as reported by the large representative sample of U.S. Army soldiers who participated in the Consolidated All‐Army Survey of the Army Study to Assess Risk and Resilience in Servicemembers (N  =  29,982). We also examine associations of key Army career characteristics with these outcomes. Prevalence estimates for lifetime suicide ideation are 12.7% among men and 20.1% among women, and for lifetime suicide attempts are 2.5% and 5.1%, respectively. Retrospective age‐of‐onset reports suggest that 53.4%–70% of these outcomes had preenlistment onsets. Results revealed that, for both men and women, being in the Regular Army, compared with being in the National Guard or Army Reserve, and being in an enlisted rank, compared with being an officer, is associated with increased risk of suicidal behaviors and that this elevated risk is present both before and after joining the Army.     

SCALING PERFORMANCE UTILITY TO ENHANCE SELECTION/CLASSIFICATION DECISIONS

For purposes of selection and classification there are two general reasons for scaling the mean and variance of the utility of performance across jobs. First, if differential utility across jobs does exist, then the payoff from a selection and classification system will be enhanced to the extent that accurate utility values are incorporated in the assignment system. Second, a valid utility metric would permit a more meaningful comparison of the gains achieved by alternative selection and classification procedures. It is argued in this paper that the Army context, and perhaps others, precludes using the dollar metric and estimates of SD_y in dollars. Consequently, Project A conducted a relatively long series of exploratory workshops with Army personnel to (a) define the utility issue, (b) pilot test a wide variety of possible scaling methods, and (c) evaluate the methods that seemed most appropriate. On the basis of exploratory analysis, a combined procedure incorporating both an interval estimation and a ratio estimation method was used to estimate the utility of five different performance levels for each of 276 jobs (MOS) in the enlisted personnel system. The psychometric properties of the resulting scale values are analyzed and discussed.