Validities of Unit-Weighted Composites of the ASVAB and the ECAT Battery

Currently, assignment to training in each entry-level enlisted job in the U.S. military is influenced by examinees’ scores on the classification composite for that job from the Armed Services Vocational Aptitude Battery (ASVAB). For a cross-service sample of the 18 occupations in the Enhanced Computer-Administered Test (ECAT) project, enhancements of their respective operational ASVAB composites (9 unique ones) are evaluated here. Analyses are reported on the gains in validity from adding 1 test with unit weight to each composite, the addition coming either from the other ASVAB tests or from the 9 tests of the ECAT battery. In more than half the schools, the best ASVAB additions were Coding Speed or Mechanical Comprehension. In the ECAT battery, 1 test of psychomotor ability (Two-Hand Tracking) and 2 tests of working memory (Mental Counters and Sequential Memory) were the most promising additions. The utility of the new composites for person–job matching is discussed.     

An Examination of Five Indexes of Test Battery Performance: Analysis of the ECAT Battery

This investigation evaluated potential revisions to the Armed Services Vocational Aptitude Battery (ASVAB). The data analyzed were collected from trainees in 17 U.S. Air Force, Army, and Navy jobs as part of the Joint-Services Enhanced Computer-Administered Test (ECAT) battery validation study. Predictors included the trainees’ preenlistment scores for the 10 tests in the current ASVAB, plus the 9 experimental ECAT battery tests. The criteria were measures of training performance. All possible combinations of tests that (a) included the Word Knowledge and Arithmetic Reasoning tests of the ASVAB and (b) could be administered in a 134- to 164-min interval were evaluated with respect to 5 indexes of test battery performance: criterion-related validity, classification efficiency, and 3 types of subgroup differences (White vs. Black, White vs. Hispanic, and male vs. female). The 5 indexes were calculated for each of the 16,437 possible combinations of tests. The standard deviations of the indexes across the combinations of tests showed that (a) values on the validity index varied little, (b) values on the classification efficiency and White versus Black and White versus Hispanic subgroup differences indexes varied moderately, and (c) values on the male versus female difference index varied substantially. The validity index of the combinations showed a moderate correlation with the classification efficiency index and a nearly zero correlation with subgroup differences. However, the classification efficiency index showed a small-to-moderate positive correlation with the subgroup difference indexes. The subgroup difference indexes showed moderate-to-high positive correlations with one another. Examinations of the top 20 combinations of tests identified by each index demonstrated that tests that optimize one type of index usually do not optimize each of the other indexes. In particular, trade-offs were observed between (a) the maximization of validity (and classification efficiency) versus the minimization of all 3 types of subgroup differences and (b) the minimization of differences between Whites and Blacks (or between Whites and Hispanics) versus the minimization of differences between men and women. These results suggest that no combination of the tests considered in this investigation simultaneously optimizes all 5 test battery performance indexes.     

Individual and group differences in intelligence and speed of information processing

A battery of eight different reaction time (RT) tests, measuring the speed with which individuals perform various elementary cognitive processes, and a group test of scholastic aptitude (the Armed Services Vocational Aptitude Battery, ASVAB) were given to 50 black and 56 white male vocational college students. The regression of the general factor scores of the ASVAB on the RT measures yielded a shrunken multiple correlation of 0.465. Although discriminant analyses, when applied separately to the ASVAB subtests and to the RT variables, showed highly comparable overall discrimination (over 70% correct classification) between the black and white groups, factor scores derived from the general factor (labeled ‘speed of information processing’) of the RT battery show only about one-third as large a mean black-white difference as the mean group difference on the general factor scores derived from the ASVAB. Comparisons were also made between the 106 vocational college students and 100 university students of higher average academic aptitude who had previously been tested on the same RT battery (Vernon, 1983a). These groups showed marked differences on the RT variables, the largest differences occuring on the tests that required more complex cognitive processing. The more complex RT tests also correlate most highly with the psychometric measures of ability within each group. The results are consistent with the hypothesis that individual differences and the mean differences between groups in psychometric abilities and scholastic achievement are related to differences in the speed of information processing as measured in elementary cognitive tasks.     

Development of a Cyber/Information Technology Knowledge Test for Military Enlisted Technical Training Qualification

An Armed Services Vocational Aptitude Battery (ASVAB) Review Panel, with expertise in personnel selection, job classification, psychometrics, and cognitive psychology developed recommendations for changes to the military enlistment test battery. One recommendation was to develop and evaluate a test of cyber/information and communications technology literacy to supplement current ASVAB content. This article summarizes a multiphased Cyber Test development process: (a) a review of information/computer technology literacy definitions and measures, (b) development and pilot testing of a cyber knowledge measure, (c) validation of test scores against final school grades (FSGs) for selected technical training courses, (d) development of an operational reporting metric and subgroup norms, and (e) examination of construct validity. Results indicate the Cyber Test has predictive validity versus technical training school grades and incremental validity comparable to the ASVAB technical knowledge tests when used with the ASVAB Armed Forces Qualification Test (AFQT) verbal/math composite as a baseline.     

Evaluation of Tests of Perceptual Speed/Accuracy and Spatial Ability for Use in Military Occupational Classification

With the exception of Assembling Objects (AO), a spatial ability test used only by the Navy in enlisted occupational classification, the Armed Services Vocational Aptitude Battery (ASVAB) is academic and knowledge-based, somewhat limiting its utility for occupational classification. This article presents the case for integrating the AO test into military classification composites and for expanding the breadth of ASVAB content by including a former ASVAB speed/accuracy test, Coding Speed (CS). Empirical evidence is presented that shows AO and CS (a) increment the validity of the ASVAB in predicting training grades for a broad array of occupations, (b) reduce adverse impact defined as test score barriers for women and minorities, and (c) improve classification in terms of matching recruits to occupations. Some cognitive theory is presented to support AO and CS, as well as nonverbal reasoning and working memory tests for inclusion in or adjuncts to the ASVAB.     

Alternative Techniques for Predicting Success in Air Controller School

The Armed Services Vocational Aptitude Battery (ASVAB) is the principal cognitive test battery used for military classification-that is, for assignment of individuals to specific job categories. Successful entrance into a specific job category requires satisfactory completion of relevant training. Therefore, it is necessary that the ASVAB accurately predict the likelihood that individuals will complete training. Presently, the U.S. Navy classifies enlisted personnel based on a unit-weighted linear combination of scores derived from a subset of tests currently composing the ASVAB test battery. This research explored the development of new classification models estimated by using linear regression, logistic regression, discriminant analysis, and artificial neural networks, and it compared their classification accuracy with models developed using unit- weighted regression. Models were estimated and cross-validated using data from individuals admitted into the Navy's Air Controlman training during fiscal years 1988 and 1989. Accuracy was measured by the proportion of individuals correctly classified as graduates or as academic failures and by the ability to rank students-that is, to predict the highest scores for graduates and the lowest scores for attrites.     

The Armed Services Vocational Aptitude Battery—A Test Battery in Transition

The widely used ASVAB is reviewed from the standpoint of career guidance in grades 9–12. The battery measures general ability, trade-technical experience, a spatial factor, and a clerical factor unlike traditional measures of clerical aptitude. The version used during the 1977–78 school year had serious limitations. ASVAB changes annually; if subsequent versions have similar limitations, the student drawing conclusions from the report form without the help of an informed counselor will likely draw incorrect conclusions about his or her vocational potential. Established procedures for test review are inadequate for generating an informed opinion regarding a test that is heavily promoted and that changes from year to year.     

Measuring Cognitive Aptitude Using Unobtrusive Knowledge Tests: A New Survey Technology

Five knowledge tests and one implicit-reasoning task were developed to be: (1) exceptionally short, (2) correlated with general cognitive aptitude, (3) unobtrusive, i.e., appear similar to attitudinal survey items as opposed to maximal performance measures, and (4) without formally “correct” answers. The intent was to design scales that could be administered in non-proctored environments to directly measure general cognitive aptitude while avoiding the possibility that participants could use references to provide “good” answers. The five knowledge tests used a Likert format to assess knowledge in verbal and practical domains, and were scored by computing distances between examinee and reference ratings. The implicit-reasoning task appeared to be a series completion “game” that required a dichotomous response. The scales were administered to 288 Air Force recruits and were validated against the Armed Services Vocational Aptitude Battery (ASVAB). Individual unobtrusive knowledge scales and ASVAB tests were substantially correlated with sample correlations ranging to .39 and population correlation estimates to .66 after correcting for range restriction. Two sets of factor scores, which were separately derived from the unobtrusive test battery and the ASVAB, were highly correlated in our sample, .54, yielding a population correlation of .80 after correcting for range restriction. This technology is important because few paper- or Internet-based surveys, and virtually no mail-based surveys accurately measure general cognitive aptitude, while many of these surveys address important social issues and commercial questions that could be better understood given an unobtrusive but accurate estimate of general cognitive aptitude.     

Correcting Differences in Answer Sheets for the 1980 Armed Services Vocational Aptitude Battery Reference Population

In the late 1970s, the Department of Defense (DoD) requested that the reference population for the Armed Services Vocational Aptitude Battery (ASVAB), at that time based on a 1944 sample, be changed and updated to reflect the current youth population. In 1980, the data for the new reference group were collected. Data analyses indicate that the new sample's speeded test scores are atypically low; therefore, the sample might be inappropriate as a reference. The problem was traced to the format of an answer sheet used in the 1980 youth population data collection. In our study, we tested the differences between performance on the 1980 youth study answer sheet and that used operationally for the ASVAB. An adjustment was developed to resolve those differences. Data for this study were scores on two ASVAB speeded tests of about 9,500 service applicants at Military Entrance Processing Stations (MEPS). Half of these applicants were tested on the operational ASVAB answer sheet; half were tested with the answer sheet used in the 1980 youth study. The speeded test scores derived from the 1980 youth study answer sheets were then equated to those derived from the operational ASVAB answer sheets. Adjustments based on these equatings resolved the speeded test anomalies observed in the 1980 youth study.     

Pilot selection in the Norwegian Air Force: a validation and meta-analysis of the test battery

The purpose of this study was to evaluate the test battery currently used for pilot selection to the Norwegian Air Force. Selection is currently based on a standard battery of 20 different psychological tests as well as on medical tests and on an interview by a licensed psychologist. First, two-factor analyses were conducted to examine the relation between the tests in the battery. Then, a correlation study was conducted to evaluate the predictive validity of the tests against two criteria of pilot performance collected during the basic training period. Finally, a small-scale meta-analysis of previous validation studies in Norway was conducted. me best predictors of success in training, based on the meta-analysis, were Instrument Comprehension (mean r = .29), Mechanical Principles (mean r = .23), and Aviation Information (mean r = .22)     

Effects of cognitive training on primary mental ability structure

We report results of the first empirical test, as far as we know, of the assumption of structural invariance of latent constructs from pretest to posttest in cognitive training research on the elderly. In all, 401 participants in the Seattle Longitudinal Study, over 62 years old, received a 5-hr test battery at pre- and posttest that included 16 ability tests, marking the five primary abilities of Spatial Orientation, Inductive Reasoning, Numerical Ability, Verbal Ability, and Perceptual Speed. A total of 229 of our subjects received 5 hr of individual training on either Spatial Orientation or Inductive Reasoning. Restricted factor analysis with the LISREL algorithm tested the hypothesis of measurement equivalence across test occasions, separately for the control subjects and for each of the training groups. When ability-specific cognitive training intervenes, no structural change is observed for abilities not subject to intervention. However, slight shifts occurred in the optimal regression weights for the different markers for the training target abilities.     

Relation of Study Factors to Performance in Navy Technical Schools

This study was conducted to determine which study factors lead to success in Navy technical schools and to ascertain whether the effects of study factors vary from one school to another. Study factors refer to all clearly definable elements that may affect student learning and that may be influenced by training. A Study Factors Survey (SFS) was developed and administered to 1,762 students in seven Navy technical schools. Scores on high-failure tests (those failed by 10% or more of the students) were collected for the beginning, middle, and end of each course. Partial correlations between mean test scores and study factor scales were calculated, controlling for ability as measured by the Armed Services Vocational Aptitude Battery (ASVAB). Of 304 possible correlations, 87 were significant. Four study factors-Concentration, Competition, Memorization, and Motivation - had the greatest number of signif- icant correlations with achievement scores. Anxiety and Mastery Beliefs had the next largest number of significant correlations.     

The predictive power of combined neuropsychological measures for attention-deficit/hyperactivity disorder in children.

The present study explores the predictive power of seven neuropsychological assessment tools used in combination in classifying children with attention-deficit/hyperactivity disorder (ADHD). Twenty-one ADHD boys and 22 community control children participated. Group differences were significant on the continuous performance test only; however, battery analysis did increase overall predictive power, which was moderate. This study highlights the difficulty in identifying consistent mean differences on tests of frontal/executive functioning across studies, as well as the need to assess the predictive validity of these tests in classifying children with ADHD. The study suggests that these tests may provide greater predictive validity when used in combination. Inconsistencies in the literature are discussed, with consideration of research methodology, the heterogeneity of the ADHD population, and comorbid diagnoses.     

The Predictive Power of Combined Neuropsychological Measures for Attention-Deficit/Hyperactivity Disorder in Children

The present study explores the predictive power of seven neuropsychological assessment tools used in combination in classifying children with attention-deficit/hyperactivity disorder (ADHD). Twenty-one ADHD boys and 22 community control children participated. Group differences were significant on the continuous performance test only; however, battery analysis did increase overall predictive power, which was moderate. This study highlights the difficulty in identifying consistent mean differences on tests of frontal/executive functioning across studies, as well as the need to assess the predictive validity of these tests in classifying children with ADHD. The study suggests that these tests may provide greater predictive validity when used in combination. Inconsistencies in the literature are discussed, with consideration of research methodology, the heterogeneity of the ADHD population, and comorbid diagnoses.     

Validity of Selection/ Classification Variables for Performance in Three Navy Ratings

This study examined speed of advancement and score on a job knowledge test as job performance criteria in the validation of selection and classification variables for three Navy ratings. Longitudinal samples spanning a 4-year period were drawn for men in radioman, machinist's mate, and hospital corpsman ratings who, within their first enlistment, advanced to E-4 by means of the Navy competitive advancement system. Validity coefficients for Navy enlisted selection and classification variables (Armed Services Vocational Aptitude Battery [ASVAB] and education) were computed for personnel in each rating, subgrouped by entry pay grade. Validity coefficients for the job knowledge test were high by historical standards, and the effects were highest for the subgroups with the highest selection standards. Validity coefficients for the advancement criterion were much lower. Advancement, the criterion that appeared to have the greater face validity, was not as good a measure of job performance as the job knowledge test. The criteria were predicted much better by mental ability than by level of education variables. Personnel who enlisted at E-2 and E-3 pay grades spent substantially more time working at the journeyman performance level during their first enlistment than did personnel who enlisted at E-1.     

Measures of the sheep-goat variable, transliminality, and their correlates

In this study a battery of pencil-and-paper tests was given to 125 first-year psychology students (27% men). This battery included as measures of belief in the paranormal (the so-called "sheep-goat variable") the Australian Sheep-Goat Scale, Tobacyk's Revised Paranormal Belief Scale (comprised of two scales--New Age Philosophy and Traditional Paranormal Beliefs), and the Anomalous Experience Inventory (comprised of five scales: Anomalous/Paranormal Experience, Belief, Ability, Fear, and Drug Use). Also included were the 29-item Transliminality Scale, a 35-item Kundalini Scale, an experimental 13-item Determinism/Free Will scale, and a number of single-question items aimed specifically at transliminality. The results were, first, that virtually all the measures of the sheep-goat variable were intercorrelated with each other (range, .34 to .77), thereby providing support for their convergent validity. Second, scores on the Kundalini Scale and Drug Use correlated significantly with scores on the sheep-goat variable, replicating previous findings. And, finally, many correlates of transliminality were found, again including scores on the Kundalini Scale and Drug Use (prescribed and illicit) as well as certain determinism-related beliefs. Beliefs, experiences, and behaviors associated with transliminality and the Kundalini experience may reflect a desire to escape a negative state of being.     

The Effect of Selection Ratio on Perceptions of the Fairness of a Selection Test Battery

This research examined the effects of the selection ratio on applicants’ perceptions of the fairness of a battery of tests. These fairness perceptions were also examined to assess whether they moderated the validity of the tests. A laboratory study was undertaken to examine the effects of the selection ratio on applicants going through a selection process and to examine the effects of being hired or rejected. Results revealed that the selection ratio did not have an effect on perceptions, but that hired individuals perceived greater distributive fairness. Furthermore, attitudes towards the cognitive ability test moderated the validity of the cognitive ability test, but attitudes toward the personality test did not moderate the validity of the personality test.     

生态化微表情识别测验EMERT的建立——对JACBART微表情识别测验的改进与发展

经典JACBART微表情识别测验只考察平静表情背景下微表情识别,生态效度不高。本研究创建生态化微表情识别测验,考察所有7种基本表情背景下的6种微表情识别特征。结果发现:(1)该测验具有良好的重测信度、校标效度和生态效度,能够稳定有效地测量生态化微表情识别。(2)信效度检验揭示了生态化微表情识别特征。某些生态化微表情识别存在训练效应。生态化微表情与经典微表情或普通表情普遍相关。恐惧、悲伤、厌恶、愤怒微表情背景主效应显著;惊讶和愉快微表情背景主效应不显著,成对比较发现各背景下惊讶/愉快微表情差异不显著,但是与普通表情有广泛的显著差异。用不同背景下微表情识别正确率的标准差定义生态化微表情识别波动,发现生态化微表情识别具有稳定的波动性。     

A test battery to assess intrasensory and intersensory development of young children.

The purpose of the study was to develop a battery of tests for use in evaluation of intra- and intersensory development of young children. A battery of 15 tests (4 visual, 4 auditory, 4 tactile-kinesthetic, and 3 intersensory) was administered to 109 normally developing 6- and 8-year-old and 32 slowly developing or learning disabled children. Interdependence of test items within each intrasensory and the intersensory category was determined; intercorrelations ranged from .00 to .78. Reliability estimates were also determined. Face validity was claimed for each item. The effects of age or developmental level on test performance were established. Based upon the interdependence of the tests, reliability estimates, and the capacity of the tests to discriminate among groups classified according to age or developmental level, a battery of 10 intra- and intersensory tests was proposed. The battery has 3 tests of visual perception-visual memory, dynamic depth perception, and size discrimination; 3 tests of auditory perception-auditory discrimination, auditory memory of related syllables, and auditory sequential memory of numbers; 2 tests of tactile-kinesthetic perception-tactile integration and movement awareness; and 2 tests of intersensory integration-auditory-tactile intergration and auditory-visual integration.     

The Correlation of General Cognitive Ability and Psychomotor Tracking Tests

A study was conducted to investigate the nexus of cognitive and psychomotor tests as might be used for personnel selection and assessment. These two domains are frequently seen as independent. A multiple aptitude cognitive test battery and a psychomotor test battery were administered to 354 United States Air Force recruits. The average multiple correlation of the cognitive tests and each psychomotor score as a criterion was 0.34, corrected for range restriction. Confirmatory factor analyses disclosed general cognitive and general psychomotor factors, three lower-order psychomotor factors, and two lower-order cognitive factors. The general cognitive factor accounted for 39% of the variance and the general psychomotor factor accounted for 29% of the variance. Residualized, the lower-order factors accounted for between 10% and 3% of the variance. The average g saturations (loadings) of the cognitive and psychomotor tests were 0.82 and 0.34 respectively. An implication for personnel selection is that the incremental validity of psychomotor tracking tests beyond the validity of cognitive tests will be small due to the commonality of measurement. A further implication of findings is the need to study the validity of the general and specific psychomotor factors.