An Examination of Five Indexes of Test Battery Performance: Analysis of the ECAT Battery

This investigation evaluated potential revisions to the Armed Services Vocational Aptitude Battery (ASVAB). The data analyzed were collected from trainees in 17 U.S. Air Force, Army, and Navy jobs as part of the Joint-Services Enhanced Computer-Administered Test (ECAT) battery validation study. Predictors included the trainees’ preenlistment scores for the 10 tests in the current ASVAB, plus the 9 experimental ECAT battery tests. The criteria were measures of training performance. All possible combinations of tests that (a) included the Word Knowledge and Arithmetic Reasoning tests of the ASVAB and (b) could be administered in a 134- to 164-min interval were evaluated with respect to 5 indexes of test battery performance: criterion-related validity, classification efficiency, and 3 types of subgroup differences (White vs. Black, White vs. Hispanic, and male vs. female). The 5 indexes were calculated for each of the 16,437 possible combinations of tests. The standard deviations of the indexes across the combinations of tests showed that (a) values on the validity index varied little, (b) values on the classification efficiency and White versus Black and White versus Hispanic subgroup differences indexes varied moderately, and (c) values on the male versus female difference index varied substantially. The validity index of the combinations showed a moderate correlation with the classification efficiency index and a nearly zero correlation with subgroup differences. However, the classification efficiency index showed a small-to-moderate positive correlation with the subgroup difference indexes. The subgroup difference indexes showed moderate-to-high positive correlations with one another. Examinations of the top 20 combinations of tests identified by each index demonstrated that tests that optimize one type of index usually do not optimize each of the other indexes. In particular, trade-offs were observed between (a) the maximization of validity (and classification efficiency) versus the minimization of all 3 types of subgroup differences and (b) the minimization of differences between Whites and Blacks (or between Whites and Hispanics) versus the minimization of differences between men and women. These results suggest that no combination of the tests considered in this investigation simultaneously optimizes all 5 test battery performance indexes.     

Recruiting Effectiveness of the ASVAB Career Exploration Program

To assess the recruiting effectiveness of the Department of Defense's Armed Services Vocational Aptitude Battery (ASVAB) Career Exploration Program, a variety of sources, including historical databases, surveys, and interviews, were used. Despite downsizing and reductions in recruiting resources over the course of the evaluation period, there is evidence that a higher percentage of program participants subsequently enlist than the U.S. youth population as a whole. The overwhelming majority of a sample of recruiters rated the program as an important lead source as well as an effective recruiting tool. Data indicate that 16% to 25% of annual enlistments started as ASVAB program leads. The program influences young people's consideration of the military as a career and creates a positive impression of the military, regardless of whether a participant decides to enlist. Although there is evidence that the ASVAB program enhances recruiting, obstacles to its effectiveness include recruiter training deficiencies.     

Standard Errors of Multivariate Range-Corrected Validities

The multivariate rather than the univariate range correction is used for estimating unrestricted applicant population validities in many military test validity studies but not uniformly. A Monte Carlo approach compared the standard errors of range-corrected validities under various experimental conditions adhering to the assumptions underlying correction accuracy. The multivariate corrected validities had smaller standard errors than both the univariate-corrected validities and the unrestricted validities. We conclude that using the univariate correction could fail to reveal the most valid selection instrument and that the multivariate correction should be used when scores for relevant predictors are available for the unrestricted population.     

测验效度理论的析义

测验的构思效度、内容效度和生态效度可以追朔到斯披曼提出的认知三定律,它们相当于现代认知心理学中的元认知和信息处理观点。本文意欲综合这两种研究,并主张人类能力是一种阶层结构。     

Validities,antivalidities and contingencies: A multi-standard approach

Journal of Philosophical Logic - It is widely accepted that classical logic is trivialized in the presence of a transparent truth-predicate. In this paper, we will explain why this point of view...     

中国人人格七因素量表(QZPS-SF)的信度与效度

本研究的目的是依据中国人人格量表（QZPS）编制一份能够迅速测查人格七个维度的短式七因素量表（QZPS—SF）。根据6000余名被试对QZPS的215个项目的评定分数,通过因素分析确定了七个因素、与QZPS的七个因素对应构成了QZPS—SF。通过被试的自我评定以及他人评定和特殊群体被试（精神分裂症病人、吸毒者以及罪犯）的人格特点比较,显示了QZPS—SF有着良好的信度和效度。文中还对QZPS—SF的应用范围以及注意事项进行了讨论。     

Validities of the Physical Estimation Scale in predicting running times

Two studies were conducted with the purpose of examining the validity of the 12-item Physical Estimation Scale, developed by Safrit and Wood (1985), as a predictor of running performance. Physical Estimation Scale scores were not significantly related to running time at the 5-mi. distance but were significantly correlated with predicted and actual marathon finishing times.     

Seven-Year Follow-up Validities of Readiness for Vocational Planning Scales

From interviews with 110 boys and girls in the 8th grade (1958) and again in the 10th grade (1961), 8 Readiness for Vocational Planning (RVP) scales were derived. Multivariate analysis of variance studies of the hypothesized dependence of vocational adjustment criteria collected 2 years out of high school (1965) on the RVP scores are reported. Most of the criteria are statistically dependent on the 8th grade RVP scores, and none on the 10th grade RVP scores, suggesting that (1) vocational maturity is a measurable syndrome as early as the 8th grade, and (2) the RVP traits for the 10th grade are of unsatisfactory validity.     

Classification Validities of a Statewide 10th Grade Test Program

If statewide test norms are useful in student counseling, state testing programs could provide predictions for individual students as a service to local schools. Since the choice of a high school curriculum is a major problem on which youth seeks guidance, the predictive validity of a statewide test battery, using state norms, is explored with curriculum choice as the criterion, and is compared with the validity of the same tests for the same criterion, but using local norms. The separation in state norms of career-goals groups and of the sexes is also explored. Results achieved are positive enough to encourage widespread and energetic application of the methods by state testing officers, and to imply their responsibility to make classification probabilities available to their customers.     

Generalization of Cognitive and Noncognitive Validities across Personality‐based Job Families

The positive relationship between complexity of work and the validity of general mental ability (GMA) measures across a variety of occupations is well supported by research and provides important practical and theoretical support for cognitive ability measures. However, there is currently no research demonstrating a systematic relationship between the size of the validities of any personality measure and the personality requirements of jobs, thus leaving open to question the predictive and construct validity of personality measures for applicant selection. We compared the validities of two biodata measures – one scored to measure social competence and one to measure GMA – across six job families that varied in social requirements. The validity of the GMA measure generalized at approximately the same level across the job families while the validity of the social competence measure decreased as social requirements diminished.     

Validation of the Micropat Battery

Micropat is a battery of computer-based aptitude tests designed to assess aspects of psychomotor coordination and information management. Initial research and development work on Micropat started in 1980 with the Army Air Corps and the tests were successfully validated against rotary wing pilot training outcome. In 1985, funding for the project was taken over by the Royal Navy (RN) with a view to improving selection for both RN Pilots and RN Observers. Validation of the main battery was carried out on Pilot and Observer trainees at Britannia Royal Naval College (BRNC), Dartmouth, UK. The paper outlines the development of the Micropat programme over the past decade, focusing on a number of key practical methodological issues. These include discussion of the task-based approach to test construction, the use made of the potential afforded by computer-based assessment and the problem of developing robust composite predictors from small samples. In particular, the relative merits of unit-weighted, rationally weighted and empirically weighted composite predictors are examined. Evidence for the validity of the tests is summarized. It is concluded that we are unlikely to improve substantially on the levels of prediction which were being obtained in the 1950s. Continual development and improvement in selection testing is needed simply to maintain levels of prediction as the demands of flying change. The new forms of test made possible by computer-based assessment technology provide the means of maintaining useful levels of prediction as flying training courses become longer and more complex.     

The Situational Specificity of Validities: Correcting for Statistical Artifacts Does Not Always Reduce the Trans-Situational Variability of Correlation Coefficients

Validity generalization research suggests that much of the variability in validity coefficients is due to statistical artifacts, such as sampling error. Correcting for these artifacts usually reduces the trans-situational variability in validity coefficients, but the reverse can happen. Conditions under which corrections for statistical artifacts can increase the variability in validity coefficients are described, examples of meta-analyses reporting larger corrected than observed variance in validity are cited, and implications for the situational specificity of validity coefficients are discussed. In general, the true variability in validity coefficients based on large samples is often larger than the observed variability in test validities.