Stepwise Selection of ECATs for Maximum Mean Validity

The problem addressed in this article is how to select a common battery of predictor tests for all armed services schools. A technique was developed for selecting a subset of tests from a larger test pool in order to maximize the mean corrected validity across occupational samples. The method involved adding or deleting tests stepwise to build a battery having a specified number of tests. The method was applied to the combined battery consisting of the Armed Services Vocational Aptitude Battery (ASVAB) and the Enhanced Computer-Administered Test (ECAT) battery. The analyses showed that, if all ASVAB tests were forced in first, validity tended to level off after 3 ECAT tests were added, each test representing a different ECAT factor. The first 3 ECAT tests selected by the procedure were Two-Hand Tracking, Mental Counters, and Assembling Objects. These 3 ECAT tests produced 76% of the validity gain from the full 9-test ECAT battery for preferred (i.e., hands-on performance oriented) school criteria. Also, a combination of 3 ECAT battery tests and 5 ASVAB tests had greater mean validity than did the full ASVAB of 10 tests. A variety of alternative combinations of 3 ECAT tests had nearly the same validity as did the optimal tests, as long as the tests measured the 3 basic ECAT battery factors of Psychomotor Ability, Working Memory, and Spatial Ability.