Estimating Creativity with a Multiple-Measurement Approach Within Scientific and Artistic Domains

This article presents the structure and the composition of a newly developed multifaceted test battery for the measurement of creativity within scientific and artistic domains. By integrating existing procedures for the evaluation of creativity, the new battery promises to become a comprehensive assessment of creativity, encompassing both domain-general and domain-specific components. In particular, the test battery was designed for the measurement of the 2 main stages of the creative thinking process: ideation and evaluation. The test battery also includes 2 measures of creative achievement and can be used to assess professional levels of creativity in artistic and scientific creativity, as well as everyday creativity. Because creative thinking is not an isolated phenomenon in human behavior, the battery includes the measurement of 2 constructs, intelligence and personality, both of which are highly relevant for creativity. Preliminary results from a vast administration campaign of this test battery are presented.     

Some uses of the square root method as a tool in multiple prediction

It is shown how the square root method can be used in the derivation of formulas in multiple prediction and in the computations necessary for their application. A formula derived by Horst indicates the validity a test must possess in order to contribute a specified amount to the multiple correlation of a battery. Then a formula is derived which relates the length of one test in a battery to the multiple correlation of the battery as a whole. The formulas hold for population parameters; in this paper, sampling theory is not discussed. A numerical example is presented; it is shown that one square root analysis provides all the data needed for the formulas.     

An algebraic solution for the communalities

Factorial analysis begins with ann ×n correlation matrixR, whose principal diagonal entries are unknown. If the common test space of the battery is under investigation, the communality of each test is entered in the appropriate diagonal cell. This value is the portion of the test's variance shared with others in the battery. The communalities must be so estimated thatR will maintain the rank determined by its side entries, after the former have been inserted. Previous methods of estimating the communalities have involved a certain arbitrariness, since they depended on selecting test subgroups or parts of the data inR. A theory is presented showing that this difficulty can be avoided in principle. In its present form, the theory is not offered as a practical computing procedure. The basis of the new method lies in the Cayley-Hamilton theorem: Any square matrix satisfies its own characteristic equation.     

Music ability

Two batteries of music tests were factored by the centroid method. From each battery three oblique factors were extracted and in each case were tentatively identified as tonal sensitivity, retentivity (memory for elements), and memory for form. The correlations of the music tests of one battery with subtests of Cattell's intelligence test and with tests of a literary nature are also reported.Karlin, J. E. A multiple factor analysis of musicality. M. A. thesis, University of Cape Town, 1939.Drake, R. M. A factorial analysis of music tests by the Spearman tetrad-difference technique.J. Musicology, 1939, 1, 1.     

Stepwise Selection of ECATs for Maximum Mean Validity

The problem addressed in this article is how to select a common battery of predictor tests for all armed services schools. A technique was developed for selecting a subset of tests from a larger test pool in order to maximize the mean corrected validity across occupational samples. The method involved adding or deleting tests stepwise to build a battery having a specified number of tests. The method was applied to the combined battery consisting of the Armed Services Vocational Aptitude Battery (ASVAB) and the Enhanced Computer-Administered Test (ECAT) battery. The analyses showed that, if all ASVAB tests were forced in first, validity tended to level off after 3 ECAT tests were added, each test representing a different ECAT factor. The first 3 ECAT tests selected by the procedure were Two-Hand Tracking, Mental Counters, and Assembling Objects. These 3 ECAT tests produced 76% of the validity gain from the full 9-test ECAT battery for preferred (i.e., hands-on performance oriented) school criteria. Also, a combination of 3 ECAT battery tests and 5 ASVAB tests had greater mean validity than did the full ASVAB of 10 tests. A variety of alternative combinations of 3 ECAT tests had nearly the same validity as did the optimal tests, as long as the tests measured the 3 basic ECAT battery factors of Psychomotor Ability, Working Memory, and Spatial Ability.     

The test–retest reliability of the latent construct of executive function depends on whether tasks are represented as formative or reflective indicators

This study investigates the test–retest reliability of a battery of executive function (EF) tasks with a specific interest in testing whether the method that is used to create a battery-wide score would result in differences in the apparent test–retest reliability of children’s performance. A total of 188 4-year-olds completed a battery of computerized EF tasks twice across a period of approximately two weeks. Two different approaches were used to create a score that indexed children’s overall performance on the battery—i.e., (1) the mean score of all completed tasks and (2) a factor score estimate which used confirmatory factor analysis (CFA). Pearson and intra-class correlations were used to investigate the test–retest reliability of individual EF tasks, as well as an overall battery score. Consistent with previous studies, the test–retest reliability of individual tasks was modest (rs ≈ .60). The test–retest reliability of the overall battery scores differed depending on the scoring approach (r_mean = .72; r_{factor_score} = .99). It is concluded that the children’s performance on individual EF tasks exhibit modest levels of test–retest reliability. This underscores the importance of administering multiple tasks and aggregating performance across these tasks in order to improve precision of measurement. However, the specific strategy that is used has a large impact on the apparent test–retest reliability of the overall score. These results replicate our earlier findings and provide additional cautionary evidence against the routine use of factor analytic approaches for representing individual performance across a battery of EF tasks.     

DEVELOPMENT OF A JOB ANALYSIS-BASED PROCEDURE FOR WEIGHTING AND COMBINING CONTENT RELATED TESTS INTO A SINGLE TEST BATTERY SCORE

By definition, content-related approaches to test validation do not rely on criterion data. As a consequence, regression and other statistical procedures for weighting and generating a composite score from a test battery are not applicable when a content-related validation strategy is used. This paper presents a procedure for determining the component weights for a test battery that has been developed on the basis of a content-related validity strategy. The Relative Content Contribution (RCC) weighting procedure is a logical extension of the conceptual basis underlying the rational developmental process used to demonstrate the validity of content-related tests. Results from field implementations of the procedure in the development of two promotional test batteries (fire and police) and an entry-level test battery (police) in two large metropolitan cities are presented to illustrate the procedure.     

A multidimensional approach for selecting child care workers

A multidimensional selection battery was designed to predict a variety of criteria important in the selection of child care workers. The battery assesses constructs related to honesty, violence, substance abuse, emotional stability and safety. A series of studies were used to test the validity of the selection battery. Scores on the test battery were compared with those from three alternative selection procedures to define the measured constructs. Three additional studies show the relation of scores on the selection battery and the behavior of child care workers. The test battery was correlated with the job performance of child care workers and identified adults convicted for sexual offenses against minors.     

The relationship between the validity of a single test and its contribution to the predictive efficiency of a test battery

Typical selection or classification testing programs should provide for improvement of the predictive efficiency of the test battery. Such provision calls for the administration of experimental tests along with the operational battery administration and follow-up analysis to determine the value of the experimental material. It is possible to determine without waiting for criterion data what the validity of the experimental test must be in order to improve the battery validity. The method together with the proof is presented.     

The Armed Services Vocational Aptitude Battery—A Test Battery in Transition

The widely used ASVAB is reviewed from the standpoint of career guidance in grades 9–12. The battery measures general ability, trade-technical experience, a spatial factor, and a clerical factor unlike traditional measures of clerical aptitude. The version used during the 1977–78 school year had serious limitations. ASVAB changes annually; if subsequent versions have similar limitations, the student drawing conclusions from the report form without the help of an informed counselor will likely draw incorrect conclusions about his or her vocational potential. Established procedures for test review are inadequate for generating an informed opinion regarding a test that is heavily promoted and that changes from year to year.     

Comparing latent distributions

We consider the problem of comparingm latent population distributions when the observed values are scores on a test battery with binary items. The latent densities are assumed to be normal densities, and we consider a test for equality of the means as well as a test equality of the variances. In addition, we consider a longitudinal model, where the test battery has been applied to the same individuals at different points in time. This model allows for correlations between the latent variable at different time points, and methods are discussed for estimating the correlation coefficient.This work was supported in part by a grant from the Danish Social Science Research Council.     

Extended workdays: Effects on performance and ratings of fatigue and alertness

The present investigation provided a laboratory test of long workdays and served as an initial step in developing a field-test battery that is sensitive to fatigue. Six subjects worked both a 12-h/4-day workweek and an 8-h/6-day week at a data-entry job simulation. Before and after the first and last days of each week, they completed a battery of brief tests measuring cognitive, perceptual-motor, task-sharing, motor, and sensory capacities, as well as subjective feelings. Results suggest that the 12-h/4-day week was more fatiguing than the 8 h/6-day week. In the data-entry job, it was easier to improve performance across the 8 h/6-day week. This result suggested that the fatigue of the 12-h days slowed the rate of improvement across the week. With respect to the test battery, two cognitive tasks (grammatical reasoning and digit addition) and several self-report scales also reflected greater fatigue in the 12-h/4day week. Performance efficiency decreased and reports of drowsiness and lack of concentration increased from the beginning to the end of the final 12-h workday. On the basis of these results, it was concluded that the test battery has utility for the assessment of the potential fatigue effects of long workdays in actual work settings.     

Fitness to drive following cerebral pathology: The Rookwood Driving Battery as a tool for predicting on‐road driving performance

The use of neuropsychological testing to determine fitness to drive in people with neuropathology is likely to be an increasingly attractive alternative to on‐road testing for many candidates for assessment. The Rookwood Driving Battery has been shown to have good predictive value for determining some who are likely to fail an on‐road test in early studies. This study replicated earlier research by examining the predictive value and theoretical validity of the battery on a larger sample of 391 participants, as well as extending earlier analysis by examining the effect of older age (over 70 years) and the interaction between age and pathology on battery and on‐road performance. The battery demonstrated good positive and negative predictive values for predicting on‐road performance. There were significant effects of older age on both the Rookwood Battery performance and the on‐road test, with older adults performing significantly poorer on both. There was no interaction between age and pathology on the Rookwood Battery but on‐road age interacted with some pathologies to produce significantly poorer performances. Furthermore, correlation and regression analysis indicate that the battery is a powerful instrument that encompasses tests of core neuropsychological functions needed for driving.     

Validation of physical ability tests for high pressure cleaning occupations

This paper describes the development and validation of a physical performance test battery to select high pressure cleaning workers at Tennessee Eastman Company. The research uses a concurrent, criterion-related validation strategy to demonstrate the relationship between test performance and job performance. The study proceeded through four stages, with data collected in one stage used in subsequent stages. The two types of predictor measures included ability tests and work sample simulations. For some analyses, the work sample simulations were also used as criteria. Four samples of incumbents (N=63) were tested on all experimental measures. Using criterion groups of successful incumbents and unsuccessful former incumbents (N=33), all of the work simulation tests and the ability-referenced tests were significant predictors of successful versus unsuccessful job status. In addition, the ability tests predicted performance on the work sample simulations, particularly for those requiring strength and endurance. A battery of three ability tests resulted in R=.68 for job status and in R=.43, .73, .80, and .84, respectively, for each of the work sample simulations. Two of the three ability tests in the final test battery are likely to result in adverse impact against females. However, data collected on female maintenance technicians indicated women, as a group, would not be screened out categorically as a result of implementing the physical ability test battery.     

PROJECT A: SPECIFICATION OF THE PREDICTOR DOMAIN AND DEVELOPMENT OF NEW SELECTION/ CLASSIFICATION TESTS

A major goal of the Army Selection and Classification Project was to develop an experimental predictor battery that would best supplement the Armed Forces Vocational Aptitude Battery for making selection and classification decisions for entry-level enlisted personnel. That is, what predictor measures would best serve the needs of all the jobs in an entire selection/classification system? This paper describes the characteristics of the new test battery and the procedures that were used to develop it. The major steps in the procedure were a structured literature search using a standard protocol, an extensive expert judgment study of expected true validities for a population of predictor variables against a population of performance components, fabrication of modularized software and a special response pedestal for computerized measurement of perceptual and psychomotor abilities, evaluations of experimental measures in three iterative pilot tests and one major field test, and a series of reviews by a panel of scientific advisers. The test battery that resulted from this 2 1/2-year development effort is described. The basic psychometric properties of each measure, as determined in a large concurrent validation sample, are also described.     

Regression weights as a function of test length

A battery of pencil-and-paper tests is commonly used for predicting a single criterion. If the score on each test is the number of correct answers, the composite battery score would normally be the sum of the weighted test scores, where the weights are the raw score regression weights. Knowing the reliability of each test, it is possible to alter the lengths of the tests in a manner such that the weights will all be equal. The composite battery score would then simply be the total number of items answered correctly and scoring would be greatly simplified. Such simplification is particularly desirable where the volume of testing is large. Section I of the article outlines the procedure for altering the lengths of the tests, and Section II gives a proof of the method.     

Measuring episodic memory: A novel approach with an indefinite number of alternative forms

Both clinical practice and clinical research settings can require successive administrations of a memory test, particularly when following the trajectory of suspected memory decline in older adults. However, relatively few verbal episodic memory tests have alternative forms. We set out to create a broad‐based memory test to allow for the use of an essentially unlimited number of alternative forms. Four tasks for inclusion in such a test were developed. These tasks varied the requirement for recall as opposed to recognition, the need to form an association between unrelated words, and the need to discriminate the most recent list from earlier lists, all of which proved useful. A total of 115 participants completed the battery of tests and were used to show that the test could differentiate between older and younger adults; a sub‐sample of 73 participants completed alternative forms of the tests to determine test–retest reliability and the amount of learning to learn. Copyright © 2009 John Wiley & Sons, Ltd.     

Determination of optimal test length to maximize the multiple correlation

If the lengths of the tests in a battery are altered, their intercorrelations and their validities or correlations with a criterion are also altered. Consequently, the multiple correlation of the battery with the criterion will also be altered. These changes are a function of the reliabilities of the tests. Suppose we have given from a set of experimental data (1) the time allowed for each test in the battery, (2) the reliability of each test, (3) the intercorrelations, and (4) the validities of all the tests. If we specify the over-all testing time we are willing to allow for the test in the future, we can determine the amount by which each test must be altered in order to give the maximum multiple correlation with the criterion. The method is presented, together with numerical examples and the mathematical proof.     

The factorial interpretation of test difficulty

This paper discusses the influence of test difficulty on the correlation between test items and between tests. The greater the difference in difficulty between two test items or between two tests the smaller the maximum correlation between them. In general, the greater the number of degrees of difficulty among the items in a test or among the tests in a battery, the higher the rank of the matrix of intercorrelations; that is, differences in difficulty are represented in the factorial configuration as additional factors. The suggestion is made that if all tests included in a battery are roughly homogeneous with respect to difficulty existing hierarchies will be more clearly defined and meaningful psychological interpretation of factors more readily attained.     

Taskomat: Evaluation of a Computerized Test Battery

A case study on a computerized test battery is presented based on more than a decade of use in military and civil settings. The study sheds light on the general issues associated with computerized testing. The accent is on the contrast between computerized and traditional paper-and-pencil testing. The computerized battery, called Taskomat, is founded on concepts arising from human performance theory. The tests of the battery are described, as well as issues related to the hardware (cost of purchasing, hardware reliability and industrial styling). Criteria used to evaluate the content of the tests were: reliability, robustness of effects, uniqueness or newness of abilities relative to paper-and-pencil testing, and validity of these abilities for predicting occupational success. A conclusion is that a theoretical foundation helps to increase the likelihood of assessing new abilities.