An automated test development of parallel tests from a seed test

Binary programming models are presented to generate parallel tests from an itembank. The parallel tests are created to match item for item an existing seed test and match user supplied taxonomic specifications. The taxonomic specifications may be either obtained from the seed test or from some other user requirement. An algorithm is presented along with computational results to indicate the overall efficiency of the process. Empirical findings based on an itembank for the Arithmetic Reasoning section of the Armed Services Vocational Aptitude Battery are given.The Office of Naval Research, Program in Cognitive Science, N00014-87-C-0696 partially supported the work of Douglas H. Jones. The Rutgers Research Resource Committee of the Graduate School of Management partially supported the work of Douglas H. Jones and Ing-Long Wu. A Thomas and Betts research fellowship partially supported the work of Ing-Long Wu. The Human Resources Laboratory, United States Air Force, partially supported the work of Ronald Armstrong. The authors benefited from conversations with Dr. Wayne Shore, Operational Technologies, San Antonio, Texas. The order of authors' names is alphabetical and denotes equal authorship.     

A taxonomy of item response models

A number of models for categorical item response data have been proposed in recent years. The models appear to be quite different. However, they may usefully be organized as members of only three distinct classes, within which the models are distinguished only by assumptions and constraints on their parameters. “Difference models” are appropriate for ordered responses, “divide-by-total” models may be used for either ordered or nominal responses, and “left-side added” models are used for multiple-choice responses with guessing. The details of the taxonomy and the models are described in this paper. The present study was supported in part by two postdoctoral fellowships awarded to Lynne Steinberg: an Educational Testing Service Postdoctoral Fellowship at ETS, Princeton, NJ and an NIMH Individual National Research Service Award at Stanford University, Stanford, CA. Helpful comments by the editor and three anonymous reviewers are gratefully acknowledged.     

Modeling item responses when different subjects employ different solution strategies

A model is presented for item responses when different subjects employ different strategies, but only responses, not choice of strategy, can be observed. Using substantive theory to differentiate the likelihoods of response vectors under a fixed set of strategies, we model response probabilities in terms of item parameters for each strategy, proportions of subjects employing each strategy, and distributions of subject proficiency within strategies. The probabilities that an individual subject employed the various strategies can then be obtained, along with a conditional estimate of proficiency under each. A conceptual example discusses response strategies for spatial rotation tasks, and a numerican example resolves a population of subjects into subpopulations of valid responders and random guessers.The first author's work was supported by Contract No. N00014-85-K-0683, project designation NR 150-539, from the Cognitive Science Program, Cognitive and Neural Sciences Division, Office of Naval Research. We are grateful to Murray Aitkin, Isaac Bejar, Neil Dorans, Frederiksen, and Marklyn Wingersky for their comments and suggestions, and to Alison Gooding, Maxine Kingston, Donna Lembeck, Joling Liang, and Kentaro Yamamoto for their assistance with Example 2.     

The percentage bend correlation coefficient

A well-known result is that the usual correlation coefficient,, is highly nonrobust: very slight changes in only one of the marginal distributions can alter by a substantial amount. There are a variety of methods for correcting this problem. This paper identifies one particular method which is useful in psychometrics and provides a simple test for independence. It is not recommended that the new test replace the usual test ofH ₀: = 0, but the new test has important advantages over the usual test in terms of both Type I errors and power.     

Identification of inconsistent variates in factor analysis

When some of observed variates do not conform to the model under consideration, they will have a serious effect on the results of statistical analysis. In factor analysis the model with inconsistent variates may result in improper solutions. In this article a useful method for identifying a variate as inconsistent is proposed in factor analysis. The procedure is based on the likelihood principle. Several statistical properties such as the effect of misspecified hypotheses, the problem of multiple comparisons, and robustness to violation of distributional assumptions are investigated. The procedure is illustrated by some examples.     

Rationally developed fake-good and fake-bad scales for the Jesness Inventory

Develops two validity scales for the Jesness Inventory using a rational approach: a fake-good scale, Lie (L); and a fake-bad scale, Overt Symptomatology (OS). Effectiveness was assessed using 293 male delinquents classified as fake-good, fake-bad, or honest based on a matched-pair MMPI-A. L was moderately effective in detecting the fake-good set, and OS tentatively effective in detecting the fake-bad set. Both correlated well with their MMPI-A counterparts. Sensitivity, specificity, positive predictive power, negative predictive power, and overall effectiveness data were reported. The L scale and OS scale were related to race but differences were less than 1 raw score point. L was unrelated to age. OS was related to age, with younger children showing more willingness to admit to symptomatology. Age-based modified T-score norms were developed for the newly constructed scales using 1142 male and 360 female delinquents. ages 13–18.     

Reliability of multiple classifications

Cohen's kappa index is reformulated for multiple classifications based on exchangeable random variables. It is found that kappa is between 0 and 1 inclusive. Two characterizations for kappa are stated in terms of the relationship between such random variables. Within the normal test score model, kappa increases with test reliability and test length. Furthermore, when based on binary classifications, kappa is an inverse U-shaped function of the cutoff score. These trends also hold for the beta-binomial test score model.Paper read at the Spring meeting of the Psychometric Society, Bell Laboratories (Murray Hill, New Jersey), March 1976.The editorial assistance and helpful comments of Leonard S. Feldt, Sarah P. Seaman-Huynh, and the three referees are gratefully acknowledged.     

Psychopathy and structure of primary mental abilities

The Comprehensive Ability Battery (CAB) was used to assess 18 primary mental abilities in 53 white, male prison inmates for whom highly reliable ratings of psychopathy were available. None of the correlations between ratings of psychopathy and scores on the CAB tests was significant, and the overall pattern or structure of abilities was the same for inmates with low and high ratings of psychopathy. The possibility is raised that the personality characteristics of psychopaths make them appear to be brighter and more creative than they actually are. Compared with norms for male high school students, the inmates as a group generally performed well, falling near the 50th percentile on 11 of the tests. The inmates performed significantly better than the normative sample on tests of verbal ability, esthetic judgment, aiming, and representational drawing, and significantly worse on tests of ideational fluency, spontaneous flexibility, and word fluency.This research was supported by Grant MT-4511 from the Medical Research Council of Canada.