Aggregate item response analysis

A stochastic postulate is given for the multiple-item, successive-intervals scaling of populations. The logistic equivalent of this postulate provides an aggregate item response model in which a unidimensional submodel may be nested. This reduction provides a subtractive conjoint measurement of several items and stimuli on the same latent scale. Generalized-least-squares methods are used to estimate and test the multiple-item model, and its unidimensional reduction, on aggregate survey responses. The entire procedure is illustrated with an analysis of semantic-differential attitude data. This analysis exhibits an item selection procedure that is applicable to various social constructs.The authors dedicate this paper to the memory and contributions of Clyde Coombs.The programming and data analyses for the present paper were carried out by José Ventura of the Department of Industrial and Systems Engineering, and Jerry Meiten of the Department of Statistics, University of Florida.The study was also supported by the College of Business Administration, University of Florida, and the Faculty of Social Sciences, Hebrew University of Jerusalem.     

Examining differential item functioning due to item difficulty and alternative attractiveness

A method for analyzing test item responses is proposed to examine differential item functioning (DIF) in multiple-choice items through a combination of the usual notion of DIF, for correct/incorrect responses and information about DIF contained in each of the alternatives. The proposed method uses incomplete latent class models to examine whether DIF is caused by the attractiveness of the alternatives, difficulty of the item, or both. DIF with respect to either known or unknown subgroups can be tested by a likelihood ratio test that is asymptotically distributed as a chi-square random variable.     

An automated test development of parallel tests from a seed test

Binary programming models are presented to generate parallel tests from an itembank. The parallel tests are created to match item for item an existing seed test and match user supplied taxonomic specifications. The taxonomic specifications may be either obtained from the seed test or from some other user requirement. An algorithm is presented along with computational results to indicate the overall efficiency of the process. Empirical findings based on an itembank for the Arithmetic Reasoning section of the Armed Services Vocational Aptitude Battery are given.The Office of Naval Research, Program in Cognitive Science, N00014-87-C-0696 partially supported the work of Douglas H. Jones. The Rutgers Research Resource Committee of the Graduate School of Management partially supported the work of Douglas H. Jones and Ing-Long Wu. A Thomas and Betts research fellowship partially supported the work of Ing-Long Wu. The Human Resources Laboratory, United States Air Force, partially supported the work of Ronald Armstrong. The authors benefited from conversations with Dr. Wayne Shore, Operational Technologies, San Antonio, Texas. The order of authors' names is alphabetical and denotes equal authorship.     

Criterion-related construct validity

Established results on latent variable models are applied to the study of the validity of a psychological test. When the test predicts a criterion by measuring a unidimensional latent construct, not only must the total score predict the criterion, but the joint distribution of criterion scores and item responses must exhibit a certain pattern. The presence of this population pattern may be tested with sample data using the stratified Wilcoxon rank sum test. Often, criterion information is available only for selected examinees, for instance, those who are admitted or hired. Three cases are discussed: (i) selection at random, (ii) selection based on the current test, and (iii) selection based on other measures of the latent construct. Discriminant validity is also discussed.This work was supported in part by Grant SES-87-01890 from the Measurement Methods and Data Improvement Program of the U.S. National Science Foundation.     

Using item-specific instructional information in achievement modeling

The problem of detecting instructional sensitivity (item basis) in test items is considered. An illustration is given which shows that for tests with many biased items, traditional item bias detection schemes give a very poor assessment of bias. A new method is proposed instead. This method extends item response theory (IRT) by including item-specific auxiliary measurement information related to opportunity-to-learn. Item-specific variation in measurement relations across students with varying opportunity-to-learn is allowed for.This paper was presented at the 1987 AERA meeting in Washington, DC. This research was supported by grant OERI-G-86-003 from the Office of Educational Research and Improvement, Department of Education. The author thanks Michael Hollis and Chih-fen Kao for valuable research assistance, and appreciates valuable comments made by an anonymous reviewer.     

A nonparametric approach for assessing latent trait unidimensionality

Assuming a nonparametric family of item response theory models, a theory-based procedure for testing the hypothesis of unidimensionality of the latent space is proposed. The asymptotic distribution of the test statistic is derived assuming unidimensionality, thereby establishing an asymptotically valid statistical test of the unidimensionality of the latent trait. Based upon a new notion of dimensionality, the test is shown to have asymptotic power 1. A 6300 trial Monte Carlo study using published item parameter estimates of widely used standardized tests indicates conservative adherence to the nominal level of significance and statistical power averaging 81 out of 100 rejections for examinee sample sizes and psychological test lengths often incurred in practice.The referees' comments were remarkably detailed and greatly enhanced the writeup and sensitized the author to certain pertinent issues. Discussions with Fritz Drasgow, Lloyd Humphreys, Dennis Jennings, Brian Junker, Robert Linn, Ratna Nandakumar, and Robin Shealy were also very useful.This research was supported by the Office of Naval Research under grant N00014-84-K-0186; NR 150-533, and by the National Science Foundation under grant DMS 85-03321.     

Reliability of test scores in nonparametric item response theory

Three methods for estimating reliability are studied within the context of nonparametric item response theory. Two were proposed originally by Mokken (1971) and a third is developed in this paper. Using a Monte Carlo strategy, these three estimation methods are compared with four classical lower bounds to reliability. Finally, recommendations are given concerning the use of these estimation methods.The authors are grateful for constructive comments from the reviewers and from Charles Lewis.     

A taxonomy of item response models

A number of models for categorical item response data have been proposed in recent years. The models appear to be quite different. However, they may usefully be organized as members of only three distinct classes, within which the models are distinguished only by assumptions and constraints on their parameters. “Difference models” are appropriate for ordered responses, “divide-by-total” models may be used for either ordered or nominal responses, and “left-side added” models are used for multiple-choice responses with guessing. The details of the taxonomy and the models are described in this paper. The present study was supported in part by two postdoctoral fellowships awarded to Lynne Steinberg: an Educational Testing Service Postdoctoral Fellowship at ETS, Princeton, NJ and an NIMH Individual National Research Service Award at Stanford University, Stanford, CA. Helpful comments by the editor and three anonymous reviewers are gratefully acknowledged.     

Fragile promise: The role of justification in promise-breaking

People who consider themselves moral sometimes use self-serving justifications to rationalize their selfish behaviours. Previous studies have tested the role of ambiguity in justifying wrongdoings, but it remains unclear whether ambiguity also plays a role in justifying promise-breaking behaviour and whether heterogeneity exists. To investigate justification in promise-breaking, we introduced a new experimental paradigm called the card-guessing task and used hierarchical cluster analysis to classify participants based on their promise-breaking decisions in unambiguous and ambiguous conditions. Experiment 1 revealed three clusters of solutions: Cluster 1 always kept their promises (i.e., keepers); Cluster 2 only exploited the vague promises and broke their promises in the ambiguous condition (i.e., intermediates); Cluster 3 tended to take advantage of vague promises and broke their promises irrespective of ambiguity (i.e., breakers). Experiment 2 confirmed that participants in the three clusters differed in their norm-abiding preferences and social value orientations. Keepers were more altruistic and had a stronger sense of norm compliance than intermediates and breakers. In conclusion, our study demonstrates that self-serving justifications were more likely to be employed by people who are moderately sensitive to deviation from social norms, which has implications for strategic interventions and policy formulation concerning unethical behaviour.     

Validity of the DSM-III-R personality disorder Clusters

This study assessed the validity of the DSM-III-R personality disorder clusters (i.e., odd-eccentric, dramatic-emotional-erratic, and anxious-fearful) by examining the relationships between self-report measures that tap the core features shared by disorders from each cluster and Cluster scores established via a semistructured interview in a sample of 57 outpatients. Results indicated a high degree of correlation among the DSM-III-R personality disorder Cluster scores. In addition, a series of regression analyses revealed that self-report scores did not account for a significant amount of variance in their respective Cluster scores over and above that accounted for by other self-report measures and other Cluster scores. These results suggest that the current DSM-III-R cluster classification scheme may not be appropriate, and it is recommended that a more empirically justifiable classification of the personality disorders be adopted in DSM-IV.