Aggregate item response analysis

A stochastic postulate is given for the multiple-item, successive-intervals scaling of populations. The logistic equivalent of this postulate provides an aggregate item response model in which a unidimensional submodel may be nested. This reduction provides a subtractive conjoint measurement of several items and stimuli on the same latent scale. Generalized-least-squares methods are used to estimate and test the multiple-item model, and its unidimensional reduction, on aggregate survey responses. The entire procedure is illustrated with an analysis of semantic-differential attitude data. This analysis exhibits an item selection procedure that is applicable to various social constructs.The authors dedicate this paper to the memory and contributions of Clyde Coombs.The programming and data analyses for the present paper were carried out by José Ventura of the Department of Industrial and Systems Engineering, and Jerry Meiten of the Department of Statistics, University of Florida.The study was also supported by the College of Business Administration, University of Florida, and the Faculty of Social Sciences, Hebrew University of Jerusalem.     

Examining differential item functioning due to item difficulty and alternative attractiveness

A method for analyzing test item responses is proposed to examine differential item functioning (DIF) in multiple-choice items through a combination of the usual notion of DIF, for correct/incorrect responses and information about DIF contained in each of the alternatives. The proposed method uses incomplete latent class models to examine whether DIF is caused by the attractiveness of the alternatives, difficulty of the item, or both. DIF with respect to either known or unknown subgroups can be tested by a likelihood ratio test that is asymptotically distributed as a chi-square random variable.     

An automated test development of parallel tests from a seed test

Binary programming models are presented to generate parallel tests from an itembank. The parallel tests are created to match item for item an existing seed test and match user supplied taxonomic specifications. The taxonomic specifications may be either obtained from the seed test or from some other user requirement. An algorithm is presented along with computational results to indicate the overall efficiency of the process. Empirical findings based on an itembank for the Arithmetic Reasoning section of the Armed Services Vocational Aptitude Battery are given.The Office of Naval Research, Program in Cognitive Science, N00014-87-C-0696 partially supported the work of Douglas H. Jones. The Rutgers Research Resource Committee of the Graduate School of Management partially supported the work of Douglas H. Jones and Ing-Long Wu. A Thomas and Betts research fellowship partially supported the work of Ing-Long Wu. The Human Resources Laboratory, United States Air Force, partially supported the work of Ronald Armstrong. The authors benefited from conversations with Dr. Wayne Shore, Operational Technologies, San Antonio, Texas. The order of authors' names is alphabetical and denotes equal authorship.     

Using item-specific instructional information in achievement modeling

The problem of detecting instructional sensitivity (item basis) in test items is considered. An illustration is given which shows that for tests with many biased items, traditional item bias detection schemes give a very poor assessment of bias. A new method is proposed instead. This method extends item response theory (IRT) by including item-specific auxiliary measurement information related to opportunity-to-learn. Item-specific variation in measurement relations across students with varying opportunity-to-learn is allowed for.This paper was presented at the 1987 AERA meeting in Washington, DC. This research was supported by grant OERI-G-86-003 from the Office of Educational Research and Improvement, Department of Education. The author thanks Michael Hollis and Chih-fen Kao for valuable research assistance, and appreciates valuable comments made by an anonymous reviewer.     

A nonparametric approach for assessing latent trait unidimensionality

Assuming a nonparametric family of item response theory models, a theory-based procedure for testing the hypothesis of unidimensionality of the latent space is proposed. The asymptotic distribution of the test statistic is derived assuming unidimensionality, thereby establishing an asymptotically valid statistical test of the unidimensionality of the latent trait. Based upon a new notion of dimensionality, the test is shown to have asymptotic power 1. A 6300 trial Monte Carlo study using published item parameter estimates of widely used standardized tests indicates conservative adherence to the nominal level of significance and statistical power averaging 81 out of 100 rejections for examinee sample sizes and psychological test lengths often incurred in practice.The referees' comments were remarkably detailed and greatly enhanced the writeup and sensitized the author to certain pertinent issues. Discussions with Fritz Drasgow, Lloyd Humphreys, Dennis Jennings, Brian Junker, Robert Linn, Ratna Nandakumar, and Robin Shealy were also very useful.This research was supported by the Office of Naval Research under grant N00014-84-K-0186; NR 150-533, and by the National Science Foundation under grant DMS 85-03321.     

Reliability of test scores in nonparametric item response theory

Three methods for estimating reliability are studied within the context of nonparametric item response theory. Two were proposed originally by Mokken (1971) and a third is developed in this paper. Using a Monte Carlo strategy, these three estimation methods are compared with four classical lower bounds to reliability. Finally, recommendations are given concerning the use of these estimation methods.The authors are grateful for constructive comments from the reviewers and from Charles Lewis.     

A taxonomy of item response models

A number of models for categorical item response data have been proposed in recent years. The models appear to be quite different. However, they may usefully be organized as members of only three distinct classes, within which the models are distinguished only by assumptions and constraints on their parameters. “Difference models” are appropriate for ordered responses, “divide-by-total” models may be used for either ordered or nominal responses, and “left-side added” models are used for multiple-choice responses with guessing. The details of the taxonomy and the models are described in this paper. The present study was supported in part by two postdoctoral fellowships awarded to Lynne Steinberg: an Educational Testing Service Postdoctoral Fellowship at ETS, Princeton, NJ and an NIMH Individual National Research Service Award at Stanford University, Stanford, CA. Helpful comments by the editor and three anonymous reviewers are gratefully acknowledged.     

Worldviews about change: Their structure and their implications for understanding responses to sustainability,technology, and political change

People hold different perspectives about how they think the world is changing or should change. We examined five of these “worldviews” about change: Progress, Golden Age, Endless Cycle, Maintenance, and Balance. In Studies 1–4 (total N = 2733) we established reliable measures of each change worldview, and showed how these help explain when people will support or oppose social change in contexts spanning sustainability, technological innovations, and political elections. In mapping out these relationships we identify how the importance of different change worldviews varies across contexts, with Balance most critical for understanding support for sustainability, Progress/Golden Age important for understanding responses to innovations, and Golden Age uniquely important for preferring Trump/Republicans in the 2016 US election. These relationships were independent of prominent individual differences (e.g., values, political orientation for elections) or context-specific factors (e.g., self-reported innovativeness for responses to innovations). Study 5 (N = 2140) examined generalizability in 10 countries/regions spanning five continents, establishing that these worldviews exhibited metric invariance, but with country/region differences in how change worldviews were related to support for sustainability. These findings show that change worldviews can act as a general “lens” people use to help determine whether to support or oppose social change.