Nonparametric item response theory axioms and properties under nonlinearity and their exemplification with knowledge space theory

This paper investigates the dichotomous Mokken nonparametric item response theory (IRT) axioms and properties under incomparabilities among latent trait values and items. Generalized equivalents of the unidimensional nonparametric IRT axioms and properties are formulated for nonlinear (quasi-ordered) person and indicator spaces. It is shown that monotone likelihood ratio (MLR) for the total score variable and nonlinear latent trait implies stochastic ordering (SO) of the total score variable, but may fail to imply SO of the nonlinear latent trait. The reason for this and conditions under which the implication holds are specified, based on a new, simpler proof of the fact that in the unidimensional case MLR implies SO. The approach is applied in knowledge space theory (KST), a combinatorial test theory. This leads to a (tentative) Mokken-type nonparametric axiomatization in the currently parametric theory of knowledge spaces. The nonparametric axiomatization is compared with the assumptions of the parametric basic local independence model which is fundamental in KST. It is concluded that this paper may provide a first step toward a basis for a possible fusion of the two split directions of psychological test theories IRT and KST.     

On the need for negative local item dependence

While negative local item dependence (LID) has been discussed in numerous articles, its occurrence and effects often go unrecognized. This is due in part to confusion over what unidimensional latent trait is being utilized in evaluating the LID of multidimensional testing data. This article addresses this confusion by using an appropriately chosen latent variable to condition on. It then provides a proof that negative LID must occur when unidimensional ability estimates (such as number right score) are obtained from data which follow a very general class of multidimensional item response theory models. The importance of specifying what unidimensional latent trait is used, and its effect on the sign of the LIDs are shown to have implications in regard to a variety of foundational theoretical arguments, to the simulation of LID data sets, and to the use of testlet scoring for removing LID.This paper is based in part on a chapter in the first author's doctoral dissertation, written at the University of Illinois at Urbana-Champaign under the supervision of William Stout. Part of this research has been presented at the annual meeting of the National Council on Measurement in Education, San Diego, California, April 14–16, 1998.The research of the first author was partially supported by a Harold Gulliksen Psychometric fellowship through Educational Testing Service and by a Research and Productive Scholarship award from the University of South Carolina.     

Two-group classification in latent trait theory: Scores with monotone likelihood ratio

This paper deals with two-group classification when a unidimensional latent trait,, is appropriate for explaining the data,X. It is shown that ifX has monotone likelihood ratio then optimal allocation rules can be based on its magnitude when allocation must be made to one of two groups related to. These groups may relate to probabilistically via a non-decreasing functionp(), or may be defined by all subjects above or below a selected value on.In the case where the data arise from dichotomous items, then only the assumption that the items have nondecreasing item characteristic functions is enough to ensure that the unweighted sum of responses (the number-right score or raw score) possesses this fundamental monotone likelihood ratio property.     

Items bundles

An item bundle is a small group of multiple choice items that share a common reading passage or graph, or a small group of matching items that share distractors. Item bundles are easily identified by paging through a copy of a test. Bundled items may violate the latent conditional independence assumption of unidimensional item response theory (IRT), but such a violation would not typically suggest the existence of a new fundamental human ability to read one specific reading passage or to interpret one specific graph. It is important, therefore, to have theoretical concepts and empirical checks that distinguish between, on the one hand, anticipated violations of latent conditional independence within item bundles, and, on the other hand, violations that cannot be attributed to idiosyncratic features of test format and instead suggest departures from unidimensionalty. To this end, two theorems on unidimensional IRT are extended to describe observable item response distributions when there is conditional independencebetween but not necessarilywithin item bundles.The author is grateful to Ivo Molenaar and the referees for many helpful suggestions, and to D. Thayer for assistance with computing.     

Adjusting for information inflation due to local dependency in moderately large item clusters

When multiple items are clustered around a reading passage, the local independence assumption in item response theory is often violated. The amount of information contained in an item cluster is usually overestimated if violation of local independence is ignored and items are treated as locally independent when in fact they are not. In this article we provide a general method that adjusts for the inflation of information associated with a test containing item clusters. A computational scheme was presented for the evaluation of the factor of adjustment for clusters in the restrictive case of two items per cluster, and the general case of more than two items per cluster. The methodology was motivated by a study of the NAEP Reading Assessment. We present a simulated study along with an analysis of a NAEP data set.The research was supported under the National Assessment of Educational Progress (Grant No. R999G30002) as administered by the Office of Educational Research and Improvement, U.S. Department of Education. This work was started when the author was at the Division of Statistics and Psychometrics at the Educational Testing Service. The author thanks Juliet Shaffer, Bob Mislevy, Eric Bradlow, three reviewers and an associate editor for their helpful comments on the paper.     

Constant latent odds-ratios models and the mantel-haenszel null hypothesis

In the present paper, a new family of item response theory (IRT) models for dichotomous item scores is proposed. Two basic assumptions define the most general model of this family. The first assumption is local independence of the item scores given a unidimensional latent trait. The second assumption is that the odds-ratios for all item-pairs are constant functions of the latent trait. Since the latter assumption is characteristic of the whole family, the models are called constant latent odds-ratios (CLORs) models. One nonparametric special case and three parametric special cases of the general CLORs model are shown to be generalizations of the one-parameter logistic Rasch model. For all CLORs models, the total score (the unweighted sum of the item scores) is shown to be a sufficient statistic for the latent trait. In addition, conditions under the general CLORs model are studied for the investigation of differential item functioning (DIF) by means of the Mantel-Haenszel procedure. This research was supported by the Dutch Organization for Scientific Research (NWO), grant number 400-20-026.     

Nonparametric IRT: Testing the bi-isotonicity of isotonic probabilistic models (ISOP)

Nonparametric tests for testing the validity of polytomous ISOP-models (unidimensional ordinal probabilistic polytomous IRT-models) are presented. Since the ISOP-model is a very general nonparametric unidimensional rating scale model the test statistics apply to a great multitude of latent trait models. A test for the comonotonicity of item sets of two or more items is suggested. Procedures for testing the comonotonicity of two item sets and for item selection are developed. The tests are based on Goodman-Kruskal's gamma index of ordinal association and are generalizations thereof. It is an essential advantage of polytomous ISOP-models within probabilistic IRT-models that the tests of validity of the model can be performed before and without the model being fitted to the data. The new test statistics have the further advantage that no prior order of items or subjects needs to be known.     

An extension of the rating scale model with an application to the measurement of change

The polytomous unidimensional Rasch model with equidistant scoring, also known as the rating scale model, is extended in such a way that the item parameters are linearly decomposed into certain basic parameters. The extended model is denoted as the linear rating scale model (LRSM). A conditional maximum likelihood estimation procedure and a likelihood-ratio test of hypotheses within the framework of the LRSM are presented. Since the LRSM is a generalization of both the dichotomous Rasch model and the rating scale model, the present algorithm is suited for conditional maximum likelihood estimation in these submodels as well. The practicality of the conditional method is demonstrated by means of a dichotomous Rasch example with 100 items, of a rating scale example with 30 items and 5 categories, and in the light of an empirical application to the measurement of treatment effects in a clinical study.Work supported in part by the Fonds zur Förderung der Wissenschaftlichen Forschung under Grant No. P6414.     

Polytomous IRT models and monotone likelihood ratio of the total score

In a broad class of item response theory (IRT) models for dichotomous items the unweighted total score has monotone likelihood ratio (MLR) in the latent trait. In this study, it is shown that for polytomous items MLR holds for the partial credit model and a trivial generalization of this model. MLR does not necessarily hold if the slopes of the item step response functions vary over items, item steps, or both. MLR holds neither for Samejima's graded response model, nor for nonparametric versions of these three polytomous models. These results are surprising in the context of Grayson's and Huynh's results on MLR for nonparametric dichotomous IRT models, and suggest that establishing stochastic ordering properties for nonparametric polytomous IRT models will be much harder.Hemker's research was supported by the Netherlands Research Council, Grant 575-67-034. Junker's research was supported in part by the National Institutes of Health, Grant CA54852, and by the National Science Foundation, Grant DMS-94.04438.     

多维题组效应Rasch模型

首先, 本文诠释了“题组”的本质即一个存在共同刺激的项目集合。并基于此, 将题组效应划分为项目内单维题组效应和项目内多维题组效应。其次, 本文基于Rasch模型开发了二级评分和多级评分的多维题组效应Rasch模型, 以期较好地处理项目内多维题组效应。最后, 模拟研究结果显示新模型有效合理, 与Rasch题组模型、分部评分模型对比研究后表明：(1)测验存在项目内多维题组效应时, 仅把明显的捆绑式题组效应进行分离而忽略其他潜在的题组效应, 仍会导致参数的偏差估计甚或高估测验信度; (2)新模型更具普适性, 即便当被试作答数据不存在题组效应或只存在项目内单维题组效应, 采用新模型进行测验分析也能得到较好的参数估计结果。     

Unbiased estimators of ability parameters,of their variance,and of their parallel-forms reliability

Given known item parameters, unbiased estimators are derived i) for an examinee's ability parameter and for his proportion-correct true score, ii) for the variances of and across examinees in the group tested, and iii) for the parallel-forms reliability of the maximum likelihood estimator .This work was supported in part by contract N00014-80-C-0402, project designation NR 150-453 between the Office of Naval Research and Educational Testing Service. Reproduction in whole or in part is permitted for any purpose of the United States Government.     

Nonunique solutions to the likelihood equation for the three-parameter logistic model

Samejima identified the possibility of multiple solutions to the likelihood equation (multiple maxima in the likelihood function) for estimating an examinee's trait value for the three-parameter logistic model. In the practical applications that Lord studied, he found that multiple solutions did not occur when the number of items was 20. In the present paper, fourteen multiple-choice achievement tests with from 20 to 50 items were examined to see if it was possible for them to produce item response vectors with multiple maxima; such vectors were found for all the tests. Examination of response vectors for large groups of real examinees found that from 0 to 3.1% of them had response vectors with multiple maxima. The implications of these results for multiple-choice tests are discussed.     

A rasch model for partial credit scoring

A unidimensional latent trait model for responses scored in two or more ordered categories is developed. This “Partial Credit” model is a member of the family of latent trait models which share the property of parameter separability and so permit “specifically objective” comparisons of persons and items. The model can be viewed as an extension of Andrich's Rating Scale model to situations in which ordered response alternatives are free to vary in number and structure from item to item. The difference between the parameters in this model and the “category boundaries” in Samejima's Graded Response model is demonstrated. An unconditional maximum likelihood procedure for estimating the model parameters is developed. Preparation of this paper was supported by grants from the Spencer Foundation and the National Institute for Justice. I would like to thank Professor Benjamin D. Wright of the University of Chicago for his very kind help with the various drafts of this paper.     

An approximation for the bias function of the maximum likelihood estimate of a latent variable for the general case where the item responses are discrete

Lord developed an approximation for the bias function for the maximum likelihood estimate in the context of the three-parameter logistic model. Using Taylor's expansion of the likelihood equation, he obtained an equation that includes the conditional expectation, given true ability, of the discrepancy between the maximum likelihood estimate and true ability. All terms of orders higher thann ^?1 are ignored wheren indicates the number of items. Lord assumed that all item and individual parameters are bounded, all item parameters are known or well-estimated, and the number of items is reasonably large. In the present paper, an approximation for the bias function of the maximum likelihood estimate of the latent trait, or ability, will be developed using the same assumptions for the more general case where item responses are discrete. This will include the dichotomous response level, for which the three-parameter logistic model has been discussed, the graded response level and the nominal response level. Some observations will be made for both dichotomous and graded response levels.     

Item information and discrimination functions for trinary pcm items

For trinary partial credit items the shape of the item information and the item discrimination function is examined in relation to the item parameters. In particular, it is shown that these functions are unimodal if ₂ – ₁ < 4 ln 2 and bimodal otherwi The locations and values of the maxima are derived. Furthermore, it is demonstrated that the value of the maximum is decreasing in ₂ – ₁. Consequently, the maximum of a unimodal item information function is always larger than the maximum of a bimodal one, and similarly for the item discrimination function.The work reported herein was partially supported under the National Assessment of Educational Progress (Grant No. R999G30002; CFDA No. 84.999G) as administered by the Office of Educational Research and Improvement, US Department of Education.     

On Holland's Dutch identity conjecture

The manifest probabilities of observed examinee response patterns resulting from marginalization with respect to the latent ability distribution produce the marginal likelihood function in item response theory. Under the conditions that the posterior distribution of examinee ability given some test response pattern is normal and the item logit functions are linear, Holland (1990a) gives a quadratic form for the log-manifest probabilities by using the Dutch Identity. Further, Holland conjectures that this special quadratic form is a limiting one for all smooth unidimensional item response models as test length tends to infinity. The purpose of this paper is to give three counterexamples to demonstrate that Holland's Dutch Identity conjecture does not hold in general. The counterexamples suggest that only under strong assumptions can it be true that the limits of log-manifest probabilities are quadratic. Three propositions giving sets of such strong conditions are given.     

Additive conjoint isotonic probabilistic models (ADISOP)

The ISOP-model or model of twodimensional or bi-isotonicity (Scheiblechner, 1995) postulates that the probabilities of ordered response categories increase isotonically in the order of subject ability and item easiness. Adding a conventional cancellation axiom for the factors of subjects and items gives the ADISOP model where the c.d.f.s of response categories are functions of an additive item and subject parameter and an ordinal category parameter. Extending cancellation to the interactions of subjects and categories as well as of items and categories (independence axiom of the category factor from the subject and item factor) gives the CADISOP model (completely additive model) in which the parallel c.d.f.s are functions of the sum of subject, item and category parameters. The CADISOP model is very close to the unidimensional version of the polytomous Rasch model with the logistic item/category characteristic(s) replaced by nonparametric axioms and statistics. The axioms, representation theorems and algorithms for model fitting of the additive models are presented.     

Robust Measurement via A Fused Latent and Graphical Item Response Theory Model

Item response theory (IRT) plays an important role in psychological and educational measurement. Unlike the classical testing theory, IRT models aggregate the item level information, yielding more accurate measurements. Most IRT models assume local independence, an assumption not likely to be satisfied in practice, especially when the number of items is large. Results in the literature and simulation studies in this paper reveal that misspecifying the local independence assumption may result in inaccurate measurements and differential item functioning. To provide more robust measurements, we propose an integrated approach by adding a graphical component to a multidimensional IRT model that can offset the effect of unknown local dependence. The new model contains a confirmatory latent variable component, which measures the targeted latent traits, and a graphical component, which captures the local dependence. An efficient proximal algorithm is proposed for the parameter estimation and structure learning of the local dependence. This approach can substantially improve the measurement, given no prior information on the local dependence structure. The model can be applied to measure both a unidimensional latent trait and multidimensional latent traits.     

A Unified Nonparametric IRT Model for <Emphasis Type="Italic">d</Emphasis>-Dimensional Psychological Test Data (<Emphasis Type="Italic">d</Emphasis>-ISOP)

The (univariate) isotonic psychometric (ISOP) model (Scheiblechner, 1995) is a nonparametric IRT model for dichotomous and polytomous (rating scale) psychological test data. A weak subject independence axiom W1 postulates that the subjects are ordered in the same way except for ties (i.e., similarly or isotonically) by all items of a psychological test. A weak item independence axiom W2 postulates that the order of the items is similar for all subjects. Local independence (LI or W3) is assumed in all models. With these axioms, sample-free unidimensional ordinal measurements of items and subjects become feasible. A cancellation axiom (Co) gives, as a result, the additive isotonic psychometric (ADISOP) model and interval scales for subjects and items, and an independence axiom (W4) gives the completely additive isotonic psychometric (CADISOP) model with an interval scale for the response variable (Scheiblechner, 1999). The d-ISOP, d-ADISOP, and d-CADISOP models are generalizations to d-dimensional dependent variables (e.g., speed and accuracy of response). The author would like to thank an Associate Editor and two anonymous referees and also Professor H.H. Schulze for their very valuable suggestions and corrections.