Exact person fit indexes for the rasch model for arbitrary alternatives

A new algorithm for obtaining exact person fit indexes for the Rasch model is introduced which realizes most powerful tests for a very general family of alternative hypotheses, including tests concerning DIF as well as model-deviating item correlations. The method is also used as a goodness-of-fit test for whole data sets where the item parameters are assumed to be known. For tests with 30 items at most, exact values are obtained, for longer tests a Monte Carlo-algorithm is proposed. Simulated examples and an empirical investigation demonstrate test power and applicability to item elimination.The author wishes to thank Elisabeth Ponocny-Seliger and the reviewers for many helpful comments. All exact goodness-of-fit tests proposed in this article are implemented in the menu-driven program T-Rasch 1.0 by Ponocny and Ponocny-Seliger (1999) which can be obtained from ProGAMMA (WWW: http://www.gamma.rug.nl) and also performs nonparametric tests.     

A relation between a between-item multidimensional IRT model and the mixture rasch model

Two generalizations of the Rasch model are compared: the between-item multidimensional model (Adams, Wilson, and Wang, 1997), and the mixture Rasch model (Mislevy & Verhelst, 1990; Rost, 1990). It is shown that the between-item multidimensional model is formally equivalent with a continuous mixture of Rasch models for which, within each class of the mixture, the item parameters are equal to the item parameters of the multidimensional model up to a shift parameter that is specific for the dimension an item belongs to in the multidimensional model. In a simulation study, the relation between both types of models also holds when the number of classes of the mixture is as small as two. The relation is illustrated with a study on verbal aggression. Frank Rijmen was supported by the Fund for Scientific Research Flanders (FWO). This research is also funded by the GOA/2000/02 granted from the KU Leuven. We would like to thank Kristof Vansteelandt for providing the data of the study on verbal aggression.     

An Efficient MCMC Algorithm to Sample Binary Matrices with Fixed Marginals

Uniform sampling of binary matrices with fixed margins is known as a difficult problem. Two classes of algorithms to sample from a distribution not too different from the uniform are studied in the literature: importance sampling and Markov chain Monte Carlo (MCMC). Existing MCMC algorithms converge slowly, require a long burn-in period and yield highly dependent samples. Chen et al. developed an importance sampling algorithm that is highly efficient for relatively small tables. For larger but still moderate sized tables (300×30) Chen et al.’s algorithm is less efficient. This article develops a new MCMC algorithm that converges much faster than the existing ones and that is more efficient than Chen’s algorithm for large problems. Its stationary distribution is uniform. The algorithm is extended to the case of square matrices with fixed diagonal for applications in social network theory. I am indebted to my colleague Gunter Maris for his suggestion to add a Metropolis–Hastings step as the finishing touch of the algorithm.     

Exact tests for the rasch model via sequential importance sampling

Rasch proposed an exact conditional inference approach to testing his model but never implemented it because it involves the calculation of a complicated probability. This paper furthers Rasch’s approach by (1) providing an efficient Monte Carlo methodology for accurately approximating the required probability and (2) illustrating the usefulness of Rasch’s approach for several important testing problems through simulation studies. Our Monte Carlo methodology is shown to compare favorably to other Monte Carlo methods proposed for this problem in two respects: it is considerably faster and it provides more reliable estimates of the Monte Carlo standard error.This Research was supported in part by National Science Foundation grant DMS-0203762 and a University of Pennsylvania Research Foundation grant.The authors are grateful to Don Burdick for helpful comments. In addition, the authors wish to thank the editor, the associate editor, and the referees for their helpful suggestions.This revised article was published online in August 2005 with the PDF paginated correctly.     

Constructing the exact significance level for a person fit statistic

This paper uses an extension of the network algorithm originally introduced by Mehta and Patel to construct exact tail probabilities for testing the general hypothesis that item responses are distributed according to the Rasch model. By assuming that item difficulties are known, the algorithm is applicable to the statistical tests either given the maximum likelihood ability estimate or conditioned on the total score. A simulation study indicates that the network algorithm is an efficient tool for computing the significance level of a person fit statistic based on test lengths of 30 items or less.     

多面Rasch模型在结构化面试中的应用

使用项目反应理论中的多面Rasch模型,对66名考生在结构化面试中的成绩进行分析,剔除了由于评委等具体测量情境因素引入的误差对原始分数的影响,得到考生的能力估计值以及个体水平的评分者一致性信息。对基于考生能力估计值和考生面试分得到的决策结果进行比较,发现测量误差的确对决策造成影响,对个别考生的影响甚至相当巨大。进一步使用Facets偏差分析以及评委宽严程度的Facets分析追踪误差源。结果表明,将来自不同面试组的被试进行面试原始成绩的直接比较,评委的自身一致性和评委彼此之间在宽严程度上的差异均将导致误差。研究表明,采用Facets的考生能力估计值作为决策的依据将提高选拔的有效性。同时,Facets分析得到的考生个体层次的评分者一致性指标,以及评委与考生的偏差分析等研究结果还可以为面试误差来源的定位提供详细的诊断信息     

Loglinear multidimensional IRT models for polytomously scored items

A loglinear IRT model is proposed that relates polytomously scored item responses to a multidimensional latent space. The analyst may specify a response function for each response, indicating which latent abilities are necessary to arrive at that response. Each item may have a different number of response categories, so that free response items are more easily analyzed. Conditional maximum likelihood estimates are derived and the models may be tested generally or against alternative loglinear IRT models.Hank Kelderman is currently affiliated with Vrije Universiteit, Amsterdam.We thank Linda Vodegel-Matzen of the Division of Developmental Psychology of the University of Amsterdam for making available the data used in the example in this article.     

Polytomous IRT models and monotone likelihood ratio of the total score

In a broad class of item response theory (IRT) models for dichotomous items the unweighted total score has monotone likelihood ratio (MLR) in the latent trait. In this study, it is shown that for polytomous items MLR holds for the partial credit model and a trivial generalization of this model. MLR does not necessarily hold if the slopes of the item step response functions vary over items, item steps, or both. MLR holds neither for Samejima's graded response model, nor for nonparametric versions of these three polytomous models. These results are surprising in the context of Grayson's and Huynh's results on MLR for nonparametric dichotomous IRT models, and suggest that establishing stochastic ordering properties for nonparametric polytomous IRT models will be much harder.Hemker's research was supported by the Netherlands Research Council, Grant 575-67-034. Junker's research was supported in part by the National Institutes of Health, Grant CA54852, and by the National Science Foundation, Grant DMS-94.04438.     

Rasch's model for reading speed with manifest explanatory variables

In educational and psychological measurement we find the distinction between speed and power tests. Although most tests are partially speeded, the speed element is usually neglected. Here we consider a latent trait model developed by Rasch for the response time on a (set of) pure speed test(s), which is based on the assumption that the test response times are approximately gamma distributed, with known shape parameters and scale parameters depending on subject ability and test difficulty parameters. In our approach the subject parameters are treated as random variables having a common gamma distribution. From this, maximum marginal likelihood estimators are derived for the test difficulties and the parameters of the latent subject distribution. This basic model can be extended in a number of ways. Explanatory variables for the latent subject parameters and for the test parameters can be incorporated in the model. Our methods are illustrated by the analysis of a simulated and an empirical data set.     

Stochastic ordering using the latent trait and the sum score in polytomous IRT models

In a restricted class of item response theory (IRT) models for polytomous items the unweighted total score has monotone likelihood ratio (MLR) in the latent trait. MLR implies two stochastic ordering (SO) properties, denoted SOM and SOL, which are both weaker than MLR, but very useful for measurement with IRT models. Therefore, these SO properties are investigated for a broader class of IRT models for which the MLR property does not hold.In this study, first a taxonomy is given for nonparametric and parametric models for polytomous items based on the hierarchical relationship between the models. Next, it is investigated which models have the MLR property and which have the SO properties. It is shown that all models in the taxonomy possess the SOM property. However, counterexamples illustrate that many models do not, in general, possess the even more useful SOL property.Hemker's research was supported by the Netherlands Research Council, Grant 575-67-034. Junker's research was supported in part by the National Institutes of Health, Grant CA54852, and by the National Science Foundation, Grant DMS-94.04438.     

Some neglected problems in IRT

The paper addresses three neglected questions from IRT. In section 1, the properties of the “measurement” of ability or trait parameters and item difficulty parameters in the Rasch model are discussed. It is shown that the solution to this problem is rather complex and depends both on general assumptions about properties of the item response functions and on assumptions about the available item universe. Section 2 deals with the measurement of individual change or “modifiability” based on a Rasch test. A conditional likelihood approach is presented that yields (a) an ML estimator of modifiability for given item parameters, (b) allows one to test hypotheses about change by means of a Clopper-Pearson confidence interval for the modifiability parameter, or (c) to estimate modifiability jointly with the item parameters. Uniqueness results for all three methods are also presented. In section 3, the Mantel-Haenszel method for detecting DIF is discussed under a novel perspective: What is the most general framework within which the Mantel-Haenszel method correctly detects DIF of a studied item? The answer is that this is a 2PL model where, however, all discrimination parameters are known and the studied item has the same discrimination in both populations. Since these requirements would hardly be satisfied in practical applications, the case of constant discrimination parameters, that is, the Rasch model, is the only realistic framework. A simple Pearsonx ² test for DIF of one studied item is proposed as an alternative to the Mantel-Haenszel test; moreover, this test is generalized to the case of two items simultaneously studied for DIF.     

协单参数单维度Rasch模型的优势与意义

相比多参数多维度IRT模型通过增加参数的方式来提升模型拟合度和解释度,Rasch模型流派强调“理论驱动研究”和“数据符合模型”,推崇单参数单维度的测量模型能最大限度地减少额外因素对真实测量目的的影响和干扰,从而保证测量的客观性和准确性。Rasch模型关注测量目标与测量工具的对应关系,它的“简单”特性有助于研究者更准确地评估和解释被测目标与测量工具间的适配性,且在将非线性数据转化为等距数据时具有天然的优势。     

Item bias detection using loglinear irt

A method is proposed for the detection of item bias with respect to observed or unobserved subgroups. The method uses quasi-loglinear models for the incomplete subgroup × test score × Item 1 × ... × itemk contingency table. If subgroup membership is unknown the models are Haberman's incomplete-latent-class models.The (conditional) Rasch model is formulated as a quasi-loglinear model. The parameters in this loglinear model, that correspond to the main effects of the item responses, are the conditional estimates of the parameters in the Rasch model. Item bias can then be tested by comparing the quasi-loglinear-Rasch model with models that contain parameters for the interaction of item responses and the subgroups.The author thanks Wim J. van der Linden and Gideon J. Mellenbergh for comments and suggestions and Frank Kok for empirical data.     

Examining differential item functioning due to item difficulty and alternative attractiveness

A method for analyzing test item responses is proposed to examine differential item functioning (DIF) in multiple-choice items through a combination of the usual notion of DIF, for correct/incorrect responses and information about DIF contained in each of the alternatives. The proposed method uses incomplete latent class models to examine whether DIF is caused by the attractiveness of the alternatives, difficulty of the item, or both. DIF with respect to either known or unknown subgroups can be tested by a likelihood ratio test that is asymptotically distributed as a chi-square random variable.     

Nonparametric polytomous IRT models for invariant item ordering, with results for parametric models

It is often considered desirable to have the same ordering of the items by difficulty across different levels of the trait or ability. Such an ordering is an invariant item ordering (IIO). An IIO facilitates the interpretation of test results. For dichotomously scored items, earlier research surveyed the theory and methods of an invariant ordering in a nonparametric IRT context. Here the focus is on polytomously scored items, and both nonparametric and parametric IRT models are considered.The absence of the IIO property in twononparametric polytomous IRT models is discussed, and two nonparametric models are discussed that imply an IIO. A method is proposed that can be used to investigate whether empirical data imply an IIO. Furthermore, only twoparametric polytomous IRT models are found to imply an IIO. These are the rating scale model (Andrich, 1978) and a restricted rating scale version of the graded response model (Muraki, 1990). Well-known models, such as the partial credit model (Masters, 1982) and the graded response model (Samejima, 1969), do no imply an IIO.     

迫选式人格测验的传统计分与IRT计分模型

迫选测验的传统计分方式会产生自模式数据, 不能进行传统的信效度检验、因素分析和方差分析等。近年来研究者提出了一些基于项目反应理论的计分模型, 如瑟斯顿IRT模型和MUPP模型等, 它们可以规避自模式数据的弊端。瑟斯顿IRT模型方便进行参数估计, 模型定义灵活; 而MUPP模型的拓展性较差, 参数估计的方法有待提高。另一方面, 已有研究者基于MUPP模型开发了一些抗作假的迫选测验, 而瑟斯顿IRT模型距离这种应用还比较远。此外, 两个模型的适用性和有效性都有待更多的实证研究来检验。     

心理测验中个人拟合研究的回顾与展望

个人拟合研究是对心理测验中偏差行为鉴别的一种新方法,它以Guttman理想模型、项目反应理论和非参数项目反应理论为基础。传统个人拟合指标是对宽泛偏差行为的检验,当前个人拟合研究更具体、更形象,针对于某种偏差行为的专门性指标和图形与回归分析受到了越来越多的关注。但无论哪种个人拟合方法都会受到各种无关因素的影响,如何克服这些影响,提高个人拟合方法的实际应用价值将成为个人拟合研究的主要方向