Stochastic ordering using the latent trait and the sum score in polytomous IRT models

In a restricted class of item response theory (IRT) models for polytomous items the unweighted total score has monotone likelihood ratio (MLR) in the latent trait. MLR implies two stochastic ordering (SO) properties, denoted SOM and SOL, which are both weaker than MLR, but very useful for measurement with IRT models. Therefore, these SO properties are investigated for a broader class of IRT models for which the MLR property does not hold.In this study, first a taxonomy is given for nonparametric and parametric models for polytomous items based on the hierarchical relationship between the models. Next, it is investigated which models have the MLR property and which have the SO properties. It is shown that all models in the taxonomy possess the SOM property. However, counterexamples illustrate that many models do not, in general, possess the even more useful SOL property.Hemker's research was supported by the Netherlands Research Council, Grant 575-67-034. Junker's research was supported in part by the National Institutes of Health, Grant CA54852, and by the National Science Foundation, Grant DMS-94.04438.     

Residualanalysis in the polytomous rasch model

Residuals for check of model fit in the polytomous Rasch model are examined. Comparisons are made between using counts for all response pattern and using item totals for score groups for the construction of the residuals. Comparisons are also, for the residuals based on score group totals, made between using as basis the item totals, or using the estimated item parameters. The developed methods are illustrated by two examples, one from a psychiatric rating scale, one from a Danish Welfare Study.     

Essential independence and likelihood-based ability estimation for polytomous items

A definition ofessential independence is proposed for sequences of polytomous items. For items satisfying the reasonable assumption that the expected amount of credit awarded increases with examinee ability, we develop a theory ofessential unidimensionality which closely parallels that of Stout. Essentially unidimensional item sequences can be shown to have a unique (up to change-of-scale) dominant underlying trait, which can be consistently estimated by a monotone transformation of the sum of the item scores. In more general polytomous-response latent trait models (with or without ordered responses), anM-estimator based upon maximum likelihood may be shown to be consistent for under essentially unidimensional violations of local independence and a variety of monotonicity/identifiability conditions. A rigorous proof of this fact is given, and the standard error of the estimator is explored. These results suggest that ability estimation methods that rely on the summation form of the log likelihood under local independence should generally be robust under essential independence, but standard errors may vary greatly from what is usually expected, depending on the degree of departure from local independence. An index of departure from local independence is also proposed.This work was supported in part by Office of Naval Research Grant N00014-87-K-0277 and National Science Foundation Grant NSF-DMS-88-02556. The author is grateful to William F. Stout for many helpful comments, and to an anonymous reviewer for raising the questions addressed in section 2. A preliminary version of section 6 appeared in the author's Ph.D. thesis.     

A person-fit index for polytomous rasch models,latent class models,and their mixture generalizations

A normally distributed person-fit index is proposed for detecting aberrant response patterns in latent class models and mixture distribution IRT models for dichotomous and polytomous data.This article extends previous work on the null distribution of person-fit indices for the dichotomous Rasch model to a number of models for categorical data. A comparison of two different approaches to handle the skewness of the person-fit index distribution is included.Major parts of this paper were written while the first author worked at the Institute for Science Education, Kiel, Germany. Any opinions expressed in this paper are those of the authors and not necessarily of Educational Testing Service. The results presented in this paper were improved by valuable comments from J. Rost, K. Yamamoto, N.D. Verhelst, E. Bedrick and two anonymous reviewers.     

基于等级反应模型的规则空间方法

本研究基于Tatsuoka的规则空间方法, 对理想反应模式与异常反应指标进行了扩展, 推导了多级评分项目下规则空间方法的算法公式。在4种属性层级结构(发散型、收敛型、线型与无结构型)×4种“失误”作答概率(2%、5%、10%与15%)测验情境下, 以属性模式判准率、被试属性判准率、敏感性与特异性为指标, 检验了多级评分项目下规则空间方法的分类准确性。结果表明：(1) 基于多级评分项目构建的异常反应指标, 能有效地对被试进行分类与解释, 且0-1评分项目下异常反应指标及其性质都是多级评分下的特例; (2) 随着“失误”作答概率的增加, 4种属性层级结构的分类准确性都会降低; (3) 线型和收敛型的分类准确性明显好于发散型与无结构型; (4) 纯规则点的分布对规则空间方法的分类准确性有显著影响。     

Multi‐Criteria Evaluation for Collaborative Circulating Farming with Collective Operations Between Arable and Cattle Farmers

Tokachi sub‐prefecture in Hokkaido is one of the most famous dairy and crop farming regions in Japan. It is known that Tokachi is faced with various difficult problems such as soil degradation, water contamination and unpleasant odours because of the excessive use of chemical fertilizers and inappropriate treatment of livestock excretion. In this paper, we focus on Shihoro town where agricultural outputs are relatively large in Tokachi, and propose collaborative circulating farming with collective operations between arable and cattle farmers. Under the assumption that the decision‐maker in this problem is a representative of a farming organization who hopes for sustainable agricultural development and values the intentions of local residents including arable and cattle farmers in this region, we employ multi‐attribute utility theory in order to evaluate multiple alternatives of the farming management problem. Copyright © 2012 John Wiley & Sons, Ltd.     

多级属性Q矩阵的验证与估计

多级属性是将诊断测验中传统的二值(即两种水平, 通常定义为0和1)属性定义为多值(多个水平可以为0, 1, …), 它不但可以描述学生对于知识属性是否掌握, 而且可以描述学生在属性上的掌握程度, 这样使得诊断测验能提供给被试更丰富的知识掌握详情。本文将适用于二级属性Q矩阵的统计量(S统计量)拓展到多级属性下的Q矩阵验证和估计, 在两种常见的条件下, 设计了两种估计算法：联合估计算法和在线估计算法。模拟实验结果表明：联合估计算法适用于对专家界定的初始Q矩阵进行验证, 当初始Q矩阵中包含较少的错误时, 通过联合估计算法有很大可能恢复正确的Q矩阵; 在线估计算法适用于对“新项目”进行属性向量和项目参数的在线标定, 基于一定数量的“基础项目”, 在线估计算法对于新项目的估计也能达到较满意的成功率。实证数据分析则进一步展示了该方法的使用。     

Empirical Priors in Polytomous Computerized Adaptive Tests: Risks and Rewards in Clinical Settings

The use of empirical prior information about participants has been shown to substantially improve the efficiency of computerized adaptive tests (CATs) in educational settings. However, it is unclear how these results translate to clinical settings, where small item banks with highly informative polytomous items often lead to very short CATs. We explored the risks and rewards of using prior information in CAT in two simulation studies, rooted in applied clinical examples. In the first simulation, prior precision and bias in the prior location were manipulated independently. Our results show that a precise personalized prior can meaningfully increase CAT efficiency. However, this reward comes with the potential risk of overconfidence in wrong empirical information (i.e., using a precise severely biased prior), which can lead to unnecessarily long tests, or severely biased estimates. The latter risk can be mitigated by setting a minimum number of items that are to be administered during the CAT, or by setting a less precise prior; be it at the expense of canceling out any efficiency gains. The second simulation, with more realistic bias and precision combinations in the empirical prior, places the prevalence of the potential risks in context. With similar estimation bias, an empirical prior reduced CAT test length, compared to a standard normal prior, in 68% of cases, by a median of 20%; while test length increased in only 3% of cases. The use of prior information in CAT seems to be a feasible and simple method to reduce test burden for patients and clinical practitioners alike.     

使用似然比D2统计量的题目属性定义方法

题目属性的定义是实施认知诊断评价的关键步骤, 通过有丰富经验的领域专家对题目的属性进行定义是当前的主要方法, 然而该方法受到许多主观经验因素的影响。寻找客观的题目属性定义或验证方法可以为主观定义过程提供策略支持或对结果进行改进, 因此已经引起研究者们的关注。本研究构建了一种简单高效的题目属性定义方法, 研究使用似然比D2统计量从作答数据中估计题目属性的方法, 实现属性掌握模式、题目参数和题目属性向量的联合估计。模拟研究结果表明, 使用似然比D2统计量可以有效地识别题目的属性向量, 该方法一方面可以实现新编制题目属性向量的在线估计, 另一方面可以验证已经定义的题目属性向量的准确性。     

一种多级评分的广义认知诊断模型

认知诊断是近些年教育测量研究中的热点,大多数的认知诊断模型仅适用于0～1评分的情况.本文提出一种有多个潜变量多个滑动参数的多级评分认知诊断模型——GP-D1NA,只要由评分标准和知识状态能确定理想反应模式,就可以利用此方法进行认知诊断分析.在该方法中,我们给出项目滑动矩阵的概念,将被试的观测得分均看成由某个理想得分的滑动,并采用EM算法估计滑动矩阵.在模拟研究中,采用每掌握一个属性得1分的评分标准,结果表明线性型、收敛型、发散型、无结构型和独立型五种属性层级结构均有较高的判准率.