Hidden Markov Item Response Theory Models for Responses and Response Times

Current approaches to model responses and response times to psychometric tests solely focus on between-subject differences in speed and ability. Within subjects, speed and ability are assumed to be constants. Violations of this assumption are generally absorbed in the residual of the model. As a result, within-subject departures from the between-subject speed and ability level remain undetected. These departures may be of interest to the researcher as they reflect differences in the response processes adopted on the items of a test. In this article, we propose a dynamic approach for responses and response times based on hidden Markov modeling to account for within-subject differences in responses and response times. A simulation study is conducted to demonstrate acceptable parameter recovery and acceptable performance of various fit indices in distinguishing between different models. In addition, both a confirmatory and an exploratory application are presented to demonstrate the practical value of the modeling approach.     

等级反应模型下项目特征曲线等值法在大型考试中的应用

在中国最大的资格考试之一的经济专业资格考试中,为保证不同年度间考试的可比性、进行题库建设和为计算机自适应考试做准备,应用项目反应理论中等级反应模型下的项目特征曲线等值法,采用铆测验等值设计,实现了4个年度考试资料的项目参数和能力参数的等值,并成功地组建了经济专业题库。在此基础上,利用等值技术对不同年份试卷的划界分数进行了比较,为经济考试的合格标准制定、确保考试的公平性提供了实证依据。     

应用项目反应理论对《中国士兵人格问卷》的项目分析

采用项目反应理论（IRT）对《中国士兵人格问卷》进行项目分析。计算机呈现中国士兵人格问卷（CSPQ）对100,523名适龄男性青年进行测验,随机抽取2676名任一维度标准分均低于70的定为合格组;将任一维度大于70分并经专业人员访谈不合格的274名定为不合格组;从精神病院抽取男性年龄相当的221名缓解期精神分裂症患者定为精神病组,并完成CSPQ测验。运用基于IRT的双参数Logistic模型进行分析;结果发现,区分度参数超过区间(0.30,4.00)的条目删除前后,被试的能力值与标准分均存在显著相关;精神病组的测验分数经IRT分析,图形曲线与不合格组有高度吻合。研究结果说明,在测验精度基本相同的条件下,应用IRT可以减少施测条目,提高测验效率,可在一定程度上更精确地区分被试的特质水平     

Profile-likelihood Confidence Intervals in Item Response Theory Models

Confidence intervals (CIs) are fundamental inferential devices which quantify the sampling variability of parameter estimates. In item response theory, CIs have been primarily obtained from large-sample Wald-type approaches based on standard error estimates, derived from the observed or expected information matrix, after parameters have been estimated via maximum likelihood. An alternative approach to constructing CIs is to quantify sampling variability directly from the likelihood function with a technique known as profile-likelihood confidence intervals (PL CIs). In this article, we introduce PL CIs for item response theory models, compare PL CIs to classical large-sample Wald-type CIs, and demonstrate important distinctions among these CIs. CIs are then constructed for parameters directly estimated in the specified model and for transformed parameters which are often obtained post-estimation. Monte Carlo simulation results suggest that PL CIs perform consistently better than Wald-type CIs for both non-transformed and transformed parameters.     

Specifying Ability Growth Models Using a Multidimensional Item Response Model for Repeated Measures Categorical Ordinal Item Response Data

When categorical ordinal item response data are collected over multiple timepoints from a repeated measures design, an item response theory (IRT) modeling approach whose unit of analysis is an item response is suitable. This study proposes a few longitudinal IRT models and illustrates how a popular compensatory multidimensional IRT model can be utilized to formulate such longitudinal IRT models, which permits an investigation of ability growth at both individual and population levels. The equivalence of an existing multidimensional IRT model and those longitudinal IRT models is also elaborated so that one can make use of an existing multidimensional IRT model to implement the longitudinal IRT models.     

反应抑制的心理加工模型与神经机制

反应抑制是指抑制不符合当前需要的或不恰当行为反应的能力, 也是执行控制加工的重要成分。解释反应抑制的心理加工模型有两种: 反应与抑制相互独立的赛马模型和交互作用的赛马模型。近年来对反应抑制神经机制的研究表明: 额叶－基底神经节系统内的超直接通路和间接通路可能共同负责对优势反应的抑制, 而额下回、辅助运动区/辅助运动前区和前部扣带回皮层等脑区可能是抑制控制的关键脑区; 反应抑制与反应选择、工作记忆和注意的神经加工之间存在密切联系, 它们的激活脑区既相互重叠, 又相互区别; 右背外侧前额皮层的激活可能反映与抑制任务相关的注意和工作记忆的加工。未来的研究需要将脑损伤、神经功能成像和经颅磁刺激等多种技术结合起来, 进一步阐明上述脑区在反应抑制中的相互作用机制。     

Limits on Log Odds Ratios for Unidimensional Item Response Theory Models

Bounds are established for log odds ratios (log cross-product ratios) involving pairs of items for item response models. First, expressions for bounds on log odds ratios are provided for one-dimensional item response models in general. Then, explicit bounds are obtained for the Rasch model and the two-parameter logistic (2PL) model. Results are also illustrated through an example from a study of model-checking procedures. The bounds obtained can provide an elementary basis for assessment of goodness of fit of these models. Any opinions expressed in this publication are those of the authors and not necessarily those of the Educational Testing Service. The authors thank Dan Eignor, Matthias von Davier, Lydia Gladkova, Brian Junker, and the three anonymous reviewers for their invaluable advice. The authors gratefully acknowledge the help of Kim Fryer with proofreading.     

Model Selection of Nested and Non-Nested Item Response Models Using Vuong Tests

Abstract

In this paper, we apply Vuong’s general approach of model selection to the comparison of nested and non-nested unidimensional and multidimensional item response theory (IRT) models. Vuong’s approach of model selection is useful because it allows for formal statistical tests of both nested and non-nested models. However, only the test of non-nested models has been applied in the context of IRT models to date. After summarizing the statistical theory underlying the tests, we investigate the performance of all three distinct Vuong tests in the context of IRT models using simulation studies and real data. In the non-nested case we observed that the tests can reliably distinguish between the graded response model and the generalized partial credit model. In the nested case, we observed that the tests typically perform as well as or sometimes better than the traditional likelihood ratio test. Based on these results, we argue that Vuong’s approach provides a useful set of tools for researchers and practitioners to effectively compare competing nested and non-nested IRT models.     

Estimation Methods for Mixed Logistic Models with Few Clusters

For mixed models generally, it is well known that modeling data with few clusters will result in biased estimates, particularly of the variance components and fixed effect standard errors. In linear mixed models, small sample bias is typically addressed through restricted maximum likelihood estimation (REML) and a Kenward-Roger correction. Yet with binary outcomes, there is no direct analog of either procedure. With a larger number of clusters, estimation methods for binary outcomes that approximate the likelihood to circumvent the lack of a closed form solution such as adaptive Gaussian quadrature and the Laplace approximation have been shown to yield less-biased estimates than linearization estimation methods that instead linearly approximate the model. However, adaptive Gaussian quadrature and the Laplace approximation are approximating the full likelihood rather than the restricted likelihood; the full likelihood is known to yield biased estimates with few clusters. On the other hand, linearization methods linearly approximate the model, which allows for restricted maximum likelihood and the Kenward-Roger correction to be applied. Thus, the following question arises: Which is preferable, a better approximation of a biased function or a worse approximation of an unbiased function? We address this question with a simulation and an illustrative empirical analysis.     

测验理论的新发展:多维项目反应理论

多维项目反应理论是基于因子分析和单维项目反应理论两大背景下发展起来的一种新型测验理论。根据被试在完成一项任务时多种能力之间是如何相互作用的,多维项目反应模型可以分为补偿性模型和非补偿性模型两类。本文在系统介绍了当前普遍使用的补偿性模型的基础上,指出后续研究者应关注多维项目反应理论中多级评分和高维空间的多维模型、补偿性和非补偿性模型的融合、参数估计程序的开发和多维测验等值四个方面的研究。     

Models for the statistics and mechanisms of response speed and accuracy

Van Breukelen offers a promising method for modeling both response speed and response accuracy. However, the underlying conception of both dependent measures is somewhat flawed, leading the author to conclude that the approach possesses limitations that, under revised assumptions, may not hold. The central misconception, and a set of related misconceptions, is addressed, and it is suggested that this approach holds a good deal of promise for application in the perceptual and cognitive sciences.     

Latent Class Models for Diary Method Data: Parameter Estimation by Local Computations

The increasing use of diary methods calls for the development of appropriate statistical methods. For the resulting panel data, latent Markov models can be used to model both individual differences and temporal dynamics. The computational burden associated with these models can be overcome by exploiting the conditional independence relations implied by the model. This is done by associating a probabilistic model with a directed acyclic graph, and applying transformations to the graph. The structure of the transformed graph provides a factorization of the joint probability function of the manifest and latent variables, which is the basis of a modified and more efficient E-step of the EM algorithm. The usefulness of the approach is illustrated by estimating a latent Markov model involving a large number of measurement occasions and, subsequently, a hierarchical extension of the latent Markov model that allows for transitions at different levels. Furthermore, logistic regression techniques are used to incorporate restrictions on the conditional probabilities and to account for the effect of covariates. Throughout, models are illustrated with an experience sampling methodology study on the course of emotions among anorectic patients. Frank Rijmen was partly supported by the Fund for Scientific Research Flanders (FWO).     

Detecting Curvilinear Relationships: A Comparison of Scoring Approaches Based on Different Item Response Models

There is a growing use of noncognitive assessments around the world, and recent research has posited an ideal point response process underlying such measures. A critical issue is whether the typical use of dominance approaches (e.g., average scores, factor analysis, and the Samejima's graded response model) in scoring such measures is adequate. This study examined the performance of an ideal point scoring approach (e.g., the generalized graded unfolding model) as compared to the typical dominance scoring approaches in detecting curvilinear relationships between scored trait and external variable. Simulation results showed that when data followed the ideal point model, the ideal point approach generally exhibited more power and provided more accurate estimates of curvilinear effects than the dominance approaches. No substantial difference was found between ideal point and dominance scoring approaches in terms of Type I error rate and bias across different sample sizes and scale lengths, although skewness in the distribution of trait and external variable can potentially reduce statistical power. For dominance data, the ideal point scoring approach exhibited convergence problems in most conditions and failed to perform as well as the dominance scoring approaches. Practical implications for scoring responses to Likert-type surveys to examine curvilinear effects are discussed.     

Unsanctioned aggression in rugby union: relationships among aggressiveness,anger, athletic identity,and professionalization

Aggressive players who intentionally cause injury to their opponents are common in many sports, particularly collision sports such as Rugby Union. Although some acts of aggression fall within the rules (sanctioned), others do not (unsanctioned), with the latter tending to be less acceptable than the former. This study attempts to identify characteristics of players who are more likely to employ unsanctioned methods in order to injure an opponent. Male Rugby Union players completed questionnaires assessing aggressiveness, anger, past aggression, professionalization, and athletic identity. Players were assigned to one of two groups based on self‐reported past unsanctioned aggression. Results indicated that demographic variables (e.g., age, playing position, or level of play) were not predictive of group membership. Measures of aggressiveness and professionalization were significant predictors; high scores on both indicated a greater probability of reporting the use of unsanctioned aggressive force for the sole purpose of causing injury or pain. In addition, players who had been taught how to execute aggressive illegal plays without detection were also more likely to report using excessive force to injure an opponent. Results provide further support that highly professionalized players may be more likely to use methods outside the constitutive rules of Rugby Union in order to intentionally injure their opponents. Results are discussed within the context of the increasing win‐at‐all‐cost attitude that is becoming more prevalent in sport and its implications for youth athletes. Aggr. Behav. 35:237–243, 2009. © 2009 Wiley‐Liss, Inc.     

Heteroscedasticity as a Basis of Direction Dependence in Reversible Linear Regression Models

Heteroscedasticity is a well-known issue in linear regression modeling. When heteroscedasticity is observed, researchers are advised to remedy possible model misspecification of the explanatory part of the model (e.g., considering alternative functional forms and/or omitted variables). The present contribution discusses another source of heteroscedasticity in observational data: Directional model misspecifications in the case of nonnormal variables. Directional misspecification refers to situations where alternative models are equally likely to explain the data-generating process (e.g., x → y versus y → x). It is shown that the homoscedasticity assumption is likely to be violated in models that erroneously treat true nonnormal predictors as response variables. Recently, Direction Dependence Analysis (DDA) has been proposed as a framework to empirically evaluate the direction of effects in linear models. The present study links the phenomenon of heteroscedasticity with DDA and describes visual diagnostics and nine homoscedasticity tests that can be used to make decisions concerning the direction of effects in linear models. Results of a Monte Carlo simulation that demonstrate the adequacy of the approach are presented. An empirical example is provided, and applicability of the methodology in cases of violated assumptions is discussed.     

The direct product model for the mtmm matrix parameterized as a second order factor analysis model

The composite direct product model for the multitrait-multimethod matrix is reparameterized as a second-order factor analysis model. This facilitates the use of widely available computer programs such as LISREL and LISCOMP for fitting the model.Bruce Bloxom. Paul Horst and Karl Jöreskog contributed helpful comments to an earlier version of this paper. Their suggestions are gratefully acknowledged.     

多水平研究中的概念转换模式--以气氛研究为例

在组织环境中开展多水平研究会涉及到对处于组织中不同水平七的概念、变量或过程进行界定的问题。在已有概念基础上提出新的概念必须保证其有效性。本文以气氛研究为例介绍了与此问题相应的五种模式。     

Outcomes as affirmation of membership value: Material compensation as an administrative response to procedural injustice

The current line of research suggests that the provision of compensation by group representatives may be an effective way to address the identity concerns resulting from procedural violations because compensation serves to reaffirm the victim’s membership value, protecting his or her identity. A series of five studies is presented, demonstrating that compensation can function symbolically as a legitimate act of concern for the injustice victim. Results showed that offers of compensation by group representatives resulted in more favorable evaluations of the group and higher identification than when no compensation was offered, but only when the compensation was construed as a benevolent gesture and only when the injustice was identity relevant. Even unsuccessful attempts to compensate the victim resulted in positive reactions towards the group. Consistent with relational models of procedural justice, these effects were mediated by perceptions of membership value.     

Classical Test Theory as a first-order Item Response Theory: Application to true-score prediction from a possibly nonparallel test

We give an account of Classical Test Theory (CTT) in terms of the more fundamental ideas of Item Response Theory (IRT). This approach views classical test theory as a very general version of IRT, and the commonly used IRT models as detailed elaborations of CTT for special purposes. We then use this approach to CTT to derive some general results regarding the prediction of the true-score of a test from an observed score on that test as well from an observed score on a different test. This leads us to a new view of linking tests that were not developed to be linked to each other. In addition we propose true-score prediction analogues of the Dorans and Holland measures of the population sensitivity of test linking functions. We illustrate the accuracy of the first-order theory using simulated data from the Rasch model, and illustrate the effect of population differences using a set of real data.This research is collaborative in every respect and the order of authorship is alphabetical. It was begun when both authors were on the faculty of the Graduate School of Education at the University of California, Berkeley.We would like to thank both Neil Dorans, Skip Livingston and two anonymous referees for many suggestions that have greatly improved this paper.