Hidden Markov Item Response Theory Models for Responses and Response Times

Current approaches to model responses and response times to psychometric tests solely focus on between-subject differences in speed and ability. Within subjects, speed and ability are assumed to be constants. Violations of this assumption are generally absorbed in the residual of the model. As a result, within-subject departures from the between-subject speed and ability level remain undetected. These departures may be of interest to the researcher as they reflect differences in the response processes adopted on the items of a test. In this article, we propose a dynamic approach for responses and response times based on hidden Markov modeling to account for within-subject differences in responses and response times. A simulation study is conducted to demonstrate acceptable parameter recovery and acceptable performance of various fit indices in distinguishing between different models. In addition, both a confirmatory and an exploratory application are presented to demonstrate the practical value of the modeling approach.     

A Multivariate Multilevel Approach to the Modeling of Accuracy and Speed of Test Takers

Response times on test items are easily collected in modern computerized testing. When collecting both (binary) responses and (continuous) response times on test items, it is possible to measure the accuracy and speed of test takers. To study the relationships between these two constructs, the model is extended with a multivariate multilevel regression structure which allows the incorporation of covariates to explain the variance in speed and accuracy between individuals and groups of test takers. A Bayesian approach with Markov chain Monte Carlo (MCMC) computation enables straightforward estimation of all model parameters. Model-specific implementations of a Bayes factor (BF) and deviance information criterium (DIC) for model selection are proposed which are easily calculated as byproducts of the MCMC computation. Both results from simulation studies and real-data examples are given to illustrate several novel analyses possible with this modeling framework. The authors thank Steven Wise, James Madison University, and Pere Joan Ferrando, Universitat Rovira i Virgili, for generously making available their data sets for the empirical examples in this paper.     

The Joint Multivariate Modeling of Multiple Mixed Response Sources: Relating Student Performances with Feedback Behavior

The present study concerns a Dutch computer-based assessment, which includes an assessment process about information literacy and a feedback process for students. The assessment is concerned with the measurement of skills in information literacy and the feedback process with item-based support to improve student learning. To analyze students’ feedback behavior (i.e. feedback use and attention time), test performance, and speed of working, a multivariate hierarchical latent variable model is proposed. The model can handle multivariate mixed responses from multiple sources related to different processes and comprehends multiple measurement components for responses and response times. A flexible within-subject latent variable structure is defined to explore multiple individual latent characteristics related to students’ test performance and feedback behavior. Main results of the computer-based assessment showed that feedback-information pages were less visited by well-performing students when they relate to easy items. Students’ attention paid to feedback was positively related to working speed but not to the propensity to use feedback.     

Modeling Differences Between Response Times of Correct and Incorrect Responses

Psychometrika - While standard joint models for response time and accuracy commonly assume the relationship between response time and accuracy to be fully explained by the latent variables of the...     

项目反应时间的对数偏正态模型

近年来,项目反应时间数据的建模是心理和教育测量领域的热门方向之一。针对反应时间的对数正态模型和Box-Cox正态模型的不足,本文在van der Linden的分层模型框架下基于偏正态分布建立一个反应时间的对数线性模型,并成功给出模型参数估计的马尔科夫链蒙特卡罗（Markov Chain Monte Carlo, MCMC）算法。模拟研究和实例分析的结果均表明,与对数正态模型和Box-Cox正态模型相比,对数偏正态模型表现出更加优良的拟合效果,具有更强的灵活性和适用性。     

潜变量交互效应建模: 告别均值结构

潜变量交互效应建模研究近年来有了长足的发展, 但模型中被认为不可缺少的均值结构往往让实际应用工作者却步。本文首先分析了潜变量交互效应模型中均值结构产生的根源; 然后讨论了指标变换与均值结构的关系; 接着提出了一个均值为零的潜变量交互结构, 所建立的模型不需要均值结构, 却不会改变主效应和交互效应等参数; 最后用模拟例子对无均值结构和有均值结构的两种模型的参数估计进行了比较, 结果符合理论预期, 困扰人们多年的均值结构问题从此可以终结。     

Item Response Theory with Estimation of the Latent Population Distribution Using Spline-Based Densities

The purpose of this paper is to introduce a new method for fitting item response theory models with the latent population distribution estimated from the data using splines. A spline-based density estimation system provides a flexible alternative to existing procedures that use a normal distribution, or a different functional form, for the population distribution. A simulation study shows that the new procedure is feasible in practice, and that when the latent distribution is not well approximated as normal, two-parameter logistic (2PL) item parameter estimates and expected a posteriori scores (EAPs) can be improved over what they would be with the normal model. An example with real data compares the new method and the extant empirical histogram approach.     

A Bivariate Generalized Linear Item Response Theory Modeling Framework to the Analysis of Responses and Response Times

A generalized linear modeling framework to the analysis of responses and response times is outlined. In this framework, referred to as bivariate generalized linear item response theory (B-GLIRT), separate generalized linear measurement models are specified for the responses and the response times that are subsequently linked by cross-relations. The cross-relations can take various forms. Here, we focus on cross-relations with a linear or interaction term for ability tests, and cross-relations with a curvilinear term for personality tests. In addition, we discuss how popular existing models from the psychometric literature are special cases in the B-GLIRT framework depending on restrictions in the cross-relation. This allows us to compare existing models conceptually and empirically. We discuss various extensions of the traditional models motivated by practical problems. We also illustrate the applicability of our approach using various real data examples, including data on personality and cognitive ability.     

A Hierarchical Framework for Modeling Speed and Accuracy on Test Items

Current modeling of response times on test items has been strongly influenced by the paradigm of experimental reaction-time research in psychology. For instance, some of the models have a parameter structure that was chosen to represent a speed-accuracy tradeoff, while others equate speed directly with response time. Also, several response-time models seem to be unclear as to the level of parametrization they represent. A hierarchical framework for modeling speed and accuracy on test items is presented as an alternative to these models. The framework allows a “plug-and-play approach” with alternative choices of models for the response and response-time distributions as well as the distributions of their parameters. Bayesian treatment of the framework with Markov chain Monte Carlo (MCMC) computation facilitates the approach. Use of the framework is illustrated for the choice of a normal-ogive response model, a lognormal model for the response times, and multivariate normal models for their parameters with Gibbs sampling from the joint posterior distribution. This study received funding from the Law School Admission Council (LSAC). The opinions and conclusions contained in this paper are those of the author and do not necessarily reflect the policy and position of LSAC. The author is indebted to the American Institute of Certified Public Accountants for the data set in the empirical example and to Rinke H. Klein Entink for his computational assistance     

Multilevel and Latent Variable Modeling with Composite Links and Exploded Likelihoods

Composite links and exploded likelihoods are powerful yet simple tools for specifying a wide range of latent variable models. Applications considered include survival or duration models, models for rankings, small area estimation with census information, models for ordinal responses, item response models with guessing, randomized response models, unfolding models, latent class models with random effects, multilevel latent class models, models with log-normal latent variables, and zero-inflated Poisson models with random effects. Some of the ideas are illustrated by estimating an unfolding model for attitudes to female work participation. We wish to thank The Research Council of Norway for a grant supporting our collaboration.     

被试作答的偶然性对IRT能力估计的影响研究

被试能力参数估计是项目反应理论应用研究最重要的技术之一。本文在理想的测验情境下,研究被试作答的偶然性对被试能力值估计的影响。研究设计了被试作答的两种偶然性情况：一是偶然做对了一道项目难度高于其能力值的试题,二是偶然做错了一道或几道项目难度低于其能力值的试题．然后分别探讨了这两种情况下对被试的能力估计所带来的影响,并且就如何消除这些偶然性所带来的影响提出了相应的方法。     

项目反应理论中潜在心理特质“填补”的参数估计方法及其演变

在心理与教育测量中, 项目反应理论(Item Response Theory, IRT)模型的参数估计方法是理论研究与实践应用的基本工具。最近, 由于IRT模型的不断扩展与EM (expectation-maximization)算法自身的固有问题, 参数估计方法的改进与发展显得尤为重要。这里介绍了IRT模型中边际极大似然估计的发展, 提出了它的阶段性特征, 即联合极大似然估计阶段、确定性潜在心理特质“填补”阶段、随机潜在心理特质“填补”阶段, 重点阐述了它的潜在心理特质“填补” (data augmentation)思想。EM算法与Metropolis-Hastings Robbins-Monro (MH-RM)算法作为不同的潜在心理特质“填补”方法, 都是边际极大似然估计的思想跨越。目前, 潜在心理特质“填补”的参数估计方法仍在不断发展与完善。     

融合反应时的多级评分IRT模型开发及其应用研究

当前大多数融合反应时的IRT模型仅适用于0-1评分数据资料,极大的限制了IRT反应时模型在实际中的应用。本文在传统的二级计分反应时IRT模型基础上,拟开发一种多级评分反应时模型。在层次建模框架下,分别采用拓广分部评分模型(GPCM)和对数正态模型构建融合反应时的多级评分IRT模型(本文记为JRT-GPCM),并采用全息贝叶斯MCMC算法实现新模型的参数估计。为验证新开发的JRT-GPCM模型的可行性及其在实践中的应用,本文开展了两项研究:研究1为模拟实验研究,研究2为新模型在大五人格-神经质分量表中的应用。研究1结果表明,JRT-GPCM模型的估计精度较高,且具有较好的稳健性。研究2表明,被试的潜在特质与作答速度具有一定的正相关,且本研究结果支持Ferrando和Lorenzo-Seva(2007)提出的“距离-困难度假设”,即当被试的潜在特质与项目的难度阈限距离越远,那么被试会花费更多的时间对项目进行作答。总之,本研究为拓展反应时信息在心理测量及教育中的应用提供新的方法支持。     

The bidirectional relation between positive mental health and psychopathology in a longitudinal representative panel study

Background: There is accumulating evidence that positive mental health and psychopathology should be seen as separate indicators of mental health. This study contributes to this evidence by investigating the bidirectional relation between positive mental health and psychopathological symptoms over time. Methods: Positive mental health (MHC-SF) and psychopathological symptoms (BSI) were longitudinally measured in a representative adult sample (N?=?1932) on four measurement occasions in nine months. A cross-lagged panel design was applied and evaluated with a latent growth model combined with an item response theory measurement model. Results: Psychopathological symptoms were longitudinally related to positive mental health and vice versa, controlling for initial levels. The changes over time were even more important than the absolute levels of psychopathological symptoms and positive mental health, respectively. Conclusions: The results underline the need for a comprehensive perspective on mental health, incorporating both the treatment of symptoms and the enhancement of well-being.     

工作记忆、加工速度和推理能力对小学儿童策略适应性的影响

研究选取89名小学三~五年级学生,探讨工作记忆、加工速度、推理能力以及年龄对小学儿童策略适应性的影响。通过路径分析发现:(1)工作记忆和推理能力对策略适应性有直接效应;工作记忆通过推理能力对策略适应性产生间接效应;加工速度通过"加工速度→工作记忆→策略适应性"和"加工速度→工作记忆→推理能力→策略适应性"两条路径对策略适应性起间接作用;在三个因素中,工作记忆对策略适应性的总效应最大,而推理能力对策略适应性的直接效应最大。(2)年龄对加工速度和推理能力有直接效应,但对工作记忆的效应不显著;年龄对策略适应性不产生直接效应,年龄通过"年龄→加工速度→工作记忆→策略适应性"、"年龄→加工速度→工作记忆→推理能力→策略适应性"和"年龄→推理能力→策略适应性"三条路径对策略适应性产生间接影响。     

When speech enhances Spatial Musical Association of Response Codes: Joint spatial associations of pitch and timbre in nonmusicians

Previous studies have shown that the effect of the Spatial Musical Association of Response Codes (SMARC) depends on various features, such as task conditions (whether pitch height is implicit or explicit), response dimension (horizontal vs. vertical), presence or absence of a reference tone, and former musical training of the participants. In the present study, we investigated the effects of pitch range and timbre: in particular, how timbre (piano vs. vocal) contributes to the horizontal and vertical SMARC effect in nonmusicians under varied pitch range conditions. Nonmusicians performed a timbre judgement task in which the pitch range was either small (6 or 8 semitone steps) or large (9 or 12 semitone steps) in a horizontal and a vertical response setting. For piano sounds, SMARC effects were observed in all conditions. For the vocal sounds, in contrast, SMARC effects depended on pitch range. We concluded that the occurrence of the SMARC effect, especially in horizontal response settings, depends on the interaction of the timbre (vocal and piano) and pitch range if vocal and instrumental sounds are combined in one experiment: the human voice enhances the attention, both to the vocal and the instrumental sounds.     

Differential Associations Between Psychopathy Dimensions,Types of Aggression,and Response Inhibition

Findings on executive functioning in psychopathy are inconsistent. Different associations between psychopathy dimensions and executive functioning might explain contradicting findings. This study examined the role of psychopathy dimensions and types of aggression in response inhibition among 117 male adolescents (53 antisocial delinquents and 64 controls). Participants completed a self‐report measure of aggression and a GoNoGo task. Psychopathy dimensions were assessed using the Psychopathy Checklist: Youth Version. Although high scores on the antisocial dimension and reactive aggression were associated with poor response inhibition, the affective–interpersonal dimension, proactive aggression, and verbal intelligence (IQ) were related to better response inhibition (two‐factor model). Associations with the affective–interpersonal dimensions did not reach significance. Exploratory analyses showed that affective and antisocial facets accounted for the obtained opposing associations of the affective–interpersonal and antisocial psychopathy dimensions with response inhibition. The interpersonal and lifestyle facets (four‐facet model) were unrelated to response inhibition. Results could not be explained by Attention Deficit Hyperactivity Disorder (ADHD). Findings suggest differential associations between the psychopathy dimensions, types of aggression, and response inhibition. Therefore, a dimensional approach to psychopathy and related concepts, such as aggression, might strongly improve diagnostic procedures. Global scores could mask important differential associations. Aggr. Behav. 38:77‐88, 2012. © 2011 Wiley Periodicals, Inc.     

基于作答数据的模型参数和Q矩阵联合估计

Q矩阵在认知诊断的模型参数估计和诊断分类中起着重要作用。本文通过研究Liu等人的方法, 设计了同时估计项目参数和Q矩阵的联合估计算法。在DINA模型下, 对项目参数未知时开展模拟研究。研究假设项目为20个, 考察的属性个数分别是3、4和5, 初始Q矩阵中分别存在3、4和5个属性界定错误的项目。结果表明, 联合估计算法能在错误的初始Q矩阵基础上以很高的概率得到正确的Q矩阵。另外, 当专家认定测验的属性个数存在错误时, 该方法推导的Q矩阵和模型参数能提供很好的鉴别Q矩阵错误的信息。     

Evaluation of Tests of Perceptual Speed/Accuracy and Spatial Ability for Use in Military Occupational Classification

With the exception of Assembling Objects (AO), a spatial ability test used only by the Navy in enlisted occupational classification, the Armed Services Vocational Aptitude Battery (ASVAB) is academic and knowledge-based, somewhat limiting its utility for occupational classification. This article presents the case for integrating the AO test into military classification composites and for expanding the breadth of ASVAB content by including a former ASVAB speed/accuracy test, Coding Speed (CS). Empirical evidence is presented that shows AO and CS (a) increment the validity of the ASVAB in predicting training grades for a broad array of occupations, (b) reduce adverse impact defined as test score barriers for women and minorities, and (c) improve classification in terms of matching recruits to occupations. Some cognitive theory is presented to support AO and CS, as well as nonverbal reasoning and working memory tests for inclusion in or adjuncts to the ASVAB.