Joint Modeling of Ability and Differential Speed Using Responses and Response Times

With computerized testing, it is possible to record both the responses of test takers to test questions (i.e., items) and the amount of time spent by a test taker in responding to each question. Various models have been proposed that take into account both test-taker ability and working speed, with the many models assuming a constant working speed throughout the test. The constant working speed assumption may be inappropriate for various reasons. For example, a test taker may need to adjust the pace due to time mismanagement, or a test taker who started out working too fast may reduce the working speed to improve accuracy. A model is proposed here that allows for variable working speed. An illustration of the model using the Amsterdam Chess Test data is provided.     

A Bivariate Generalized Linear Item Response Theory Modeling Framework to the Analysis of Responses and Response Times

A generalized linear modeling framework to the analysis of responses and response times is outlined. In this framework, referred to as bivariate generalized linear item response theory (B-GLIRT), separate generalized linear measurement models are specified for the responses and the response times that are subsequently linked by cross-relations. The cross-relations can take various forms. Here, we focus on cross-relations with a linear or interaction term for ability tests, and cross-relations with a curvilinear term for personality tests. In addition, we discuss how popular existing models from the psychometric literature are special cases in the B-GLIRT framework depending on restrictions in the cross-relation. This allows us to compare existing models conceptually and empirically. We discuss various extensions of the traditional models motivated by practical problems. We also illustrate the applicability of our approach using various real data examples, including data on personality and cognitive ability.     

Statistical Tests of Conditional Independence Between Responses and/or Response Times on Test Items

Three plausible assumptions of conditional independence in a hierarchical model for responses and response times on test items are identified. For each of the assumptions, a Lagrange multiplier test of the null hypothesis of conditional independence against a parametric alternative is derived. The tests have closed-form statistics that are easy to calculate from the standard estimates of the person parameters in the model. In addition, simple closed-form estimators of the parameters under the alternatives of conditional dependence are presented, which can be used to explore model modification. The tests were applied to a data set from a large-scale computerized exam and showed excellent power to detect even minor violations of conditional independence.     

Hidden Markov Item Response Theory Models for Responses and Response Times

Current approaches to model responses and response times to psychometric tests solely focus on between-subject differences in speed and ability. Within subjects, speed and ability are assumed to be constants. Violations of this assumption are generally absorbed in the residual of the model. As a result, within-subject departures from the between-subject speed and ability level remain undetected. These departures may be of interest to the researcher as they reflect differences in the response processes adopted on the items of a test. In this article, we propose a dynamic approach for responses and response times based on hidden Markov modeling to account for within-subject differences in responses and response times. A simulation study is conducted to demonstrate acceptable parameter recovery and acceptable performance of various fit indices in distinguishing between different models. In addition, both a confirmatory and an exploratory application are presented to demonstrate the practical value of the modeling approach.     

Modeling Response Times for Two-Choice Decisions

The diffusion model for two-choice real-time decisions is applied to four psychophysical tasks. The model reveals how stimulus information guides decisions and shows how the information is processed through time to yield sometimes correct and sometimes incorrect decisions. Rapid two-choice decisions yield multiple empirical measures: response times for correct and error responses, the probabilities of correct and error responses, and a variety of interactions between accuracy and response time that depend on instructions and task difficulty. The diffusion model can explain all these aspects of the data for the four experiments we present. The model correctly accounts for error response times, something previous models have failed to do. Variability within the decision process explains how errors are made, and variability across trials correctly predicts when errors are faster than correct responses and when they are slower.     

Using Response Times for Joint Modeling of Response and Omission Behavior

Abstract

For adequate modeling of missing responses, a thorough understanding of the nonresponse mechanisms is vital. As a large number of major testing programs are in the process or already have been moving to computer-based assessment, a rich body of additional data on examinee behavior becomes easily accessible. These additional data may contain valuable information on the processes associated with nonresponse. Bringing together research on item omissions with approaches for modeling response time data, we propose a framework for simultaneously modeling response behavior and omission behavior utilizing timing information for both. As such, the proposed model allows (a) to gain a deeper understanding of response and nonresponse behavior in general and, in particular, of the processes underlying item omissions in LSAs, (b) to model the processes determining the time examinees require to generate a response or to omit an item, and (c) to account for nonignorable item omissions. Parameter recovery of the proposed model is studied within a simulation study. An illustration of the model by means of an application to real data is provided.     

计算机化多维测验中作答时间和作答精度数据的联合分析

随着心理与教育测量研究的发展和科技的进步,计算机化(大规模)测验逐渐受到人们的关注。为探究在计算机化多维测验中如何利用作答时间数据来辅助评估多维潜在能力,以及为我国义务教育阶段教育质量监测提供数据分析方法上的理论支持。本研究以2012年和2015年国际学生能力评估(PISA)计算机化数学测验数据为例,提出了一种可同时利用作答时间和作答精度数据的联合作答与时间的多维Rasch模型。根据新模型对PISA数据的分析结果,表明引入作答时间数据,不仅有助于提高模型参数的估计精度,还有助于数据分析者利用被试的作答时间信息来做进一步的决策和干预(e.g., 对异常作答行为或预备知识的诊断)。     

Response Mixture Modeling: Accounting for Heterogeneity in Item Characteristics across Response Times

In item response theory modeling of responses and response times, it is commonly assumed that the item responses have the same characteristics across the response times. However, heterogeneity might arise in the data if subjects resort to different response processes when solving the test items. These differences may be within-subject effects, that is, a subject might use a certain process on some of the items and a different process with different item characteristics on the other items. If the probability of using one process over the other process depends on the subject’s response time, within-subject heterogeneity of the item characteristics across the response times arises. In this paper, the method of response mixture modeling is presented to account for such heterogeneity. Contrary to traditional mixture modeling where the full response vectors are classified, response mixture modeling involves classification of the individual elements in the response vector. In a simulation study, the response mixture model is shown to be viable in terms of parameter recovery. In addition, the response mixture model is applied to a real dataset to illustrate its use in investigating within-subject heterogeneity in the item characteristics across response times.     

Aesthetic Responses and Earliest Memories: The Differences Between Music and Literature

Aesthetic responses to music involve intense emotions that may be mitigated by musical training. The purpose of this study was to ascertain whether early or intense music memory and accessibility differed from literature memories and whether such reactions were differentiated by music training in college students (n = 90). Results showed no statistically significant differences among early or intense music and literature comparisons for analyses of music training or intensity ratings. Subjects used significantly more words to refer to music than to literature memories. Negative anecdotes were more likely for intense, rather than for early, memories. Literary memories were tied to specific books and authors, and music memories were triggered more often by the emotional context than by a specific composition.     

Modeling Dependence Structures for Response Times in a Bayesian Framework

A multivariate generalization of the log-normal model for response times is proposed within an innovative Bayesian modeling framework. A novel Bayesian Covariance Structure Model (BCSM) is proposed, where the inclusion of random-effect variables is avoided, while their implied dependencies are modeled directly through an additive covariance structure. This makes it possible to jointly model complex dependencies due to for instance the test format (e.g., testlets, complex constructs), time limits, or features of digitally based assessments. A class of conjugate priors is proposed for the random-effect variance parameters in the BCSM framework. They give support to testing the presence of random effects, reduce boundary effects by allowing non-positive (co)variance parameters, and support accurate estimation even for very small true variance parameters. The conjugate priors under the BCSM lead to efficient posterior computation. Bayes factors and the Bayesian Information Criterion are discussed for the purpose of model selection in the new framework. In two simulation studies, a satisfying performance of the MCMC algorithm and of the Bayes factor is shown. In comparison with parameter expansion through a half-Cauchy prior, estimates of variance parameters close to zero show no bias and undercoverage of credible intervals is avoided. An empirical example showcases the utility of the BCSM for response times to test the influence of item presentation formats on the test performance of students in a Latin square experimental design.

     

物体受力分析正误样例组合的学习效果

分别采用正误样例组合、有无错误标记的正误样例组合和有标记的正误样例配对组合设计方法,以物体受力分析正误样例为实验材料,以初中三年级学生为被试进行3项实验,考察了被试物体受力分析图正误样例组合的学习效果。结果表明,正误样例组合的学习成绩显著优于正确样例组合的学习成绩;有标记的正误样例组合的学习成绩显著优于无标记样例组合的学习成绩;有标记正误样例配对组合的学习成绩明显优于正确样例组合的学习成绩。     

Correct Responses and the Priority of the Normative

The ‘Wrong Kind of Reason’ problem for buck-passing theories (theories which hold that the normative is explanatorily or conceptually prior to the evaluative) is to explain why the existence of pragmatic or strategic reasons for some response to an object does not suffice to ground evaluative claims about that object. The only workable reply seems to be to deny that there are reasons of the ‘wrong kind’ for responses, and to argue that these are really reasons for wanting, trying, or intending to have that response. In support of this, it is pointed out that awareness of pragmatic or strategic considerations, unlike awareness of reasons of the ‘right kind’, are never sufficient by themselves to produce the responses for which they are reasons. I argue that this phenomenon cannot be used as a criterion for distinguishing reasons-for-a-response from reasons-for-wanting-to-have-a-response. I subsequently investigate the possibility of basing this distinction on a claim that the responses in question (e.g. admiration or desire) are themselves inherently normative; I conclude that this approach is also unsuccessful. Hence, the ‘direct response’ phenomenon cannot be used to rule out the possibility of pragmatic or strategic reasons for responses; and the rejection of such reasons therefore cannot be used to circumvent the Wrong Kind of Reason Problem.

兼顾两种匹配的考试抄袭统计量研究评述

考试抄袭统计量是专门用于识别抄袭的统计指标。它们是以被怀疑抄袭者和抄袭来源匹配反应的数目为基础建立的,按其考虑的匹配反应信息可分为两类:一类只考虑被怀疑抄袭者和抄袭来源间的错误答案匹配;另一类同时考虑两者错误答案匹配和正确答案匹配。由于第一类抄袭统计量对那些只从抄袭来源处抄袭正确答案的抄袭者检测不敏感,所以存在缺陷;第二类抄袭统计量兼顾两种匹配,考虑的信息更全面。s2、g2、ω是具有代表性的第二类抄袭统计量,它们在测量理论基础、被怀疑抄袭者和抄袭来源的异常相似反应模式确定方法及考试抄袭检测效能等方面存在差异,在抄袭识别应用中应合理选择。     

碳氢共价键结构式正误样例组合的学习效果

为考察错误样例在样例组合学习中的作用,以烃类物质碳氢共价键结构式为实验材料,以高中一年级学生为被试,对正确和错误样例组合的学习效果进行了实验。结果显示,正、误样例组合的学习成绩显著优于正确样例组合的学习成绩。在正、误样例组合的学习中,正、误样例数量相等的学习成绩明显优于数量不等的学习成绩;“对比”呈现正、误样例的学习成绩明显优于“分块”呈现正、误样例的学习成绩。     

正误样例组合学习的相似性效应和认知加工深度效应

为了考察物体受力分析图正、误样例的相似程度以及被试对错误受力分析的认知加工(即对错误受力分析的辨别、解释和改正)水平对物体受力分析规则正误样例组合学习迁移效果的影响,以初中三年级学生为被试进行两项实验研究。实验结果表明:在物体受力分析图正误样例组合学习中存在明显的相似性效应,即正、误样例的相似程度越高,学习迁移的效果越好;对错误受力分析进行"改正"加工的学习迁移效果明显优于对错误的"辨别"加工,即存在明显的认知加工深度效应。     

Comparing Mathematics Interventions: The Effects of Cover-Copy-Compare Alone and Combined with Performance Feedback on Digits Correct and Incorrect

The purpose of this study was to compare the isolated effects of cover-copy-compare (CCC) with the combined effects of CCC and two types of performance feedback (i.e., digits correct and incorrect per minute) on sixth grade students’ mathematics fluency and accuracy. An alternating treatments design was employed to compare treatments across 16 weeks. Participants included three general education sixth grade students whose teachers recommended them for additional help with mathematics calculation. No differentiation between treatments was demonstrated for any of the participants and individual differences pertaining to the number of sessions needed to reach and consistently perform at mastery levels were found. Follow-up data at 4 and 12 days following termination of the intervention phase indicated that all participants continued to perform at mastery levels. Generalization to a slightly more difficult mathematics skill yielded pre-intervention to post-intervention increases, albeit small. Students rated all three treatment conditions favorably.     

Examining Interindividual Differences in Cyclicity of Pleasant and Unpleasant Affects Using Spectral Analysis and Item Response Modeling

Weekly cycles in emotion were examined by combining item response modeling and spectral analysis approaches in an analysis of 179 college students' reports of daily emotions experienced over 7 weeks. We addressed the measurement of emotion using an item response model. Spectral analysis and multilevel sinusoidal models were used to identify interindividual differences in intraindividual cyclic change. Simulations and incomplete data designs were used to examine how well this combination of analysis techniques might work when applied to other practical data problems. Empirically, we found systematic individual differences in the extent to which individuals' emotions follow a weekly cycle, and in how such cycles are exhibited. Weekly cycles accounted for very little variance in day to day emotions at the individual level. Analytically, we illustrate how measurement, change, and interindividual difference models from different traditions may be combined in a practical manner to describe some of the complexities of human behavior. The authors gratefully acknowledge the support provided by grant T32 AG20500 from the National Institute on Aging in the preparation of this article. Special thanks to those at the Institute for Developmental and Health Research Methodology at the University of Virginia and to Paul De Boeck and the reviewers for helpful comments on earlier versions of this work.     

A Note on the Hierarchical Model for Responses and Response Times in Tests of van der Linden (2007)

Findings suggest that in psychological tests not only the responses but also the times needed to give the responses are related to characteristics of the test taker. This observation has stimulated the development of latent trait models for the joint distribution of the responses and the response times. Such models are motivated by the hope to improve the estimation of the latent traits by additionally considering response time. In this article, the potential relevance of the response times for psychological assessment is explored for the model of van der Linden (Psychometrika 72:287–308, 2007) that seems to have become the standard approach to response time modeling in educational testing. It can be shown that the consideration of response times increases the information of the test. However, one also can prove that the contribution of the response times to the test information is bounded and has a simple limit.     

Maternal Care and Individual Differences in Defensive Responses

Abstract— Familial transmission of mental illness is common. Recent studies in behavioral neuroscience and biological psychiatry reveal the importance of epigenetic mechanisms of transmission that center on the developmental consequences of variations in parental care. Studies with rats suggest that environmental adversity results in patterns of parent–offspring interactions that increase stress reactivity through sustained effects on gene expression in brain regions known to regulate behavioral, endocrine, and autonomic responses to stress. While such effects might be adaptive, the associated cost involves an increased risk for stress-related illness.     

Interdependency Between Risk Assessments for Self and Other in the Field of Comparative Optimism: The Contribution of Response Times

The Psychological Record - By introducing a response-time measure in the field of comparative optimism, this study was designed to explore how people estimate risk to self and others depending on...