计算机化多维测验中作答时间和作答精度数据的联合分析

随着心理与教育测量研究的发展和科技的进步,计算机化(大规模)测验逐渐受到人们的关注。为探究在计算机化多维测验中如何利用作答时间数据来辅助评估多维潜在能力,以及为我国义务教育阶段教育质量监测提供数据分析方法上的理论支持。本研究以2012年和2015年国际学生能力评估(PISA)计算机化数学测验数据为例,提出了一种可同时利用作答时间和作答精度数据的联合作答与时间的多维Rasch模型。根据新模型对PISA数据的分析结果,表明引入作答时间数据,不仅有助于提高模型参数的估计精度,还有助于数据分析者利用被试的作答时间信息来做进一步的决策和干预(e.g., 对异常作答行为或预备知识的诊断)。     

双因子模型MCAT中多级评分项目选题策略的比较

双因子模型假设测验考察一个一般因子和多个组因子,符合很多教育和心理测验的因素结构。“维度缩减”方法将参数估计中多维积分计算化简为多个迭代二维积分,是双因子模型的重要特征。本文针对考察多级评分项目的计算机化自适应测验,首先推导双因子等级反应模型下Fisher信息量的计算,然后推导“维度缩减”方法在项目选择方法中的应用,最后在低、中、高双因子模式题库中比较D-优化方法、后验加权Fisher信息D优化方法(PDO)、后验加权Kullback-Leibler方法(PKL)、连续熵(CEM)和互信息(MI)方法在能力估计的相关、均方根误差、绝对值偏差和欧氏距离的表现。模拟研究表明：(1)双因子模式越强,即一般因子和组因子在项目上的区分度的差异越小,一般因子估计精度降低,组因子估计精度增加,整体能力的估计精度提高;(2)相同实验条件下,连续熵方法的测量精度最高,PKL方法的能力估计精度最低,其它方法的测量精度没有显著差异。     

Combining mixture distribution and multidimensional IRTree models for the measurement of extreme response styles

Personality constructs, attitudes and other non-cognitive variables are often measured using rating or Likert-type scales, which does not come without problems. Especially in low-stakes assessments, respondents may produce biased responses due to response styles (RS) that reduce the validity and comparability of the measurement. Detecting and correcting RS is not always straightforward because not all respondents show RS and the ones who do may not do so to the same extent or in the same direction. The present study proposes the combination of a multidimensional IRTree model with a mixture distribution item response theory model and illustrates the application of the approach using data from the Programme for the International Assessment of Adult Competencies (PIAAC). This joint approach allows for the differentiation between different latent classes of respondents who show different RS behaviours and respondents who show RS versus respondents who give (largely) unbiased responses. We illustrate the application of the approach by examining extreme RS and show how the resulting latent classes can be further examined using external variables and process data from computer-based assessments to develop a better understanding of response behaviour and RS.     

Evaluation on types of invariance in studying extreme response bias with an IRTree approach

In recent years, item response tree (IRTree) approaches have received increasing attention in the response style literature for their ability to partial out response style latent variables as well as associated item parameters. When an IRTree approach is adopted to measure extreme response styles, directional and content invariance could be assumed at the latent variable and item parameter levels. In this study, we propose to evaluate the empirical validity of these invariance assumptions by employing a general IRTree model with relaxed invariance assumptions. This would allow us to examine extreme response biases, beyond extreme response styles. With three empirical applications of the proposed evaluation, we find that relaxing some of the invariance assumptions improves the model fit, which suggests that not all assumed invariances are empirically supported. Specifically, at the latent variable level, we find reasonable evidence for directional invariance but mixed evidence for content invariance, although we also find that estimated correlations between content-specific extreme response latent variables are high, hinting at the potential presence of a general extreme response tendency. At the item parameter level, we find no directional or content invariance for thresholds and no content invariance for slopes. We discuss how the variant item parameter estimates obtained from a general IRTree model can offer useful insight to help us understand response bias related to extreme responding measured within the IRTree framework.     

Scoring Depression on a Common Metric: A Comparison of EAP Estimation,Plausible Value Imputation,and Full Bayesian IRT Modeling

There are a growing number of item response theory (IRT) studies that calibrate different patient-reported outcome (PRO) measures, such as anxiety, depression, physical function, and pain, on common, instrument-independent metrics. In the case of depression, it has been reported that there are considerable mean score differences when scoring on a common metric from different, previously linked instruments. Ideally, those estimates should be the same. We investigated to what extent those differences are influenced by different scoring methods that take into account several levels of uncertainty, such as measurement error (through plausible value imputation) and item parameter uncertainty (through full Bayesian IRT modeling). Depression estimates from different instruments were more similar, and their corresponding confidence/credible intervals were larger when plausible value imputation or Bayesian modeling was used, compared to the direct use of expected a posteriori (EAP) estimates. Furthermore, we explored the use of Bayesian IRT models to update item parameters based on newly collected data.     

Evaluating Factorial Invariance: An Interval Estimation Approach Using Bayesian Structural Equation Modeling

In this study, we introduce an interval estimation approach based on Bayesian structural equation modeling to evaluate factorial invariance. For each tested parameter, the size of noninvariance with an uncertainty interval (i.e. highest density interval [HDI]) is assessed via Bayesian parameter estimation. By comparing the most credible values (i.e. 95% HDI) with a region of practical equivalence (ROPE), the Bayesian approach allows researchers to (1) support the null hypothesis of practical invariance, and (2) examine the practical importance of the noninvariant parameter. Compared to the traditional likelihood ratio test, simulation results suggested that the proposed Bayesian approach could offer additional insight into evaluating factorial invariance, thus, leading to more informative conclusions. We provide an empirical example to demonstrate the procedures necessary to implement the proposed method in applied research. The importance of and influences on the choice of an appropriate ROPE are discussed.     

Differential effects of alcohol on associative versus item memory

Alcohol has detrimental effects on a range of cognitive processes, the most prominent being episodic memory. These deficits appear functionally similar to those observed within the normal aging population. We investigated whether an associative memory deficit, as found in older adults, would also be evident in young adults moderately intoxicated by alcohol. Participants were shown unrelated word pairs and then tested on both their item recognition (old/new item?) and associative recognition (intact/recombined pair?). Half the participants were under the influence of alcohol whereas the other half were sober. Alcohol impaired memory performance but significantly more so for associative than for item memory. Moreover, within the alcohol group, the associative memory deficit was significantly related to the amount of alcohol consumed. The findings suggest that not all aspects of episodic memory are equally impaired by alcohol, which may have practical implications for criminal investigations involving eye witnesses who have consumed alcohol.     

情绪对项目记忆和来源记忆的影响

最近几十年以来, 情绪对记忆的影响越来越受到认知心理学家的关注。本文总结了我在中科院心理研究所师从傅小兰研究员攻读博士学位期间所开展的主要研究工作。我的研究以中文词语为学习材料, 分别考察了在编码过程中和编码结束后诱发情绪对项目记忆和来源记忆的影响。当情绪在编码阶段被诱发, 主要的研究发现如下：1)负性情绪增强自由回忆, 但正性情绪只呈现出增强自由回忆的趋势。2)负性情绪降低总体再认记忆, 但正性情绪对总体再认记忆没有影响。此外, 正性和负性情绪均不影响“记得”反应的正确率; 然而, 负性情绪降低了“知道”反应的正确率, 正性情绪对“知道”反应的正确率没有影响。3)正性和负性情绪均对来源记忆(即对词语字体颜色的记忆)没有影响。当情绪在编码结束后被诱发, 主要的研究发现如下：1)对女性被试而言, 负性情绪增强了项目记忆的巩固, 然而, 对男性被试而言, 正性和负性情绪均不影响项目记忆的巩固。2)无论对男性还是女性被试, 正性和负性情绪对来源记忆的巩固均没有影响。本研究的上述发现对建立情绪和记忆的关系具有理论意义, 对如何在教学情境中如何利用情绪诱发促进学生的记忆巩固也具有现实指导意义。     

多面Rasch模型理论及其在结构化面试中的应用

针对影响面试效度的各种误差来源,该文引入了一种新颖的面试结果处理方法：多面Rasch模型。这一模型在结构化面试中的应用不但有利于有效测量被试的能力水平,而且为识别问题评委、进一步完善评分规则、实现面试等值等问题都提供了全新的解决思路。文章在对结构化面试信、效度研究进展进行综述的基础上,介绍了多面Rasch模型的理论及其在结构化面试中的应用框架。     

The generalized Logit-Linear Item Response Model for Binary-Designed Items

This paper introduces the generalized logit-linear item response model (GLLIRM), which represents the item-solving process as a series of dichotomous operations or steps. The GLLIRM assumes that the probability function of the item response is a logistic function of a linear composite of basic parameters which describe the operations, and the coefficients depend on three design matrices X, Y and Z. The GLLIRM provides a tool for testing hypotheses on the item-solving process and generalizes existing models. An empirical application is included, in which the model is applied to evaluate sources of difficulty and pairwise item interactions in a logical analysis test. This research was supported by the Comunidad de Madrid grant CCG06-UAM/ESP-0043.