一个新的测量过程框架——对引入认知加工模型的再思考

从认知任务分析出发的测验设计,核心在于从认知加工的角度对项目底层作答机制作出解释。其中,能够刻画任务难度的模型最有可能与心理计量模型相结合。因而,具备成熟难度法则的各种小型理论或通用理论可以进入测量过程,实现从纯粹误差结构控制的测量到内容导引的测量的转变。在这样一个框架之下,理论对任务上作答过程的理解是否恰当,刺激特性变量与任务难度关系的揭示(即难度法则)是否准确,可以在测量的过程中进行证伪。项目反应理论还未及很好回答的效度问题,可望在这一拓展的框架中获得圆满解决。本文为新的测量过程框架提供了一个示意图。     

IRT框架下追踪数据的测量不变性分析——————以4至5岁儿童认知能力测验为例

本研究以4岁~5岁儿童认知能力测验为例,在IRT框架下探讨了如何进行追踪数据的测量不变性分析。分析模型采用项目间多维项目反应理论模型(between-item MIRT model)和项目内（within-item MIRT model）多维two-tier model,被试为来自全国的882名48个月的儿童,工具为自编4岁~5岁儿童认知能力测验。经测验水平 分析和项目水平分析,结果表明：(1)本文对追踪数据的测量不变性分析方法合理有效; (2)该测验在两个时间点上满足部分测量不变性要求,测验的潜在结构稳定; (3)“方位题”的区分度和难度参数都发生变化,另有4题难度参数出现浮动; (4)儿童在4岁~5岁期间认知能力总体呈快速发展趋势,能力增长显著。     

混合IRT潜在模型及其应用轨迹

项目反应理论是测量被试潜在特质的现代测量理论, 潜在类别分析是基于模型的潜在特质分类技术。混合项目反应理论将项目反应理论与潜在类别分析相结合, 能够同时对被试分类并量化其潜在特质。在阐述混合项目反应理论概念、原理的基础上, 介绍了MRM、mNRM和mPCM等几种常见混合模型及其参数估计方法, 并从心理与行为特征分类、项目功能差异检测、测验效度评价等方面评述了其在心理测验中的应用发展轨迹。     

分部评分模型与其它几种多级模型的比较

项目反应理论(IRT)是当前教育与心理测量领域的一个重大进展,也日益成为当今测验领域的一个热点。本文对IRT理论中的二个重要评分模型PCM与GPCM作了简要的介绍。并较详细地对PCM与GPCM模型的逻辑结构及其适用范围进行了论述,指出了它们与其它几种重要模型之间的关系及不同之处。     

多水平IRT的发展与应用述评

阶层线性模型是处理阶层结构数据的高级统计方法, 项目反应理论是精确测量被试能力的现代测量理论。多水平项目反应理论将阶层线性模型和项目反应理论相结合, 将项目反应模型嵌套在阶层线性模型内, 实现了项目参数和不同水平能力参数的估计, 对回归系数和误差项变异的估计也更加精确。作者概述了多水平项目反应理论的发展历程, 并从项目功能差异、测验等值、学校效能研究等方面评述了多水平项目反应理论在心理与教育测量中的应用, 总结了多水平项目反应理论的价值, 同时展望了今后的研究趋势。     

人格测量：从累积式模型到展开式模型

当前大多数人格测量都采用的是累积式反应模型方法,该模型假设被试在测验上的得分随其能力或特质提高而增加,但是随着人格测量技术的不断发展,这一模型的实施效果遭到了质疑,研究者们开始关注展开式模型,该模型认为被试的反应取决于被试能力和项目阈值的匹配程度,当被试能力与项目阈值完全匹配时,被试做出肯定回答的概率达到最高点,称之为“理想点”,展开式模型的目的就是找到被试的理想点,从而寻找其真正的态度强度或人格特质水平。GGUM作为一种比较成熟的展开式模型,已经开始应用于人格测量的各个领域,但仍需要进行大规模的试测,在评估和预测效度方面积累经验,建立业界认可的心理测量学标准,不断探讨和开发相应的心理测量理论和简便易行的统计程序。     

计算机化自适应测验选题策略述评

计算机化自适应测验(computerized adaptive testing, CAT)是基于测量理论和计算机技术的一种测验模式。它根据考生的作答反应自适应地选择测验项目。选题策略是CAT的重要组成部分之一, 关系到测量效率、测验安全和测验信、效度等重要问题。根据CAT是否具有非统计约束对传统CAT和认知诊断CAT的选题策略进行了分类介绍, 未来研究应进一步提高选题策略的综合表现、深入探讨多级评分项目和认知诊断CAT的选题策略。     

速度测量能否预测PASS模型所指的智力?

ＰＡＳＳ模型是达斯（Ｊ．Ｐ．Ｄａｓ）等人提出的新智力模型。该模型以认知加工过程为基础，主张智力是由四个相互关联的过程：计划过程、注意过程和编码过程（合同时性加工和维时性加工过程）构成。据此，他们编制了ＤＮ认知评价系统来测量智力，这一系统中的许多分测验以完成任务的时间为指标。因此，对该评价系统有效性的考察除去讨论其理论的合理性和内容效度外，时间指标运用的恰当与否是一重要问题。时间多少往往是速度的体现，这样，速度与智力的关系问题又一次展现在我们面前。自智力测量初始，智力与速度两者之间存在何种关系的问…     

心理科学领域内的客观测量——Rasch模型之特点及发展趋势

Rasch模型是在国外学术界受到广泛关注和深入研究的一个潜在特质模型。该模型为解决心理科学领域内测量的客观性问题提供了一个可行性很高的解决方案。而国内关于Rasch模型的理论探讨和应用研究却并不多见。不同于一般项目反应理论, Rasch模型要求所收集的数据必须符合模型的先验要求, 而不是使用不同的参数去适应数据的特点。Rasch模型的主要特点(包括个体与题目共用标尺、线性数据、参数分离)确保了客观测量的实现。未来关于Rasch模型的研究方向包括多维度Rasch模型、测验的等值与链接、计算机自适应性考试, 大型应用测量系统(比如Lexile系统)等等。     

瑞文测验项目认知难度因素分析及LLTM拟合验证

该文在以认知和测量相结合为特征的新一代测验理论的指导下,针对项目刺激特征,对影响瑞文测验项目认知难度的因素进行了分析,提取出图块繁简度、整体布局、变化维度和题序等4个因素,然后应用线性逻辑潜在特质模型（LLTM）对这四个因素进行了拟合检验,结果证明提取出的四个因素是有效的。文章还进一步探讨了研究结果在智力测验中的应用。     

Generating items during testing: Psychometric issues and models

On-line item generation is becoming increasingly feasible for many cognitive tests. Item generation seemingly conflicts with the well established principle of measuring persons from items with known psychometric properties. This paper examines psychometric principles and models required for measurement from on-line item generation. Three psychometric issues are elaborated for item generation. First, design principles to generate items are considered. A cognitive design system approach is elaborated and then illustrated with an application to a test of abstract reasoning. Second, psychometric models for calibrating generating principles, rather than specific items, are required. Existing item response theory (IRT) models are reviewed and a new IRT model that includes the impact on item discrimination, as well as difficulty, is developed. Third, the impact of item parameter uncertainty on person estimates is considered. Results from both fixed content and adaptive testing are presented.This article is based on the Presidential Address Susan E. Embretson gave on June 26, 1999 at the 1999 Annual Meeting of the Psychometric Society held at the University of Kansas in Lawrence, Kansas. —Editor     

Invariance Properties for General Diagnostic Classification Models

In item response theory (IRT), the invariance property states that item parameter estimates are independent of the examinee sample, and examinee ability estimates are independent of the test items. While this property has long been established and understood by the measurement community for IRT models, the same cannot be said for diagnostic classification models (DCMs). DCMs are a newer class of psychometric models that are designed to classify examinees according to levels of categorical latent traits. We examined the invariance property for general DCMs using the log-linear cognitive diagnosis model (LCDM) framework. We conducted a simulation study to examine the degree to which theoretical invariance of LCDM classifications and item parameter estimates can be observed under various sample and test characteristics. Results illustrated that LCDM classifications and item parameter estimates show clear invariance when adequate model data fit is present. To demonstrate the implications of this important property, we conducted additional analyses to show that using pre-calibrated tests to classify examinees provided consistent classifications across calibration samples with varying mastery profile distributions and across tests with varying difficulties.     

项目反应理论中模型-资料拟合检验常用统计量

项目反应理论(IRT)模型依据项目与被试的特征预测被试的作答表现, 是常用的心理测量模型。但IRT的有效运用依赖于所选用IRT模型与实际数据资料相符合的程度(即模型?资料拟合度, goodness of fit)。只有当所采用IRT分析模型与实际数据资料拟合较好时, IRT的优点和功能才能真正发挥出来(Orlando & Thissen, 2000)。而当所采用IRT模型与资料不拟合或选择了错误的模型, 则会导致如参数估计、测验等值及项目功能差异分析等具有较大误差(Kang, Cohen & Sung, 2009), 给实际工作带来不良影响。因此, 在使用IRT分析时, 应首先充分考察及检验所选用模型与实际数据是否相匹配/相拟合(McKinley & Mills, 1985)。IRT领域中常用模型?资料拟合检验统计量可从项目拟合、测验拟合两个角度进行阐述并比较, 这是心理、教育测量领域的重要主题, 也是测验分析过程中较易忽视的环节, 目前还未见此类公开发表的文章。未来的研究可以在各统计量的实证比较研究以及在认知诊断领域的拓展方面有所发展。     

An Introduction to Network Psychometrics: Relating Ising Network Models to Item Response Theory Models

In recent years, network models have been proposed as an alternative representation of psychometric constructs such as depression. In such models, the covariance between observables (e.g., symptoms like depressed mood, feelings of worthlessness, and guilt) is explained in terms of a pattern of causal interactions between these observables, which contrasts with classical interpretations in which the observables are conceptualized as the effects of a reflective latent variable. However, few investigations have been directed at the question how these different models relate to each other. To shed light on this issue, the current paper explores the relation between one of the most important network models—the Ising model from physics—and one of the most important latent variable models—the Item Response Theory (IRT) model from psychometrics. The Ising model describes the interaction between states of particles that are connected in a network, whereas the IRT model describes the probability distribution associated with item responses in a psychometric test as a function of a latent variable. Despite the divergent backgrounds of the models, we show a broad equivalence between them and also illustrate several opportunities that arise from this connection.     

Automatic Generation of Rasch-Calibrated Items: Figural Matrices Test GEOM and Endless-Loops Test EC

The future of test construction for certain psychological ability domains that can be analyzed well in a structured manner may lie—at the very least for reasons of test security—in the field of automatic item generation. In this context, a question that has not been explicitly addressed is whether it is possible to embed an item response theory (IRT) based psychometric quality control procedure directly into the process of automatic item generation. Research in this area was conducted using 2 item generators (for the 2 domains of reasoning and spatial ability) that were developed and based on relevant models of cognitive psychology. During the course of the 4 studies reported here, those parts of the generators that check for possible violations of psychometric quality ("constraints") were improved. The main findings indicate that quality control procedures can be embedded in automatic item generators depending on (a) the degree to which the domain to be measured can be structured; (b) item-specific, content-based analyses; and (c) the degree to which the constraints can be implemented in software. Furthermore, beyond the global check of given model fit via IRT, the content-based analysis of items may be valuable in terms of finding such item properties that may lead to violations of psychometric quality.     

Asymmetric Item Characteristic Curves and Item Complexity: Insights from Simulation and Real Data Analyses

While item complexity is often considered as an item feature in test development, it is much less frequently attended to in the psychometric modeling of test items. Prior work suggests that item complexity may manifest through asymmetry in item characteristics curves (ICCs; Samejima in Psychometrika 65:319–335, 2000). In the current paper, we study the potential for asymmetric IRT models to inform empirically about underlying item complexity, and thus the potential value of asymmetric models as tools for item validation. Both simulation and real data studies are presented. Some psychometric consequences of ignoring asymmetry, as well as potential strategies for more effective estimation of asymmetry, are considered in discussion.     

Psychometric Approaches Help Resolve Competing Cognitive Models: When Less Is More Than It Seems

Simple arithmetic word problems are often featured in elementary school education. One type of problem, "compare with unknown reference set," ranks among the most difficult to solve. Differences in item difficulty for compare problems with unknown reference set are observed depending on the direction of the relational statement (more than vs. less than). Various cognitive models have been proposed to account for these differences. We employed item response theory (IRT) to compare competing cognitive models of student performance. The responses of 100 second-grade students to a series of compare problems with unknown reference set, along with other measures of individual differences, were fit to IRT models. Results indicated that the construction integration model (Kintsch, 1988, 1998) provided the best fit to the data. We discuss the potential contribution of psychometric approaches to the study of thinking.     

Loglinear Rasch model tests

Existing statistical tests for the fit of the Rasch model have been criticized, because they are only sensitive to specific violations of its assumptions. Contingency table methods using loglinear models have been used to test various psychometric models. In this paper, the assumptions of the Rasch model are discussed and the Rasch model is reformulated as a quasi-independence model. The model is a quasi-loglinear model for the incomplete subgroup × score × item 1 × item 2 × ... × itemk contingency table. Using ordinary contingency table methods the Rasch model can be tested generally or against less restrictive quasi-loglinear models to investigate specific violations of its assumptions.     

An Introduction to Item Response Theory Using the Need for Cognition Scale

This paper provides an introduction to two commonly used item response theory (IRT) models (the two-parameter logistic model and the graded response model). Throughout the paper, the Need for Cognition Scale (NCS) is used to help illustrate different features of the IRT model. After introducing the IRT models, I explore the assumptions these models make as well as ways to assess the extent to which those assumptions are plausible. Next, I describe how adopting an IRT approach to measurement can change how one thinks about scoring, score precision, and scale construction. I briefly introduce the advanced topics of differential item functioning and computerized adaptive testing before concluding with a summary of what was learned about IRT generally, and the NCS specifically.