Bayesian Estimation of the DINA Q matrix

Cognitive diagnosis models are partially ordered latent class models and are used to classify students into skill mastery profiles. The deterministic inputs, noisy “and” gate model (DINA) is a popular psychometric model for cognitive diagnosis. Application of the DINA model requires content expert knowledge of a Q matrix, which maps the attributes or skills needed to master a collection of items. Misspecification of Q has been shown to yield biased diagnostic classifications. We propose a Bayesian framework for estimating the DINA Q matrix. The developed algorithm builds upon prior research (Chen, Liu, Xu, & Ying, in J Am Stat Assoc 110(510):850–866, 2015) and ensures the estimated Q matrix is identified. Monte Carlo evidence is presented to support the accuracy of parameter recovery. The developed methodology is applied to Tatsuoka’s fraction-subtraction dataset.     

A general diagnostic model applied to language testing data

Probabilistic models with one or more latent variables are designed to report on a corresponding number of skills or cognitive attributes. Multidimensional skill profiles offer additional information beyond what a single test score can provide, if the reported skills can be identified and distinguished reliably. Many recent approaches to skill profile models are limited to dichotomous data and have made use of computationally intensive estimation methods such as Markov chain Monte Carlo, since standard maximum likelihood (ML) estimation techniques were deemed infeasible. This paper presents a general diagnostic model (GDM) that can be estimated with standard ML techniques and applies to polytomous response variables as well as to skills with two or more proficiency levels. The paper uses one member of a larger class of diagnostic models, a compensatory diagnostic model for dichotomous and partial credit data. Many well‐known models, such as univariate and multivariate versions of the Rasch model and the two‐parameter logistic item response theory model, the generalized partial credit model, as well as a variety of skill profile models, are special cases of this GDM. In addition to an introduction to this model, the paper presents a parameter recovery study using simulated data and an application to real data from the field test for TOEFL^® Internet‐based testing.     

Q矩阵包含错误的诊断测验分类准确性比较

Q矩阵是认知诊断测验的重要组成部分之一,围绕Q矩阵构建的诊断模型对Q矩阵中包含的错误较敏感。贝叶斯网分类模型是基于网络结点之间的关系构建的模型,将朴素贝叶斯网作为诊断模型,与DINA模型进行比较。模拟实验结果表明：Q矩阵中是否包含可达矩阵和错误界定的项目数量对DINA模型影响较大,对贝叶斯网模型影响较小;项目数量对DINA和贝叶斯网模型影响都较大;样本大小对贝叶斯网模型影响较大,对DINA模型影响较小。模拟研究结果显示,当Q矩阵中不包含可达阵、包含5个以上错误项目或样本数较大时,贝叶斯网分类模型优于DINA模型;而当Q矩阵中包含可达阵和5个(以下)错误项目时,DINA模型优于贝叶斯分类模型。     

认知诊断模型的比较及其应用研究：饱和模型、简化模型还是混合方法？

GDINA是一个饱和认知诊断模型（Cognitive Diagnosis Models, CDM）,Wald检验被用于在题目水平上检验GDINA是否可以被简化模型（如DINA, DINO, ACDM和RRUM）替代,并为测验的每一个题目选择一个最恰当的CDM（简称混合CDM）。选择合适的CDM是进行诊断评估的一个关键步骤,通过Monte Carlo 模拟实验,比较了不同的测验情境下,GDINA、简化CDM和混合CDM在测验整体拟合指标、模式判准率和项目参数估计的返真性等效果,研究发现混合模型的整体表现是最好的,其次是GDINA,最后是简化CDM。     

G-DINA认知诊断模型在语言测验中的验证

G-DINA模型是DINA 模型的一般化模型,具有补偿性和饱和性两个主要特征。G-DINA模型的补偿性特征契合了语言测验的综合性和多元性,G-DINA模型的饱和性特征则可以比较理想地应对语言技能的抽象性和难区分性。此项研究以代表性的语言测验类型阅读测验为案例,应用G-DINA模型对1029名被试的PISA英语阅读测验结果进行实证分析,证明了两个假设：补偿饱和型认知诊断模型对多元抽象的语言测验的适应程度较高;G-DINA这一新生认知诊断模型可以被用来诊断较为复杂抽象的语言测验,且经得起统计学和语言学理论的双重考验。     

The conjunctive model of hierarchical classes

This paper describes the conjunctive counterpart of De Boeck and Rosenberg's hierarchical classes model. Both the original model and its conjunctive counterpart represent the set-theoretical structure of a two-way two-mode binary matrix. However, unlike the original model, the new model represents the row-column association as a conjunctive function of a set of hypothetical binary variables. The conjunctive nature of the new model further implies that it may represent some conjunctive higher order dependencies among rows and columns. The substantive significance of the conjunctive model is illustrated with empirical applications. Finally, it is shown how conjunctive and disjunctive hierarchical classes models relate to Galois lattices, and how hierarchical classes analysis can be useful to construct lattice models of empirical data.The research reported in this paper was supported by NATO (Grant CRG.921321 to Iven Van Mechelen and Seymour Rosenberg) and by the Research Fund of Katholieke Universiteit Leuven (Grants PDM92/19 and POR93/3 to Iven Van Mechelen; Grants OT89/9 and F91/56 to Paul De Boeck).     

A sequential cognitive diagnosis model for polytomous responses

This paper proposes a general polytomous cognitive diagnosis model for a special type of graded responses, where item categories are attained in a sequential manner, and associated with some attributes explicitly. To relate categories to attributes, a category‐level Q‐matrix is used. When the attribute and category association is specified a priori, the proposed model has the flexibility to allow different cognitive processes (e.g., conjunctive, disjunctive) to be modelled at different categories within a single item. This model can be extended for items where categories cannot be explicitly linked to attributes, and for items with unordered categories. The feasibility of the proposed model is examined using simulated data. The proposed model is illustrated using the data from the Trends in International Mathematics and Science Study 2007 assessment.     

多维题组效应认知诊断模型

当前认知诊断领域还缺少对包含题组的测验进行诊断分析的研究, 即已开发的认知诊断模型无法合理有效地处理含有题组效应的测验数据, 且已开发的题组反应模型也不具有对被试知识结构或认知过程进行诊断的功能。针对该问题, 本文尝试性地将多维题组效应向量参数引入线性Logistic模型中, 同时开发了属性间具有补偿作用的和属性间具有非补偿作用的多维题组效应认知诊断模型。模拟研究结果显示新模型合理有效, 与线性Logistic模型和DINA模型对比研究后表明：(1)作答数据含有题组效应时, 忽略题组效应会导致项目参数的偏差估计并降低对目标属性的判准率; (2)新模型更具普适性, 即便当作答数据不存在题组效应时, 采用新模型进行测验分析亦能得到很好的项目参数估计结果且不影响对目标属性的判准率。整体来看, 新模型既具有认知诊断功能又可有效处理题组效应。     

Conjunctive item response theory kernels

Conjunctive item response models are introduced such that (a) sufficient statistics for latent traits are not necessarily additive in item scores; (b) items are not necessarily locally independent; and (c) existing compensatory (additive) item response models including the binomial, Rasch, logistic, and general locally independent model are special cases. Simple estimates and hypothesis tests for conjunctive models are introduced and evaluated as well. Conjunctive models are also identified with cognitive models that assume the existence of several individually necessary component processes for a global ability. It is concluded that conjunctive models and methods may show promise for constructing improved tests and uncovering conjunctive cognitive structure. It is also concluded that conjunctive item response theory may help to clarify the relationships between local dependence, multidimensionality, and item response function form.I appreciate the many helpful suggestions that were given by the reviewers and Ivo Molenaar.     

使用验证性补偿多维IRT模型进行认知诊断评估

随着人们对测验反馈结果精细化的需求逐渐提高, 具有认知诊断功能的测量方法逐渐受到人们的关注。在认知诊断模型(CDMs)闪耀着光芒的同时, 另一类能够在连续量尺上提供精细反馈的多维IRT模型(MIRTMs)似乎受到些许冷落。为探究MIRTMs潜在的认知诊断功能, 本文以补偿模型为视角, 聚焦于分别属于MIRTMs的多维两参数logistic模型(M2PLM)和属于CDMs的线性logistic模型(LLM); 之后为使两者具有可比性, 可对补偿M2PLM引入验证性矩阵(Q矩阵)来界定题目与维度之间的关系, 进而得到验证性的补偿M2PLM (CC-M2PLM), 并通过把潜在特质按切点划分为跨界属性, 以期使CC-M2PLM展现出其本应具有的认知诊断功能; 预研究表明logistic量尺上的0点可作为相对合理的切点; 然后, 通过模拟研究对比探究CC-M2PLM和LLM的认知诊断功能, 结果表明CC-M2PLM可用于分析诊断测验数据, 且认知诊断功能与直接使用LLM的效果相当; 最后, 以两则实证数据为例来说明CC-M2PLM在实际诊断测验分析中的可行性。     

Nonlinear,near photo-realistic caricatures using a parametric facial appearance model

A mathematical model previously developed for use in computer vision applications is presented as an empirical model for face space. The term appearance space is used to distinguish this from previous models. Appearance space is a linear vector space that is dimensionally optimal, enables us to model and describe any human facial appearance, and possesses characteristics that are plausible for the representation of psychological face space. Randomly sampling from a multivariate distribution for a location in appearance space produces entirely plausible faces, and manipulation of a small set of defining parameters enables the automatic generation of photo-realistic caricatures. The appearance space model leads us to the new concept of nonlinear caricatures, and we show that the accepted linear method for caricature is only a special case of a more general paradigm. Nonlinear methods are also viable, and we present examples of photographic quality caricatures, using a number of different transformation functions. Results of a simple experiment are presented that suggest that nonlinear transformations can accurately capture key aspects of the caricature effect. Finally, we discuss the relationship between appearance space, caricature, and facial distinctiveness. On the basis of our new theoretical framework, we suggest an experimental approach that can yield new evidence for the plausibility of face space and its ability to explain processes of recognition.     

重参数化多分属性DINA模型的多级评分拓广——基于等级反应模型

多分属性认知诊断模型（CDMs）比传统的二分属性CDMs提供更详细的诊断反馈信息,但现有大部分多分属性CDMs并不具备直接分析多级（或混合）评分数据的功能。本文基于等级反应模型对重参数化多分属性DINA模型进行多级评分拓广,开发一个可处理多级评分数据的等级反应多分属性DINA模型。首先通过实证数据分析呈现新模型的现实可应用性;然后通过模拟研究探究新模型的参数估计返真性。结果表明,新模型满足同时处理多分属性和多级评分数据的现实需求;且具备良好的心理计量学性能,但对测验质量有一定要求（e.g., 题目质量较高且测验Qp矩阵具有完备性等）。     

认知诊断中多分属性与二分属性的对比研究

摘 要：Karelitz（2004）和詹沛达等（2016）认为1个多分属性内部(Lk+1)个水平的关系相当于Lk个部分满足线型层级关系的二分属性。本研究的目的是通过比较多分属性模型和二分属性模型的判准率,从而验证多分属性和二分属性间是否存在以上关系。结果表明：当属性个数较少时,两个模型的模式判准率相当,随着属性个数增加,多分属性模型的模式判准率高于二分属性模型的模式判准率。结论：在一定程度上,多分属性和二分属性之间确实存在以上关系,但两者并非完全等价,二者间的差异随着属性个数增加更加明显。     

认知诊断模型中项目水平模型比较统计量的健壮性

使用模拟研究方法比较了以往研究中提出的基于观察信息矩阵、三明治矩阵的Wald（分别表示为W_Obs、W_Sw）、似然比（Likelihood Ratio）统计量以及新提出的基于经验交叉相乘信息矩阵的Wald统计量（W_XPD）在模型——数据失拟条件下进行项目水平上模型比较时的表现。结果显示：（1）W_Sw的一类错误控制率有很强的健壮性。（2）W_XPD在Q矩阵错误设定的大多数条件下的表现优于W_Sw。结论：模型—数据拟合良好时可以使用W_Sw进行项目水平上的模型比较,当模型与数据失拟时W_XPD可能是更好的选择。     

认知诊断模型中项目水平模型比较统计量的健壮性

使用模拟研究方法比较了以往研究中提出的基于观察信息矩阵、三明治矩阵的Wald（分别表示为W_Obs、W_Sw）、似然比（Likelihood Ratio）统计量以及新提出的基于经验交叉相乘信息矩阵的Wald统计量（W_XPD）在模型——数据失拟条件下进行项目水平上模型比较时的表现。结果显示：（1）W_Sw的一类错误控制率有很强的健壮性。（2）W_XPD在Q矩阵错误设定的大多数条件下的表现优于W_Sw。结论：模型—数据拟合良好时可以使用W_Sw进行项目水平上的模型比较,当模型与数据失拟时W_XPD可能是更好的选择。     

现代测量理论下四大认知诊断模型述评

该文介绍并比较了现代测量理论下四大认知诊断模型的思想方法、模型结构及各自的特点性能等。LLTM是一个较早的认知诊断模型,它实现了认知与测量的结合;规则空间模型实现了对认知结构的诊断,并创造性地提出了Q矩阵理论;统一模型与融合模型是同一类模型：两者均沿用了规则空间模型的Q矩阵方法,但克服了规则空间模型中的一些不足;融合模型被认为是二十一世纪初创立的一个很成功的认知诊断模型。     

几种常用非补偿型认知诊断模型的比较与选用：基于属性层级关系的考量

当前国际上开发了60多种认知诊断计量模型(Fu &; Li, 2007), 各种模型各具特点, 实际应用者应根据实际情况选用恰当的模型。本研究以属性层级关系为切入点, 采用Monte Carlo模拟的研究方法, 比较了属性层级关系正确及有误两种情况下, 当前国际上常用的五种认知诊断模型的性能, 以充分考察不同认知诊断模型对属性层级关系的依赖程度, 及属性层级关系的错误界定对各认知诊断模型诊断正确率的影响, 从而为实际运用者在认知诊断模型选用上提供借鉴和参考。     

认知诊断中的Q矩阵及其作用

Q矩阵作为连接认知和测量的桥梁,在认知诊断中起重要作用。本文梳理了应用Q矩阵解决认知诊断相关问题的理论与方法。首先整理Q矩阵的相关概念、算法、性质及其在认知诊断中的作用;并根据Q矩阵可计算理论构念效度、可以构成格等,指出Q矩阵是特殊的关联矩阵;接着介绍Q矩阵理论研究方面的几个近期发展;并对Q矩阵未来的应用研究作出展望。期望本文能为测量工作者更灵活地利用Q矩阵提供参考和帮助。     

认知诊断中的Q矩阵及其作用

Q矩阵作为连接认知和测量的桥梁，在认知诊断中起重要作用。本文梳理了应用Q矩阵解决认知诊断相关问题的理论与方法。首先整理Q矩阵的相关概念、算法、性质及其在认知诊断中的作用；并根据Q矩阵可计算理论构念效度、可以构成格等，指出Q矩阵是特殊的关联矩阵；接着介绍Q矩阵理论研究方面的几个近期发展；并对Q矩阵未来的应用研究作出展望。期望本文能为测量工作者更灵活地利用Q矩阵提供参考和帮助。