一种基于Q矩阵理论朴素的认知诊断方法

现有的认知诊断方法均是在复杂的统计测量学知识基础上构建的, 需要经过大量的运算才可实现对被试的诊断分类。这使得相关研究者及一线教师在理解和运用某一认知诊断方法时困难重重。相比之下, 孙佳楠、张淑梅、辛涛和包钰(2011)提出的广义距离判别法(GDD)较其他认知诊断方法更简单易用且分类准确率高。本研究在改进的Q矩阵理论(丁树良, 祝玉芳, 林海菁, 蔡艳, 2009; 丁树良, 杨淑群, 汪文义, 2010)的基础上, 借鉴GDD的思路, 提出一种无需进行参数估计的朴素的认知诊断方法, 即海明距离判别法(HDD)。根据判别方式的不同将其分为R方法和B方法。采用Monte Carlo模拟的研究方法, 以模式判准率(PMR)和属性平均判准率(AAMR)作为衡量被试知识状态分类准确率的指标, 与GDD进行比较。结果表明, HDD具有更简便的操作步骤和更好的分类准确率。

A Simple Cognitive Diagnosis Method Based on Q-Matrix Theory

Cognitive diagnosis has recently gained prominence in educational assessment, psychiatric evaluation, and many other fields. Researchers have been trying their best to develop a new Cognitive Diagnosis Model (CDM) or to improve existing ones’ performance for respondent classification. As a new CDM, GDD (Sun, Zhang, Xin, & Bao, 2011) receives more and more attention due to its classification accuracy which is as high as DINA. This article introduces a new approach called Hamming Distance Discrimination (HDD) which is inspired by GDD and based on the Q-matrix theory (Tatsuoka, 1991) modified by Leighton et al. (2004) and Ding et al. (2009, 2010). HDD uses Hamming Distance (HD) to measure the distance between an examinee’s Observed Response Pattern (ORP) and an Expected Response Pattern (ERP). When there are more than one ERPs with the same minimum HD for an examinee’s ORP, two solutions based on HD are proposed: the random method (Method R) and the Bayesian method (Method B). Method R randomly chooses one ERP from those share the same minimum HD whereas in method B, we apply Bayesian Discriminant to distinguish which ERP the examinee belongs to. Monte Carlo simulation was used to compare the accuracy of respondent classification between HDD and GDD. In the Monte Carlo simulation study, the pattern match ratio and average attribute match ratio were used as criteria to evaluate the classification accuracy of GDD and HDD. Five attribute hierarchical structures in attribute hierarchical model (AHM) of Leighton et al. (2004) and Tatsuoka (1995, 2009) with 6 attributes were simulated. Under each type of Q-matrix, we set the slip at four levels (2%, 5%, 10%, 15%) to simulate ORPs of examinees (N=1000). The results of this study demonstrate that HDD is superior, especially under the unstructured hierarchy and independent structure. Moreover, method B presented higher classification accuracy than method R. Further research on HDD’s validity and performance in other situations is warranted.