非参数认知诊断方法下诊断结果的概率化表征

非参数认知诊断分类方法非常适合课堂评估，其诊断结果采用0-1形式而缺乏概率化表征，不能精细地区分被试属性掌握程度的差异或变化，还缺乏可用于评价真实测验分类结果的信度和效度指标。要刻画被试属性掌握程度的差异，首要的问题是要为非参数认知诊断方法提供一种可以量化属性掌握概率的方法。针对此问题，基于二项分布和玻尔兹曼分布提出非参数认知诊断方法下诊断结果的概率化表征方法，并用于构建分类准确性和分类一致性指标。模拟研究与实测数据分析结果显示：概率化表征方法与非参数认知诊断方法的分类结果高度一致；概率化表征方法与认知诊断模型所得的属性掌握概率十分接近；概率化表征方法所得的属性（模式）掌握概率可用于计算属性（模式）分类准确性和分类一致性指标，在实际测验情景下可作为信度和效度指标，评价诊断结果的重测一致率和判准率。

A Probabilistic Representation Approach for the Nonparametric Classification Method to Cognitive Diagnosis

Cognitive diagnostic assessment (CDA) can be regarded as a kind of formative assessments because it is intended to promote assessment for learning in classrooms by providing the diagnostic information about students’ cognitive strengths and weaknesses. CDA has received increasing attention in recent years. Statistical pattern recognition and classification methodology are two critical parts in CDA. Q-matrix corresponding to feature generation is of paramount importance in CDA. Therefore, Q-matrix becomes a popular research area in the field of CDA. Beside of the study of Q-matrix, many of the studies have focused on cognitive diagnostic models (CDMs). CDMs required a large sample size for calibrating item parameters. Therefore, the nonparametric classification method has become a popular method for classroom assessments. It is very suitable for classroom assessments, because it does rely on a large sample size for the calibration of item parameters. However, classification results for nonparametric classification method are often expressed as the form of 0-1 vector. The classification results lack a probabilistic representation, and cannot finely distinguish the differences in the degree of mastery of attributes among the subjects. At the same time, it lacks the reliability and validity indicators for evaluating the quality of classification results. To solve this problem, a probabilistic representation based approach for the nonparametric classification method to cognitive diagnosis is proposed based on binomial distribution and Boltzmann distribution. The probabilistic representation method handles attributes mastery status at a probability level. A simulation study was conducted to investigate the performance of the probabilistic representation method under three factors (slip parameter, guessing parameter, and two CDMs). Five independent attributes and the reduced Q-matrix as a test Q-matrix were considered in the simulation study. Item response data was generated by the deterministic inputs, noisy “and” gate (DINA) model or the deterministic inputs, noisy “or” gate (DINO) model. Simulation results showed that the performance of the new method is promising in terms of pattern matched rates and attribute matched rates. Results showed that the 0-1 classification results transformed from the probabilistic representation are highly consistent with those of the nonparametric classification method; the attribute posterior probability obtained from the probabilistic representation method is very closed with that of DINA or DINA model; the marginal posterior probability of attributes obtained from the new method can be used to evaluate the validity of classification results. From the simulation and real data study, the major findings and implications are the following: (a) compared with the DINA or DINO model and the conjunctive or disjunctive nonparametric classification method, the accuracy of the nonparametric method is slightly lower than that of the DINA or DINO model when the slip and guessing parameters of test items are quite different; (b) under most conditions, binomial distribution and Boltzmann distribution can be applied to accurately estimate the attribute mastery probability for the nonparametric classification method; (c) classification consistency and accuracy in CDMs and the probabilistic representation can be combined to construct the reliability and validity index for the evaluation of the classification quality of the nonparametric classification method.