重参数化的多分属性诊断分类模型及其判准率影响因素

多分属性比传统的二分属性提供更多更详细的诊断反馈信息, 符合对知识技能的多水平要求, 具有较好的应用前景。本文首先介绍了多分属性和多分Q矩阵的概念; 之后重参数化了3个分别满足连接、分离和补偿缩合规则的多分属性诊断分类模型并研究了其判准率影响因素, 结果发现它们的判准率(1)均随多分属性数量的增加而降低, 建议实际使用中不宜高于5个; (2)均随多分属性的最高水平数增加而降低, 建议实际使用中不宜高于4水平; (3)均随多分属性间统计相关性增加而增加, 但影响不大; (4)受多分属性层级结构的影响较大; (4)受被试量影响不大; (5)均随题目数量增加而增加且影响较大。最后, 针对“多分属性与多级评分的关系”和“多分属性与二分属性之间的关系”这两个问题进行了讨论。以期为实证研究者提供相关的理论支持和使用建议。

Factors affecting the classification accuracy of reparametrized diagnostic classification models for expert-defined polytomous attributes

Diagnostic classification assessment (DCA) utilizes latent class models to provide fine-grained information about students’ strengths and weaknesses in the learning process. In the past decades, extensive research has been conducted in the area of DCA and many statistical models based on a probabilistic approach have been proposed. At present, several diagnostic classification models (DCMs) for dichotomous attributes exist, which include the deterministic inputs, noisy “and” gate (DINA; Junker & Sijtsma, 2001); the deterministic inputs, noisy “or” gate (DINO; Templin & Henson, 2006); and the linear logistic model (LLM; Maris, 1999). In contrast, only a few DCMs can be used to deal with the polytomous attributes, such as the model based on the ordered-category attribute coding (OCAC; Karelitz, 2004), and the polytomous generalized DINA (pG-DINA; Chen & de la Torre, 2013).
Polytomous attributes, particularly those defined as part of the test development process, can provide additional diagnostic information. The present research proposes three reparametrized reduced models of pG-DINA (Chen & de la Torre, 2013), which include the reparametrized polytomous attributes DINA (RPa-DINA), the reparametrized polytomous attributes DINO (RPa-DINO), and the reparametrized polytomous attributes LLM (RPa-LLM). Furthermore, to better understand the classification accuracy of the new models, the impact of 6 factors was investigated, namely, the number of polytomous attributes, the highest level of polytomous attributes, the correlations among polytomous attributes, the hierarchical structure, the sample size, and the number of items. Results of the simulation study indicated that:
(1) more polytomous attributes led to lower classification. Their effects, in descending order, were the RPa-LLM, the RPa-DINO, and the RPa-DINA. Less than 5 polytomous attributes used in empirical research is suggested;
(2) for the number of attribute levels, more levels resulted in worse performance. Less than 4 levels within one attribute used in empirical research is suggested;
(3) the higher the correlations among polytomous attributes, the higher the classification accuracy would be;
(4) different hierarchical structure had different influences on the classification accuracy. No matter what structure we had, the performance of RPa-DINA was quite well behaved. However, other 2 models, especially the RPa-DINO, were recommended for the analysis of response data from independent hierarchical structure;
(5) the sample size has little impact on the classification accuracy; and
(6) the number of items was inversely proportional to the classification accuracy.