属性不等权重的多级评分属性层级方法

本文给出基于属性不等权重的等级反应模型(Grade Response Model, GRM)的属性层级方法(Attribute Hierarchy Method, AHM), 简记为属性不等权重的GRM-AHM。在属性层级结构下, 本文利用贝叶斯网与最小二乘两种方法, 提出了被试掌握属性的条件概率与属性权重的计算方法, 发现并解决了属性在不同的项目内权重有可能不相等的问题。本研究进一步将认知诊断推广到多级评分的情形。试验证明, 属性不等权重的GRM-AHM具有较高的判准率。

Attribute Hierarchy Method Based on Graded Response Model with Different Scoring-Weight for Attributes

Compared to the traditional test, the value of test for diagnostic assessment test lies in its ability to reveal each student’s specific cognitive strengths and weaknesses and further helps design effective remedy for individual student. More information for cognitive diagnose could be provided by polytomous scoring than dichotomous scoring. So far, the Polytomous Extension of diagnostic assessment still remains at the stage that all the attributes share the same scoring-weight. It is contrary to the fact that attributes are very likely to have different weights. On the assumption that two students respectively grasp the same number of attributes in an item, but not the same attributes, rater should give more scores to the student who answers the more difficult or key attributes correctly, rather than give the same score. It’s imperative for us to study the Cognitive Diagnostic Models(CDM) based on the attributes with different scoring-weight. In this paper, a method derived from Bayesian Networks and Least Squares Distance theories is proposed to calculate the score weight of attributes. Additionally, this paper discovers and solves a problem that the weight of the same attributes in different items may not be the same. The cognitive diagnostic model in this paper is Weighted Attribute Hierarchy Method (WAHM) with score weights of attributes, which is based on Graded Response Model (GRM), briefly, it is called WAHM-GRM. Four kinds of attribute hierarchies were separately used as the basis for the simulation. A sample of 5000 expected item response vectors was generated based on each of the four expected response patterns which are normally distributed. Each of the four samples consists of expected response patterns which are free from slips, the observed item response patterns were generated by randomly adding slips to each of the expected response patterns. In this study, the percentage of random errors was manipulated to 5%, 10%,15% and 20% of the total number of item responses to examine whether the number of random errors has an impact on the accuracy of classification methods. Simulation results showed that under the condition that attributes with different weights, very high classification accuracy rates remain for all classification methods, including methods A and B, proposed by Leighton et al.(2004) and ration of logarithm likelihood method (LL),proposed by Zhu et al.(2009). Especially for A and B methods, classification accuracy rate of AHM-GRM remains above 90% even when slip is as high as 20%. In Conclusion, AHM-GRM with different weighted attributes has a very high classification accuracy rate. In addition, score weights of attribute can guide item builders to distribute scores to the item attributes at the stage of developing item tests.