变长CD-CAT中的曝光控制与终止规则

本研究借鉴传统计算机化自适应测验的思想, 并结合认知诊断的特点, 在认知诊断框架下提出了4种变长CD-CAT的终止规则, 分别是属性标准误法(SEA)、邻近后验概率之差法(DAPP)、二等分法(HA)以及混合法(HM)。在未控制曝光和采用不同曝光控制条件下, 与HSU法及KL法进行了比较。研究结果表明：(1) 终止条件越严格, 平均测验长度越长, 按测验长度最大值终止的测验百分比越大, 模式判准率越高。(2) 当未加入曝光控制时, 4种新的终止规则均有较好表现, 与HSU法十分接近。随着最大后验概率预设值的增加或e的减小, 模式判准率呈上升趋势, 平均测验长度逐渐增加, 但在题库使用率方面均较差。(3) 当加入项目曝光控制时, 6种变长终止规则下的题库使用率有了极大的提升, 仍能保持较高的模式判准率, 并且不同的曝光控制方法对终止规则的影响是不同的。其中, 相对标准终止规则极易受到曝光控制方法的影响。(4) 综合来看, SEA、HM以及HA法在各项指标上的表现与HSU法基本一致, 其次为KL法和DAPP法。

Exposure Control Methods and Termination Rules in Variable-Length Cognitive Diagnostic Computerized Adaptive Testing

Comparing to the nonadaptive testing, the major advantage of computerized adaptive testing (CAT) is that the examinees achieve the same degree of measurement precision (i.e., fixed precision). But few studies are devoted to the termination rules in variable-length cognitive diagnostic computerized adaptive testing (CD-CAT). Inspired by the termination rule research in traditional CAT, this paper proposed four termination rules for variable-length CD-CAT. The new termination rules were standard error of attribute method (SEA), difference of the adjacent posterior probability method (DAPP), halving algorithm (HA) and hybrid method (HM), respectively. Then, the four new termination rules were compared with the HSU and KL method under two scenarios: with and without item exposure control. Three exposure control methods were considered, i.e., simple, modified restrictive progressive (MRP) and modified restrictive threshold (MRT) method. The MRP and MRT methods were extension of the Wang et al.’s (2011) work to the variable-length CD-CAT scenario. The results indicated that: (1) When the criterion of variable-length termination rule was conservative, the mean of the test length and the percentage of examinees reaching the maximum test length were large, and the classification accuracy rate for examinees who finished the CAT using fixed precision was high. (2) Without the item exposure control, the four new variable-length termination rules had a similar performance compared to the HSU method. With the increase of maximum posterior probability and the decrease of e, the classification accuracy rate and the mean test length presented a increasing trend. But the item pool usage was unsatisfactory. (3) With the item exposure control, item pool usage was greatly improved in the six variable-length termination rules while the classification accuracy rates were maintained. Different exposure control methods had a different effect on the different variable-length termination rules. The relative criterion termination rules such as DAPP and KL methods were easily affected by the item exposure control. (4) Taken all together, the SEA, HM, and HA methods were comparable to the HSU method, and followed by the KL and DAPP method. Some future directions were suggested in the end of this paper.