建构反应题中能力估计准确性的影响因素:评分者人数和项目个数的交互作用

采用康春花、孙小坚和曾平飞(2016)提出的等级反应多水平侧面模型探讨了评分者人数和项目个数对被试能力估计准确性的影响。模拟研究的结果表明:(1)随着项目个数的增加,估计值与真值之间的相关也不断增加;(2)评分者人数和项目个数在平均绝对偏差(MAB)和误差均方根(RMSE)上的主效应均显著,两者间的交互效应也显著;(3)简单效应分析发现,当项目较少时,3个评分者条件下的能力估计准确性最好; 随着项目个数的增加,4个评分者的估计误差迅速下降,且表现变为最好。

The Impact of Number of Raters and Number of Items on Ability Estimation in Constructed-response Items

Abstract:Using the grade response multilevel facets model(GR-MLFM),which formulated by Kang,Sun and Zeng(2016),to explore the impact of number of raters and number of items on the ability estimation.Results based on simulation show that:(1)with the increase of number of items,the correlation between estimates and true values become larger;(2)The main effect of both number of items and number of raterson MAB and RMSE are significant,also the interaction effect between these two variables is significant;(4)Post hoc analyses show that when the number of items are small,3 raters can lead to the best ability estimation; with the increase of the number of items,the error of estimates for 4 raters decreased dramatically,and yields the smallest errors.