托幼机构教育质量评价中评委偏差的多侧面Rasch分析

采用多侧面Rasch模型对28位评委在托幼机构教育质量评价中的评委偏差进行了分析。分析结果显示：28名评委评分宽严度差异显著；3名评委内部一致性较差，其余25名评委内部一致性较稳定；评委与评价班级的交互作用不显著，与评价项目的交互作用显著。研究结果表明MFRM可以对托幼机构教育质量评价的评委偏差进行个体层面的具体分析，从项目反应理论的视角为托幼机构教育质量评价的评委针对性培训、评估评委的合格性从而建立合格评委库等提供现代教育、心理测量学依据。

Many-Facet Rasch Model analysis of Rater Bias in the Measurement Quality of the Early Childhood Program

The measurement quality of the early childhood program is of great importance for the children’s development. 155 classrooms sampled from the eastern part of China, were observed and assessed by 28 raters. The scale in the study which is called Chinese Early Childhood Environment Rating Scale(CECERS) is made up of 46 items, 8 subscales, they are Space and Furnishing, Person Care Routines, Curriculum Planning and Implementation, Whole-Group Instruction, Activities, Language-Reasoning, Guidance and Interactions, Parents and Staff. Usually, one classroom was observed and assessed by 2 or 3 raters alone at the same time. In this measurement situation, rater severity, inter-rater reliability, the bias between the rater and the class, and bias between the rater and the item were analyzed from three facets, they are the class, the rater and the item respectively.Facets software (MFRM3.68.1) were adopted in this study. The results of the analysis show that: there were significantly different among the 28 raters; the consistency from the 3 raters was high, at the same time, the internal consistency was stable from the rest of 25 raters; a further analysis of the unexpected responses from the three raters showed that, there was no common attribute of the classes and the same of the items；the interaction between the rater and classroom was not different significantly, the ratio of the significant bias is 8.02%, the further analysis based of the significant bias showed that, there were no common attribute among these classes；the interaction between the rater and the item was significantly different, the items with significant bias was computed, it found that there was a large deviation in the item 45 and item 13;the deviation was less in the items such as item 22, 25 and so on, which were more familiar for the raters.All of these provided the psychometric basis for the training, the standards and the constructing of the rater bank from the perspective of IRT.