多面Rasch模型在结构化面试中的应用

使用项目反应理论中的多面Rasch模型，对66名考生在结构化面试中的成绩进行分析，剔除了由于评委等具体测量情境因素引入的误差对原始分数的影响，得到考生的能力估计值以及个体水平的评分者一致性信息。对基于考生能力估计值和考生面试分得到的决策结果进行比较，发现测量误差的确对决策造成影响，对个别考生的影响甚至相当巨大。进一步使用Facets偏差分析以及评委宽严程度的Facets分析追踪误差源。结果表明，将来自不同面试组的被试进行面试原始成绩的直接比较，评委的自身一致性和评委彼此之间在宽严程度上的差异均将导致误差。研究表明，采用Facets的考生能力估计值作为决策的依据将提高选拔的有效性。同时，Facets分析得到的考生个体层次的评分者一致性指标，以及评委与考生的偏差分析等研究结果还可以为面试误差来源的定位提供详细的诊断信息

A Many-faceted Rasch Model Analysis of Structured Interview

Being one of the most important techniques in personnel selection, structured interview has attracted more and more research interest in improving its reliability and validity. Some researches focus on the standardization of its content and dimensions, others try to decrease rater bias by intensive rater training. The third one, to handle the possible bias in statistic way, has attracted more and more attention. Many-faceted Rasch Model (MFRM), an extension to Rasch model, served as such kind of techniques. By parameterizing not only interviewee’s ability and item difficulty but also judge severity, MFRM offers an effective way to estimate interviewee’s latent trait, that is, the ability, and provides detailed information of inter-rater reliability as far as a specific interviewee is concerned. This study used MFRM to analyze the result of a structured interview and demonstrated a creative way to locate the source of bias for a specific interviewee. Data came from a structured interview. There were 7 raters in each interview panel. Since the interview last two days, these 21 raters were randomized into 3 panels in the morning of each day in order to prevent cheating. Rating scores of two panels were used in this study. A、B、C、D、E、F、G were raters of one panel and interviewed interviewees numbered 1-34. A、E、H、I、J、K、L were raters in the other panel that interviewees numbered 35-66 were interviewed. Each rater rated each interviewee independently on five dimensions using a 10 points rating scale. Using Facets 3.62.0, a computer program based on MFRM, the abilities of 66 interviewees was estimated, accompanied with a Infit MnSq, which demonstrated the degree to which raters in the panel agreed with each other on the evaluation of a specific interviewee. The ranking order based on interview raw scores and Facets estimated logits values were compared. Difference were found between those them. To track the source of error for interviewee numbered 56, bias analysis of Facets was also made. (1) The ability of 66 interviewees were reported with infit MnSq, showing inter-rater reliability at individual level; (2) The ranking order based on interview raw score and estimated ability score were quite different, especially for some interviews. Taking interviewee numbered 56 for example, the ranking difference was as large as 15. (3) Bias analysis aimed at locating the source of error for interviewee number 56 show that not only rater consistency, but also rater severity contribute to the ranking difference. The results confirmed the utility of MFRM analysis. The application of MFRM in the analysis of structured interview was proved to be not only an effective way in personnel selection, but also provided diagnostic information for sources of error locating at individual level