结构化面试中多源变异的概化分析

用多元概化理论分析由75名应聘者和7名面试官组成的结构化面试中多种变异来源,结果表明:面试中5个测评因子的概化系数约为0.81~0.88之间,说明该面试7名考官对应聘者评分并不太一致,但勉强可以接受,产生误差的主要来源是应聘者与面试官交互作用影响;继续增加面试官数量可有效提高概化系数,但是并不符合实际,对面试官在评分标准上进行统一培训才是解决问题的最佳方法。概化理论具有经典测量理论不可及的优势,适合广泛应用于结构化面试的数据分析中。     

国家公务员录用面试初步研究

本文结合湖北省公务员考试的实践对公务员录用面试进行了初步研究。本研究从参与 1 997年湖北省省直公务员考试的 3 9个部门中抽取了 8个部门共 1 5 3名被试的面试数据 ,分析表明 ,在本研究所采用的结构化面试中 ,测评要素间具有较高的一致性 ,经过严格培训的考官的评分者信度是可靠和有效的。本研究还对如何设计、实施此类面试提出了有益的建议。     

中层管理人员结构化面试测评效度的现场研究

通过对某上市公司随机抽取的43位中层管理人员素质测评的现场研究,探讨结构化面试的信度效度问题。研究设计基于岗位分析与关键事件分析,采用3人小组面试的方法,同时实施情景面试与行为描述面试,综合测评被试岗位胜任能力。分析结果表明,评委要素评价内部一致性和评委间内部一致性都比较高,并与面试半年后上级评定的任务绩效和总体绩效显著相关,结构化面试具有较高的信度与预测效度。进一步比较情景面试和行为描述面试发现,这两种结构化面试有类似的信度,但是行为描述面试具有更高的效度。     

基于胜任特征的结构化面试初探

基于胜任特征的结构化面试是对传统结构化面试的优化,有很高的信度和效度。本文从基于胜任特征的结构化面试的内涵、特点、程序以及实施过程中需要注意的问题几个方面进行了深入探讨,对于改善结构化面试的测评功能和深化胜任特征的应用有一定价值。     

基于GT和多面Rasch模型的结构化面试分析

该研究应用GT和多面Rasch模型对结构化面试数据进行分析,并提出一些建议针对某辅导员招聘面试数据,运用GT从宏观上分析应聘者、考官和项目所带来的总体误差大小,在此基础上,运用多面Rasch模型从微观上进一步探查考官严厉度、应聘者能力差异、项目难易度及侧面偏差．结果表明：1）GT分析表明应聘者产生的变异较大（90．65％）,说明面试可靠性较高,且当考官数为2时可靠性已较好.2）多面Rasch模型分析出了各侧面效应中的非拟合因素及交互效应中的偏差因素,表明面试误差主要来自考官间严厉度的差异及其自身一致性的不稳定。将GT与多面Rasch模型相结合分析面试数据不仅能测查出评价过程各方面的问题因素,并能更好地作整体把握。     

面试考官的权重策略对评分和录用的影响

从个体水平研究面试考官的有效性是面试研究的一个热点问题．本研究对面试考官权重策略的类型及类型的比较进行了实证研究。研究运用线性规划策略捕捉方法,分析了公务员考试中28名考官在结构化面试和无领导小组讨论中的权重策略,研究了权重策略对评分和录用的影响,结果发现：（1）在结构化面试中存在三种策略,在无领导小组讨论中存在两种策略,这些策略大部分是规范的（符合评分规则的要求）,它们会随面试情景发生变化;（2）权重策略影响了考官的评分区分度和最终考生的录用结果,规范性策略具有较高的区分度和录用预测效度。研究最后对结果的理论和实践意义进行了讨论。     

用结构方程建模(SEM)估计概化理论(GT)中的评分者信度

评分者的信度分析 ,已从经典测量理论的方法 (Kendall和谐系数 )发展为基于现代测量理论 (如GT)的方法 ,但由于GT中方差分量估计的工具和途径有限 ,使该方法的推广受到限制。本文介绍了结构方程建模 (SEM )估计GT中方差分量的必要性及原理 ,并通过对某省直国家机关公务员录用考试结构化面试的评分者信度分析 ,阐述用SEM估计概化理论中不同设计下的评分者信度研究     

基于多元概化理论的中小学教师资格考试质量分析——以《综合素质》(中学)科目为例

教师资格考试是国家教育规划纲要确定的改革项目之一。本文运用多元概化理论对中小学教师资格考试笔试科目《综合素质》(中学)试卷的测量信度、试卷总分合成、试卷结构优化等问题进行探讨。研究发现:本次测试的测量信度较高,为0.707;各测量分项对全域总分的方差贡献比例与试卷赋分意图基本一致,除写作能力外,其余部分误差在7%以内;可以采取减少客观题数量,增加主观题数量的方式来提高信度。     

用多元概化理论对普通话的测试

用多元概化理论 (MGT)研究了国家语委编制的普通话测验。在G研究中 ,利用香港人普通话测试的数据 ,估计了引起分数变异的各种来源的方差与协方差分量。在D研究中 ,首先估计了该测验 3个部分的全域分数和各自的概化系数等技术指标 ,然后估计了全域合成分数及其概化系数、信噪比等指标。结果表明 ,该测验的信度从总体上讲是较高的 ,把三个部分的全域分数进行合成也是合理的 ,但从细节上看其第 3部分的信度较低。另外 ,当评分者个数为 3、试题数量为 2 8时 ,测验的第 1、2部分的信度已经较高 ,因此 ,在实测时减少这两部分的题量并不会有太大问题     

用结构方程建模（SEM）估计概论理论（GT）中的评分者信度

评分者的信度分析,已从经典测量理论的方法（Kendall和谐系数）发展为基于现代测量理论（如GT）的方法,但由于GT中方差分量估计的工具和途径有限,使该方法的推广受到限制。本文介绍了结构方程建模（SEM）估计GT中方差分量的必要性及原理,并通过对某省直国家机关公务员录用考试结构化面试的评分者信度分析,阐述用SEM估计概化理论中不同设计下的评分者信度研究。     

THE INFLUENCE OF RELATIONAL DEMOGRAPHY ON PANEL INTERVIEW RATINGS: A FIELD EXPERIMENT

The influence of relational demography (assessor race, candidate race, and the racial composition of rating panels) was examined in a structured interview setting. Twenty assessors (10 White and 10 Black) comprising five, 4-person panels of all possible racial compositions, evaluated videotaped responses of police officers participating in a promotion process. Each panel rated the same 73 (36 White and 37 Black) candidates' responses to a complex, structured interview question. An examination of mean overall ratings revealed a same race bias and a significant difference between panels based upon the relational demography of the interview panel; nevertheless, the size of these effects was small. Net reconciliation (i.e., between initial and final scores) differed significantly between minority and majority panel conditions for only Black assessors and, again, the effects here were very small. The practical implications of these findings are discussed.     

Employment Interview Reliability: New meta‐analytic estimates by structure and format

This study sought to provide an update on evidence regarding the interrater reliability of employment interviews. Using a final dataset of 125 coefficients with a total sample size of 32,428, our results highlight the importance of taking all three sources of measurement error (random response, transient, and conspect) into account. For instance, the mean interrater reliability was considerably higher for panel interviews than for separate interviews conducted by different interviewers (.74 vs. .44). A strong implication of our findings is that interview professionals should not base perceptions of the psychometric properties of their interview process on interrater estimates that do not include all three sources. A number of directions for future research were identified, including the influence of cues in medium structure panel interviews (e.g., changes in tone or pitch) and the lower than expected reliability for highly structured interviews conducted separately by different interviewers.     

Psychometric Properties of a Structured Behavioral Interview to Hire Private Security Personnel

This paper reports on a study about the reliability and validity of a structured behavioral interview to assess private security personnel. Reliability was estimated using interrater coefficients. Two independent interviewers were used to rate each interviewee. Results show a reliability coefficient of .81 (N = 43) and .89 with Spearman-Brown correction for two raters. Validity was estimated using a content validation approach. This strategy was suggested by Lawshe (1975) to estimate the content validity of selection tests. So far, only two studies carried out by Schmitt and Ostroff (1986) and Carrier et al. (1990) have used Lawshe's strategy in the structured behavioral interview case. The interview consisted of seven questions and each was rated by 11 experts in the job. Results show a significant content validity ratio (CVR) for majority of the questions in the interview and a content validity index (CVI) of .89. Implications of these findings for the practice of the structured behavioral interview are discussed and future research is suggested.     

A counterintuitive hypothesis about employment interview validity and some supporting evidence

This study found mixed support for the hypothesis that the difference in criterion-related validity between unstructured and structured employment interviews is due solely to the greater reliability of structured interviews. Using data from prior meta-analyses, this hypothesis was tested in 4 data sets by using standard psychometric procedures to remove the effects of measurement error in interview scores from correlations with rated job performance and training performance. In the 1st data set. support was found for this hypothesis. However, in a 2nd data set structured interviews had higher true score correlations with performance ratings, and in 2 other data sets unstructured interviews had higher true score correlations. We also found that averaging across 3 to 4 independent unstructured interviews provides the same level of validity for predicting job performance as a structured interview administered by a single interviewer. Practical and theoretical implications are discussed.     

Methods for Assessing Quality of Life and Well-Being in Frail Older People

The appropriateness, reliability and convergence of differing methods and associated instruments for the assessment of quality of life and well-being in frail older people was examined in 60 ( S 65 years of age) nursing home residents. The feasibility of measuring quality of life and well-being via a variety of instruments was explored through observation (an adaptation of Dementia Care Mapping), structured interview (Schedule for the Evaluation of Individual Quality of Life - Direct Weighting), and proxy questionnaire (Pleasant Events Schedule - Alzheimer Disease; Apparent Emotion Rating Scale). The observational assessment and Pleasant Events Schedule demonstrated good reliability. Significant associations were found between well-being as assessed by observation, and active social interaction and engagement in recreation as assessed by observation, quality of life as assessed by structured interview, and enjoyment of activities as assessed by proxy questionnaire. These findings indicate the importance of social activity in later life, and the potential of different approaches for the appropriate assessment of key aspects of quality of life in frail older people.     

STRUCTURED INTERVIEWING: RAISING THE PSYCHOMETRIC PROPERTIES OF THE EMPLOYMENT INTERVIEW

A highly structured employment interviewing technique is proposed, which includes the following steps: (1) develop questions based on a job analysis, (2) ask the same questions of each candidate, (3) anchor the rating scales for scoring answers with examples and illustrations, (4) have an interview panel record and rate answers, (5) consistently administer the process to all candidates, and (6) give special attention to job relatedness, fairness, and documentation in accordance with testing guidelines. Examination of psychometric properties for hiring entry-level production employees ( n = 149) reveals high interrater reliability ( r = .88) and predictive validity (uncorrected r = .34, corrected r = .56), as well as evidence for test fairness and utility. The levels of these properties are comparable to those of a comparison battery of typical employment tests, and correlations with the tests suggest that the interview has a strong cognitive aptitude component. Potential explanations for the effectiveness of this structured interviewing technique are discussed.     

Acceptance of Structured Diagnostic Interviews for Mental Disorders in Clinical Practice and Research Settings

The objective of this study was to investigate the acceptance of structured diagnostic interviews in clinical practice, as well as research, settings. Using the Diagnostisches Interview bei Psychischen Störungen (the modified and extended German version of the Anxiety Disorders Interview Schedule for DSM-IV), 10 certified interviewers conducted 183 structured interviews in different inpatient, outpatient, and research settings in Switzerland and Germany. After each interview, patients and interviewers filled out a questionnaire asking for their evaluation of the interview. Patients' and interviewers' reactions to the interview were highly positive. On a scale measuring overall satisfaction with the interview (0 = not at all satisfied, 100 = totally satisfied) the mean patient rating was 86.55 (SD = 13.18), and the mean interviewer rating was 85.82 (SD = 12.84). The procedure used by the interviewer was rated by 142 (78.5%) patients as being helpful, and 176 (96.7%) rated the relationship as being positive. Less than 16% of the interviews were described as exhausting by the patients and interviewers. A majority of the interviewers (92.6%) indicated that during the interview they could respond adequately to the patient. The results of this study indicate that structured diagnostic interviews are highly accepted by interviewers and patients in a variety of settings. These findings, together with the existing evidence of the reliability and validity of structured interviews, should encourage their use in the diagnostic process, in outpatient and inpatient clinical settings as well as in research studies.     

Development of a Structured Psychiatric Interview for Children: Agreement Between Child and Parent on Individual Symptoms

To test the reliability of children's reporting as compared with that of their mothers, a highly structured psychiatric diagnostic interview was used with 307 subjects, ages 6 through 16. Another interviewer gave each mother a similar interview about the child. Responses of each mother-child pair to 168 questions were compared using the kappa statistic. Highest agreement was found on questions concerning symptoms that are concrete, observable, severe, and unambiguous. Mothers tended to report significantly more behavioral symptoms, and children more subjective symptoms. Reasons for low kappas and asymmetrical reporting of symptoms are discussed.     

Development of a structured psychiatric interview for children: Agreement between child and parent on individual symptoms

To test the reliability of children's reporting as compared with that of their mothers, a highly structured psychiatric diagnostic interview was used with 307 subjects, ages 6 through 16. Another interviewer gave each mother a similar interview about the child. Responses of each mother-child pair to 168 questions were compared using the kappa statistic. Highest agreement was found on questions concerning symptoms that are concrete, observable, severe, and unambiguous. Mothers tended to report significantly more behavioral symptoms, and children more subjective symptoms. Reasons for low kappas and asymmetrical reporting of symptoms are discussed.