中层管理人员结构化面试测评效度的现场研究

通过对某上市公司随机抽取的43位中层管理人员素质测评的现场研究,探讨结构化面试的信度效度问题。研究设计基于岗位分析与关键事件分析,采用3人小组面试的方法,同时实施情景面试与行为描述面试,综合测评被试岗位胜任能力。分析结果表明,评委要素评价内部一致性和评委间内部一致性都比较高,并与面试半年后上级评定的任务绩效和总体绩效显著相关,结构化面试具有较高的信度与预测效度。进一步比较情景面试和行为描述面试发现,这两种结构化面试有类似的信度,但是行为描述面试具有更高的效度。     

多面Rasch模型在结构化面试中的应用

使用项目反应理论中的多面Rasch模型,对66名考生在结构化面试中的成绩进行分析,剔除了由于评委等具体测量情境因素引入的误差对原始分数的影响,得到考生的能力估计值以及个体水平的评分者一致性信息。对基于考生能力估计值和考生面试分得到的决策结果进行比较,发现测量误差的确对决策造成影响,对个别考生的影响甚至相当巨大。进一步使用Facets偏差分析以及评委宽严程度的Facets分析追踪误差源。结果表明,将来自不同面试组的被试进行面试原始成绩的直接比较,评委的自身一致性和评委彼此之间在宽严程度上的差异均将导致误差。研究表明,采用Facets的考生能力估计值作为决策的依据将提高选拔的有效性。同时,Facets分析得到的考生个体层次的评分者一致性指标,以及评委与考生的偏差分析等研究结果还可以为面试误差来源的定位提供详细的诊断信息     

结构化面试预测效度的贡献成分分析

以情境面试和行为面试为主要形式的结构化面试,被广泛证明能够预测员工绩效,但其构念效度一直没被清晰地证明.面试的结构性、应聘者印象管理行为、动机、认知能力和人格特征等成分对结构化面试的预测效度都有积极贡献,这种对结构化面试预测效度贡献成分的分析有助于认清结构化面试的构念效度.未来的研究需关注结构化面试构念是否以人际技能为主,以及人际技能、动机、认知能力各自所占比重.     

Selection Interview: A Review of Validity Evidence,Adverse Impact and Applicant Reactions

This article is focused on the evidence related to the criterion and construct validity of interview, its adverse impact on minority groups and the applicant reactions. Based on the content of the questions included in personnel interview, two types of interview made by found: conventional structured interview and behavioural structured interview. With regard to criterion validity, evidence shows that, in general, interviews may be used to predict job performance and, specifically, behavioural structured interviews show the highest validity coefficients. Although construct validity was less well investigated, there is currently a number of studies carried out in order to clarify what constructs are assessed by interviews. It was shown that conventional structured interviews and behavioural structured interviews clearly measure different constructs. With regard to group differences, interviews show only a small adverse impact, but this impact decreases if behavioural structured interviews are used. In connection with applicant reactions, more negative applicant reactions appear with behavioural structured interviews than conventional structured interviews. Practical implications and future lines of research are suggested.     

基于胜任特征的结构化面试初探

基于胜任特征的结构化面试是对传统结构化面试的优化,有很高的信度和效度。本文从基于胜任特征的结构化面试的内涵、特点、程序以及实施过程中需要注意的问题几个方面进行了深入探讨,对于改善结构化面试的测评功能和深化胜任特征的应用有一定价值。     

SMALL GROUP BEHAVIORAL TRAINING TO IMPROVE THE JOB INTERVIEW SKILLS REPERTOIRE OF MILDLY RETARDED ADOLESCENTS

Four retarded adolescents, enrolled in a short-term residential treatment program, received behavioral job interview skills training. Although potentially employable, each was unable to present himself effectively in standard employment interviews. Treatment consisted of a series of behavioral group sessions using instructions, modeling and rehearsal procedures to increase, in multiple baseline fashion, such skills as the adolescents' ability to disclose positive information about their experience and background, convey interest in the position and direct relevant questions to an interviewer. Effectiveness of treatment for each client was assessed by: (1) Objective ratings of performance during individual, structured role-play job interviews following each treatment group; (2) objective ratings of pre- and posttraining performance during tape recorded in vivo generalization job interviews at a fast-food restaurant; and (3) global evaluations of pre- and posttraining in vivo generalization interviews made by experienced personnel interviewers unfamiliar with the nature of the treatment. The results indicated that potentially employable retarded citizens can be successfully taught appropriate job interview behavior using a small group behavioral procedure. The need for such techniques in community and rehabilitation centers for retarded citizens and other clinical populations is discussed.     

Individual differences in effective interview design: Factors affecting question choice

Despite the evidence that structured interviews are superior to unstructured interviews, interviewers remain reluctant to use them for personnel selection. The current research replicated and extended recent research examining how individual differences relate to the ability to identify effective interview questions. Question judgments were made across questions that varied substantially in quality and across two different jobs. Across two samples of data, respondents evaluated past behavioral and traditional interview questions more favorably than oddball questions and questions about job‐irrelevant competencies. Furthermore, general mental ability and social aptitude were strongly related to skill in identifying effective interview questions, with results suggesting these traits are associated with customizing questions to specific job contexts.     

An interview to measure vital exhaustion. II. reliability and validity of the interview and correlations of vital exhaustion with personality characteristics

Abstract

In the previous paper (Meesters and Appels, this issue) we described the development of an interview to measure feelings of vital exhaustion (VE). The current paper presents data on the reliability and the validity of the interview, which is labeled the Maastricht Interview for Vital Exhaustion (MIVE). Correlations with the Type A behavior pattern (TABP) and hostility are also discussed in this paper. The data were collected in a case-control study of 81 males with first MI and 168 healthy neighborhood controls. The interview was found to have a good reliability, Cronbach's alpha being .90 in the case group and .86 in the control group. The kappa coefficient was .95 in the case group and .91 in the control group. The validity was tested by computing the estimated relative risk of VE, controlling for angina pectoris and the classic risk factors. The odds ratio was found to be 8.11 (p<.001). This indicates that the interview makes a valid assessment of the mental precursors of MI. The interview correlated significantly with TABP (structured interview) and two self-report measures of hostility (Cook-Medley Hostility Scale and Buss-Durkee Hostility Inventory). Although the absolute magnitude of the observed odds ratios may be affected to some extent because the interviews were not scored blind, these results clearly suggest that the MIVE is a reliable and valid method to assess VE.     

The Validity of the Employment Interview: A Meta-Analysis

The interview is the most widely used personnel selection method, but has revealed low reliability and validity compared with other selection methods (Mayfield 1964; Ulrich and Trumbo 1965; Schmitt 1976; Arvey 1979). Thirty-one studies on the validity of the interview were meta-analyzed. The result was an average validity coefficient of .27. The estimated true validity of the interview was calculated to be .38. These values indicated that the interview has moderate validity as a personnel selection device. Six characteristics of the interview were also examined in relation to the validity of the interview: structure of the interview; number of interviewers; length of the interview; gender of the applicant pool; blue-collar/white-collar jobs; and use of college students versus job applicants. The six study characteristics accounted for 30.9% of the variance in the validity of the interview. Structure of the interview appeared to be the only characteristic that moderated the validity of the interview. The relationship of this study to other meta-analyses of the employment interview is discussed.     

领导干部结构化面试信度的多元概括化理论分析

本研究尝试运用多元概括化理论对北京市某区副处级干部准入资格结构化面试测评数据进行测量信度分析,为提高领导干部考试与测评工作科学化水平提供了有益的实证依据。主要结论有:(1)本次结构化面试难度适中,区分度较高;(2)各测评要素及合成分数的类信度系数均较高,合成分数的测量信度高于单个测评要素的测量信度;(3)各测评要素及合成分数的类信度系数随着考官数量的增加而增加,且从确保信度和降低成本考虑,考官数量以5-9位为宜;(4)在这次面试测评中,各项测评要素间的相关系数较高,这为目前在选拔面试中将各项测评要素得分进行合成提供了依据,说明用合成分数计算总分具有一定的合理性。     

Development of a Structured Psychiatric Interview for Children: Agreement Between Child and Parent on Individual Symptoms

To test the reliability of children's reporting as compared with that of their mothers, a highly structured psychiatric diagnostic interview was used with 307 subjects, ages 6 through 16. Another interviewer gave each mother a similar interview about the child. Responses of each mother-child pair to 168 questions were compared using the kappa statistic. Highest agreement was found on questions concerning symptoms that are concrete, observable, severe, and unambiguous. Mothers tended to report significantly more behavioral symptoms, and children more subjective symptoms. Reasons for low kappas and asymmetrical reporting of symptoms are discussed.     

The Type A Structured Interview: A behavioral assessment in the rough

The Type A behavior pattern is the focus of considerable research in behavioral medicine because of its causal relevance to coronary heart disease. The primary assessment of Type A behavior is a global rating made from a structured interview. From the perspective of behavioral assessment, this Structured Interview (SI) is important for two reasons. First, it provides an example of the superiority of an observed behavioral measure compared to self-report questionnaires for the assessment of an overt behavior pattern. Second, it provides a unique opportunity for the application of behavioral assessment technology. However, while the SI has demonstrated its clinical and research utility as a diagnostic procedure, its usefulness could be further enhanced in the areas of treatment selection and evaluation. The historical relationship between behavior and coronary heart disease is described. The development of the SI and the strengths of this assessment relative to other measurement approaches are reviewed. The standard questions and administration procedures for the interview are presented, along with several variations and modifications. Following this overview, the SI is discussed from the perspective of behavioral assessment.     

Development of a structured psychiatric interview for children: Agreement between child and parent on individual symptoms

To test the reliability of children's reporting as compared with that of their mothers, a highly structured psychiatric diagnostic interview was used with 307 subjects, ages 6 through 16. Another interviewer gave each mother a similar interview about the child. Responses of each mother-child pair to 168 questions were compared using the kappa statistic. Highest agreement was found on questions concerning symptoms that are concrete, observable, severe, and unambiguous. Mothers tended to report significantly more behavioral symptoms, and children more subjective symptoms. Reasons for low kappas and asymmetrical reporting of symptoms are discussed.     

Development and validation of the Eating Disorder Diagnostic Scale: a brief self-report measure of anorexia, bulimia, and binge-eating disorder

This article describes the development and validation of a brief self-report scale for diagnosing anorexia nervosa, bulimia nervosa, and binge-eating disorder. Study 1 used a panel of eating-disorder experts and provided evidence for the content validity of this scale. Study 2 used data from female participants with and without eating disorders (N = 367) and suggested that the diagnoses from this scale possessed temporal reliability (mean kappa = .80) and criterion validity (with interview diagnoses; mean kappa = .83). In support of convergent validity, individuals with eating disorders identified by this scale showed elevations on validated measures of eating disturbances. The overall symptom composite also showed test-retest reliability (r = .87), internal consistency (mean alpha = .89), and convergent validity with extant eating-pathology scales. Results implied that this scale was reliable and valid in this investigation and that it may be useful for clinical and research applications.     

Development of Standard Anchors for Scoring Generic Past-Behaviour Questions in Structured Interviews

A structured interviewing system was developed which included an item bank of pre-written past-behaviour questions. Scoring anchors for each of 168 questions in the item bank were created from analysis of protocols from individual assessments. A concurrent validation study was conducted, focusing on the job of highway patrol sergeant. Job knowledge experts used a job analysis tool to select a subset of twelve questions from the item bank. Panel interviews were conducted with 32 highway patrol sergeants as subjects. Average inter-rater reliability was 0.65 and interview validity was 0.42 for group consensus scores and 0.44 for panel average scores. Discussion focused on practical issues affecting the interviewing system.     

Behavioral skills training to increase interview skills of adolescent males in a juvenile residential treatment facility

Some studies suggest that acquiring employment following release from prison may reduce recidivism; however, few studies have evaluated procedures for teaching job-related skills to adolescents in residential detention facilities. Stocco et al. (2017) used behavioral skills training (BST) to improve interview skills of college students. The current study used a nonconcurrent multiple baseline design across participants to evaluate the extent to which BST improved interview skills for 7 adolescents who had been adjudicated for sexual offenses. Results show that BST increased appropriate responses to interview questions for 4 students and BST plus modifications (i.e., stimulus and response prompts) increased correct responding to questions for the other 3. In addition, BST increased appropriate questions, correct posture, and smiling, and decreased fidgeting for all 7 students. We briefly discuss the social implications of teaching interview skills to adjudicated adolescents, as well as the limitations of the findings.     

EXPERIENCE-BASED AND SITUATIONAL INTERVIEW QUESTIONS: STUDIES OF VALIDITY

This research compared the validity of two different types of structured interview questions (i.e., experience-based and situational) under tightly controlled conditions. The experience-based interview questions required that 108 study participants relate how they had handled situations in the past requiring skills and abilities necessary for effective performance on the job. Situational questions, administered to another group of 108 study participants, provided interviewees with hypothetical job-relevant situations and asked them how they would respond if they were confronted with these problems. The experience-based interview questions yielded higher levels of validity than the situational questions. Additional analyses showed that the interview added incrementally to the prediction of performance beyond the variance accounted for by a cognitive ability test. There were small differences in subgroup performance (White, Black, Hispanic, male, and female) on the experience-based interview, though it was equally valid for all subgroups.     

STRUCTURED INTERVIEWING: RAISING THE PSYCHOMETRIC PROPERTIES OF THE EMPLOYMENT INTERVIEW

A highly structured employment interviewing technique is proposed, which includes the following steps: (1) develop questions based on a job analysis, (2) ask the same questions of each candidate, (3) anchor the rating scales for scoring answers with examples and illustrations, (4) have an interview panel record and rate answers, (5) consistently administer the process to all candidates, and (6) give special attention to job relatedness, fairness, and documentation in accordance with testing guidelines. Examination of psychometric properties for hiring entry-level production employees ( n = 149) reveals high interrater reliability ( r = .88) and predictive validity (uncorrected r = .34, corrected r = .56), as well as evidence for test fairness and utility. The levels of these properties are comparable to those of a comparison battery of typical employment tests, and correlations with the tests suggest that the interview has a strong cognitive aptitude component. Potential explanations for the effectiveness of this structured interviewing technique are discussed.     

ACCURACY OF INTERVIEWER JUDGMENTS OF JOB APPLICANT PERSONALITY TRAITS

This study investigated whether interviewers can assess Big Five personality traits during a job interview. Four raters (self, interviewer, friend, and stranger) assessed the applicant's personality. Results from ratings for 73 applicants demonstrated that interviewer ratings of applicant personality correlate higher with self-ratings ( = .28) than do stranger ratings ( = .09) but less than ratings from close friends ( = .39). However, correlations between interviewer ratings and self-ratings were smaller for the two job-relevant personality traits, Conscientiousness ( = .16, n.s.) and Emotional Stability ( = .17, n.s.) than for the other three personality traits. Variance in ratings suggested the applicants managed their self-presentation on these two traits during the interview. Thus, although interviewers can and do assess personality during the interview, they are not able to assess those traits that would best predict later job success. Finally, the moderating effect of interview design (i.e., structure and content) was assessed. The results revealed that job-relevant interviews, situational interviews, and behavioral interviews did not affect an interviewer's ability to assess personality. Although there was a small (positive) effect for more structured interviews, this affect was modest. The results of this study suggest that future research should examine whether the interview can be designed to assess personality directly, and what the gains to predictive validity are by doing so.     

前景理论视角下结构化面试问题的设计

前景理论回答了类似应聘者在结构化面试中所面临的不确定场景下,人们是如何决策并行动的问题。论文采用思辨方法就结构化面试问题设计中如何恰当应用前景理论的基本要．董,有效提高结构化面试效度进行了探讨,结论如下：1）问题句子越短、问题越具体,效度有望越高;2）问题导向性要与岗位胜任特征充分契合;3）回溯性行为面试问题效度要高于虚拟情景面试问题;4）由于年龄、性别及个体认知能力等方面的差异,问题设计不仅要因需而变,也要因人而异。