CORRESPONDENCE BETWEEN ESTIMATES OF CONTENT AND CRITERION-RELATED VALIDITY VALUES

The correspondence between inferences made using two validation strategies–content and criterion-related–were examined in a specific personnel selection application. Empirical validity values and Law-she's (1975) content validity ratios (CVR) were obtained for items from three structured interview guides used in the selection of insurance agents. Ratings of each item by over 300 field managers were used to calculate the CVR values. Statistically significant, yet modest correlations were found between empirical item validities and content validities for an interview guide used to select applicants with prior insurance sales experience. No significant differences were found among these correlations by comparing job experts of different levels of managerial experience and experience in selection. Data for the interview guide used to select experienced applicants also indicated that a content validity approach can be useful in developing a selection instrument with an empirically valid composite rating. The hypotheses were not confirmed for interview guides used to select applicants with no prior insurance sales experience. The practical importance of these results are discussed, as are plans for future research.     

Alternative Predictors for Dealing with the Diversity–Validity Dilemma in Personnel Selection: The constructed response multimedia test

In the context of the diversity–validity dilemma in personnel selection, the present field study compared ethnic subgroup differences on an innovative constructed response multimedia test to other commonly used selection instruments. Applicants (N = 245, 27% ethnic minorities) for entry‐level police jobs completed a constructed response multimedia test, cognitive ability test, language proficiency test, personality inventory, structured interview, and role play. Results demonstrated minor ethnic subgroup differences on constructed response multimedia test scores as compared to other instruments. Constructed response multimedia test scores were related to the selection decision, and no evidence for predictive bias was found. Subgroup differences were also examined on the dimensional level, with cognitively loaded dimension scores displaying larger differences.     

Psychometric Properties of a Structured Behavioral Interview to Hire Private Security Personnel

This paper reports on a study about the reliability and validity of a structured behavioral interview to assess private security personnel. Reliability was estimated using interrater coefficients. Two independent interviewers were used to rate each interviewee. Results show a reliability coefficient of .81 (N = 43) and .89 with Spearman-Brown correction for two raters. Validity was estimated using a content validation approach. This strategy was suggested by Lawshe (1975) to estimate the content validity of selection tests. So far, only two studies carried out by Schmitt and Ostroff (1986) and Carrier et al. (1990) have used Lawshe's strategy in the structured behavioral interview case. The interview consisted of seven questions and each was rated by 11 experts in the job. Results show a significant content validity ratio (CVR) for majority of the questions in the interview and a content validity index (CVI) of .89. Implications of these findings for the practice of the structured behavioral interview are discussed and future research is suggested.     

履历数据测评的效度分析

履历数据是人员测评的重要手段, 但其组织情景性限制使得国外同类工具不能适用于国内人事选拔, 国内也缺乏实证研究验证其信度、效度。本研究结合某通信企业的招募选拔工作, 开发履历数据分析工具, 选取250名应聘者的履历数据、一般认知能力、大五个性和面试结果数据, 通过效标关联效度、增量效度思想, 分析履历数据有效性。结果表明, 以面试结果为效标, 履历数据具有良好的效标关联效度, 与其他测评工具组合使用时有良好的增量效度。     

Assessing personality at risk in personnel selection and development

This paper demonstrates the validity and usefulness of a count technique to screen for potential personality dysfunctioning in NEO‐PI‐R ratings obtained in selection and professional development assessments. The usefulness of this screening technique for Industrial, Work and Organizational (IWO) psychologists is demonstrated in five different samples that were administered the NEO‐PI‐R for selection or development purposes. Three additional samples served as normative data to compute FFM PD count cut‐offs that can be used for selection and career development decisions. Evidence for the construct validity of 6 out of 10 FFM PD counts was provided, and all FFM PD compound scales were significantly related to important criteria, including the final selection decision, the results of a behaviourally oriented selection interview and self‐rated work competencies. The practical utility and limitations of this count technique for personnel selection and development are discussed. Copyright © 2008 John Wiley & Sons, Ltd.     

多面Rasch模型在结构化面试中的应用

使用项目反应理论中的多面Rasch模型,对66名考生在结构化面试中的成绩进行分析,剔除了由于评委等具体测量情境因素引入的误差对原始分数的影响,得到考生的能力估计值以及个体水平的评分者一致性信息。对基于考生能力估计值和考生面试分得到的决策结果进行比较,发现测量误差的确对决策造成影响,对个别考生的影响甚至相当巨大。进一步使用Facets偏差分析以及评委宽严程度的Facets分析追踪误差源。结果表明,将来自不同面试组的被试进行面试原始成绩的直接比较,评委的自身一致性和评委彼此之间在宽严程度上的差异均将导致误差。研究表明,采用Facets的考生能力估计值作为决策的依据将提高选拔的有效性。同时,Facets分析得到的考生个体层次的评分者一致性指标,以及评委与考生的偏差分析等研究结果还可以为面试误差来源的定位提供详细的诊断信息     

Transparency in Structured Interviews: Consequences for Construct and Criterion-Related Validity

Although researchers agree on the use of structured interviews in personnel selection, past research has been undecided on whether these interviews need to be conducted nontransparently (i.e., without giving interviewees any indication of the evaluated criteria) or transparently (i.e., by revealing to interviewees the dimensions assessed in the interview). This article presents two independent studies examining the effects of interview transparency on interviewees' performance and on the interview's construct and criterion-related validity in the context of an application training program. Results from both Study 1 (N = 123) and Study 2 (N = 269) indicate an improvement in interviewees' performance under transparent interview conditions. Both studies further support the assumption that transparent interviews show satisfactory construct validity, whereas nontransparent interviews do not. Moreover, Study 2 showed no significant difference between the interview's criterion-related validity under transparent versus nontransparent conditions. Implications and directions for future research are discussed.     

多面Rasch模型理论及其在结构化面试中的应用

针对影响面试效度的各种误差来源,该文引入了一种新颖的面试结果处理方法：多面Rasch模型。这一模型在结构化面试中的应用不但有利于有效测量被试的能力水平,而且为识别问题评委、进一步完善评分规则、实现面试等值等问题都提供了全新的解决思路。文章在对结构化面试信、效度研究进展进行综述的基础上,介绍了多面Rasch模型的理论及其在结构化面试中的应用框架。     

Decision bias and personnel selection strategies

The purpose of this study was to examine the influence of two decisional biases—framing and cost salience—on personnel selection decisions. One hundred twenty-eight graduate and undergraduate students participated in a personnel selection simulation. Framing was manipulated by inducing participants to use either a “rejecting” strategy (identify those applicants whom you would not interview) or an “accepting” strategy (list those applicants whom you would interview). Cost salience was manipulated by making selection-related costs either implicit or explicit. Results showed that “accepting” strategy subjects selected less applicants to be interviewed than “rejecting” strategy subjects, but only when selection-related costs were made salient. More time was required for subjects to make their selection decisions when selection-related costs were made salient. Framing and cost salience also influenced the success probability thresholds used by subjects to select applicants. Limitations of this research and directions for future study were discussed.     

An ecological study of the reliability of clinical judgment in executive appraisal

This research was a study of the reliability of clinical judgment findings (multitrait) across three different information sources (psychometric tests, structured interview, and psychometric tests and interview). Subjects (N = 74) were middle and senior executives of Western Canadian technical companies; clinicians (N = 3) were trained and experienced industrial psychologists. The study investigated the similarity of clinical evaluation of personological characteristics (based on an 18-factor multitrait paradigm) across the three different information sources. Subjects were independently rated by a single clinician on 18 criterion factors in each of the three information source categories. Test information source categories required the administration of approximately 12 hrs of standardized psychological assessment questionnaires to each of the 74 subjects. Interview source category involved a 1.5-hr structured interview per subject. Combined condition pooled both test and interview conditions. Generalizability of the findings was maximized by the undertaking of the experiment in a natural situation thus increasing ecological validity. Statistical treatments used were designed to assess the similarity of a clinician's evaluation of a subject based on the different category of information available about that client. Convergence (intrarater reliability) indexes range from a high of .64 to a low of .05. Results indicate a varying degree of convergence of multitrait clinical ratings dependent on clinician and trait being rated. Results are discussed in terms of implications for practitioners involved in executive personnel selection.     

Development and preliminary validation of a semi-structured interview for the screening of law enforcement candidates

A standard practice among most law enforcement agencies is to include psychological screening in the selection of job candidates. Although the use of personality tests in predicting job performance of law enforcement officers has received empirical support (e.g., the Minnesota Multiphasic Personality Inventory, the Inwald Personality Inventory), there is a conspicuous absence of data regarding the reliability and validity of interview procedures. The present study represents the first step in the development of a standardized, semi-structured interview for use in the screening of law enforcement personnel. This interview, the Law Enforcement Candidate Interview, was constructed using content areas drawn from measures used in the screening of law enforcement personnel (e.g., Inwald Personality Inventory) and assessment of personality functioning (Structured Clinical Interview for DSM-III-R Personality Disorders). This interview was then administered by two doctoral students to a group of law enforcement academy cadets (n=34). A modest degree of inter-rater reliability was achieved, although internal consistency was somewhat lacking. Interview scores were correlated with measures of academy performance (final grade-point average and peer and supervisor ranking of academy performance). Results of these analyses suggest the LECI is a modest predictor of academy performance. Potential uses of the LECI and future research indications are outlined.     

A Narrative Review of Post-1989 Employment Interview Research

This narrative review covers both selection- and recruitment/com-munication-oriented employment interview research published since 1989. Brief comparisons of older reviews' findings to recent findings are presented. Key topics reviewed include validity and reliability evidence, decision making in the interview, applicant characteristics, applicant reactions, and interview communication processes. Key findings of the present review include: interview validity, under certain conditions, may rival that of mental ability tests; more structure in selection interviews is better; previewing applicant information (e.g., resumes) by the interviewer can harm selection interview effectiveness; and certain recruiter behaviors can influence applicants' willingness to work for the firm.     

基于三角互动模型评判人员选拔的有效性

分析了评判人员选拔的有效性存在的误区,指出评判人员选拔的有效性不应局限于评判素质测评的有效性,还应该全面地评判工作分析、人员管理、绩效评估等环节的有效性。提出了评判人员选拔有效性的三角互动模型,并指出这一模型所揭示的提高人员选拔有效性的途径。     

结构化面试中的评分一致性问题初探

面试由于其直观、灵活的方式及良好的效度而被广泛应用于各类组织机构的人员招聘与录用中。但在多个评分员对一个应试者的面试中 ,评分员间的评分一致性问题是一个关键问题。本文以中央国家机关某部二组评分员对应试者的评分为研究对象 ,对国家公务员录用考试结构化面试中的评分员评分一致性问题进行了初步的探讨与分析。得到的结论如下 :1 .从总体上看 ,两组评分员的成员间在对应聘同一职位的不同应试者的评分在评分标准上是基本一致的 ;2 .不同的评分维度 (测评要素 )对评分员的评分一致性有不同的影响 ;3 .应试者在面试时的表现也会影响评分员的评分一致性 ;4.评分员有关面试的知识和经验水平对评分一致性也有重要的影响。     

Applicant impression management in job interview: The moderating role of interviewer affectivity

In light of the organizational need to obtain talented personnel, an appropriate evaluation of applicant behaviour in the selection interview is crucial. Extending past research on applicant use of impression management (IM) tactics, this study examines the effects of a broad set of IM tactics in a field setting, and also investigates the moderating roles of two rarely tested interviewer characteristics: interviewer positive affectivity (PA) and negative affectivity (NA). Due to the nested nature of the data, consisting of 142 job applicants and 33 interviewers, we adopt hierarchical linear modelling (HLM) to examine the proposed hypotheses. The results indicate that three IM tactics (self‐focused IM, SFIM; other‐focused IM, OFIM; and non‐verbal IM, NVIM) are significantly positively related to interviewer evaluations. Furthermore, interviewer PA appears to strengthen the positive effects of SFIM tactics on interviewer evaluations, while the effects of NVIM tactics may be weaker when interviewers are high in NA. In addition, these findings suggest the importance of interviewer trait affectivity in explaining interviewer's decision‐making variability, which may lead to low inter‐rater reliability and in turn restrict the level of achievable validity.     

Evaluation of video-based assessment in transit operator selection

The use of video testing is relatively new in personnel selection. Initial research has shown that the visual presentation of behaviorial incidents to job applicants may be a practical alternative to paper-and-pencil selection tests. Little research, however, has been conducted on the psychometric properties of video testing. To address this shortcoming in the personnel selection literature, this study reports the results of a validation study conducted on a video test for transit operators conducted in a large Canadian transit authority. The test, the Metro Seattle Video Test (MSVT), was designed to assess the interpersonal skills required of transit operators. The results show that, although content validation evidence was supportive, other psychometric evidence (i.e., reliability, criterion-related validation evidence, and construct-oriented validation evidence) is not consistent with an adequate selection test of interpersonal skills. Recommendations are made for future development and use of video testing in transit operator selection, as well as for improving the reliability and validity of video-based assessment of interpersonal skills.We gratefully acknowledge the assistance of Kevin Kelloway and Maury Getkate for their assistance with this paper, as well as Rick Hackett and Peter Hausdorf for their involvement in the data collection phase of this study.     

Impact of Assessments of Validity Generalization and Situational Specificity on the Science and Practice of Personnel Selection

The application of meta-analysis, in particular validity generalization (VG) analysis, to the cumulative literature on the validity of selection tests has fundamentally changed the science and practice of personnel selection. VG analyses suggest that the validities of standardized tests and other structured assessments are both higher and more consistent across jobs and organizations than was previously believed. As a result, selection researchers and practitioners can draw on the research literature to make reasonably accurate forecasts about the validity and usefulness of different tests in particular applications. Distinctions between tests of validity generalization and tests of situational specificity are described, and difficulties in demonstrating that validity is constant across the different settings where tests are used are outlined.     

Fairness Reactions to Selection Methods: An Italian study

This study examined the fairness reactions to 10 personnel selection methods in a sample of Italian students. University students (N=137) were asked to rate the favorability of these selection procedures and then to evaluate them on eight procedural justice dimensions. Work‐sample tests were the most favorably rated of the selection methods, followed by résumés, written ability tests, interviews and personal references. Graphology was perceived negatively. Opportunity to perform and the perceived face validity of selection procedures were the strongest procedural justice dimensions for predicting the process favorability ratings. The results of this study are compared with those from similar studies conducted in other countries. Similar results have been found in the various countries where such research has been conducted to date.     

A Position Statement by the International Graphonomics Society on the Use of Graphology in Personnel Selection Testing

Among the various tests employed in personnel selection, handwriting analysis, or graphology, has enjoyed long-standing international popularity despite being highly contentious. This report contains not only an evaluation of the current published scientific reviews on the use of graphology in personnel selection, but also an evaluation of several additional studies graphologists provided that seemed to have been overlooked. The latter were obtained by contacting nine of the foremost institutes offering graphological training, consulting services, or both to ensure that the graphologists themselves would be fairly represented. Even with this additional information we found no reason to counter conclusions the scientific community has reached, namely that (a) the continued use of graphology in personnel selection could prove harmful to many individuals and firms, and (b) it fails to approach the level of criterion validity of other widely available and less expensive screening devices used for personnel selection. This article ends with a position statement about this matter, which the International Graphonomics Society1 endorsed.