人事测评效度验证的多方法途径

本文围绕我国人事测评研究与应用中存在的方法论问题,结合人事心理学理论与方法的最新发展,讨论了人事测评效度验证的若干模型与具体方法。文章在此基础上提出了人事测评效度验证的多方法途径及其对人力资源管理的意义。     

人事测评中的概化理论应用

概化理论是现代心理与教育测量理论之一,可应用在各种人事测评中,如表现性评价、多源评估、心理测验、结构化面试、水平测试、工作分析、评价中心等.与经典测量理论相比,概化理论应用于人事测评,表现出较强的优势,能够同时考察多种因素、确定多个维度权重等,其应用对象主要包括两大类,即企业和机构.概化理论应用于人事测评,存在应用领域、样本数据、测评效度和微观分析等问题.     

评价中心结构效度的多元概化理论分析研究

评价中心技术是现代人事测评的一种主要形式,主要通过多种模拟任务来测评高级管理人才。然而,少有研究能证明它具有理想的结构效度。该研究运用多质多法分析了某金融企业一次真实的评价中心测评项目,发现测评的结构效度并不理想。接着,通过多元概化理论的分析,探讨了结构效度不理想的原因,并指出了评价中心任务与所测维度之间的关系,以及对各维度的测量信度。最后,根据分析结果,探讨了优化评价中心结构效度和提升整体测量信度的途径。     

无领导小组讨论的多侧面Rasch模型应用*

采用项目反应理论(IRT)的多侧面Rasch模型(MFRM),分析评价中心技术中无领导小组讨论(LGD)的测评结果,探讨被试能力水平、评委评分宽严度、评分内部一致性、维度难度和评定等级等问题,进而讨论各种偏差。通过 MFRM 分析人事测评结果,可深入了解被试能力的真实差异、甑别维度难度、探查测评误差源,从而完善测评试题编制、评估或诊断评委合格性、提高测评维度与测评目的匹配性,为拓展项目反应理论在人事测评中的应用提供独特视角。     

效度概化：预测效度元分析的30年成果述评

效度概化是通过元分析技术对普遍化的预测效度的估计。元分析是研究效度概化问题的技术,是对具有“预测因子-效标”特征的相关数据进行定量综合的方法。效度概化促进了预测因子与效标之间关系的理论研究和应用研究,是应用心理学领域近30年（1977~2007）来最重要的进展之一。30年来的效度概化研究表明,认知能力测验、知识和技能测验、人格测验、结构化面试和评价中心技术等的预测效度具有鲁棒性、对应性和联合增值性     

履历数据测评的效度分析

履历数据是人员测评的重要手段, 但其组织情景性限制使得国外同类工具不能适用于国内人事选拔, 国内也缺乏实证研究验证其信度、效度。本研究结合某通信企业的招募选拔工作, 开发履历数据分析工具, 选取250名应聘者的履历数据、一般认知能力、大五个性和面试结果数据, 通过效标关联效度、增量效度思想, 分析履历数据有效性。结果表明, 以面试结果为效标, 履历数据具有良好的效标关联效度, 与其他测评工具组合使用时有良好的增量效度。     

应聘者的选拔标准识别能力对人事选拔效果的影响

应聘者对选拔标准的识别能力(ability to identify criteria, 简写为ATIC)是指个体在参与测评的过程中, 正确识别评价标准的能力, 现有测量选拔标准识别能力变量的主要方式有被试自评和研究者评分两种。众多研究发现, 选拔标准识别能力对应聘者的表现有促进作用, 它可以解释选拔的预测效度并影响其结构效度。当公开选拔标准, 提高透明度, 消除选拔标准识别能力个体差异的影响之后, 选拔的预测效度会降低, 而且对应聘者的成绩和选拔的结构效度也有影响, 但目前有关研究的结果并不统一。未来研究应关注选拔标准识别能力与其他构念的关系、完善测量方法、探索它在其他人事选拔手段中的作用, 并继续考察改变选拔标准识别能力对人事选拔的影响。     

LGD在选拔企业中高级管理人才方面的实证研究

本研究利用在为某企业人才招聘活动中提供大型人才测评服务时所收集到的数据,分析研究了无领导小组讨论测评技术的信效度。结果表明,在LGD测评技术中,评价者经过系统培训后,其评分者信度较高,但在不同要素的评价方面存在一定的差异性;在效度方面,通过多种途径的数据检验都表明LGD是一种比较有效的人才测评技术。同时,本研究还得出结论．与个性品质方面的评价相比,LGD在评价个人的能力素质特征方面更有优势。另外,结合本次人才测评的实际操怍经验．研究者还提出了有关LGD测评实施过程的应用建议。     

评价中心的结构效度研究

评价中心虽然具备很高的预测效度,但其结构效度指标却不太理想,如研究普遍发现其汇聚效度和区分效度较低。影响评价中心结构效度的因素众多,如评分维度因素（数量和类型）、评价者因素（培训方式和人员类型）、测评方法因素（情景导向特征、特质激活潜力、测评活动形式）以及系统的观察与评价程序。该文从上述因素出发,综述了评价中心结构效度的相关研究,总结了提高评价中心结构效度的措施,并指出了未来的研究方向     

仿真模拟测评技术信度和效度的初步研究

本研究对典型的仿真测评技术——“角色扮演”测评方法的评分者信度、重测信度和效标关联效度进行了初步研究。结果表明：仿真模拟测评技术具有较强的岗位针对性,在相似情境中具有较高的预测效度,在某些要素上具有测评和培训的双重功能。     

The utility of the NEO-PI-R validity scales to detect response distortion: a comparison with the MMPI-2

In this psychometric study, we compared the recently developed Validity Scales from the Revised NEO Personality Inventory (NEO PI-R; Costa & McCrae, 1992b) with the MMPI-2 (Butcher, Dahstrom, Graham, Tellegen, & Kaemmer, 1989) Validity Scales. We collected data from clients (n = 74) who completed comprehensive psychological evaluations at a university-based outpatient mental health clinic. Correlations between the Validity Scales of the NEO-PI-R and MMPI-2 were significant and in the expected directions. The relationships provide support for convergent and discriminant validity of the NEO-PI-R Validity Scales. The percent agreement of invalid responding on the two measures was high, although the diagnostic agreement was modest (kappa = .22-.33). Finally, clients who responded in an invalid manner on the NEO-PI-R Validity Scales produced significantly different clinical profiles on the NEO-PI-R and MMPI-2 than clients with valid protocols. These results provide additional support for the clinical utility of the NEO-PI-R Validity Scales as indicators of response bias.     

Practical guidelines in the use of symptom validity and other psychological tests to measure malingering and symptom exaggeration in traumatic brain injury cases

The detection of malingering or symptom exaggeration has become an essential component in forensic neuropsychological evaluations, particularly in cases involving personal injury claims. Symptom Validity Tests refer to those measures that can be utilized to detect test performance that is so poor that it is below the level of probability, oftentimes even among brain-damaged populations. This article outlines legal standards for expert testimony in regard to forensic neuropsychological personal injury evaluations. The article provides an outline of specific Symptom Validity Tests and Indicators, and reviews literature supporting test sensitivity and validity. In addition, the use of symptom checklists and questionnaires is discussed, as well as the appropriate use of Symptom Validity Tests and Indicators to establish the presence or absence of malingering or symptom exaggeration.     

A three dimensional test for White's effectance motive

A scale was developed based on Robert W. White's construct of effectance motivation. Effectance was conceptualized as involving three dimensions--Effect on Self, Effect on Objects, and Effect on People. It was found that reliabilities for the three subscales were good. Validity was determined using three methods: criterion groups, multitrait-multimethod matrix, and convergent and discriminant validity. Validity was supported under all three methods.     

Measurement of intimacy attitudes: the Intimacy Attitude Scale--revisited

The study evaluated the reliability and validity of a revised version of the Intimacy Attitude Scale originally developed by Amidon. The Cronbach alpha measure of internal consistency showed alpha values between .78 and .86 in five different groups. The test-retest correlation was .57 which improved to .84 after exclusion of three extreme scores. Validity was examined by correlating with several variables. It was concluded that the revised Intimacy Attitude Scale is a reliable instrument with Some degree of Validity for use in research, diagnosis, and training.     

Diagnostic sex-role bias: how can we measure it?

Summary: A new instrument was designed to measure sex-role bias towards children. The instrument, the Chasen Diagnostic Sex-Role Bias Scale, is composed of case histories focusing on the "somewhat unhealthy" aspects of two main factors of behavior: activity (composed of independence and aggression); and passivity (composed of dependence and nonaggression). Validity and reliability coefficients were reported as well as a formula by which to use the scale to diagnose individual bias, in addition to group bias. In using the scale on a national sample of school psychologists, it was determined that differences in Diagnostic Sex-Role Bias scores exist, as do wide variations about what is considered healthy and unhealthy behavior among girls and boys.     

A time for specificity

McGrath's (2005/this issue) article, "Conceptual Complexity and Construct Validity," is a happy sign that the serious reform necessary for theoretical progress in the study of human personality and psychopathology may be entering a formative phase.     

Validity of Selection/ Classification Variables for Performance in Three Navy Ratings

This study examined speed of advancement and score on a job knowledge test as job performance criteria in the validation of selection and classification variables for three Navy ratings. Longitudinal samples spanning a 4-year period were drawn for men in radioman, machinist's mate, and hospital corpsman ratings who, within their first enlistment, advanced to E-4 by means of the Navy competitive advancement system. Validity coefficients for Navy enlisted selection and classification variables (Armed Services Vocational Aptitude Battery [ASVAB] and education) were computed for personnel in each rating, subgrouped by entry pay grade. Validity coefficients for the job knowledge test were high by historical standards, and the effects were highest for the subgroups with the highest selection standards. Validity coefficients for the advancement criterion were much lower. Advancement, the criterion that appeared to have the greater face validity, was not as good a measure of job performance as the job knowledge test. The criteria were predicted much better by mental ability than by level of education variables. Personnel who enlisted at E-2 and E-3 pay grades spent substantially more time working at the journeyman performance level during their first enlistment than did personnel who enlisted at E-1.     

Statement Validity Analysis

Summary

All information regarding 103 Dutch sexual abuse cases was analyzed using the Validity Checklist, the second part of the Statement Validity Analysis procedure. The fulfillment of the VC items was compared for four different age groups. Only two VC items differed significantly between the age groups. The systematic summarization of content analysis and validity checks revealed only a weak relationship between the fulfillment of the CBCA criteria and the VC items together and the outcome of the cases. The results suggest that SVA cannot yet be used as a scientifically validated instrument for judging the truthfulness of allegations of child sexual abuse.     

Implied body action directs spatial attention

Research confirms that the body influences perception, but little is known about the embodiment of attention. We investigated whether the implied actions of others direct spatial attention, using a lateralized covert-orienting task with nonpredictive central cues depicting static, right/left-facing bodies poised in midaction. Validity effects (decreased response times for validly compared with invalidly cued trials) indicated orienting in the direction of the implied action. In Experiment 1, we compared action (running, throwing) with nonaction (standing) cues. Only the action cues produced validity effects, suggesting that implied action directs attention. The action cues produced faster responses overall, suggesting that action cues prime motor responses. In Experiment 2, we determined whether action cues shifted attention in a specific direction rather than to a general side of space: Two cues had similar action speed and motor effort but differed in implied direction (jumping, vertical; throwing, horizontal). Validity effects were found only for the throw cues for which the implied motion direction was consistent with lateralized target locations. In Experiment 3, we compared block-like stimuli to the throwing action stimuli to examine whether lower level perceptual information could account for the attention effects alone. Validity effects were found only for the human-action stimuli. Overall, the results suggest that predictive simulations of action shift attention in action-consistent directions.     

Morningness‐eveningness and educational outcomes: the lark has an advantage over the owl at high school

Background. Chronotype refers to individuals’ preference for morning or evening activities. Its two dimensions (morningness and eveningness) are related to a number of academic outcomes. Aims. The main goal of the study was to investigate the incremental validity of chronotype as a predictor of academic achievement after controlling for a number of traditional predictors. In so doing, a further aim was ongoing validation of a chronotype questionnaire, the Lark‐Owl Chronotype Indicator. Sample. The sample comprised 272 students attending 9th and 10th grades at five German high schools. Data was also obtained from 132 parents of these students. Method. Students were assessed in class via self‐report questionnaires and a standardized cognitive test. Parents filled out a questionnaire at home. The incremental validity of chronotype was investigated using hierarchical linear regression. Validity of the chronotype questionnaire was assessed by correlating student ratings of their chronotype with behavioural data on sleep, food intake, and drug consumption and with parent ratings of chronotype. Results. Eveningness was a significant (negative) predictor of overall grade point average (GPA), math–science GPA, and language GPA, after cognitive ability, conscientiousness, need for cognition, achievement motivation, and gender were held constant. Validity evidence for the chronotype measure was established by significant correlations with parent‐ratings and behavioural data. Conclusions. Results point to the possible discrimination of adolescents with a proclivity towards eveningness at school. Possible explanations for the relationship between chronotype and academic achievement are presented. Implications for educational practice are also discussed.