情景判断测验效度的研究述评

文章在回顾近年来情景判断测验研究的基础上,总结了情景判断测验的效标关联效度、结构效度、增量效度和情景判断测验效度的影响因素.研究发现情景判断测验有较高的效标关联效度,是一种较好的人才选拔工具;情景判断测验是一种测量方法,可以用来测量指定的结构;情景判断测验对认知能力、人格、工作知识等变量具有增量效度;试题特性、测验开发模式、研究设计、评分方式等会影响情景判断测验的效度.     

幼儿教师胜任力研究之二:情景判断测验的编制

采用行为事件访谈获得幼儿教师工作事件,通过开放式调查获得教师对这些事件的真实行为反应,并采用德尔菲法对事件和行为反应进行筛选,经验证性因素分析,最终得到由34个情景共103个行为反应项的幼儿教师胜任力情景判断测验。测验的信、效度检验表明:(1)测验各维度的内部一致性信度在0.72~0.85之间,全测验为0.95;(2)验证性因素分析表明测验的结构效度符合测量学要求,效标关联效度交叉验证表明,测验具有较好的效标关联效度。     

情境判断测验的研究进展

目前,情境判断测验的相关研究主要呈现两大趋势———效度研究和跨文化比较研究。其中,效度研究主要分为效标关联效度和构想效度研究,跨文化比较研究则主要探讨测验的文化公平性,以及对不同种族员工工作绩效的预测作用。文章还将介绍研究呈现出的新趋势———对情境判断测验本质的研究,即探讨测验形式特点和被试信息加工过程对测验结果的影响。     

审美的测量:美术能力倾向测验的回顾与展望

美术能力倾向测验对美术人才的识别和选拔具有重要意义。现有测验可分为审美能力测量和艺术创作能力测量两种,其中审美能力测量的测验又可以分为审美判断测验和判断后选择判断理由的两种形式。但以往研究缺乏对审美知觉能力维度的实证研究、没有区分“主观美”和“客观美”,以及缺乏对各类型美术能力倾向测验效度的比较研究。未来可加深对审美知觉能力维度的实证研究,开发多类型美术能力判断测验并比较其效度,开发适用于小学生的美术能力倾向测验,以及开发适合我国国情的美术能力倾向测验。     

计算机模拟动态心理测验

本文通过对心理测验的回顾与计算机模拟文献综合,把传统的纸笔测验与动态心理测验相比较,认为动态心理测验具有目标性、过程性与情境性三个特点,能够适应动态复杂的环境与群体作业任务,并且能够对阶段性的问题作出考察,具有传统的纸笔测验所不具备的优点.同时也提出,计算机模拟测验要更加注意外部效度与构思效度,注意计算机与人的交互作用的局限,才能够避免实验室实验的缺点,达到良好的模拟.     

情境判断测验的效度研究述评

情境判断测验是一种为作答者呈现工作相关的典型情境以及该情境下可能的行为反应, 要求根据指导语提示进行选择或评价的测验形式。随着其理论和实践的发展, 研究者越来越关注情境判断测验的效度研究, 包括对其构想效度、效标关联效度和递增效度的探讨, 以及指导语类型、情境保真度以及计分方式等因素对其效度的影响。基于这些研究进展, 未来情境判断测验实践领域可能的方向是：(1)开发针对特定构想的情境判断测验; (2)结合具体构想选用相应的指导语; (3)应用作假和培训对效度影响的研究结果指导实践。     

运用项目参数调节测验效度的试验研究

1　引言　　测验信度和效度是衡量测验编制质量的两个主要参数。测验信度和效度受项目难度、区分度以及被试能力分布等多方面因素的制约。IRT利用信息函数的概念提出了用项目参数来调节测验信度的具体方法 ,这是IRT在心理和教育测量学上的一大贡献。但对于如何提高测验效度 ,至今人们还是凭经验来选择测验项目 ,缺乏客观有效的方法。另外 ,项目难度与区分度是密切地关联着的 ,它们协同影响着测验效度。为此 ,在研究项目参数与测验效度间的关系前 ,首先应该研究项目难度与项目区分度间的关系。2　区分度对难度的回归关系的模拟试验2 .1　…     

效度概化：预测效度元分析的30年成果述评

效度概化是通过元分析技术对普遍化的预测效度的估计。元分析是研究效度概化问题的技术,是对具有“预测因子-效标”特征的相关数据进行定量综合的方法。效度概化促进了预测因子与效标之间关系的理论研究和应用研究,是应用心理学领域近30年（1977~2007）来最重要的进展之一。30年来的效度概化研究表明,认知能力测验、知识和技能测验、人格测验、结构化面试和评价中心技术等的预测效度具有鲁棒性、对应性和联合增值性     

基于行为事件的履历资料评估

行为事件的履历资料评估能有效克服传统履历数据构思效度弱、情景限制多等问题。本研究结合某通信企业招聘工作,选取250名应聘者数据,结合关键事件法,通过对履历事件的行为锚定来评估应聘者的胜任特征。研究在多重比较行为履历资料、履历表数据、认知能力等多个指标组合对录用结果预测的回归模型基础上,检验新增指标的预测效度。结果表明,行为事件的履历资料评估具有效标关联效度,与其他工具组合使用时具有增量效度。     

关于“考试行为困扰与心理障碍测验”的研究报告

在学校教书育人和心理卫生工作中,青年学生在考试问题上存在着行为困扰与心理障碍的事实是不可忽视的。本文主要介绍了作者关于编制“考试行为困扰与障碍心理测验”的初步研究,包括对测验因素及题目的筛选,对测验信度与效度证据的搜集,以及对测验样本的数据分析等。研究者认为,尽管该测验的相关研究有待深入,但测验的设计是沿着科学与可行的思路进行的,其信度与效度也是基本可接受的,故该测验具有一定的实用价值和发展前途。     

SITUATIONAL JUDGMENT TESTS: CONSTRUCTS ASSESSED AND A META-ANALYSIS OF THEIR CRITERION-RELATED VALIDITIES

Situational judgment tests (SJTs) are a measurement method that may be designed to assess a variety of constructs. Nevertheless, many studies fail to report the constructs measured by the situational judgment tests in the extant literature. Consequently, a construct-level focus in the situational judgment test literature is lacking, and researchers and practitioners know little about the specific constructs typically measured. Our objective was to extend the efforts of previous researchers (e.g., McDaniel, Hartman, Whetzel, & Grubb, 2007 ; McDaniel & Ngyuen, 2001 ; Schmitt & Chan, 2006 ) by highlighting the need for a construct focus in situational judgment test research. We identified and classified the construct domains assessed by situational judgment tests in the literature into a content-based typology. We then conducted a meta-analysis to determine the criterion-related validity of each construct domain and to test for moderators. We found that situational judgment tests most often assess leadership and interpersonal skills and those situational judgment tests measuring teamwork skills and leadership have relatively high validities for overall job performance. Although based on a small number of studies, we found evidence that (a) matching the predictor constructs with criterion facets improved criterion-related validity; and (b) video-based situational judgment tests tended to have stronger criterion-related validity than pencil-and-paper situational judgment tests, holding constructs constant. Implications for practice and research are discussed.     

A SITUATIONAL JUDGMENT TEST OF PERSONAL INITIATIVE AND ITS RELATIONSHIP TO PERFORMANCE

Although situational judgment tests have been found to be valid predictors of performance, they have rarely been used to measure particular constructs. In this study, we apply the situational judgment test method to the measurement of personal initiative, a construct defined as situated action. We used respondents' situated preferences in mental simulations of work scenarios as formative indicators of their overall level of personal initiative at work. Results from a validation study showed that the situational judgment test of personal initiative (SJT-PI) had adequate validity and complemented a Likert-type self-report measure of personal initiative in predicting behavioral criteria. Situated preferences for personal initiative were hypothesized to be proximal predictors of actual behavior and were accordingly found to mediate the relationship between generalized self-efficacy, felt responsibility, and actual behavior. Furthermore, situated preferences for personal initiative could be differentiated empirically from organizational citizenship behavior. We conclude that situational judgment tests are a promising method for measuring personal initiative and may be a general means of improving the validity of measurement in organizations.     

Use of situational judgment tests to predict job performance: a clarification of the literature

Although situational judgment tests have a long history in the psychological assessment literature and continue to be frequently used in employment contexts, there has been virtually no summarization of this literature. The purpose of this article is to review the history of such tests and present the results of a meta-analysis on criterion-related and construct validity. On the basis of 102 coefficients and 10,640 people, situational judgment tests showed useful levels of validity (rho = .34) that were generalizable. A review of 79 correlations between situational judgment tests and general cognitive ability involving 16,984 people indicated that situational judgment tests typically evidence relationships with cognitive ability (rho = .46). On the basis of the literature review and meta-analytic findings, implications for the continued use of situational judgment tests are discussed, particularly in terms of recent investigations into tacit knowledge.     

False consensus in situational judgment tests: What would others do?

We introduce an alternative response instruction to reduce the fakability of situational judgment tests. This novel instruction is based on the false consensus effect, a robust social psychological bias whereby people infer that the majority of other people’s thoughts, attitudes, and behaviors are aligned with their own. In four studies, including both field and laboratory data (total N = 882), we demonstrate that participants show a false consensus bias when asked what others would do in situational judgment tests. Furthermore, the situational judgment test based on the false consensus effect turned out to relatively difficult to be fake, and produced scores that were meaningfully correlated with conceptually related traits, as well as both self-reported and behavioral outcomes.     

Measuring Relationships between Personality,Knowledge, and Performance using Single‐response Situational Judgment Tests

We report two studies that investigate single‐response situational judgment tests (SJTs) as measures of job knowledge. Study 1 examines relationships between job knowledge measured by a single‐response SJT, personality, and performance for museum tour guides. Study 2 extends Study 1's findings with a sample of volunteers using a single‐response SJT about volunteerism. In both studies, personality was related to knowledge, and knowledge predicted performance. In Study 2, knowledge accounted for incremental variance in performance beyond personality, but personality added no incremental variance beyond knowledge. Results suggest that knowledge of effective behavior and knowledge of ineffective behavior are separate constructs. These studies demonstrate the validity of single‐response SJTs and provide evidence that knowing what to do and what not to do are separate domains.     

Work Sample Exams and Gender Adverse Impact Potential: The influence of self‐concept,social skills,and written skills

We clarify the existing literature on gender differences in work sample exams by disentangling such differences on work sample exams from other predictors (e.g., situational judgment tests). In fact, we note that there are only two articles specifically related to gender differences in work sample exams. Based on theory and literature from social psychology, neuropsychology, and applied psychology, we propose that the three constructs of self‐concept, social skills, and writing skills are likely to influence work sample gender differences. We tested our hypotheses on two samples of managers. We found in one instance that males scored higher on a technical exercise, but there was stronger support for females, on average, scoring higher on exercises that involved social skills and on exercises that involved writing skills. Work samples that targeted a broad array of knowledge, skills, and abilities were associated with higher overall scores for females (ds of ?.37 and ?.34) and, thus, were unlikely to be associated with gender‐based adverse impact against females.     

An Ideal Student Factor and the validity of noncognitive measures of student potential

The presence of a general factor has been found in previous research on the utilization of personality tests in selection contexts. Termed the Ideal Employee Factor (IEF), this factor is thought to represent an effort to present oneself as an ideal employee to some degree on all items in the test. Examining the responses to biodata and situational judgment items, we provide evidence for the existence of a factor similar to the IEF within the college admissions context, termed the Ideal Student Factor (ISF), but the magnitude of this factor relative to specific factors was small. Mixed support was found for the hypothesized differences in the factor loadings on the ISF for different constructs included in the model. The presence of the ISF was not found to substantially impact the validity of other predictors within this study. The overall conclusion is that this general factor would not impact the use of these biodata and situational judgment items in college admissions.     

Situational Tests in Student Selection: An Examination of Predictive Validity,Adverse Impact,and Construct Validity

The Flemish Admission Exam ‘Medical and Dental Studies’ is comprised of four cognitive ability tests and four situational tests, namely two work samples (i.e., a lecture and a medical text) and two video‐based situational judgement tests (i.e., a physician–patient interaction and a medical expert discussion). On the basis of the Admission Exam scores of 941 candidates (359 men, 582 women) this study shows that situational tests significantly can predict better than cognitive ability tests, with lecture and text emerging as significant predictors. When situational tests are combined with cognitive ability tests, there are no mean gender differences. Situational tests also enable us to measure a broader range of constructs. For example, in this study, the personality factor Openness is related to better situational test performance. Overall, this study demonstrates that situational tests may be a useful complement to traditional student selection procedures.     

FURTHER STUDIES OF SITUATIONAL TESTS

In the present paper, results are reported for two different situational judgment tests (SJT) used in validation studies with almost 4,000 employees in 7 different organizations. Across the 2 studies, it was shown that situational test scores were significantly related to performance (weighted average r = .19), cognitive ability (weighted average r = .45), and experience (weighted average r = .20). In one study, there was a slight tendency for experience and cognitive ability to interact in the prediction of situational judgment, such that cognitive ability became less predictive as experience increased. Situational judgment fully mediated the effects of cognitive ability in one study, but not in the other. Finally, SJT race effect sizes were consistent with past research and, while not trivial, smaller than those typically observed for cognitive ability tests. The studies yielded very similar results despite the use of different keying methods (empirical vs. subject matter expert opinion). The preponderance of the evidence indicates that situational judgment measures mediate avariety of job relevant skills. Limitations to these studies and directions for future research are reviewed.     

Incremental Validity of Situational Judgment Tests for Task and Contextual Job Performance

This paper has three goals. First, it responds to calls for additional research on subgroup differences in situational judgment tests. Second, it expands the cumulative knowledge on the incremental validity of situational judgment tests beyond cognitive ability and personality. Third, it examines the validity and incremental validity of various predictors for both task and contextual performance.