Cognitive Tests,Constructs, and Content Validity: A commentary on Schmidt ()

This commentary discusses a number of issues that build on Schmidt's (International Journal of Selection and Assessment, 20, 1‐13 (2012)) perspective on content validity and cognitive tests. First, it elaborates on the relationship between the treatment of content validity in various professional standards and government guidelines. Second, it offers a differing perspective on the definition of ‘construct’ than that taken by Schmidt. Third, it elaborates on the settings in which content validity can and cannot be used to support the use of a given test in the cognitive ability domain.     

Content Validity and Cognitive Tests: Response to Kehoe (), Ployhart (), and Sackett ()

The most important fact emerging from the combination of my article and the three commentaries is the consensus judgment that content validity is appropriate scientifically and professionally for use with tests of specific cognitive skills used in job performance. This is important because the 1978 Uniform Guidelines on Employee Selection Procedures have typically been interpreted as not permitting such usage, and this is particularly the case in the interpretation given to the Guidelines by federal government enforcement agencies. Although the Society for Industrial and Organizational Psychology Principles and the Standards do not prohibit such usage, many industrial–organizational psychologists believe that it is not professionally or scientifically appropriate to employ content validity methods with cognitive measures. The hope is that this series will convince them otherwise. On this point, all four authors in the series are in agreement. The major disagreement among us concerns whether specific cognitive skills used in content valid tests must be considered constructs or not. My position, and apparently that of Kehoe, is that they need not be so considered. I argue that constructs must be invoked only in the context of a substantive theory. Sackett and Ployhart, on the other hand, argue that all measures taken on people must be viewed as constructs, regardless of whether any theoretical propositions and assumptions are involved. In this response, I present reasons why this need not be the case.     

情境判断测验的效度研究述评

情境判断测验是一种为作答者呈现工作相关的典型情境以及该情境下可能的行为反应, 要求根据指导语提示进行选择或评价的测验形式。随着其理论和实践的发展, 研究者越来越关注情境判断测验的效度研究, 包括对其构想效度、效标关联效度和递增效度的探讨, 以及指导语类型、情境保真度以及计分方式等因素对其效度的影响。基于这些研究进展, 未来情境判断测验实践领域可能的方向是：(1)开发针对特定构想的情境判断测验; (2)结合具体构想选用相应的指导语; (3)应用作假和培训对效度影响的研究结果指导实践。     

面试预测效度和构想效度研究述评

面试是人才甄选中最常用的测量工具。大量研究证实,面试的预测效度比较理想,但不同类型面试的预测效度存在差异。虽然能够证实面试的预测效度较好,但对面试的测量构想却知之甚少。研究面试的构想效度,对于提高面试的递增效度有着重要的实践价值。相对人格成分而言,以往研究对面试能够测量到认知成分形成了更加一致地认识。     

Cognitive Tests Used in Selection Can Have Content Validity as Well as Criterion Validity: A broader research review and implications for practice

Many industrial/organizational (I/O) psychologists, both academics and practitioners, believe that the content validity model is not appropriate for cognitive ability measures used in personnel selection. They believe that cognitive tests can have criterion validity and construct validity but not content validity. Based on a review of the broader differential psychology research literature on cognitive skills, aptitudes, and abilities, this article demonstrates that with the proper content validity procedures, cognitive ability measures, including, ultimately, some de facto measures of general cognitive ability, can have content validity in addition to criterion and construct validity. Finally, the article considers, critiques, and refutes the specific arguments contending that content validity is inappropriate for use with cognitive skills and abilities. These research facts have implications for I/O practice, professional standards, and legal defensibility of selection programs.     

The Ecological Validity of Neuropsychological Tests: A Review of the Literature on Everyday Cognitive Skills

Evaluating the ecological validity of neuropsychological tests has become an increasingly important topic over the past decade. In this paper, we provide a comprehensive review of the research on the ecological validity of neuropsychological tests, as it pertains to everyday cognitive skills. This review is presented in the context of several theoretical issues facing ecological validity research. Overall, the research suggests that many neuropsychological tests have a moderate level of ecological validity when predicting everyday cognitive functioning. The strongest relationships were noted when the outcome measure corresponded to the cognitive domain assessed by the neuropsychological tests. Several other factors that may moderate the degree of ecological validity established for neuropsychological tests are in need of further exploration. These factors include the effects of the population being tested, the approach utilized (verisimilitude vs. veridicality), the person completing the outcome measure (significant other vs. clinician), illness severity, and time from injury until evaluation. In addition, a standard measurement of outcome for each cognitive domain is greatly needed to allow for comparison across studies.     

情景判断测验的开发程序、构思效度及研究趋势

章详细介绍情景判断测验开发的一般程序,对情景判断测验的多种形式和记分方法进行了总结和比较。同时,从情景判断测验的结果与认知能力、个性和工作经验等关系的角度出发,分析了情景判断测验的构思效度,认为情景判断测验测量的是多维构思。文章最后认为,需要从与其他构思的关系、测评指定构思、影响效度的因素以及跨文化比较等四个方面对情景判断测验开展进一步研究。     

Shades of Awareness on the Mechanisms Underlying the Quality of Conscious Representations: A Commentary to Fazekas and Overgaard ()

Fazekas and Overgaard ( 2018 ) present a novel, multidimensional model that explains different ways in which conscious representations can be degraded. Moreover, the authors discuss possible mechanisms that underlie different kinds of degradation, primarily those related to attentional processing. In this letter, we argue that the proposed mechanisms are not sufficient. We propose that (1) attentional mechanisms work differently at various processing stages; and (2) factors that are independent of attentional ones, such as expectation, previous experience, and context, should be accounted for if we are aiming to construct a comprehensive model of conscious visual perception.     

The Within‐Setting Variability of Validity in Cognitive Ability Tests

This study was directed to check the second prediction of the situational specificity hypothesis (SSH). According to this prediction, if the setting is held constant, then validity coefficients will not vary from study to study. This prediction was tested using data from a series of studies in which cognitive ability tests were used. These tests are subjected to a cultural bias that could affect the validity variability. The results are contrary to the second prediction of the SSH because a great variability was found in the validity coefficients within‐setting. However, the findings are consistent with the hypothesis of validity generalization (VGH) that predicts similar variability in both across settings and within‐setting. Also, the cultural context does not seem to affect the variability of validity and this variability is similar to that observed in previous studies of validity generalization.     

Situational Tests in Student Selection: An Examination of Predictive Validity,Adverse Impact,and Construct Validity

The Flemish Admission Exam ‘Medical and Dental Studies’ is comprised of four cognitive ability tests and four situational tests, namely two work samples (i.e., a lecture and a medical text) and two video‐based situational judgement tests (i.e., a physician–patient interaction and a medical expert discussion). On the basis of the Admission Exam scores of 941 candidates (359 men, 582 women) this study shows that situational tests significantly can predict better than cognitive ability tests, with lecture and text emerging as significant predictors. When situational tests are combined with cognitive ability tests, there are no mean gender differences. Situational tests also enable us to measure a broader range of constructs. For example, in this study, the personality factor Openness is related to better situational test performance. Overall, this study demonstrates that situational tests may be a useful complement to traditional student selection procedures.     

The Validity of Certain Mechanical Ability Tests for Selecting Cotton Mill Machine Fixers

Abstract

Sherif, M., &; Cantril, H. The Psychology of Ego-Involvements. New York: Wiley, 1947. Pp. 525. Reviewed by Louise Bates Ames

Willkie, H. F. A Rebel Yells. New York: Van Nostrand, 1946. Pp. 311. Reviewed by Samuel Renshaw

Wylie, P. An Essay on Morals. New York: Rinehart, 1947. Pp. 204. Reviewed by Albert Ellis     

A Comparative Study of Integrity Tests: The Criterion-Related Validity of Personality-Based and Overt Measures of Integrity

In the field of personality assessment, one construct or set of constructs that is emerging as important for both publishers and users of psychological testing has been labelled variously as integrity, conscientiousness, reliability, delinquency, or responsibility. The associated organizational outcome has been broadly labelled counterproductivity. In the present study, four integrity scales, selected scales from mainstream personality inventories, derived molar integrity factors and optimal scale linear combinations were correlated with admissions of counterproductivity. It was found that (a) all four integrity scales were significantly correlated with the counterproductivity criterion, (b) individual personality scales from normal personality inventories, as well as unit-weighted linear combinations of them, were approximately as highly correlated with counterproductivity as were the integrity tests, and (c) personality traits conceptually distinct from the domain of integrity were also related to counterproductivity.     

Response Time Measures as Supplementary Validity Indicators in Forced-Choice Recognition Memory Performance Validity Tests: A Systematic Review

Neuropsychology Review - Performance validity tests (PVTs) based on the forced-choice recognition memory (FCRM) paradigm are commonly used for the detection of noncredible performance....     

Cognitive Ability Tests and the Situational Interview: A Test of Incremental Validity

This study investigated the validity and incremental validity of a situational interview beyond that of a composite measure of cognitive ability. Forty-seven factory service technicians underwent an interview and took four cognitive ability tests. Supervisors rated the performance of these subjects in a concurrent validation study. The interview was found to be a valid predictor of a supervisor rating of performance (r = 0.32, p < 0.05 uncorrected), however, was unable to show incremental validity over ability tests (Incremental R²= 0.05, n.s.). Limitations of the present study and directions for future research are discussed.