On the Construct Validity of Integrity Tests: Individual and Situational Factors as Predictors of Test Performance

Although integrity tests are widely applied in screening job applicants, there is a need for research for examining the construct validity of these tests. In the present study, a theoretical model examining the causes of destructive behavior in organizational settings was used to develop background data measures of individual and situational variables that might be related to integrity test scores. Subsequently, 692 undergraduates were asked to complete these background data scales along with (a) two overt integrity tests – the Reid Report and the Personnel Selection Inventory, and (b) two personality‐based measures – the delinquency and socialization scales of the California Psychological Inventory. When scores of these measures were correlated with and regressed on the background data scales, it was found that relevant individual variables, such as narcissism and power motives, and relevant situational variables, such as alienation and exposure to negative peer groups, were related to scores on both types of integrity tests. However, a stronger pattern of validity evidence was obtained for the personality‐based measures and, in all cases, situational variables were found to be better predictors than individual variables. The implications of these findings for the validity of inferences drawn from overt and personality‐based integrity tests are discussed.     

Assessment of Creative Thinking Across Cultures Using the Torrance Tests of Creative Thinking (TTCT): Translation and Validity Issues

As part of a project to translate and administer the Torrance Tests of Creative Thinking (TTCT) to Turkish elementary and secondary students, 35 professionals were trained in a full-day workshop to learn to score the verbal TTCT. All trainees scored the same 4 sets of TTCT verbal criterion tests for fluency, flexibility, and originality by filling out specially designed forms to allow item analysis of each of the criterion responses. This analysis outlines a protocol to monitor the responses of scorers at the item level, not a level of analysis that is normally performed on the TTCT. The results of item level analyses for fluency and flexibility data from the 35 trainees reveal new information about translation, training, and scoring. Based on these results, this article makes the case for item-level analysis for cross-cultural applications to prevent artificially congruent scores that will not predict accuracy in future scoring.     

Sampling Bias and Other Methodological Threats to the Validity of Health Survey Research

Data from a longitudinal occupational health survey of professional fire fighters were used to explore the potential impact of two types of methodological bias: sample selection and reactivity. No significant differences on demographic variables were observed between the group who first responded after a within-study change in survey administration format (Delayed) and respondents who had completed surveys since the study's inception (Initial). However, statistically significant differences in the study's 26 outcome measures provided some evidence that between-group differences did exist and that an administration format type of response bias was also potentially present. The effect sizes associated with the 37 observed significant differences ranged from small to medium. These results provide a context for a reexamination of standard techniques for the identification and interpretation of survey research biases. Methods are suggested to strengthen tests for selection bias and to minimize the impact of response biases.     

On The Validity of a Simple Argument for Moral Error Theory

In The Myth of Morality Richard Joyce presents a simple and very influential argument for the truth of moral error theory. In this paper I point out that (i) the argument does not have the form Joyce attributes to it, (ii) the argument is not valid in an extensional propositional logic and (iii) on the most natural way of explicating the meanings of the involved terms, it remains invalid. I conclude that more explanation is needed if we are to accept this particular argument for moral error theory.     

情景判断测验的开发程序、构思效度及研究趋势

章详细介绍情景判断测验开发的一般程序,对情景判断测验的多种形式和记分方法进行了总结和比较。同时,从情景判断测验的结果与认知能力、个性和工作经验等关系的角度出发,分析了情景判断测验的构思效度,认为情景判断测验测量的是多维构思。文章最后认为,需要从与其他构思的关系、测评指定构思、影响效度的因素以及跨文化比较等四个方面对情景判断测验开展进一步研究。     

Cognitive Ability Tests and the Situational Interview: A Test of Incremental Validity

This study investigated the validity and incremental validity of a situational interview beyond that of a composite measure of cognitive ability. Forty-seven factory service technicians underwent an interview and took four cognitive ability tests. Supervisors rated the performance of these subjects in a concurrent validation study. The interview was found to be a valid predictor of a supervisor rating of performance (r = 0.32, p < 0.05 uncorrected), however, was unable to show incremental validity over ability tests (Incremental R²= 0.05, n.s.). Limitations of the present study and directions for future research are discussed.     

Simple additivity of stochastic psychological processes: Tests and measures

In this article, some additive models of behavioral measures are defined, and distributional tests of them are proposed. Major theoretical results include (a) conditions for additivity of components to predict the highest level of dominance in a model-free stochastic dominance hierarchy, (b) methods of identifying the shape of the hazard rate function of an added component from certain relationships among the observable density and distribution functions, and (c) effects of stochastic dependence between components on the distributional tests. Feasibility and usefulness of the methods are demonstrated by application to choice RT and ratings experiments.The author was supported by grants MH44640 to Roger Ratcliff and AFOSR 90-0246 (jointly funded by NSF) to Gail McCoon. Parts of this work were presented at the European Mathematical Psychology meetings of 1991. My thanks to F. Gregory Ashby, Bruce Bloxom, Roger Ratcliff, W. Schwarz, Jim Townsend, and an anonymous reviewer for their many helpful suggestions.     

The Influence of Work Personality on Job Satisfaction: Incremental Validity and Mediation Effects

ABSTRACT Drawing from recent developments regarding the contextual nature of personality (e.g., D. Wood & B. W. Roberts, 2006 ), we conducted 2 studies (1 cross-sectional and 1 longitudinal over 1 year) to examine the validity of work personality in predicting job satisfaction and its mediation of the effect of global personality on job satisfaction. Study 1 showed that (a) individuals vary systematically in their personality between roles— they were significantly more conscientious and open to experience and less extraverted at work compared to at home; (b) work personality was a better predictor of job satisfaction than both global personality and home personality; and (c) work personality demonstrated incremental validity above and beyond the other two personality measures. Study 2 further showed that each of the work personality dimensions fully mediated the association between its corresponding global personality trait and job satisfaction. Evidence for the discriminant validity of the findings is also presented.     

Threats to the Operational Use of Situational Judgment Tests in the College Admission Process

This study examined the coachability of two situational judgment tests, the College Student Questionnaire (CSQ) and the Situational Judgment Inventory (SJI), developed for consideration as selection instruments in the college admission process. Strategies for raising scores on each test were generated, and undergraduates were trained in the use of the strategies using a video‐based training program. Results indicated that the CSQ was susceptible to coaching. In addition, the scoring format of the CSQ was found to be easily exploited, such that trainees could increase their scores by greater than 1 SD simply by avoiding extreme responses on that test. The results as a whole sounded a note of caution for the potential use of the CSQ in the college admission process.     

Situational Tests in Student Selection: An Examination of Predictive Validity,Adverse Impact,and Construct Validity

The Flemish Admission Exam ‘Medical and Dental Studies’ is comprised of four cognitive ability tests and four situational tests, namely two work samples (i.e., a lecture and a medical text) and two video‐based situational judgement tests (i.e., a physician–patient interaction and a medical expert discussion). On the basis of the Admission Exam scores of 941 candidates (359 men, 582 women) this study shows that situational tests significantly can predict better than cognitive ability tests, with lecture and text emerging as significant predictors. When situational tests are combined with cognitive ability tests, there are no mean gender differences. Situational tests also enable us to measure a broader range of constructs. For example, in this study, the personality factor Openness is related to better situational test performance. Overall, this study demonstrates that situational tests may be a useful complement to traditional student selection procedures.     

论马克思主义哲学的再生产难题——以阿尔都塞的解答方案为视角

阿尔都塞认为,马克思主义哲学虽然存在,但又不能按照传统哲学的生产方式进行再生产,因而是一个"悖论"。由于马克思主义哲学的"实践"原则是传统哲学不可兼并的"外部存在",因此绝不能通过改造"实践整体"并赋之以某种理论秩序的方式来实现马克思主义哲学的再生产。更为复杂的是,哲学作为政治在理论领域的延续,发挥着统一诸种意识形态以保障统治阶级领导权的功能,因此马克思主义哲学的再生产必然受到政治斗争的牵制。阿尔都塞深刻地揭示了"实践"与"理论"之间的错位关系,尤其是他以"哲学实践"来作为纾解马克思主义哲学"再生产悖论"的方案更是极富原创性。     

On the Validity of Idiographic and Generic Self‐Concept Implicit Association Tests: A Core‐Concept Model

We present a core‐concept model (CCM) suggesting that stimulus centrality is an important factor in category representations in implicit measures. We tested the hypothesis that idiographic stimuli (first name, birthday) are more central and therefore assess self‐concept in Implicit Association Tests (IATs) more validly than generic and nonspecific stimuli (me, you). Superior validity of the idiographic variant emerged across three different domains of self‐concept. First, an idiographic self‐esteem IAT displayed higher correlations than a generic IAT with self‐assessments and observer‐assessments of self‐esteem. Second, an idiographic body scheme‐IAT predicted subjective ratings of body image and objective body‐mass index. Third, an idiographic aggressiveness‐IAT had higher incremental validity for unprovoked aggression when interacting with explicit measures of aggressiveness. We conclude that idiographic stimuli focus participants' attention on the core features of the self, hence, tapping into self‐related associations to a stronger degree than generic stimuli. Copyright © 2011 John Wiley & Sons, Ltd.     

Applicant Reactions to Personality Tests: Effects of Item Invasiveness and Face Validity

In a simulated employee selection exercise, two hundred and ten participants responded to a personality test that varied in terms of item invasiveness and item face validity. A third factor of empirical job-relatedness was manipulated via test instructions. Reactions to the test, the organization, and behavioral intentions about accepting a job offer were measured. Results indicated that item invasiveness and face validity interacted in the prediction of all dependent variables. Specifically, item invasiveness affected the dependent variables only when face validity was low. Implications for the use of personality tests and integrity tests were discussed.     

情境判断测验的效度研究述评

情境判断测验是一种为作答者呈现工作相关的典型情境以及该情境下可能的行为反应, 要求根据指导语提示进行选择或评价的测验形式。随着其理论和实践的发展, 研究者越来越关注情境判断测验的效度研究, 包括对其构想效度、效标关联效度和递增效度的探讨, 以及指导语类型、情境保真度以及计分方式等因素对其效度的影响。基于这些研究进展, 未来情境判断测验实践领域可能的方向是：(1)开发针对特定构想的情境判断测验; (2)结合具体构想选用相应的指导语; (3)应用作假和培训对效度影响的研究结果指导实践。     

Higher Cost,Lower Validity and Higher Utility: Comparing the Utilities of Two Tests that Differ in Validity,Costs and Selectivity

Traditional approaches to comparing the utility of two tests have not systematically considered the effects of different levels of selectivity that are feasible and appropriate in various selection situations. For example, employers who hope to avoid adverse impact often find they can be more selective with some tests than with others. We conducted two studies to compare the utilities of two tests that differ in costs, validity, and feasible levels of selectivity which can be employed. First, an analytical solution was conducted starting with a standard formula for utility. This analysis showed that for both fixed and variable hiring costs, a higher-cost, lower-validity procedure can have higher utility than a lower-cost, higher-validity procedure when the selection ratios permissible using the two procedures are sufficiently (yet realistically) different. Second, using a computer simulation method, several combinations of the critical variables were varied systematically to detect the limits of this effect in a finite set of specific selection situations. The results showed that the existence of more severe levels of adverse impact greatly reduced the utility of a written test with relatively high validity and low cost in comparison with an assessment center with lower validity and higher cost. Both studies showed that the consideration of selectivity can yield surprising conclusions about the comparative utility of two tests. Even if one test has lower validity and higher cost than a second test, the first may yield higher utility if it allows the organization to exercise stricter levels of selectivity.     

The Content Validity of Cognitively Oriented Tests: Commentary on Schmidt ()

Schmidt (International Journal of Selection and Assessment, 20, 1–13 (2012)) argues that it is possible for scores based on measures of general cognitive ability (GCA) to have content validity evidence. This commentary examines this argument further. I first decompose the various lines of validity evidence that may exist for GCA scores. Next, I consider whether GCA scores can have content validity evidence and whether they cannot. I conclude with several observations about the meaning of content validity within GCA research and practice. The bottom line is that although I agree with Schmidt that GCA scores can have content validity evidence, I am not sure such evidence tells us much about the overall validity of GCA.     

On Conducting Construct Validity Meta-Analyses for the Rorschach: A Reply to Tibon Czopp and Zeligman (2016)

We respond to Tibon Czopp and Zeligman's (2016) critique of our systematic reviews and meta-analyses of 65 Rorschach Comprehensive System (CS) variables published in Psychological Bulletin (2013). The authors endorsed our supportive findings but critiqued the same methodology when used for the 13 unsupported variables. Unfortunately, their commentary was based on significant misunderstandings of our meta-analytic method and results, such as thinking we used introspectively assessed criteria in classifying levels of support and reporting only a subset of our externally assessed criteria. We systematically address their arguments that our construct label and criterion variable choices were inaccurate and, therefore, meta-analytic validity for these 13 CS variables was artificially low. For example, the authors created new construct labels for these variables that they called “the customary CS interpretation,” but did not describe their methodology nor provide evidence that their labels would result in better validity than ours. They cite studies they believe we should have included; we explain how these studies did not fit our inclusion criteria and that including them would have actually reduced the relevant CS variables’ meta-analytic validity. Ultimately, criticisms alone cannot change meta-analytic support from negative to positive; Tibon Czopp and Zeligman would need to conduct their own construct validity meta-analyses.