A REVIEW OF RECENT DEVELOPMENTS IN INTEGRITY TEST RESEARCH

A sizable body of new literature on integrity tests has appeared since the last review of this literature by Sackett and Wanek (1996) . Understanding of the constructs underlying integrity tests continues to grow, aided by new work at the item level. Validation work against a growing variety of criteria continues to be carried out. Work on documenting fakability and coachability continues, as do efforts to increase resistance to faking. New test types continue to be developed. Examination of subgroup differences continues, both at the test and facet level. Research addressing applicant reactions and cross-cultural issues is also reviewed.     

Stereotype Threat and Employment Testing: A Commentary

This article comments on Sackett, Schmitt, Ellingson, and Kabin's (2001) suggestion that stereotype threat may not generalize to employment testing, and the 4 articles of this issue following from it. We argue that each experiment lacked the no-stereotype threat control group needed to experimentally test stereotype threat effects; that sizeable stereotype threat effects occurred nonetheless; and that research not reviewed by Sackett et al. suggests that stereotype threat may be the sole cause of important real-life test-score gaps. The 4 experiments, we suggest, reveal important mediators of stereotype threat effects; their test-specific and general form, and their generalizability to IQ testing. We offer a framework for assessing its generalizability to real-life employment tests.     

Understanding how and why adding valid predictors can decrease the validity of selection composites: A generalization of Sackett,Dahlke, Shewach,and Kuncel (2017)

It is usually assumed that adding more valid predictors will increase the predictive power of a selection test battery. Sackett, Dahlke, Shewach, and Kuncel showed that when selection tests are combined using unit weights, adding a valid predictor can lead to a decrease in validity. Situating the Sackett et al. approach in a more general multivariate framework I show how: (a) it is the tradeoff between predictor validity and predictor intercorrelations, and not the differences in predictor validities that determines whether adding a valid predictor to a composite will cause the validity of that composite to increase or decrease; and (b) this same dynamic applies across a wide range of non‐optimal schemes for weighting predictors and/or criteria.     

INTEGRITY TESTING FOR PERSONNEL SELECTION: AN UPDATE

Sackett and M. M. Harris (1984) reviewed published and unpublished literature on the use of commercially available paper-and-pencil integrity tests for employee selection. This paper reviews recent developments in this area. Legal issues related to the use of the polygraph, and integrity tests are discussed. Empirical research on the reliability, criterion-related validity, construct validity, fakability, and adverse impact of integrity tests is reviewed. Major changes since the previous review include (1) the emergence of two distinct types of tests, labeled overt integrity tests and personality-oriented measures, (2) a broadening of the criteria used for test validation to include global performance, absence, turnover, and forms of counterproductivity other than theft, and (3) an increase in the use of external criteria (e.g., detected theft, turnover) rather than self-report criteria (e.g., admissions of theft) in validation research. Difficulties in interpreting research findings and practical constraints on the use of integrity tests are discussed.     

EFFECTS OF APPLICANT RACE, SEX, SUITABILITY, AND ANSWERS ON INTERVIEWER'S QUESTIONING STRATEGY AND RATINGS

Impression formation research (Snyder & Swann, 1978) suggests that people engage in information seeking strategies designed to confirm their impressions of others. In an extension of this work to the selection interview, Sackett (1982) found, however, that subjects did not consistently adopt a confirmatory information seeking strategy. This study is a further examination of Snyder and Swann's theory. In simulated interviews 170 subjects (1) read resumes of hypothetical applicants, (2) selected ten questions to ask each applicant, (3) received written responses to each question immediately on its selection, and (4) rated each applicant's suitability. An interaction accounting for 73% of variance indicated that applicant ratings were based primarily on the content of applicants' responses, but initial impressions also played a part. No clear-cut, confirmatory information seeking strategy was observed.     

Some Comments on Pareto Thinking,Test Validity,and Adverse Impact: When ‘and’ is optimal and ‘or’ is a trade‐off

De Corte, Lievens, and Sackett add to the literature on selection test validity and adverse impact (AI). Their Pareto‐based weighting scheme essentially asks organizations if they are willing to give up some validity to hopefully achieve some reduction in AI. We considered their approach and conclusions in relation to the regression weighting method we used, and we offer five points that reflect our observations as well as our shared goals. We hope our comments, like their work in this field, will invigorate the pursuit of new ways of examining, and one day resolving, the persistent concern regarding the AI associated with valid selection tests.     

Toward an understanding of situational judgment item validity and group differences

This paper evaluates 2 adjustments to common scoring approaches for situational judgment tests (SJTs). These adjustments can result in substantial improvements to item validity, reductions in mean racial differences, and resistance to coaching designed to improve scores. The first adjustment, applicable to SJTs that use Likert scales, controls for elevation and scatter (Cronbach & Gleser, 1953). This adjustment improves item validity. Also, because there is a White-Black mean difference in the preference for extreme responses on Likert scales (Bachman & O'Malley, 1984), these adjustments substantially reduce White-Black mean score differences. Furthermore, this adjustment often eliminates the score elevation associated with the coaching strategy of avoiding extreme responses (Cullen, Sackett, & Lievens, 2006). Item validity is shown to have a U-shaped relationship with item means. This holds both for SJTs with Likert score response formats and for SJTs where respondents identify the best and worst response option. Given the U-shaped relationship, the second adjustment is to drop items with midrange item means. This permits the SJT to be shortened, sometimes dramatically, without necessarily harming validity.     

Intelligence Tests and Social Policy

Snyderman, M., & Herrnstein, R.J. (1983). Intelligence tests and the Immigration Act of 1924. American Psychologist, 38, 986–995.     

A factor analysis and item response theory analysis of an employee honesty test

Although paper and pencil tests of employee honesty are becoming increasingly widespread in industry, a paucity of research exists regarding them. In a recent review of this literature, Sackett and Harris (1984) noted that scant psychometric evidence is available as to their merits or weaknesses. The aim of this paper is to report on the factor and item analysis of one such test. A principal axis solution and item response theory model (1-parameter) were used to examine the data. The factor analysis revealed four readily interpretable factors. With regard to the item analysis, the results indicated that on the whole most of the 40 items showed a reasonable fit to the model. The implications of this research are addressed.     

EVIDENTIARY STANDARDS IN EMPLOYMENT DISCRIMINATION: A VIEW TOWARD THE FUTURE

This article reviews recent changes in evidentiary standards in employment discrimination litigation as they relate to disparate impact theory. Precedents established in Watson v. Fort Worth Bank and Wards Cove Packing Co. v. Atonio , altering the employee's and the employer's responsibilities in discrimination cases are analyzed. In reaction to these and other Supreme Court rulings, Congress introduced a civil rights bill which was eventually signed into law after numerous compromises. The debate surrounding the development of the 1991 Civil Rights Act and the legislation itself are reviewed. With respect to disparate impact, the Act addresses the criteria for establishing a prima facie case (causation), burden of proof standards, and a definition of business necessity. However, a close reading of the Act and a review of recent disparate impact cases suggest the causation and business necessity issues are not fully resolved. Potential implications for managing personnel decision systems in this new environment are discussed.     

SITUATIONAL JUDGMENT TESTS: CONSTRUCTS ASSESSED AND A META-ANALYSIS OF THEIR CRITERION-RELATED VALIDITIES

Situational judgment tests (SJTs) are a measurement method that may be designed to assess a variety of constructs. Nevertheless, many studies fail to report the constructs measured by the situational judgment tests in the extant literature. Consequently, a construct-level focus in the situational judgment test literature is lacking, and researchers and practitioners know little about the specific constructs typically measured. Our objective was to extend the efforts of previous researchers (e.g., McDaniel, Hartman, Whetzel, & Grubb, 2007 ; McDaniel & Ngyuen, 2001 ; Schmitt & Chan, 2006 ) by highlighting the need for a construct focus in situational judgment test research. We identified and classified the construct domains assessed by situational judgment tests in the literature into a content-based typology. We then conducted a meta-analysis to determine the criterion-related validity of each construct domain and to test for moderators. We found that situational judgment tests most often assess leadership and interpersonal skills and those situational judgment tests measuring teamwork skills and leadership have relatively high validities for overall job performance. Although based on a small number of studies, we found evidence that (a) matching the predictor constructs with criterion facets improved criterion-related validity; and (b) video-based situational judgment tests tended to have stronger criterion-related validity than pencil-and-paper situational judgment tests, holding constructs constant. Implications for practice and research are discussed.     

Mindfulness facets,trait emotional intelligence,emotional distress,and multiple health behaviors: A serial two‐mediator model

In the present study, we tested a serial mindfulness facets‐trait emotional intelligence (TEI)‐emotional distress‐multiple health behaviors mediation model in a sample of N = 427 German‐speaking occupational therapists. The mindfulness facets‐TEI‐emotional distress section of the mediation model revealed partial mediation for the mindfulness facets Act with awareness (Act/Aware) and Accept without judgment (Accept); inconsistent mediation was found for the Describe facet. The serial two‐mediator model included three mediational pathways that may link each of the four mindfulness facets with multiple health behaviors. Eight out of 12 indirect effects reached significance and fully mediated the links between Act/Aware and Describe to multiple health behaviors; partial mediation was found for Accept. The mindfulness facet Observe was most relevant for multiple health behaviors, but its relation was not amenable to mediation. Implications of the findings will be discussed.     

Application patterns when applicants know the odds: implications for selection research and practice

Unlike previous research that found small differences between population standard deviations and applicant pool standard deviations (P. R. Sackett & D. J. Ostgaard, 1994; D. S. Ones & C. Viswesvaran, 2003), this study revealed a 23% disparity between Law School Admission Test (LSAT) scores of all LSAT test takers and those of LSAT test takers who applied to law school. This study also illustrated robust applicant self-selection behavior across different law school ranks. These findings are important, because predictor scores of applicants who know their scores in advance and perceive small selection ratios necessitate substantially smaller range restriction corrections than those that would be required by population standard deviations. Furthermore, these findings more generally reveal that applicants who know their scores in advance behave quite differently from applicants who do not.     

A Phenomenological Inquiry of Clients’ Meaningful Experiences in Counseling With Counselors‐in‐Training

This study is a follow‐up to Sackett, Lawson, and Burge's ( 2012 ) study that explored the meaningful experiences of clients and counselors‐in‐training (CITs). Using the previous study's themes of clients’ experiences in counseling with CITs as a launching pad, the authors examined in greater depth the meaningful experiences of clients who are in counseling with CITs through qualitative phenomenological interviews. Findings, as well as implications for counselor education, supervision, and counseling, are discussed.     

Content Validity and Cognitive Tests: Response to Kehoe (), Ployhart (), and Sackett ()

The most important fact emerging from the combination of my article and the three commentaries is the consensus judgment that content validity is appropriate scientifically and professionally for use with tests of specific cognitive skills used in job performance. This is important because the 1978 Uniform Guidelines on Employee Selection Procedures have typically been interpreted as not permitting such usage, and this is particularly the case in the interpretation given to the Guidelines by federal government enforcement agencies. Although the Society for Industrial and Organizational Psychology Principles and the Standards do not prohibit such usage, many industrial–organizational psychologists believe that it is not professionally or scientifically appropriate to employ content validity methods with cognitive measures. The hope is that this series will convince them otherwise. On this point, all four authors in the series are in agreement. The major disagreement among us concerns whether specific cognitive skills used in content valid tests must be considered constructs or not. My position, and apparently that of Kehoe, is that they need not be so considered. I argue that constructs must be invoked only in the context of a substantive theory. Sackett and Ployhart, on the other hand, argue that all measures taken on people must be viewed as constructs, regardless of whether any theoretical propositions and assumptions are involved. In this response, I present reasons why this need not be the case.     

USE OF APTITUDE SCORES AS PREDICTORS OF SUCCESS IN OCCUPATIONAL TRAINING UNDER THE MDTA

Results from studies utilizing the General Aptitude Test Battery (GATB) to predict success in specific vocational training programs authorized by the Manpower Development and Training Act of 1962 are summarized. The summary involves over 1,200 trainees from a number of states in a variety of occupational areas. Trainees from all included studies were combined and then subgrouped on the basis of number of years of formal education claimed. Good prediction was obtained for the 0–7, 8, 9, 10, 11, and 12 years-of-education groupings. The prediction obtained with the 13–16 years-of-education grouping is not at an acceptable level of significance, but this may be a result of the small sample available for this grouping. The results are seen as supporting Manpower Administration efforts to devise ways in which aptitude tests can be used to help match applicants with low levels of reported educational attainment with suitable vocational or training opportunities.     

A further examination of the constructs underlying assessment center ratings

Previous factor analytic work (Sackett and Dreher, 1982) has found exercise factors, rather than the expected dimension factors, underlying ratings made in managerial assessment centers. The present paper responds to criticisms that inappropriate analyses were conducted and that unrepresentative assessment centers were studied. Exercise factors were found to predominate in two new sets of assessment data which we obtained and in three studies published by other authors, thus indicating that the results were not idiosyncratic to the three centers studied by Sackett and Dreher. Analyses using confirmatory, rather than exploratory, factor analysis also support the dominance of exercise factors. Although dimension factors were found in three of four data sets studied, variables loading on a dimension factor consistently loaded more highly on an exercise factor as well.     

AGE DISCRIMINATION AND PERSONNEL PSYCHOLOGY: A REVIEW AND SYNTHESIS OF THE LEGAL LITERATURE WITH IMPLICATIONS FOR FUTURE RESEARCH

One hundred and fifty-two court cases were reviewed to determine the standards set by the courts for establishing a claim of age discrimination under the Age Discrimination in Employment Act, as amended. Fifty-three are discussed. Two major issues were examined in Part 1 of the review: (1) the type and extent of the evidence sufficient to establish a complaint of age discrimination and (2) available defenses against age discrimination complaints. Part 2 discussed employer strategies for averting and/or dealing with discrimination complaints. Also discussed were the research-related implications of the age discrimination case law, as well as the role of professionals in future research.     

INCREASING THE SUPPRESSIVE EFFECT OF DELAYED PUNISHERS: A REVIEW OF BASIC AND APPLIED LITERATURE

Although punishment is a common feature in programs designed to suppress inappropriate responding, the delivery of punishment may be delayed for a variety of reasons. When punishment is delayed, the result may be a decrease in response suppression. It is therefore important to identify strategies to increase the effectiveness of delayed punishers. In this paper, basic and applied research on delayed punishment is reviewed. The delay‐of‐punishment gradient is described, and strategies are provided to increase the suppressive effect of delayed punishers. For each strategy identified, a conceptual account is provided describing how the strategy enables a delayed punisher to exert a suppressive effect. Furthermore, this paper suggests additional uninvestigated but potentially effective strategies and identifies problems with current research into delayed punishment. Finally, recommendations regarding future research into delayed punishers are provided. Copyright © 2012 John Wiley & Sons, Ltd.