CORRESPONDENCE BETWEEN ESTIMATES OF CONTENT AND CRITERION-RELATED VALIDITY VALUES

The correspondence between inferences made using two validation strategies–content and criterion-related–were examined in a specific personnel selection application. Empirical validity values and Law-she's (1975) content validity ratios (CVR) were obtained for items from three structured interview guides used in the selection of insurance agents. Ratings of each item by over 300 field managers were used to calculate the CVR values. Statistically significant, yet modest correlations were found between empirical item validities and content validities for an interview guide used to select applicants with prior insurance sales experience. No significant differences were found among these correlations by comparing job experts of different levels of managerial experience and experience in selection. Data for the interview guide used to select experienced applicants also indicated that a content validity approach can be useful in developing a selection instrument with an empirically valid composite rating. The hypotheses were not confirmed for interview guides used to select applicants with no prior insurance sales experience. The practical importance of these results are discussed, as are plans for future research.     

The Effects of Social Desirability and Faking on Personality and Integrity Assessment for Personnel Selection

A review of the extant literature and new empirical research suggests that social desirability is not much of a concern in personality and integrity testing for personnel selection. In particular, based on meta-analytically derived evidence, it appears that social desirability influences do not destroy the convergent and discriminant validity of the Big Five dimensions of personality (Emotional Stability, Extraversion, Openness to Experience, Agreeableness, and Conscientiousness). We also present new empirical evidence regarding gender and age differences in socially desirable re- sponding. Although social desirability predicts a number of important work variables such as job satisfaction, organizational commitment, and supervisor ratings of training success, social desirability does not seem to be a predictor of overall job performance and is only very weakly related to specific dimensions of job performance such as technical proficiency (r = -.07) and personal discipline ( r = .05). Large sample investigations of the moderating influences of social desirability in actual work settings indicate that social desirability does not moderate the criterion-related validities of personality variables or integrity tests. The criterion-related validity of integrity tests for overall job performance with applicant samples in predictive studies is .41. Controlling for social desirability in integrity or personality test scores leaves the operational validities intact, thereby suggesting that social desirability functions neither as a mediator nor as a suppressor variable in personality-performance.     

Openness to Experience: Its Lower Level Structure,Measurement, and Cross-Cultural Equivalence

Openness to Experience is an important but relatively poorly understood personality construct. Advances in openness research require further construct clarification as well as establishment of a common framework for conceptualizing and measuring the lower level structure of the construct. In this article, we present data from 3 studies to address this research need. In Study 1, we identify 6 facets of Openness to Experience—intellectual efficiency, ingenuity, curiosity, aesthetics, tolerance, and depth—based on a factor analysis of 36 existing Openness-related scales. In Study 2, we present further validity evidence for the 6-facet structure based on a newly developed measure of Openness. Data from this study also suggest the presence of 2 intermediate-level factors (i.e., aspects) of Openness: intellect and culture. In Study 3, we present a short form of the newly developed measure, retaining items that showed the highest internal consistency and measurement invariance across 3 samples: U.S. undergraduates, Chinese MBA students, and Chinese undergraduates. Together these 3 studies offer a more nuanced understanding of the multifaceted nature of the Openness construct.     

Doing It All At Once: Multitasking as a predictor of call center agents’ performance and performance‐based dismissal

Multitasking has become an important skill in many jobs. Still, the predictive validity of job‐applicants’ multitasking abilities has rarely been tested empirically. The current study focuses on the multitasking ability of call center applicants. Results from a Dutch call center show that applicants’ multitasking ability as assessed during personnel selection indeed predicts their later job performance as call center agents as well as their likelihood of losing their job for poor performance. While some of these relationships could be explained via applicants’ fluid intelligence, results also support the usefulness of including measures of multitasking ability in the current selection procedure.     

JOB-RELATEDNESS OF A POSTTRAINING JOB KNOWLEDGE CRITERION USE TO ASSESS VALIDITY AND TEST FAIRNESS

Significant job-relatedness was found for a posttraining job knowledge test criterion using an application of Lawshe's content validity method. The aide test was used as a criterion to assess the predictive validity of a vocabulary test and a civil service test with samples of black ( N = 43) and white ( N = 62) psychiatric aides. Significant validities were found on both tests, but a vocabulary test proved to be the better predictor of the criterion in both samples. The obtained validities were discussed in terms of differential validity, test fairness, and sample size. This study demonstrated that a content validity method could be applied to criteria as well as selection tests. It was concluded that content validity methods may be able to help solve the problem of criterion relevance in validation research by providing quantitative evidence of the job-relatedness of criteria.     

MMPI Scales for measuring Eysenck's personality factors

Four new MMPI Scales were constructed to identify Eysenck's personality factors of psychoticism, extraversion, and neuroticism as well as a lie score. The scales showed good construct validities against the Eysenck Personality Questionnaire and the Friedman Overlap Scales developed from items in the MMPI. The study also supported the construct validity of the Friedman overlap scales.     

Effects of the testing situation on item responding: cause for concern

The effects of faking on personality test scores have been studied previously by comparing (a) experimental groups instructed to fake or answer honestly, (b) subgroups created from a single sample of applicants or nonapplicants by using impression management scores, and (c) job applicants and nonapplicants. In this investigation, the latter 2 methods were used to study the effects of faking on the functioning of the items and scales of the Sixteen Personality Factor Questionnaire. A variety of item response theory methods were used to detect differential item/test functioning, interpreted as evidence of faking. The presence of differential item/test functioning across testing situations suggests that faking adversely affects the construct validity of personality scales and that it is problematic to study faking by comparing groups defined by impression management scores.     

Reducing MMPI-2 defensiveness: the effect of specialized instructions on retest validity in a job applicant sample

The MMPI-2 is often used for screening job applicants when public safety or security are at risk. Inherent in such applications is concern for profile validity and test defensiveness. In this study, we examine the impact of revised instructions on profile validity for a group of job applicants who initially produced invalid profiles. Participants were 271 male applicants for airline pilot positions. Of these, 72 produced invalid defensive MMPI-2 profiles during preemployment screening. The MMPI-2 was readministered to these applicants with instructions informing them of validity scales and instructing them to respond in a more open, honest manner. Comparisons were made between valid and invalid profiles for initial administrations and between valid and invalid profiles at readministration. Some clinical scales were more elevated for valid, nondefensive profiles. Most content scales showed more elevation for valid profiles, and 12% of the applicants who were retested produced significant elevations (T>or=65) on the content scales. Profiles were similar to those produced by employed pilots of a previous study.     

Development of a Japanese version of the emotional skills and competence questionnaire

The present study described development of a Japanese version of the Emotional Skills and Competence Questionnaire and examined the relations of scores with those on Big Five scales of personality and self-esteem scales. The participants were 615 undergraduates. Factor analysis led to the shortened version of 24 items in three subscales. Although Cronbach alphas were low for the subscale, Manage and Regulate Emotion, values were satisfactory for the other two subscales, Express and Label Emotion and Perceive and Understand Emotion. Total scores of this version were positively correlated with score for self-esteem, Extraversion, and Openness but negatively correlated with scores on Neuroticism. This shorter Japanese versions shows suitable internal consistency and content validity, but other reliabilities and validities must be examined precisely.     

On the relation between personality and job performance of airline pilots

The validity of a personality questionnaire for the prediction of job success of airline pilots is compared to validities of a simulator checkflight and of flying experience data. During selection, 274 pilots applying for employment with a European charter airline were examined with a multidimensional personality questionnaire (Temperature Structure Scales; TSS). Additionally, the applicants were graded in a simulator checkflight. On the basis of training records, the pilots were classified as performing at standard or below standard after about 3 years of employment in the hiring company. In a multiple-regression model, this dichotomous criterion for job success can be predicted with 73.8% accuracy through the simulator checkflight and flying experience prior to employment. By adding the personality questionnaire to the regression equation, the number of correct classifications increases to 79.3%. On average, successful pilots score substantially higher on interpersonal scales and lower on emotional scales of the TSS.     

Racial differences in socially desirable responding in selection contexts: magnitude and consequences

Two studies were conducted to examine the magnitude and consequences of racial differences on social desirability (SD) scales. Study 1 included 1,063 job applicants, and Study 2 included 3 sets of incumbents (total N = 534). In both studies, participants were administered several personality measures and an SD scale. Across all samples, Whites scored lower on the SD scale than Blacks (average d = .37), Hispanics (average d = .47), and Asians (average d = 1.04), but these differences were not observed on the personality scales. The consequence of differences in socially desirable responding (SDR) is that fewer minority group members would be selected if SD scales were used to derive cut scores to eliminate individuals from the applicant pool or if the scales were used to correct personality test scores for SDR. However, applying the SD correction did not affect the validity of the personality test for any of the racial groups. Overall, our findings suggest that researchers and practitioners should consider the use of SD scales very carefully, as their use may have unintended consequences. These studies also demonstrate a need to closely examine the construct validity of SD measures across diverse groups.     

A measure of autonomy

This article reports the development of a measure of indicidual differences in autonomous rule compliance. The autonomy scale (a short, easily administered CPI based test) was developed within the framework of a multidimensional, role-theoretical model of moral development. Five samples were used in the construction of the scale. Two of the samples (total n = 111) were used to derive the autonomy scale. The items for the scale were derived through the sequential use of two common item selection stategies: criterion keying and factor analysis. An initial set of 55 CPI items were derived using an "ideal" autonomy Q-sort profile as a selection criterion, and an Alpha factor solution was used to reduce this initial pool to a final set of 25 items. Several analyses were conducted using three additional samples (total n = 245) to estimate the reliability of the scale and determine its validity. The results of these analyses provide initial evidence for the content, criterion-realted, and construct validity of the scale and indicate that the measure has an adequate reliability.     

True or False? Keying Direction and Acquiescence Influence the Validity of Socio-Emotional Skills Items in Predicting High School Achievement

What type of items, keyed positively or negatively, makes social-emotional skill or personality scales more valid? The present study examines the different criterion validities of true- and false-keyed items, before and after correction for acquiescence. The sample included 12,987 children and adolescents from 425 schools of the State of São Paulo Brazil (ages 11–18 attending grades 6–12). They answered a computerized 162-item questionnaire measuring 18 facets grouped into five broad domains of social-emotional skills, i.e.: Open-mindedness (O), Conscientious Self-Management (C), Engaging with others (E), Amity (A), and Negative-Emotion Regulation (N). All facet scales were fully balanced (3 true-keyed and 3 false-keyed items per facet). Criterion validity coefficients of scales composed of only true-keyed items versus only false-keyed items were compared. The criterion measure was a standardized achievement test of language and math ability. We found that coefficients were almost as twice as big for false-keyed items’ scales than for true-keyed items’ scales. After correcting for acquiescence coefficients became more similar. Acquiescence suppresses the criterion validity of unbalanced scales composed of true-keyed items. We conclude that balanced scales with pairs of true and false keyed items make a better scale in terms of internal structural and predictive validity.     

Generalization of Cognitive and Noncognitive Validities across Personality‐based Job Families

The positive relationship between complexity of work and the validity of general mental ability (GMA) measures across a variety of occupations is well supported by research and provides important practical and theoretical support for cognitive ability measures. However, there is currently no research demonstrating a systematic relationship between the size of the validities of any personality measure and the personality requirements of jobs, thus leaving open to question the predictive and construct validity of personality measures for applicant selection. We compared the validities of two biodata measures – one scored to measure social competence and one to measure GMA – across six job families that varied in social requirements. The validity of the GMA measure generalized at approximately the same level across the job families while the validity of the social competence measure decreased as social requirements diminished.     

Individual differences in hedonic capacity

Hedonic capacity--an individuals ability to experience pleasurable affect--is important, not Only in accounting for the development of psychopathology but in understanding normal personality as well. In this article, the development and validation of three scales of hedonic capacity are described. One scale consists of MMPI items, a second consists of CPI items, and the third combines items from both inventories. These scales demonstrated acceptable internal consistency and test-retest reliability. Initial support for the construct validity of the scales was also obtained. Because they have been constructed from the MMPI and CPI, the scales described in this article can be used to address important questions in research on schizophrenia, depression, and normal personality with a wide variety of data that has already been collected.     

Dimensionality and Validity of a Structured Telephone Reference Check Procedure

A structured telephone reference check (STRC) was developed to assess three job-relevant personality/interpersonal dimensions (Conscientiousness, Agreeableness, customer focus) for a family of entry-level, customer-contact jobs within a service organization. The procedure was designed to be brief (a 10–15 minute telephone call with each referee) and easy to administer by untrained recruitment staff. Confirmatory factor analyses on referee ratings for 223 applicants indicated that referees were able to adequately discriminate between dimensions of applicants' past performance. A predictive, criterion-related validation study on those applicants who were hired demonstrated that the STRC predicted supervisory ratings of job performance (   r = .25, p < .01,  uncorrected; .36 corrected for range restriction and criterion unreliability). No evidence was found that referees with shorter working relationships with applicants provided less valid ratings.     

A Hostility Scale for the California Psychological Inventory: MMPI,Observer Q-Sort,and Big-Five Correlates

Using two samples, we developed and validated a hostility scale that can be scored from the California Psychological Inventory (CPI) and serves as an alternate for the Cook-Medley Hostility Scale (Ha; Cook & Medley, 1954). The CPI Hostility (H) scale consists of 33 items that are either duplicates or close equivalents of specific He items, and the two scales correlate at least .90 in samples differing in sex. The H and Ho scales show a similar pattern of correlations with conceptually relevant MMPI scales and with observer-rated personality attributes tapping Barefoot, Peterson, et al.'s (1991) five hostility categories of Hostile Affect, Cynicism, Aggressive Responding, Social Avoidance, and Hostile Attributions. These findings provide evidence for the equivalence of the two hostility scales, as well as external validation for those personality characteristics that are purported to underlie the construct of hostility as tapped by both the original Ho and the new CPI H scale.     

Reducing MMPI-2 Defensiveness: The Effect of Specialized Instructions on Retest Validity in a Job Applicant Sample

The MMPI-2 is often used for screening job applicants when public safety or security are at risk. Inherent in such applications is concern for profile validity and test defensiveness. In this study, we examine the impact of revised instructions on profile validity for a group of job applicants who initially produced invalid profiles. Participants were 271 male applicants for airline pilot positions. Of these, 72 produced invalid defensive MMPI-2 profiles during preemployment screening. The MMPI-2 was readministered to these applicants with instructions informing them of validity scales and instructing them to respond in a more open, honest manner. Comparisons were made between valid and invalid profiles for initial administrations and between valid and invalid profiles at readministration. Some clinical scales were more elevated for valid, nondefensive profiles. Most content scales showed more elevation for valid profiles, and 12% of the applicants who were retested produced significant elevations on the content scales. Profiles were similar to those produced by employed pilots of a previous study.     

Personality and Situational Judgment Tests Across Applicant and Incumbent Settings: An Examination of Validity,Measurement, and Subgroup Differences

The effects of concurrent versus predictive designs were examined in 2 large samples of incumbents (N = 2,989) and applicants (N = 7,259) for 3 measures of personality and a situational judgment test (SJT). Results showed that applicants scored significantly higher than incumbents on the personality measures and significantly lower on the SJT. Regarding measurement equivalence, the results indicate that the same form of factor structure is found in both settings, with the factor structures themselves being generally invariant. Importantly, the criterion-related validities were equivalent across samples. Subgroup differences were small to moderate in both settings, though varying across race and gender. Limitations and directions for future research are offered.     

Scoring Biodata: Is it rational to be quasi‐rational?

Five quasi‐rational biodata scales were developed by empirically keying biodata items to predict scores on a measure of the Big Five dimensions of personality. The criterion‐related validities of the quasi‐rational scales were compared to empirical and rational biodata keying methods using supervisory ratings of job performance as the criterion. Empirical keying outperformed the quasi‐rational and rational methods (which had similar validities).