A large sample investigation of the presence of nonlinear personality-job performance relationships

We examine 123 data sets from validation studies of a single five-factor model-based occupational personality measure for evidence of curvilinear relationships with job performance. Research has produced discrepant findings about whether and when to expect curvilinear relationships between normal range personality measures and job performance. Previous studies have relied on small and unsystematic sampling, a variety of noncomparable performance criteria, the use of personality inventories for which construct validity evidence is not immediately available, and a focus on only one or two of the Big Five personality factors. We report minimal evidence of curvilinearity, suggesting that these effects are unlikely to undermine typical uses of personality test scores in decision making. Any expected declines in performance at high ends of the predictor range were very small on average and would be highly unlikely to produce scenarios in which those passing a realistic cut score would underperform those screened out. Indices of job complexity and the importance of the personality trait did not moderate the forms of each personality–performance relationship. The results are useful for evaluating whether curvilinearity is likely to be an issue when self-report personality assessments are used to make decisions with tangible employment consequences.     

Impact of Rater Beliefs Regarding Performance Appraisal and Its Organizational Context on Appraisal Quality

The relationship between attitudes and beliefs regarding performance appraisal and its organizational context and three measures of rating quality (i.e., rating level, discrimination among ratees, discrimination among rating dimensions) were examined. Eighteen of twenty-four directional predictions about the correlations between attitudinal measures and rating behavior measures were supported. However, because of the expected weak to moderate relationships between predictors and criteria and the use of small samples, statistical power was generally low, and many of the individual observed correlations failed to reach significance. Nevertheless, the consistent pattern of support for the directional predictions suggests that the attitudes and beliefs studied here provide a useful basis for understanding performance appraisal behaviors.     

Personality in teams: Its relationship to social cohesion,task cohesion,and team performance

This study continued past research on the relationship between personality composition in teams and social cohesion and team performance (Barrick, Stewart, Neubert, & Mount, 1998). Results from the Barrick et al. sample (N = 50) were compared with data from two new samples, one comprising drilling teams in the US (N = 24), and the other comprising student teams in The Netherlands (N = 25). Furthermore, this study examined the relationship between personality composition and task cohesion, usually considered to be a stronger predictor of team performance than social cohesion. Results partly confirmed the relationships between personality composition, cohesion, and team performance that were found previously. Minimum levels of conscientiousness and agreeableness contributed positively to both task cohesion and team performance. High mean levels of extraversion and emotional stability contributed positively to social cohesion. Some results differed across the three samples, stressing the importance of task characteristics as a factor influencing relationships between team personality, team processes, and team performance. Although significant relationships were found between social cohesion, task cohesion, and performance, cohesion measures did not mediate relationships between personality composition and team performance.     

New paradigms for assessing emotional intelligence: theory and data

This article describes the development and validation of 2 measures of emotional intelligence (EI): the Situational Test of Emotional Understanding (STEU) and the Situational Test of Emotion Management (STEM). Study 1 (N=207 psychology students) examines multiple sources of validity evidence: relationships with EI, vocabulary, personality, and emotion-related criteria. Study 2 (N=149 white-collar volunteers) relates STEU and STEM scores to clinical symptoms, finding relationships to anxiety and stress for both tests, and to depression for the STEM. It is concluded that new performance-based approaches to test development, such as the present ones, might be useful in distinguishing between test and construct effects. Implications for expanding theory and for developing EI interventions are discussed.     

Does Conscientiousness Moderate the Relationship Between Attitudes and Beliefs Regarding Performance Appraisal and Rating Behavior?

Attitudes, beliefs, and orientations of 253 managers toward performance appraisal, along with the personality dimension conscientiousness, were used to predict rating behavior (i.e., mean ratings and measures of performance dimensions discrimination and ratee discrimination). As predicted, relationships between rater attitudes and rating behavior were moderated by conscientiousness; correlations between the attitudes and rating behavior were strongest for raters who were relatively low on conscientiousness.     

Psychological determinants of health and performance: the tangled web of desirable and undesirable characteristics.

Correlations were determined for male (n = 225) and female (n = 242) college students between sets of undesirable personality traits (anxiety, stress reactivity, anger, and alienation) and desirable personality traits (instrumentality, achievement strivings, and optimism measured by the Scheier-Carver [1987] Life Orientation Test), and a series of outcome variables related to health (self-reported health complaints and health maintenance behaviors and beliefs) and academic performance (academic expectations and actual grade point average). Significant correlations were found between many of the personality variables and the outcome variables. However, partial correlations revealed different relationships for the various criteria. With other variables held constant, health complaints were related to several undesirable characteristics, whereas health maintenance behaviors and beliefs were related to several desirable attributes. Only achievement strivings made an independent contribution in both sexes to the 2 measures of academic performance. The theoretical and methodological wisdom of using measures of multiple personality constructs and outcome variables is discussed.     

Validation of physical ability tests for high pressure cleaning occupations

This paper describes the development and validation of a physical performance test battery to select high pressure cleaning workers at Tennessee Eastman Company. The research uses a concurrent, criterion-related validation strategy to demonstrate the relationship between test performance and job performance. The study proceeded through four stages, with data collected in one stage used in subsequent stages. The two types of predictor measures included ability tests and work sample simulations. For some analyses, the work sample simulations were also used as criteria. Four samples of incumbents (N=63) were tested on all experimental measures. Using criterion groups of successful incumbents and unsuccessful former incumbents (N=33), all of the work simulation tests and the ability-referenced tests were significant predictors of successful versus unsuccessful job status. In addition, the ability tests predicted performance on the work sample simulations, particularly for those requiring strength and endurance. A battery of three ability tests resulted in R=.68 for job status and in R=.43, .73, .80, and .84, respectively, for each of the work sample simulations. Two of the three ability tests in the final test battery are likely to result in adverse impact against females. However, data collected on female maintenance technicians indicated women, as a group, would not be screened out categorically as a result of implementing the physical ability test battery.     

EXPLORING RELATIONS BETWEEN TYPICAL AND MAXIMUM PERFORMANCE RATINGS AND THE FIVE FACTOR MODEL OF PERSONALITY

The study tests the distinction between typical and maximum criteria with ratings of transformational leadership performance, and examines whether the criterion-related validities of the five factor model differ for the two types of criteria. Using an East Asian military sample ( n = 1,259) where multiple ratings of typical and maximum performance were obtained from different sources, we used structural equation modeling to test the typical/maximum performance distinction. Results found that typical and maximum performance are different latent constructs and that this distinction is present even after considering rating method factors (i.e., rater source, time). The importance of this distinction is shown by the fact that validities for the personality constructs were not equally predictive of both criteria: Openness was most predictive of maximum performance, Neuroticism was most predictive of typical performance, and Extroversion was predictive of both. By distinguishing typical from maximum performance constructs, relationships between personality and transformational leadership were found to be stronger than previous research suggested.     

An Alternate Method for Content-Oriented Test Construction: An Empirical Evaluation

This study empirically evaluates the relationships between rationally and empirically estimated item indices used in the development of a test of land navigation knowledge for the US Marine Corps. Three land navigation instructors provided ratings on three item indices (item content validity ratios [CVRs], difficulty, and discrimination). These ratings were correlated with empirically derived item indices (difficulty, discrimination, and item-criterion correlations) obtained from the scores of 359 Marines on measures of land navigation knowledge, skill, and performance. Contrary to previous research, item CVRs were significantly correlated with all three empirically determined item indices. The increased effectiveness of rational item indices obtained in the present study over those from previous research appears to be related to the following areas: qualifications of expert raters, match of raters to the rating task, and the nature of the rating task. The implications for content-oriented test design and for judgmental test validation strategies are discussed.     

The Great Eight competencies: a criterion-centric approach to validation

The author presents results of a meta-analysis of 29 validation studies (N=4,861) that uses the Great Eight competency factors (Kurz & Bartram, 2002) as the criterion measurement framework. Predictors of the Great Eight competencies based only on personality scales show moderate to good correlations with line-manager ratings for all 8 of the competencies. On their own, ability tests correlate with 4 of the 8 competencies, and together ability and personality data yield operational validities ranging from 0.20 to 0.44 for the 8 competencies. Operational validities for aggregated predictors with aggregated criteria were estimated to be 0.53. The value of differentiating the criterion space and of relating predictor variables to criterion variables in a one-to-one fashion is discussed.     

Validation of an ipsative personality measure (DISCUS)

The purpose of this study was to examine the construct validity of an ipsative personality test (DISCUS), and various effects of the ipsative format of the test. Both an ipsative and a normative version of the DISCUS test was administered to a sample of undergraduate students (N = 103), along with an adjective based personality test that measures the Big-Five personality traits (5PFa). The results indicated that the normative and the ipsative version of DISCUS are not equivalent, and caution is needed when using the ipsative version for psychometric evaluations as in validation studies. The four DISCUS dimensions (Dominance, Influence, Stability, and Carefulness) represented combinations of the Big-Five traits rather than independent traits as indicated by the correlations with the Big-Five measure.     

Validation of personality and cognitive measures for insurance claims examiners

Claims Examiners are critical for the profitability of large insurance companies. Mistakes in hiring these people can result in improperly processed claims and/or turnover, and this yields negative financial consequences for the company. This paper describes research evaluating the validity of selection procedures for the position of claims examiner at Blue Cross and Blue Shield of Oklahoma. A job analysis indicated that both cognitive skills and personal abilities are necessary for successful performance as a claims examiner. The job analysis also indicated that both subjective and objective criterion measures reflect important aspects of job performance. The criterion data included nine supervisory ratings, employee nominations, average percent of performance, absences, disciplinary actions, and sick leave. In a concurrent validation strategy, we tested a sample of experienced incumbents (N=50) using four cognitive ability tests (Basic Skills Tests: BST) and two measures of personality (Hogan Personality Inventory and PROFILE). Scales from two personality inventories were combined to form the Claims Examiner Inventory (CEI). Scores from the CEI and three of the four cognitive tests correlated significantly with the average percent of performance and six of the supervisory ratings, including the average supervisory rating. In addition to the cognitive measures, the personality measure contributed significantly to the prediction of percent of performance achieved with R=.64. These results are consistent with other research findings that support the use of both personality and cognitive measures as predictors of job performance.     

Personality,Intelligence and Assessment Centre Expert Ratings

Just under 3000 middle managers were rated on seven dimensions/competencies by consultants after a 1‐day assessment centre and structured interview. They also completed two cognitive ability tests and two well‐established personality tests (NEO PI‐R; MBTI). Correlational analysis showed that the rated dimensions of ‘drive to achieve’, ‘internal locus of control’ and ‘resilience’ all modestly negatively correlated with Neuroticism. All seven ratings were significantly correlated with Extraversion and three each with Openness and Agreeableness. There were few significant correlations with the MBTI but the Thinking–Feeling dimension correlated significantly with combined ratings. Correlations with the intelligence test scores were mostly non‐significant. The seven ratings factored into three identifiable factors: labelled personal assertiveness, toughness and determination and curiosity. Regressions using both a total, overall computed rating of general aptitude, as well as the three scores as criterion variables and ability, personality, participant‐gender and management level as predictor variables demonstrated that they accounted for between a quarter and a third of the variance. Predictors varied considerably as a function of the different criteria measures but ability and gender accounted for little of the variance.     

Why Openness to Experience is not a Good Predictor of Job Performance

Recent meta‐analyses investigating the relationship between personality and job performance have found that openness to experience is the least predictive of the Big Five factors. Unlike other research that has sought to explain the low criterion‐validity with relation to job performance, this study explores the actual construct of openness to experience, suggesting that it consists of two dimensions that relate differentially to job performance thus reducing correlations between overall measures of openness to experience and performance criteria. Exploratory factor analysis of the six sub‐dimensions, or facets, of the NEO PI‐R (a popular measure of the Big Five factors) produced two factors of openness to experience corresponding to different areas to which people are open. A confirmatory factor analysis on a second set of data provided some support for this result. A pattern of differential relationships between the two factors and other variables including personality, biodata and supervisor‐rated performance offered further support for the multidimensionality of openness to experience. The implications of these findings for future research in the selection context are discussed.     

Predictive validation of an insurance agent support person selection battery

This study investigated the predictive validity of a selection battery for insurance agent support persons. The selection battery included cognitive tests, personality scales, and biodata scales. Predictor data were collected as part of the agent support person selection process. Behavior summary scale ratings made by immediate supervisors of agent support persons were used as criteria in this study. Uncorrected correlations for six matched predictor composite-criterion dimension combinations identified in a previous concurrent validation study (Ashworth, Barge, Dunnette, & Carter, 1987) ranged from .17 (p<. 01) to .28 (p< .01) (M=.21, N=357).     

Current Directions in Personnel Selection Research

Abstract— For many decades, the focus of personnel selection research was on developing selection tests that maximized prediction of job performance; the approach was typically lacking in theoretical bases. The past two decades saw significant shifts in research to a focus on the nature of constructs and their interrelationships, characterized by an approach that emphasizes theoretical understanding of the phenomena under investigation. This article provides an overview of how a construct-oriented approach underlies major current directions in scientific research on personnel selection. Emerging trends that are likely to constitute issues of enduring importance are discussed.     

The honeymoon effect in job performance: temporal increases in the predictive power of achievement motivation

Correlations between a job performance criterion and personality measures reflecting achievement motivation and an interpersonal orientation were examined at three points in time after completion of job training for a sample of airline reservations agents. Although correlations between the personality predictors and performance were small and nonsignificant for the 3-month period after beginning the job, by the end of 6 and 8 months a number of significant relationships had emerged. Implications for the utility of personality measures in selection and performance prediction are discussed.     

METAANALYSES OF VALIDITY STUDIES PUBLISHED BETWEEN 1964 AND 1982 AND THE INVESTIGATION OF STUDY CHARACTERISTICS

Review and metaanalyses of published validation studies for the years 1964-1982 of Journal of Applied Psychology and Personnel Psychology were undertaken to examine the effect of (1) research design; (2) criterion used; (3) type of selection instrument used; (4) occupational group studies; and (5) predictor-criterion combination on the level of observed validity coefficients. Results indicate that concurrent validation designs produce validity coefficients roughly equivalent to those obtained in predictive validation designs and that both of these designs produce higher validity coefficients than does a predictive design which includes use of the selection instrument. Of the criteria examined, performance rating criteria generally produced lower validity coefficients than did the use of other more "objective" criteria. In comparing the validities of various types of predictors, it was found cognitive ability tests were not superior to other predictors such as assessment centers, work samples, and supervisory/peer evaluations as has been found in previous metaanalytic work. Personality measures were clearly less valid. Compared to previous validity generalization work, much unexplained variance in validity coefficients remained after corrections for differences in sample size. Finally, the studies reviewed were deficient for our purposes with respect to the data reported. Selection ratios, standard deviations, reliabilities, predictor and criterion intercorrelations were rarely and inconsistently reported. There are also many predictor-criterion relationships for which very few validation efforts have been undertaken.     

Likability and Similarity as Potential Sources of Predictor-Related Criterion Bias in Validation Research

Previous research on the effects of bias in criterion-related validation research has typically involved the use of statistical corrections for halo, leniency, and/or central tendency. We present arguments that likability and similarity of raters to ratees may constitute a form of predictor-related criterion bias. One cannot investigate this form of bias without clear understanding of method, predictor, and criterion constructs and careful direct measurement of each. Measurement and theorizing of method constructs is rarely, if ever, undertaken in criterion-related validation work. The results of a criterion-related validation of the use of quantitative and verbal ability and interview and role-play simulations to predict the performance of 372 federal investigative agents are reported. Using the all-Y LISREL model (Williams & Anderson, 1994), we found that likability and similarity factors were related to interview and role play measures. However, none of these potential "biases" affected both predictor and criterion constructs, hence there was no effect on the estimates of the relationships between the predictors and criteria. Limitations with respect to the generalizability of these results to criterion-related research in which performance data are not as carefully collected as well as advantages and disadvantages of more traditional regression and correlational analyses are discussed.     

Reconsidering vocational interests for personnel selection: the validity of an interest-based selection test in relation to job knowledge, job performance, and continuance intentions

Although vocational interests have a long history in vocational psychology, they have received extremely limited attention within the recent personnel selection literature. We reconsider some widely held beliefs concerning the (low) validity of interests for predicting criteria important to selection researchers, and we review theory and empirical evidence that challenge such beliefs. We then describe the development and validation of an interests-based selection measure. Results of a large validation study (N = 418) reveal that interests predicted a diverse set of criteria—including measures of job knowledge, job performance, and continuance intentions—with corrected, cross-validated Rs that ranged from .25 to .46 across the criteria (mean R = .31). Interests also provided incremental validity beyond measures of general cognitive aptitude and facets of the Big Five personality dimensions in relation to each criterion. Furthermore, with a couple exceptions, the interest scales were associated with small to medium subgroup differences, which in most cases favored women and racial minorities. Taken as a whole, these results appear to call into question the prevailing thought that vocational interests have limited usefulness for selection.