The construct and criterion validity of emotional intelligence and its potential utility for management studies

In this study, the authors reviewed the definition of emotional intelligence (EI) and argued that El is conceptually distinct from personality. In Study 1, the authors showed that EI was related to yet distinct from personality dimensions and that it had incremental predictive power on life satisfaction. The authors examined the construct validity of self-reports and others' ratings of EI using two samples in Study 2. In a student sample, parents' ratings explained additional variance in the students' life satisfaction and feelings of powerlessness after controlling for the Big Five personality dimensions. In the work sample, peer ratings were found to be significant predictors of job performance ratings provided by supervisors after controlling for the Big Five personality dimensions. Other implications for future research on EI are discussed.     

Team effectiveness: beyond skills and cognitive ability.

On the basis of job analysis results, the validity of using measures of general cognitive ability, job-specific skills, and personality traits jointly at both the individual level and the group level to predict the performance of 79 four-person, human resource work teams was evaluated. Team member trait and job skill scores were aggregated with a conjunctive model of task performance. At the individual level of analysis, measures of personality (i.e., Agreeableness and Conscientiousness) predicted peer ratings of team member performance beyond measures of job-specific skills and general cognitive ability. Similarly, at the group level of analysis, both Agreeableness and Conscientiousness predicted supervisor ratings of work team performance, objective measures of work team accuracy, and work completed. At both the individual and group levels, the trait of Agreeableness predicted Interpersonal Skills.     

Construct Validity of Multi‐Source Performance Ratings: An Examination of the Relationship of Self‐, Supervisor‐, and Peer‐Ratings with Cognitive and Personality Measures

Although more and more organizations prefer using multi‐source performance ratings or 360° feedback over traditional performance appraisals, researchers have been rather skeptical regarding the reliability and validity of such ratings. The present study examined the validity of self‐, supervisor‐, and peer‐ratings of 195 employees in a Dutch public organization, using scores on an In‐Basket exercise, an intelligence test, and a personality questionnaire as external criterion measures. Interrater agreement ranged from .28 to .38. Variance in the ratings was explained by both method and content factors. Support for the external construct validity was rather weak. Supervisor‐ratings were not found to be superior to self‐ and peer‐ratings in predicting the scores on the external measures.     

A Meta-Analysis of Incremental Validity and Nomological Networks for Subordinate and Peer Rating

Use of subordinates and peers for job performance ratings has increased markedly in recent years, on the assumption that these sources cover different parts of the criterion space and provide incremental validity over and above supervisors. This study directly tested this assumption using a meta-analytically derived correlation matrix including supervisor-, subordinate-, and peer-ratings as well as objective performance measures. Hierarchical regression results showed that both subordinates and peers accounted for significant variance in objective measures, over and above other sources. Further meta-analyses investigated nomological networks of subordinate and peer ratings, specifically relations with personality and ability measures. Ability correlations for peers and especially subordinates were lower than those previously found for supervisors. Correlations for personality dimensions affiliation and agreeableness, although low, were somewhat higher than previous supervisor values. Results provide justification for the added expense of including subordinates and peers in a rating system.     

Sentence Completion Correlates of Airline Pilot Attitude and Proficiency

Sentence completion scores for dependency, anxiety, and hostility were correlated with sentence completion measures of good flying attitude and omissions for 300 male airline pilot candidates. The scoring-by-example method used with the first three variables was highly reliable and avoided scoring overlap. All sentence completion scores were also correlated with 15 traditional measures of pilot qualification such as mechanical information and signal reaction time. While the proficiency measures were generally unrelated to the projective measures, a good attitude towards flying was positively associated with coping (not omitting sentence completion items) and the extreme suppression of anxiety and hostility under stress. Civilian airline pilots appear similar to military pilots in their exaggerated display of good adjustment.     

Progress Toward Understanding the Structure and Determinants of Job Performance: A focus on task and citizenship performance

Job performance criterion models have become more popular and more often investigated. The current research adds to this literature by testing performance models focusing on ability and personality predictors of task and citizenship job performance. Air traffic controllers (N = 928) completed cognitive ability and personality predictor measures, were administered a performance test assessing two possible mediators, and were evaluated using supervisor and peer ratings of two task performance, one citizenship performance, and an overall performance dimension. Path analyses largely confirmed that ability predicts primarily task performance and personality predicts primarily citizenship performance. The mediation related to task knowledge and skill on the task performance side was confirmed, and the overall fit of the performance models was quite good. Discussion focuses on how the present research furthers progress toward mapping relations among individual differences and individual performance constructs, beyond using overall performance as the endogenous performance variable.     

On the relation between personality and job performance of airline pilots

The validity of a personality questionnaire for the prediction of job success of airline pilots is compared to validities of a simulator checkflight and of flying experience data. During selection, 274 pilots applying for employment with a European charter airline were examined with a multidimensional personality questionnaire (Temperature Structure Scales; TSS). Additionally, the applicants were graded in a simulator checkflight. On the basis of training records, the pilots were classified as performing at standard or below standard after about 3 years of employment in the hiring company. In a multiple-regression model, this dichotomous criterion for job success can be predicted with 73.8% accuracy through the simulator checkflight and flying experience prior to employment. By adding the personality questionnaire to the regression equation, the number of correct classifications increases to 79.3%. On average, successful pilots score substantially higher on interpersonal scales and lower on emotional scales of the TSS.     

The Impact of Contextual Self‐Ratings and Observer Ratings of Personality on the Personality–Performance Relationship1

This study examined 2 possible ways of increasing the predictive validity of personality measures: using observer (i.e., supervisor and coworker) ratings and work‐specific self‐ratings of Big Five personality factors. Results indicated that among general self‐ratings of Big Five personality dimensions, Conscientiousness was the best predictor of in‐role performance, and Agreeableness and Emotional Stability were the best predictors of organizational citizenship behavior (OCB). Observer ratings of personality accounted for incremental variance in job performance (in‐role performance and OCB) beyond that accounted for by general self‐ratings. However, contrary to our expectations, work‐specific (i.e., contextual) self‐ratings of personality generally did not account for incremental variance in job performance beyond that accounted for by general self‐ratings.     

SELF-MONITORING AS A MODERATOR OF THE RELATIONSHIPS BETWEEN PERSONALITY TRAITS AND PERFORMANCE

This study examined the hypothesis that self-monitoring moderates the relationship between Big Five personality traits and interpersonal performance. The findings from a sample of 102 employed Executive MBA students reveal that when self-monitoring was high the relationships between 3 of the Big Five personality traits (Extraversion, Emotional Stability, and Openness to Experience) and supervisory ratings of interpersonal performance were attenuated. These effects were replicated using peer ratings of interpersonal performance for Extraversion and Emotional Stability but not for Openness to Experience. Further, as expected, self-monitoring did not moderate the relationships between personality traits and supervisory or peer ratings of task performance. Implications for future research in the area of personality and other motivational theories are discussed.     

Examination of the Discriminant,Convergent, and Criterion‐Related Validity of Self‐Ratings on the Emotional Competence Inventory

We found that self‐ratings on the Emotional Competence Inventory (ECI) had small relationships with, but a distinct factor structure from, the Big‐Five personality dimensions. ECI self‐ratings were unrelated to academic performance and general mental ability. ECI self‐ratings had significant, albeit small, correlations with EC‐related behaviors and peer nominations of influence during a leaderless group discussion, coworkers' ratings of managerial skills, and number of promotions received. However, with one exception, these significant relationships disappeared after controlling for personality and age.     

Concurrent and longitudinal correlates of preschool peer sociometrics: Comparing rating scale and nomination measures

Nomination and rating scale measures of preschool sociometric status were compared with respect to their patterns of concurrent and longitudinal developmental correlates. The study was undertaken to help fill a void in the empirical literature on young children's peer adjustment. Subjects were 79 4–5 year-old children, currently enrolled in preschool classes. In addition to the sociometric interviews, diverse measures of children's social and cognitive competence were administered concurrently, and longitudinally one year later. These measures included teacher ratings of peer acceptance and behavior problems, and performance measures of social problem-solving ability, impulse control, and vocabulary competence. As expected, the reliability of the rating scale technique was superior to that of the nomination measures. Furthermore, all three sociometric measures had modest but meaningful patterns of concurrent and longitudinal correlates. However, the negative nomination measure was distinguished from the others by its consistent association with measures of impulsivity, and its predictive link with aggressive social problem solving. Therefore, negative peer nomination measures supply unique information about children's social functioning that should be represented in studies of children at risk for social maladjustment.     

Are observer ratings of applicants’ personality also faked? Yes,but less than self‐reports

Although self‐report personality tests are a comparatively cheap and easy‐to‐administer personnel selection tool, researchers have criticized them for not predicting enough criterion‐related variance. Researchers have suggested using observer‐ratings of personality (e.g., as part of a reference check from a supervisor) because observer‐ratings have been reported to be more predictive. However, it is theoretically and empirically unclear whether supervisors also engage in faking (the intentional distortion of responses). Study 1 explored faking among managers who were first asked to imagine that a subordinate had to leave his/her job for private reasons and then to rate the personality of the subordinate. A week later, managers rated their subordinates honestly. A repeated‐measures MANOVA indicated that managers did fake. Study 2 (among supervisors of working students) replicated the above findings but also showed that there is less faking in supervisor‐ratings than in self‐ratings. Furthermore, we found no evidence that the validity of personality scales for predicting academic performance depends on self‐ versus observer‐ratings or on an applicant versus an honest condition. These two studies thus show that practitioners should not equate personality ratings obtained from observers in a selection context with honest personality ratings.     

On Being the Same in Different Places: Evaluating Frame‐of‐Reference Effects Across Two Social Contexts

Frame‐of‐reference (FOR) effects in personality assessment are demonstrated when self‐rated items oriented to specific contexts (e.g., workplace) show better predictive validity than noncontextualized items. Empirical support of FOR effects typically relies on job performance ratings or academic grades for criteria. The current study evaluates FOR effects using ratings of personality provided by informants from the home or school context. Items from the NEO Five‐Factor Inventory (NEO‐FFI; Costa & McCrae, 1992) were contextualized to the home and school contexts to create NEO‐Home and NEO‐School versions. One hundred fifty‐eight college students completed the NEO‐Home and NEO‐School questionnaires, and 161 college students completed the standard, noncontextualized NEO‐FFI. All participants recruited one peer from college and at least one parent to complete standard rater versions of the NEO‐FFI. Contextualized self‐ratings did not show FOR effects. NEO‐Home self‐ratings did not correlate higher with parent ratings than with peer ratings, and NEO‐School self‐ratings did not correlate higher with peer ratings than with parent ratings. Standard NEO‐FFI self‐ratings generally showed higher self‐informant agreement with both types of informants than contextualized self‐ratings. The pattern of correlations suggests that validity is enhanced more by specific trait‐informant combinations than by the contextualization of items to social contexts.     

The predictive power of first-grade peer and teacher ratings of behavior: Sex differences in antisocial behavior and personality at adolescence

Peer and teacher assessments of aggression, social withdrawal, and likability using the Pupil Evaluation Inventory were recorded for 104 French Canadian girls and boys in grade 1. Self-reported delinquency and personality measures were administered to these children when they were in junior high school 7 years later. Linear regression analyses revealed significant predictive differences between the grade 1 assessment of girls and boys: For antisocial behavior, teacher and peer assessments of boys were equally good predictors, and the combination of the two assessments did not improve prediction; for girls, peer and teacher assessments taken separately were weak predictors, but, taken together, they were better predictors for girls than for boys; for personality, peers were better predictors than teachers both for girls and for boys. When categorical analyses were used to predict extreme antisocial behavior, peer and teacher assessments were equally good predictors for girls and boys. The use of peer and teacher ratings together, however, decreased the number of false positives. The implications of these findings for research and clinical work are discussed.     

Resilients,Overcontrollers, and Undercontrollers: The replicability of the three personality prototypes across informants

The study of patterns in personality structure reveals three replicable prototypes: Resilients, Overcontrollers, and Undercontrollers. The three prototypes were first identified in children using ratings based on the California Child Q‐set (see Block, 1971). Only recently, the three prototypes were replicated in self‐reports on questionnaires intended to assess the Big Five (see e.g. Asendorpf, Borkenau, Ostendorf, & van Aken, 2001). This paper addresses the question of whether the three prototypes are replicable across different data sources. Cluster structures in self‐, peer, and behaviour ratings, all based on the Big Five, were examined in a sample of 600 monozygotic and dizygotic twins ranging in age from 18 to 70 years. The three prototypes could be clearly identified in the self‐reports only, whereas in ratings by others only the Resilient prototype could be replicated. In both peer and behaviour ratings, the second and the third cluster reflected a Non‐desirable and an Average type. The analysis of cross‐data consistency revealed only moderate agreement in assignments of individual subjects to types. The findings suggest that personality types depend strongly on personality measures and informants. Copyright © 2003 John Wiley & Sons, Ltd.     

A New Old Question: Do contextual factors relate to rating behavior: An investigation with peer evaluations

The personality (dispositional) characteristic of self‐monitoring, beliefs and orientations toward the performance appraisal of 62 peer raters were collected in an attempt to predict peer rating behavior (i.e., mean ratings, rater discrimination, and performance appraisal dimensions discrimination). The beliefs and orientation toward appraisal systems emerged moderately related to rating behavior measures. Specific variables included in the category termed rating context revealed further illuminating effects. Furthermore, the moderating effect of self‐monitoring on the relationship rating the context‐rating behavior of peers was solidly supported. Theoretical interpretations of the results were discussed.     

Beyond cognitive ability: Improving the prediction of performance

New measures of practical intelligence and personality traits were assessed for their ability to improve the prediction of performance beyond cognitive ability. Measures of practical intelligence from the newly developed Constructive Thinking Inventory and traits measuring emotional stability and expedience were used as independent variables in regression analyses predicting academic performance, training performance and leadership ratings of 86 student leaders. The constructive thinking scales added significantly to the prediction of academic performance, beyond measures of cognitive ability. Constructive thinking also accounted for unique variance predicting training performance and leadership ratings. Personality traits were unrelated to performance or leadership.     

Predictive Criterion-Related Validity of Observer Ratings of Personality and Job-Related Competencies Using Multiple Raters and Multiple Performance Criteria

The purpose of this study was to assess the predictive validity of observer ratings of personality and job-related competencies in a selection setting. Based on ratings from multiple raters of both the predictors and the criteria in a sample of MBA students, results indicated that observer ratings of Conscientiousness, Emotional Stability, leadership, and interpersonal skills predicted work performance, team performance, and academic performance. For work performance and team performance, a composite of the four predictors had incremental predictive validity over general mental ability, even after controlling for how well the rater knew the ratee.