DOES PERFORMANCE IMPROVE FOLLOWING MULTISOURCE FEEDBACK? A THEORETICAL MODEL,META‐ANALYSIS,AND REVIEW OF EMPIRICAL FINDINGS

We review evidence showing that multisource feedback ratings are related to other measures of leadership effectiveness and that different rater sources conceptualize performance in a similar manner. We then describe a meta-analysis of 24 longitudinal studies showing that improvement in direct report, peer, and supervisor ratings over time is generally small. We present a theoretical framework and review empirical evidence suggesting performance improvement should be more likely for some feedback recipients than others. Specifically, improvement is most likely to occur when feedback indicates that change is necessary, recipients have a positive feedback orientation, perceive a need to change their behavior, react positively to the feedback, believe change is feasible, set appropriate goals to regulate their behavior, and take actions that lead to skill and performance improvement.     

Multisource Feedback and Leaders' Goal Performance: Moderating effects of rating purpose,rater perspective,and performance dimension

The main purposes of this study are to examine whether multisource feedback ratings predict leaders' organizational goal performance, and whether the relationships are consistent across the two rating purposes (developmental, administrative), two leadership dimensions behaviors (Consideration, Initiating Structure), and three rating perspectives (supervisor, self, and ‘other’ raters, i.e., peers and subordinates). Leaders (n=396) in a large organization in the transportation industry participated in two multisource feedback programs, the first for developmental purposes and the second 8 months later for administrative purposes. Approximately 1 month later, they were rated by their supervisor on their effectiveness in attaining five organizational performance goals (financial, safety, customer satisfaction, employee satisfaction, diversity). Results revealed that both developmental ratings and administrative ratings uniquely predicted leaders' goal performance. However, both leadership dimension and rater perspective moderated these relationships. Leadership behaviors associated with Consideration were stronger predictors of goal performance for supervisor ratings, whereas behaviors associated with Initiating Structure were stronger predictors of goal performance for self and other ratings.     

The predictive validity of individual psychological assessments in selecting UK public sector senior managers

Individual Psychological Assessment (IPA) is a very widely offered service in Organizational Psychology. It generally consists of a psychologist or HR practitioner using a combination of interview and psychometrics to arrive at a detailed assessment of an individual's capabilities in relation to a job they are being considered for. Although much used, this practice has limited supporting evidence of its validity—not least because of the methodological difficulties in conducting research on this subject—and has been criticized accordingly. The current study examines the use of IPAs with 115 middle and senior management level candidates in a civil service context. All candidates completed a set of psychometric measures and had an in‐depth interview with a psychologist as part of a standardized process. The ratings made by the assessors were correlated with a criterion measure of potential for promotion derived from multisource feedback ratings obtained on these candidates some months later. Analysis of the results indicated that three of the four attributes rated by assessors correlated significantly with the criterion measure. Further, assessors’ ratings were found to show incremental validity over that provided by psychometric test scores alone. These findings are discussed in terms of the use of IPAs in senior level assessment.     

CAN WORKING WITH AN EXECUTIVE COACH IMPROVE MULTISOURCE FEEDBACK RATINGS OVER TIME? A QUASI‐EXPERIMENTAL FIELD STUDY

This study examined the effects of executive coaching on multisource feedback over time. Participants were 1,361 senior managers who received multisource feedback; 404 of these senior managers worked with an executive coach (EC) to review their feedback and set goals. One year later, 1,202 senior managers (88% of the original sample) received multisource feedback from another survey. Managers who worked with an EC were more likely than other managers to set specific (rather than vague) goals ( d = .16) and to solicit ideas for improvement from their supervisors ( d = .36). Managers who worked with an EC improved more than other managers in terms of direct report and supervisor ratings, however, the effect size ( d = .17) was small.     

Construct Validity of Multi‐Source Performance Ratings: An Examination of the Relationship of Self‐, Supervisor‐, and Peer‐Ratings with Cognitive and Personality Measures

Although more and more organizations prefer using multi‐source performance ratings or 360° feedback over traditional performance appraisals, researchers have been rather skeptical regarding the reliability and validity of such ratings. The present study examined the validity of self‐, supervisor‐, and peer‐ratings of 195 employees in a Dutch public organization, using scores on an In‐Basket exercise, an intelligence test, and a personality questionnaire as external criterion measures. Interrater agreement ranged from .28 to .38. Variance in the ratings was explained by both method and content factors. Support for the external construct validity was rather weak. Supervisor‐ratings were not found to be superior to self‐ and peer‐ratings in predicting the scores on the external measures.     

Does interview anxiety predict job performance and does it influence the predictive validity of interviews?

Interview anxiety is negatively related to interview performance; however, its relation to job performance is unknown. It could correlate negatively with job performance, and it could moderate the interview performance–job performance relation. In a sample of applicants for Residence Assistant positions, interview anxiety had near‐zero correlations with job performance, rated by supervisors and supervisees. It moderated the relation between interview performance and supervisor‐rated facilitating peer and team performance, such that interview performance did not predict this job performance component for anxious applicants. The moderation was not found for supervisor‐rated task proficiency, or for supervisee ratings of either job performance component, suggesting that the impact of interview anxiety depends on rater source and which job performance component is rated.     

Predicting Transformational Leadership in Naval Cadets: Effects of Personality Hardiness and Training

The present study investigated whether personality hardiness predicts peer ratings of leadership style in Navy officer cadets. Cadets ( n  = 71) completed the Dispositional Resiliency (hardiness) Scale, and later rated their peers using the Multifactor Leadership Questionnaire (full range of leadership model). Results show the hardy–commitment facet predicted peer ratings of all leadership styles covered in the model, both before and after and intensive military exercise. The challenge facet was a positive predictor of transformational and transactional leadership and was negatively related to passive-avoidant leadership. Transformational leadership predicted leader performance, as indexed by military development grades; and a partial mediation effect was found for hardiness on the relation between transformational leadership and leader performance.     

Measurement Equivalence and Multisource Ratings for Non-managerial Positions: Recommendations for Research and Practice

ABSTRACT: The present investigation applies a comprehensive sequence of confirmatory factor analysis tests (Vandenberg and Lance, Organisational Research Methods, 3, 4–69, 2000) to the examination of the measurement equivalence of self, peer, and supervisor ratings of non-managerial targets across several performance dimensions. Results indicate a high degree of measurement equivalence across rater sources and performance dimensions. The paper illustrates how this procedure can identify very specific areas of non-equivalence and how the complexity of a multisource feedback system may be represented using such procedures. Implications of these results and recommendations for both research and practice are offered.     

Can a manager have a life and a career? International and multisource perspectives on work-life balance and career advancement potential

The present study was the first cross-national examination of whether managers who were perceived to be high in work-life balance were expected to be more or less likely to advance in their careers than were less balanced, more work-focused managers. Using self ratings, peer ratings, and supervisor ratings of 9,627 managers in 33 countries, the authors examined within-source and multisource relationships with multilevel analyses. The authors generally found that managers who were rated higher in work-life balance were rated higher in career advancement potential than were managers who were rated lower in work-life balance. However, national gender egalitarianism, measured with Project GLOBE scores, moderated relationships based on supervisor and self ratings, with stronger positive relationships in low egalitarian cultures. The authors also found 3-way interactions of work-life balance ratings, ratee gender, and gender egalitarianism in multisource analyses in which self balance ratings predicted supervisor and peer ratings of advancement potential. Work-life balance ratings were positively related to advancement potential ratings for women in high egalitarian cultures and men in low gender egalitarian cultures, but relationships were nonsignificant for men in high egalitarian cultures and women in low egalitarian cultures.     

Self‐Enhancement in Military Leaders: Its Relevance to Officer Selection and Performance

We report on two studies in which we measured army cadets' tendencies to engage in two types of self‐enhancement: communal self‐enhancement (a moralistic bias in self‐presentation) and agentic self‐enhancement (an egotistic bias in self‐presentation). These self‐presentation styles were then related to their selection and performance as military leaders. In Study 1, scores on self‐enhancement questionnaires were used to predict selection decisions for 206 applicants to an army officer training program. We found that applicants who were higher in either communal or agentic self‐enhancement were more likely to be accepted for leadership training. In Study 2, we evaluated peer and superior ratings of 94 military cadets' leadership, reflecting leadership emergence and leadership effectiveness, respectively. We found that communal self‐enhancement negatively predicted leadership emergence, with those ratings becoming more negative over a 3‐year time‐span, whereas agentic self‐enhancement positively predicted leadership effectiveness. Our results imply that, at least in the present military context, people making selection decisions should be particularly aware of the relations between (a) applicant self‐enhancement tendencies and those decisions, and (b) high communal self‐enhancement in officer trainees and negative evaluations by their cadet peers.     

Narcissism and Leadership: A Meta‐Analytic Review of Linear and Nonlinear Relationships

Past empirical studies relating narcissism to leadership have offered mixed results. This study integrates prior research findings via meta‐analysis to make 4 contributions to theory on narcissism and leadership, by (a) distinguishing between leadership emergence and leadership effectiveness, to reveal that narcissism displays a positive relationship with leadership emergence, but no relationship with leadership effectiveness; (b) showing narcissism's positive effect on leadership emergence can be explained by leader extraversion; (c) demonstrating that whereas observer‐reported leadership effectiveness ratings (e.g., supervisor‐report, subordinate‐report, and peer‐report) are not related to narcissism, self‐reported leadership effectiveness ratings are positively related to narcissism; and (d) illustrating that the nil linear relationship between narcissism and leadership effectiveness masks an underlying curvilinear trend, advancing the idea that there exists an optimal, midrange level of leader narcissism.     

Self‐ratings in Training Programs: An Examination of Level of Performance and the Effects of Feedback

The level of agreement (mean differences and correlations) between self, peer and training staff ratings were examined in this study. The sample consisted of 545 participants who were undertaking a Royal Australian Airforce officer training program. Consistent with previous research there was strong agreement between training staff and peers and weak agreement between self‐ratings and ratings by others (training staff and peers). Accuracy of ratings was examined by (a) comparing the mean ratings of outstanding, average and below‐average performers; and (b) correlating difference scores with a measure of performance. The findings showed that below‐average performers have a less accurate view of themselves compared to outstanding performers. Finally, we examined the effects of negative feedback on self‐perceptions. The analyses indicated that after receiving negative feedback, average performers adjusted their self‐ratings. Various explanations were proposed together with practical implications for training.     

RATER SOURCE EFFECTS ARE ALIVE AND WELL AFTER ALL

Recent research has questioned the importance of rater perspective effects on multisource performance ratings (MSPRs). Although making a valuable contribution, we hypothesize that this research has obscured evidence for systematic rater source effects as a result of misspecified models of the structure of multisource performance ratings and inappropriate analytic methods. Accordingly, this study provides a reexamination of the impact of rater source on multisource performance ratings by presenting a set of confirmatory factor analyses of two large samples of multisource performance rating data in which source effects are modeled in the form of second-order factors. Hierarchical confirmatory factor analysis of both samples revealed that the structure of multisource performance ratings can be characterized by general performance, dimensional performance, idiosyncratic rater, and source factors, and that source factors explain (much) more variance in multisource performance ratings whereas general performance explains (much) less variance than was previously believed. These results reinforce the value of collecting performance data from raters occupying different organizational levels and have important implications for research and practice.     

DEVELOPMENT ENGAGEMENT WITHIN AND FOLLOWING DEVELOPMENTAL ASSESSMENT CENTERS: CONSIDERING FEEDBACK FAVORABILITY AND SELF–ASSESSOR AGREEMENT

This study sought to understand employees' level of behavioral engagement in response to feedback received in developmental assessment center (DAC) programs. Hypotheses were drawn from theories of self‐enhancement and self‐consistency and from findings in the multisource feedback and assessment center literatures regarding recipients' perceptions of feedback. Data were gathered from 172 U.S. middle managers participating in a DAC program. Results suggested that more favorable feedback was related to higher behavioral engagement. When discrepancies between self‐ and assessor ratings were examined, overraters (participants whose overall self‐ratings were higher than their assessor ratings) tended to show less engagement in the program compared to underraters. However, pattern agreement on the participant's dimension profile did not significantly correlate with behavioral engagement. Based on these findings, avenues for future research are presented and practical implications are discussed.     

An Empirical Review of the Employment Interview Construct Literature

The purpose of this paper is to summarize available empirical research on the association between employment interview ratings and various construct measures using a theoretical model developed as part of the study. The model posits 3 main sources of construct‐related variance in interview ratings: job‐related interview content (e.g., job knowledge), interviewee performance (e.g., impression management tactics), and personal/demographic characteristics (e.g., candidate attractiveness). Results suggest some potentially important findings, including that the mean correlation with interview ratings is twice as large for constructs related to interviewee performance as it is for constructs pertaining to job‐related interview content. Directions for future construct research are also identified. For instance, despite being central to the interpersonal make‐up of the candidates, research regarding the influence of interests, goals, and values on interview ratings is almost nonexistent.     

Self–Other Agreement: Comparing its Relationship with Performance in the U.S. and Europe

The use of multi‐source feedback has proliferated in the United States in recent years; however, its usefulness in other countries is unknown. Using a large sample of American managers (n=3793), this study first replicated earlier studies demonstrating that simultaneous consideration of self and other ratings of leadership skills is important for managerial performance ratings. In addition, the impact of self–other agreement on performance was investigated among 2732 managers in five European countries (U.K., Germany, France, Denmark, Italy). Results indicated that the effect of self and other ratings in the prediction of performance differs between the U.S. and the European countries in that the simultaneous inclusion of both self and other ratings is generally less useful in those countries than in the U.S. Further, the effect of self–other agreement varies among the European countries. Implications for multi‐source feedback interventions as well as multi‐national personnel management are discussed.     

同伴团体对儿童青少年学业成就和社会功能关系的影响

运用同伴评价、教师评估和学校记录等多种渠道收集关于同伴团体、社会功能、同伴接纳以及学业成就的资料,旨在考察同伴团体与儿童青少年学业成就和社会功能的关系。研究发现：绝大多数儿童青少年都有自己的同伴团体。这些同伴团体在学业成就上有很高的同质性。学业成就与社会适应无论在团体内的个体水平还是在团体水平上都密切相关。而且,团体的学业表现是个体学业成就和社会适应间的重要调节因素,表明团体学业规范对个体水平上学业成就与社会功能的关系有增强或是减弱的作用。这项研究再一次证明同伴团体在儿童个体发展中的重要意义。     

Rating leniency and halo in multisource feedback ratings: testing cultural assumptions of power distance and individualism-collectivism

This study extends multisource feedback research by assessing the effects of rater source and raters' cultural value orientations on rating bias (leniency and halo). Using a motivational perspective of performance appraisal, the authors posit that subordinate raters followed by peers will exhibit more rating bias than superiors. More important, given that multisource feedback systems were premised on low power distance and individualistic cultural assumptions, the authors expect raters' power distance and individualism-collectivism orientations to moderate the effects of rater source on rating bias. Hierarchical linear modeling on data collected from 1,447 superiors, peers, and subordinates who provided developmental feedback to 172 military officers show that (a) subordinates exhibit the most rating leniency, followed by peers and superiors; (b) subordinates demonstrate more halo than superiors and peers, whereas superiors and peers do not differ; (c) the effects of power distance on leniency and halo are strongest for subordinates than for peers and superiors; (d) the effects of collectivism on leniency were stronger for subordinates and peers than for superiors; effects on halo were stronger for subordinates than superiors, but these effects did not differ for subordinates and peers. The present findings highlight the role of raters' cultural values in multisource feedback ratings.     

The curvilinear effect of benevolent leadership on team performance: The mediating role of team action processes and the moderating role of team commitment

Leaders are encouraged to show benevolence to followers in paternalistic cultures. Yet, there remains debate about whether the influence of increasingly benevolent leadership on follower outcomes is linearly favorable. Grounded in the too‐much‐of‐a‐good‐thing effect and resource allocation theory, we developed and tested a model considering a potential curvilinear relationship between benevolent leadership and team performance while also examining the mediating role of team action processes. We further reasoned that this curvilinear indirect effect would be moderated by team commitment, which could neutralize the diminishing performance returns resulting from excessive benevolent leadership. To test these ideas, we carried out two studies. In the first study, multisource and time‐lagged data collected from 381 employees working in 104 research and development teams showed that benevolent leadership exhibited an inverted U‐shaped relationship with team performance, but this curvilinear relationship disappeared in teams with high team commitment. In the second study, we replicated and extended our results using a sample of 417 employees from 101 hotel management teams of a large hotel chain. Specifically, we found an inverted U‐shaped relationship between benevolent leadership and team action processes, which mediated the inverted U‐shaped relationship between benevolent leadership and team performance. Moreover, this indirect curvilinear effect only held in teams with low team commitment. We discuss the implications of our findings for both theory and practice.     

Self–Observer Rating Discrepancies of Managers in Asia: A study of derailment characteristics and behaviors in Southern and Confucian Asia

Antecedents to self–observer rating discrepancies in multisource instruments have been established at the individual and organizational level. However, research examining cultural antecedents is limited, which is particularly relevant as multisource instruments gain popularity around the world. We investigated multisource ratings of 860 Asian managers from the regions of Southern Asia (n=261) and Confucian Asia (n=599) and analyzed cultural differences in self–observer rating discrepancies. Multivariate regression procedures revealed that the self–observer rating discrepancy was wider for managers from Southern Asia as compared with Confucian Asia. The reason for the discrepancy was driven by managers' self‐ratings being different across cultures than by observer ratings from managers' bosses, direct reports, or peers; the predictor is related to self‐ratings not observer ratings, producing differential self–observer ratings due to self‐ratings. We discuss cultural differences in self‐ and observer ratings within Asia and provide implications for the practice of multisource assessments.