Multisource Assessments of Behavioral Competencies and Selection Interview Performance

This study sought to determine whether and how ratings of leadership in a developmental multisource feedback program compare against those obtained by leadership selection decision makers. Seventy‐seven senior managers received feedback on eight competencies that also formed the basis of a subsequent promotional interview for senior executive positions. Each rating source displayed a unique perspective on the participants, but only the supervisor and peer ratings predicted performance on the interview. As well, peer, subordinate, and self‐ratings failed to improve upon the supervisors' predictions. The results also suggested a general performance factor underlying both the multisource assessments and the interview ratings.     

The forgotten ones? The validity of consideration and initiating structure in leadership research

This study provided a meta-analysis of the relationship of the Ohio State leadership behaviors--Consideration and Initiating Structure--with leadership. Overall, 163 independent correlations for Consideration and 159 correlations for Initiating Structure were analyzed. Results revealed that both Consideration (.48) and Initiating Structure (.29) have moderately strong, nonzero relations with leadership outcomes. Consideration was more strongly related to follower satisfaction (leader satisfaction, job satisfaction), motivation, and leader effectiveness, and Initiating Structure was slightly more strongly related to leader job performance and group-organization performance. Validities did vary by leadership measure, but in most cases validities generalized regardless of the measure used. Overall, the results provide important support for the validity of Initiating Structure and Consideration in leadership research.     

Ambidextrous Leadership and Employees' Self‐Reported Innovative Performance: The Role of Exploration and Exploitation Behaviors

The ambidexterity theory of leadership for innovation proposes that leaders' opening and closing behaviors positively predict employees' exploration and exploitation behaviors, respectively. The interaction of exploration and exploitation behaviors, in turn, is assumed to influence employee innovative performance, such that innovative performance is highest when both exploration and exploitation behaviors are high. The goal of this study was to provide the first empirical test of these hypotheses at the individual employee level. Results based on self‐report data provided by 388 employees were consistent with ambidexterity theory, even after controlling for employee reports of their leaders' transformational and transactional leadership behaviors as well as employees' openness to experience, conscientiousness, and positive affect. The findings extend previous research on ambidexterity at the team and organizational levels and suggest a possible way for leaders to enhance employee self‐reported innovative performance.     

UNDERSTANDING RESPONSES TO MULTI-SOURCE FEEDBACK: THE ROLE OF CORE SELF-EVALUATIONS

This longitudinal study examines the role of core self-evaluations in responses to multisource feedback. Consistent with past research, feedback recipients were most satisfied when others' ratings were high. Core self-evaluations was not related to satisfaction with feedback, but was positively linked to goal commitment 4 months after feedback. Polynomial regression results reveal a complex pattern of associations among core self-evaluations, ratings, satisfaction, and commitment to developmental goals. Specifically, individuals with high core self-evaluations were most committed to developmental goals when self and others' ratings were discrepant. In contrast, individuals with low core self-evaluations were most committed to developmental goals when self and others' ratings were in agreement. Our results question the notion that satisfaction with feedback affects motivation to improve and highlight the importance of personality in motivation to improve following feedback. We discuss practical and theoretical implications of our findings and provide a theoretical framework for future research.     

The moderating influence of job performance dimensions on convergence of supervisory and peer ratings of job performance: unconfounding construct-level convergence and rating difficulty

Meta-analysis was used to cumulate the correlations between supervisor and peer ratings for different dimensions of job performance. The general pattern in the data suggested that raters from the same organizational level disagree as much as raters from different levels. Methods were used to separate the effects of rating difficulty and lack of construct-level convergence on the correlation between supervisor and peer ratings. The authors found complete construct-level convergence for ratings of overall job performance, productivity, effort, job knowledge, quality, and leadership but not for ratings of administrative competence, interpersonal competence, and compliance or acceptance of authority. Higher rating difficulty was more strongly associated with lower mean observed peer-supervisor correlations than were construct-level disagreements between peers and supervisors. Implications for research and practice are discussed.     

An exploration of women's expectations of effective male and female leadership

University women (N=188) of varying ages (19–53) and work experience described their behavioral expectations of an effective male and female leader for levels of Consideration, Initiating Structure, Role Assumption, and Production Emphasis. In line with a sex role congruency view, it was hypothesized that an effective female leader would be expected to exhibit higher levels of Consideration behaviors, while an effective male leader would be expected to exhibit more Initiating Structure, Role Assumption, and Production Emphasis behaviors. The results provided only partial support for the sex role congruency hypothesis since an effective female leader was expected to exhibit higher levels of Consideration as well as higher levels of Initiating Structure. No sex differences were detected for Role Assumption or Production Emphasis behaviors. Further, while many similarities were noted among the women in their views of effective leadership, there were also systematic age-related differences among the women in their expectations, particularly with regard to their views of effective female leadership.Portions of this paper were presented at the annual meeting of the Southeastern Psychological Association, Orlando, Florida, March, 1986.     

Self–Observer Rating Discrepancies of Managers in Asia: A study of derailment characteristics and behaviors in Southern and Confucian Asia

Antecedents to self–observer rating discrepancies in multisource instruments have been established at the individual and organizational level. However, research examining cultural antecedents is limited, which is particularly relevant as multisource instruments gain popularity around the world. We investigated multisource ratings of 860 Asian managers from the regions of Southern Asia (n=261) and Confucian Asia (n=599) and analyzed cultural differences in self–observer rating discrepancies. Multivariate regression procedures revealed that the self–observer rating discrepancy was wider for managers from Southern Asia as compared with Confucian Asia. The reason for the discrepancy was driven by managers' self‐ratings being different across cultures than by observer ratings from managers' bosses, direct reports, or peers; the predictor is related to self‐ratings not observer ratings, producing differential self–observer ratings due to self‐ratings. We discuss cultural differences in self‐ and observer ratings within Asia and provide implications for the practice of multisource assessments.     

Rating leniency and halo in multisource feedback ratings: testing cultural assumptions of power distance and individualism-collectivism

This study extends multisource feedback research by assessing the effects of rater source and raters' cultural value orientations on rating bias (leniency and halo). Using a motivational perspective of performance appraisal, the authors posit that subordinate raters followed by peers will exhibit more rating bias than superiors. More important, given that multisource feedback systems were premised on low power distance and individualistic cultural assumptions, the authors expect raters' power distance and individualism-collectivism orientations to moderate the effects of rater source on rating bias. Hierarchical linear modeling on data collected from 1,447 superiors, peers, and subordinates who provided developmental feedback to 172 military officers show that (a) subordinates exhibit the most rating leniency, followed by peers and superiors; (b) subordinates demonstrate more halo than superiors and peers, whereas superiors and peers do not differ; (c) the effects of power distance on leniency and halo are strongest for subordinates than for peers and superiors; (d) the effects of collectivism on leniency were stronger for subordinates and peers than for superiors; effects on halo were stronger for subordinates than superiors, but these effects did not differ for subordinates and peers. The present findings highlight the role of raters' cultural values in multisource feedback ratings.     

Self‐Supervisor Agreement: The Influence of Feedback Seeking on the Relationship Between Self and Supervisor Ratings of Performance1

Self‐assessment research has continued to search for those factors that increase self‐other rating agreement. The current field study investigated the feedback‐seeking strategies (i. e., monitoring and inquiry) used by 125 employees to obtain performance information, as well as the relationship between feedback‐seeking strategy use and self‐supervisor performance‐rating agreement. Results indicate that the frequency of monitoring reported by employees significantly moderated the relationship between self and supervisor ratings of performance. Individuals who reported higher levels of feedback seeking through monitoring were more likely to have self‐assessments that were congruent with their supervisors' ratings of performance.     

360 Degree Feedback and Developmental Outcomes: The Role of Feedback Characteristics,Self‐Efficacy and Importance of Feedback Dimensions to Focal Managers' Current Role

This longitudinal study investigates whether developmental changes following 360 degree feedback are predicted by the favourability of ratings received, and moderated by focal individuals' self‐efficacy and perceived importance of feedback. Five developmental criteria are investigated longitudinally: (i) self‐assessments, (ii) line managers' ratings, (iii) amount of developmental activity, (iv) global self‐efficacy and (iv) self‐efficacy for development. Feedback ratings from certain rater groups predicted changes in ratings, but not changes in self‐efficacy or amount of developmental activity. Self‐efficacy significantly moderated the feedback–performance association for certain rater groups, but feedback importance did not. Contrary to expectations, the focal individual's initial self‐assessment predicted changes in self‐efficacy, over the favourability of ratings received. The implications of these findings for organizations using 360 degree feedback for developmental purposes are discussed.     

The use of snowball sampling for multi source organizational research: Some cause for concern

In snowball sampling for multisource studies, researchers ask target participants to recruit informants. Despite its widespread use, especially for recruiting informants for multisource research, virtually no published research has addressed possible biases snowball sampling may cause in findings of this type of research. Such potential biases were tested empirically in a multisource study with a sample of 1,058 employed students asked to collect online ratings from their supervisors and coworkers. Informant ratings were obtained for 358 target participants. Objective indicators were employed to identify informant ratings suspicious of being fabricated. Results indicated that target participants who report (a) better relationship quality with informants, (b) fewer organizational constraints, and (c) more favorable self‐evaluations on behaviors to be rated by informants were more likely to be included in multisource data unsuspicious of being fabricated. Inclusion of informant ratings suspicious of being fabricated led to inflated estimates of self‐other consensus and of variance accounted for in criterion measures, to deflated informant rating means, but also to a target sample less restricted in terms of relevant organizational variables. In sum, the present findings suggest that potential biases should be identified in future uses of snowball sampling. Some practical recommendations toward that end are derived.     

Self‐Engagement Moderates the Mediated Relationship Between Organizational Constraints and Organizational Citizenship Behaviors Via Rated Leadership

We examined self‐engagement in job performance in a moderated mediation model where engagement moderated the relationship between organizational constraints and ratings of leadership effectiveness, and ratings of leadership effectiveness mediated the relationship between organizational constraints and organizational citizenship behaviors (OCBs). University employees representing diverse occupations completed measures of self‐engagement, organizational constraints, and leader effectiveness. Supervisors provided ratings of OCB. Tests of mediated moderation using random coefficient modeling revealed that leadership mediated the relationship between constraints and OCB for highly engaged employees. Results are discussed in terms of highly engaged workers being attuned to stressors in the work environment that may compromise performance, potentially blaming leaders for the presence of these obstacles, thus causing withdrawal of effort on nonessential performance tasks.     

DOES PERFORMANCE IMPROVE FOLLOWING MULTISOURCE FEEDBACK? A THEORETICAL MODEL,META‐ANALYSIS,AND REVIEW OF EMPIRICAL FINDINGS

We review evidence showing that multisource feedback ratings are related to other measures of leadership effectiveness and that different rater sources conceptualize performance in a similar manner. We then describe a meta-analysis of 24 longitudinal studies showing that improvement in direct report, peer, and supervisor ratings over time is generally small. We present a theoretical framework and review empirical evidence suggesting performance improvement should be more likely for some feedback recipients than others. Specifically, improvement is most likely to occur when feedback indicates that change is necessary, recipients have a positive feedback orientation, perceive a need to change their behavior, react positively to the feedback, believe change is feasible, set appropriate goals to regulate their behavior, and take actions that lead to skill and performance improvement.     

The Impact of Contextual Self‐Ratings and Observer Ratings of Personality on the Personality–Performance Relationship1

This study examined 2 possible ways of increasing the predictive validity of personality measures: using observer (i.e., supervisor and coworker) ratings and work‐specific self‐ratings of Big Five personality factors. Results indicated that among general self‐ratings of Big Five personality dimensions, Conscientiousness was the best predictor of in‐role performance, and Agreeableness and Emotional Stability were the best predictors of organizational citizenship behavior (OCB). Observer ratings of personality accounted for incremental variance in job performance (in‐role performance and OCB) beyond that accounted for by general self‐ratings. However, contrary to our expectations, work‐specific (i.e., contextual) self‐ratings of personality generally did not account for incremental variance in job performance beyond that accounted for by general self‐ratings.     

Leader-Member Exchange and Employee Creativity: Test of a Multilevel Moderated Mediation Model

Using multisource data, this article examined the relationship of two types of employee perceptions of overall fairness (i.e., fairness of the organization and fairness of the department), with supervisor ratings of in-role task performance and organizational citizenship behaviors (OCB). We also examined whether high-quality leader-member exchange (LMX) relationships compensate for negative effects of fairness on job performance and citizenship behaviors. Focusing on a sample of 154 health care workers, we found that only employee perceptions of overall fairness regarding the department were related to supervisory ratings of OCB and in-role task performance. Moreover, LMX moderated the relationships between fairness perceptions and (a) task performance, (b) organization-focused OCB, but not (c) individual-focused OCB. We discuss implications for research and the potential for reducing negative effects of low fairness perceptions on job performance through positive LMX.     

Transformational leadership, initiating structure, and substitutes for leadership: a longitudinal study of research and development project team performance

Transformational leadership, initiating structure, and selected substitutes for leadership were studied as longitudinal predictors of performance in 118 research and development (R&D) project teams from 5 firms. As hypothesized, transformational leadership predicted 1-year-later technical quality, schedule performance, and cost performance and 5-year-later profitability and speed to market. Initiating structure predicted all the performance measures. The substitutes of subordinate ability and an intrinsically satisfying task each predicted technical quality and profitability, and ability predicted speed to market. Moderator effects for type of R&D work were hypothesized and found whereby transformational leadership was a stronger predictor of technical quality in research projects, whereas initiating structure was a stronger predictor of technical quality in development projects. Implications for leadership theory and research are discussed.     

Measurement Equivalence and Multisource Ratings for Non-managerial Positions: Recommendations for Research and Practice

ABSTRACT: The present investigation applies a comprehensive sequence of confirmatory factor analysis tests (Vandenberg and Lance, Organisational Research Methods, 3, 4–69, 2000) to the examination of the measurement equivalence of self, peer, and supervisor ratings of non-managerial targets across several performance dimensions. Results indicate a high degree of measurement equivalence across rater sources and performance dimensions. The paper illustrates how this procedure can identify very specific areas of non-equivalence and how the complexity of a multisource feedback system may be represented using such procedures. Implications of these results and recommendations for both research and practice are offered.     

RATER SOURCE EFFECTS ARE ALIVE AND WELL AFTER ALL

Recent research has questioned the importance of rater perspective effects on multisource performance ratings (MSPRs). Although making a valuable contribution, we hypothesize that this research has obscured evidence for systematic rater source effects as a result of misspecified models of the structure of multisource performance ratings and inappropriate analytic methods. Accordingly, this study provides a reexamination of the impact of rater source on multisource performance ratings by presenting a set of confirmatory factor analyses of two large samples of multisource performance rating data in which source effects are modeled in the form of second-order factors. Hierarchical confirmatory factor analysis of both samples revealed that the structure of multisource performance ratings can be characterized by general performance, dimensional performance, idiosyncratic rater, and source factors, and that source factors explain (much) more variance in multisource performance ratings whereas general performance explains (much) less variance than was previously believed. These results reinforce the value of collecting performance data from raters occupying different organizational levels and have important implications for research and practice.     

Placing perceptions of politics in the context of the feedback environment, employee attitudes, and job performance

The authors proposed a model suggesting that organizational environments supporting high levels of informal supervisor and coworker feedback are associated with lower employee perceptions of organizational politics. Furthermore, these lowered perceptions of politics were proposed to result in higher employee morale (as reflected in job satisfaction and affective commitment) and, through morale, to higher levels of task performance and organizational citizenship. The proposed mediational model was supported with empirical results from 150 subordinate-supervisor dyads sampled across a variety of organizations. Higher quality feedback environments were associated with lower perceptions of organizational politics, and morale mediated the relationships between organizational politics and various aspects of work performance. These findings suggest that when employees have greater access to information regarding behaviors that are acceptable and desired at work, perceptions of politics are reduced and work outcomes are enhanced.     

Is Transformational Leadership Always Perceived as Effective? Male Subordinates' Devaluation of Female Transformational Leaders1

This study investigated the impact of the gender composition of the leader–subordinate dyad on the relationship between leaders' transformational leadership behavior and their subordinates' ratings of the leaders' effectiveness. There were 109 dyads of leaders (58 male, 51 female) paired with a subordinate who was either the same or a different gender from themselves. The relationship between a leader's self‐report on transformational leadership and their subordinates' evaluation of their performance was significantly less positive for female leaders with male subordinates than for female leaders with female subordinates. The male and female subordinates of male leaders rated their performance as equally effective, regardless of their levels of transformational leadership.