Multisource Assessments of Behavioral Competencies and Selection Interview Performance

This study sought to determine whether and how ratings of leadership in a developmental multisource feedback program compare against those obtained by leadership selection decision makers. Seventy‐seven senior managers received feedback on eight competencies that also formed the basis of a subsequent promotional interview for senior executive positions. Each rating source displayed a unique perspective on the participants, but only the supervisor and peer ratings predicted performance on the interview. As well, peer, subordinate, and self‐ratings failed to improve upon the supervisors' predictions. The results also suggested a general performance factor underlying both the multisource assessments and the interview ratings.     

DEVELOPMENT ENGAGEMENT WITHIN AND FOLLOWING DEVELOPMENTAL ASSESSMENT CENTERS: CONSIDERING FEEDBACK FAVORABILITY AND SELF–ASSESSOR AGREEMENT

This study sought to understand employees' level of behavioral engagement in response to feedback received in developmental assessment center (DAC) programs. Hypotheses were drawn from theories of self‐enhancement and self‐consistency and from findings in the multisource feedback and assessment center literatures regarding recipients' perceptions of feedback. Data were gathered from 172 U.S. middle managers participating in a DAC program. Results suggested that more favorable feedback was related to higher behavioral engagement. When discrepancies between self‐ and assessor ratings were examined, overraters (participants whose overall self‐ratings were higher than their assessor ratings) tended to show less engagement in the program compared to underraters. However, pattern agreement on the participant's dimension profile did not significantly correlate with behavioral engagement. Based on these findings, avenues for future research are presented and practical implications are discussed.     

Multisource Feedback and Leaders' Goal Performance: Moderating effects of rating purpose,rater perspective,and performance dimension

The main purposes of this study are to examine whether multisource feedback ratings predict leaders' organizational goal performance, and whether the relationships are consistent across the two rating purposes (developmental, administrative), two leadership dimensions behaviors (Consideration, Initiating Structure), and three rating perspectives (supervisor, self, and ‘other’ raters, i.e., peers and subordinates). Leaders (n=396) in a large organization in the transportation industry participated in two multisource feedback programs, the first for developmental purposes and the second 8 months later for administrative purposes. Approximately 1 month later, they were rated by their supervisor on their effectiveness in attaining five organizational performance goals (financial, safety, customer satisfaction, employee satisfaction, diversity). Results revealed that both developmental ratings and administrative ratings uniquely predicted leaders' goal performance. However, both leadership dimension and rater perspective moderated these relationships. Leadership behaviors associated with Consideration were stronger predictors of goal performance for supervisor ratings, whereas behaviors associated with Initiating Structure were stronger predictors of goal performance for self and other ratings.     

Adaptive Self-Regulation: Meeting Others' Expectations of Leadership and Performance

Abstract

The authors used longitudinal multisource field data to examine core aspects of the adaptive self-regulation model (A. S. Tsui & S. J. Ashford, 1994) in terms of linkages between self-monitoring, discrepancy in manager match-to-position, 5 measures of leadership, and manager performance. At Time 1,64 superiors of focal managers rated the managers' matches to their positions within the organization; at Time 3, they rated the managers' performance. At Time 2, the 64 focal managers completed a measure of self-monitoring, and 192 subordinates rated the focal managers' leadership behaviors. Results of partial least squares analysis revealed that discrepancy in manager match-to-position was associated with reductions in laissez faire and passive management-by-exception behaviors and increases in transformational leadership behavior. Self-monitoring was positively associated with all 5 leadership behaviors. Performance was related positively to transformational leadership behavior and negatively to passive management-by-exception and contingent-reward behaviors     

Transferring More than Learned in Training: Employees' and managers' (over)generalization of skills

Across two studies, we examine the extent to which individuals accurately report their learning and transfer in a training context. In Study 1, we examine self (subordinate) and supervisory estimations of training transfer, 6–12 weeks after employees (subordinates) attended training in an organization in the United States. Using ratings of skills unrelated to training programs attended, we compared managers' and employees' ratings to determine the extent to which respondents report skill overgeneralization. Individual differences and skill visibility were examined as predictors of training transfer overgeneralization. Subordinates, particularly those high on conscientiousness, agreeableness, and emotional stability, are more likely to report transfer in areas not covered in training. Conversely, when managers rate skills, which are used in their subordinates' day‐to‐day activities (more observable, visible, or transparent skills), training transfer estimates are more accurate. In Study 2, we determine that conscientiousness is the main factor driving raters' overgeneralization (for self‐ratings), and we further demonstrate that other personality dimensions – insecurity and perfectionism – accentuate the influence of conscientiousness on trainees' overgeneralization. From a practical standpoint, employers need to be aware of potential for biases in training transfer ratings when evaluating training transfer, particularly for employees with specific personality characteristics and for those who are in jobs with low skill visibility.     

Male and female managers' ability to read emotions: Relationships with supervisor's performance ratings and subordinates' satisfaction ratings

The present study examines the direct and indirect effect of managers' accuracy in ‘reading’ non‐verbal emotional expressions and their supervisors' performance ratings and subordinates' satisfaction with the manager. Data from working managers and their supervisors and subordinates were used to test the study's hypotheses. Although managers' accuracy was not directly related to others' ratings, a significant interactive effect was found: Female but not male managers who more accurately perceived non‐verbal emotional expressions received higher performance ratings from their supervisor and higher satisfaction ratings from their subordinates, perhaps due to gender stereotypes prescribing emotional sensitivity to female managers. Furthermore, how managers used emotional information to enhance subordinates' satisfaction varied for female and male managers. Results suggested that male managers who were more accurate at emotion perception received higher satisfaction ratings if they used the information to be more persuasive, whereas more emotionally perceptive female managers received higher satisfaction ratings when they demonstrated more supportiveness.     

360 Degree Feedback and Developmental Outcomes: The Role of Feedback Characteristics,Self‐Efficacy and Importance of Feedback Dimensions to Focal Managers' Current Role

This longitudinal study investigates whether developmental changes following 360 degree feedback are predicted by the favourability of ratings received, and moderated by focal individuals' self‐efficacy and perceived importance of feedback. Five developmental criteria are investigated longitudinally: (i) self‐assessments, (ii) line managers' ratings, (iii) amount of developmental activity, (iv) global self‐efficacy and (iv) self‐efficacy for development. Feedback ratings from certain rater groups predicted changes in ratings, but not changes in self‐efficacy or amount of developmental activity. Self‐efficacy significantly moderated the feedback–performance association for certain rater groups, but feedback importance did not. Contrary to expectations, the focal individual's initial self‐assessment predicted changes in self‐efficacy, over the favourability of ratings received. The implications of these findings for organizations using 360 degree feedback for developmental purposes are discussed.     

Self–Other Agreement: Comparing its Relationship with Performance in the U.S. and Europe

The use of multi‐source feedback has proliferated in the United States in recent years; however, its usefulness in other countries is unknown. Using a large sample of American managers (n=3793), this study first replicated earlier studies demonstrating that simultaneous consideration of self and other ratings of leadership skills is important for managerial performance ratings. In addition, the impact of self–other agreement on performance was investigated among 2732 managers in five European countries (U.K., Germany, France, Denmark, Italy). Results indicated that the effect of self and other ratings in the prediction of performance differs between the U.S. and the European countries in that the simultaneous inclusion of both self and other ratings is generally less useful in those countries than in the U.S. Further, the effect of self–other agreement varies among the European countries. Implications for multi‐source feedback interventions as well as multi‐national personnel management are discussed.     

A study of the discrepancy between self- and observer-ratings on managerial derailment characteristics of European managers

Managerial derailment is costly to managers, their co-workers, and their organization. Knowing whether discrepancies (i.e., differences, dissimilarity, disagreement, incongruity) exist between self- and observer- (subordinates, peers, and bosses) ratings about derailment may help to lessen or prevent the detrimental outcomes of derailment on managers, their co-workers, and their organization. Results from 1742 European managers revealed a statistically significant difference between a manager's self-ratings and observer-ratings on the extent to which a manager displayed derailment behaviours and characteristics. The discrepancy also widened as managerial level increased, and was mostly due to inflated self-ratings. In addition, discrepancies between self – boss, self – direct report, and self – peer were examined, as well as differences between European and American managers. A discussion of these findings and implications for practice conclude this article.     

SCL‐90‐R profiles in a sample of severely violent psychiatric inpatients

A sample of 39 patients who had committed serious violent acts toward others were assessed with the revised Symptom Checklist (SCL‐90‐R). The SCL‐90‐R is a self‐report symptom inventory for the measurement of psychopathology in psychiatric and medical patients. In addition to the patients' self‐report, an observer‐rated SCL‐90‐R was obtained. This was accomplished by letting one pair of nurses complete SCL‐90‐R ratings for each patient. The first aim of the study was to compare the SCL‐90‐R self‐report scores of the patient sample with the psychiatric inpatient norms [Derogatis LR (1992): Clinical Psychometric Research Inc]. Another purpose of this study was to explore possible trends of discordance between the observer ratings and the self‐reports of the study group. Always considering the limitations of the small sample, it was nevertheless also of interest to look for possible sex differences and differences between violent subgroups and between diagnostic groups in the self‐reported scores as well as in the observer ratings. The most striking findings of the present study were that the self‐reported scores were lower than the inpatient norms for SCL‐90‐R and that the patients' self‐reported levels of distress were significantly lower than those found in the observer ratings. Underreporting of psychopathology as a marker of violence risk is discussed in light of these findings. In this study, women reported higher distress levels than men in the Interpersonal Sensitivity symptom dimension. There were no significant differences concerning SCL‐90‐R ratings between patients who had committed homicide, attempted homicide, or physically assaulted another person in a serious but not life‐threatening way. Aggr. Behav. 28:446–457, 2002. © 2002 Wiley‐Liss, Inc.     

Romantic partners and body image disturbance: Further evidence for the role of perceived—actual disparities

In this study, female and male college dyads (primarily Caucasian) involved in a “romantic relationship” used schematic figures to rate their conceptions of body size on a variety of rating protocols, including actual self, ideal self, ideal opposite gender, perceived partner's ideal opposite gender, perceived partner's rating of self, and rating of partner's body size. Based on the reflected-appraisal literature and previous body image research on perceived—actual disparities in body size ratings, five discrepancy indices were created and related to dispositional levels of body image disturbance, eating dysfunction, and general psychological function (self-esteem, depression). The results clearly indicated gender differences in the particular predictor associated with all measures of disturbance. For females, the discrepancy between the Self rating and the Perceived Partner's Ideal Other Gender rating accounted for almost all of the variance associated with clinical indices of disturbance. For males, the discrepancy between the Self rating and the Actual Partner's Ideal Other Gender rating was the only important predictor to emerge in regression analyses. The findings are interpreted in light of recent research on actual/perceived differences in body size ratings.     

The impact of rater agreeableness and rating context on the evaluation of poor performance

We tested the effects of rater agreeableness on the rating of others’ poor performance in performance appraisal (PA). We also examined the interactions between rater agreeableness and two aspects of the rating context: ratee self‐ratings and the prospect of future collaboration with the ratee. Participants (n= 230) were allocated to one of six experimental groups (a 3 × 2 between‐groups design) or a control group (n= 20). Participants received accurate, low‐deviated, or high‐deviated self‐ratings from the ratee. Half were notified they would collaborate with the ratee in a future task. High rater agreeableness, positive deviations in self‐rating, and the prospect of future collaboration were all independent predictors of higher PA ratings. The interactions between rater agreeableness and rating context were very small. We argue that conflict avoidance is an important motivation in the PA process.     

Well‐being in cross‐cultural transitions: discrepancies between acculturation preferences and actual intergroup and intragroup contact

Two studies examined the effect of acculturation–contact discrepancies on well‐being. Specifically, we tested the prediction that well‐being will be compromised when cultural minorities' acculturation preferences are not met by the intergroup and intragroup contact in a new society. Study 1 found that for Polish immigrants (n = 55) acculturation–contact discrepancies were associated with compromised well‐being. Study 2 followed a cohort of international students (n = 106) for a period of two academic years. Results suggested that discrepancies in students' acculturation–contact in their first year had harmful consequences for their well‐being 1 year later. Overall, the two studies show that discrepancies between acculturation preferences and actual contact have negative implications for the psychological adaptation of acculturating individuals.     

Effects of personality and gender on self–other agreement in ratings of leadership

We explore the role of leader personality (i.e., the Big 5 traits: Conscientiousness, Agreeableness, Openness, Extraversion, and Neuroticism) and gender in self–other (dis)agreement (SOA) in ratings of leadership. We contend that certain aspects of the leader's persona may be more or less related to self‐ or other‐ratings of the leader's behaviour if those aspects are (1) more or less observable by others, (2) more or less related to internal thoughts versus external behaviours, (3) more or less prone to self‐enhancement or self‐denigrating biases, or (4) more or less socially desirable. We utilize statistical methodologies that capture fully the effects of multiple independent variables on the congruence between two dependent variables (Edwards, 1995 , Organizational Behavior and Human Decision Processes, 64, 307), which previously have not been applied to this area of research. Our results support hypotheses predicting less SOA as leader Conscientiousness increases and greater SOA as Agreeableness and Neuroticism increase. Additionally, we found gender to be an important factor in SOA; female leaders exhibited greater SOA than did their male counterparts. We discuss the implications of these findings, limitations, and future research directions.

Practitioner points

• Popular practices such as 360‐degree feedback may reveal discrepancies between a person's self‐ratings and other's ratings.
• Although often attributed to a lack of self‐awareness, these discrepancies also may be explained by factors such as the personality and gender of the focal individual.

     

Associations between rating content and self-other agreement in multi-source feedback

In a study of the ratings of managers' behaviour made by themselves, their supervisors, subordinates, and peers, it was found that degree of similarity between self-ratings and ratings by colleagues was significantly associated with three aspects of rating content: the observability of a behaviour, its desirability, and the degree to which it concerned technical rather than interpersonal activities. However, these associations with rating content depended on which of two indices of between-rater similarity was studied, in terms of either the correlation or the difference between ratings. The desirability of behaviour more influenced correlation measures of similarity, and the type of behaviour had more influence on difference scores: managers over-rated their interpersonal activities but not their technical effectiveness.     

Are observer ratings of applicants’ personality also faked? Yes,but less than self‐reports

Although self‐report personality tests are a comparatively cheap and easy‐to‐administer personnel selection tool, researchers have criticized them for not predicting enough criterion‐related variance. Researchers have suggested using observer‐ratings of personality (e.g., as part of a reference check from a supervisor) because observer‐ratings have been reported to be more predictive. However, it is theoretically and empirically unclear whether supervisors also engage in faking (the intentional distortion of responses). Study 1 explored faking among managers who were first asked to imagine that a subordinate had to leave his/her job for private reasons and then to rate the personality of the subordinate. A week later, managers rated their subordinates honestly. A repeated‐measures MANOVA indicated that managers did fake. Study 2 (among supervisors of working students) replicated the above findings but also showed that there is less faking in supervisor‐ratings than in self‐ratings. Furthermore, we found no evidence that the validity of personality scales for predicting academic performance depends on self‐ versus observer‐ratings or on an applicant versus an honest condition. These two studies thus show that practitioners should not equate personality ratings obtained from observers in a selection context with honest personality ratings.     

Are supervisors and coworkers likely to witness employee counterproductive work behavior? An investigation of observability and self–observer convergence

Supervisor and coworker ratings (i.e., “observer ratings”) remain a common manner of measuring counterproductive work behavior (CWB) despite long‐standing doubts that observers have the opportunity to witness the work behaviors they are expected to rate. We conducted 2 studies that evaluated the observability of CWB items and consequences of observability. First, we show that many CWBs are unlikely to be witnessed by supervisors or coworkers—specifically, behaviors such as “spends too much time fantasizing or daydreaming instead of working” and “discussed confidential company information with an unauthorized person” were found to be lowest in observability, whereas “cursed at someone at work” or “acted rudely toward someone at work” were relatively higher in observability (though observability was generally low). Second, a meta‐analysis revealed variability in item‐level relationships (correlations and mean differences) between self‐ratings and observer ratings for specific CWB scale items (i.e., items from Bennett & Robinson, 2,000). Important, this variability was partially explained by observability—behaviors with low self–observer convergence tend to have low levels of observability, whereas behaviors with higher levels of convergence tend to have higher levels of observability. This study demonstrates that supervisor and coworker ratings of CWB may be susceptible to an observability bias resulting from rating behaviors they have not likely witnessed.     

When Being Discrepant from One's Ideal or Ought Selves Hurts: The Moderating Role of Neuroticism

Self‐discrepancy theory posits that people experience emotional consequences when they perceive discrepancies between their actual and possible selves. However, the extent to which people react emotionally to these self‐discrepancies (i.e. ideal, ought and undesired) may be a function of individual differences in neuroticism. Across both experimental (Study 1; N = 155) and correlational designs (Study 2; N = 139) involving college students, the authors demonstrated that neuroticism moderated the discrepancy–emotion associations such that high‐neuroticism individuals showed elevated depression and anxiety symptoms when their self‐discrepancies were activated. The heightened symptoms were maintained over time. Negative repetitive thoughts (i.e. rumination and worry) were examined as potential mediating mechanisms between the discrepancy × neuroticism interaction and symptoms. Partial support was obtained in that rumination mediated between undesired discrepancy × neuroticism interaction and anxious/depressive symptoms. Implications and possible theoretical extensions for self‐discrepancy theory are discussed. Copyright © 2012 John Wiley & Sons, Ltd.     

Assessing Innovation: A 360‐degree appraisal study

This study examines the evaluation of innovation in a wider competency framework and within a 360‐degree rating procedure among managerial‐level job holders. The total sample of 2,979 individuals consisted of 296 target employees and their 318 bosses, 1208 peers, 828 direct reports, and 329 others who provided ratings on a competency framework. The results showed significant differences in innovation‐related competence ratings between different raters. Self‐ratings were significantly lower compared to the overall observer ratings and were correlated only with peer ratings. Different patterns of results were found for the lower and upper quartiles based on self‐ratings. For instance, no correlations were observed between self‐ratings and the ratings of any observers in the group of best self‐rated individuals. Implications for practice and future research in assessment and evaluation of innovation are discussed in conclusion.     

When “who we are” and “who I desire to be” appear disconnected: Introducing collective/personal self‐discrepancies and investigating their relations with minority students' psychological health

According to Self‐Discrepancy Theory research, perceiving mismatches between personal aspects of the self‐concept is associated with negative psychological consequences, including depression and anxiety. However, the impact of perceiving mismatches between collective and personal self‐aspects is still unknown. In a first step to address this gap, we introduce collective/personal self‐discrepancies—perceived mismatches between a desired self‐aspect and a collective identity. For cultural minority group members (n = 147), collective/personal self‐discrepancies were associated with more severe anxiety and depression symptoms. Bootstrapping analyses suggest that these relations are mediated by self‐discrepancies experienced at the personal level, but only for group members presenting average or high levels of ethnic identification. This study reaffirms the importance of collective identities, especially as potential antecedents of personal aspects of the self‐concept. The findings are further discussed in terms of their significance for cultural minority group members, who often highly identify with their minority groups.