RATER-RATEE RACE EFFECTS IN DEVELOPMENTAL PERFORMANCE RATINGS OF MANAGERS

The effects of rater and ratee race on performance ratings of managers were examined. Ratings were obtained from peers, subordinates and bosses as part of a multirater, developmental feedback program for managers. Two data sets were created for purposes of this study. The between-subjects data set consisted of ratings from over 20,000 bosses, over 50,000 peers, and over 40,000 subordinates. The repeated measures data set was substantially smaller because it included only those Black and White managers who were rated by both a Black and White rater from each of the three perspectives. Results for rater race indicated that Black raters from all perspectives (peers, subordinates, and bosses) assigned more favorable ratings to ratees of their own race. Results for White raters differed according to the particular rating source. White bosses assigned more favorable ratings to ratees of their own race, but White subordinates did not. White peers assigned more favorable ratings to Whites in the repeated measures analysis, but not in the between-subjects analysis. Results for ratee race indicated that both White and Black managers received higher ratings from Black raters than from White raters, and the effect was more pronounced for ratings assigned to Black managers.     

Rater information acquisition processes: Tracing the effects of prior knowledge, performance level, search constraint, and memory demand

Research on performance ratings has tended to neglect factors affecting raters' acquisition of ratee performance information. The present research examined the dynamic effects of this rater information acquisition process. Study 1 examined the effects of prior ratee knowledge (familiarity) and ratee performance level on subsequent acquisition of ratee performance information under varying levels of constraint. After exposure to prior ratee information, raters acquired additional information concerning a set of ratees using a computer controlled information board procedure. The results indicated significant effects on the amount of search for prior knowledge, search constraint, and the interaction between prior knowledge and performance level. Study 2 extended the research by including a memory component (time delay) between the exposure to prior information and subsequent search. Results were congruent with Study 1. They indicated significant main effects on the amount of search for prior knowledge, ratee performance level, search constraint, and on the interaction between prior knowledge and performance level. In addition, the results indicated a significant interaction between memory demand and prior knowledge. Implications of these results, limitations, and extensions of this research approach are discussed.     

多源评价的特点与内在机制

多源评价在国外企业中的运用已日益成熟, 但在我国还停留在探索与发展阶段。基于已有的研究发现, 围绕评价过程、评价源及被评价者三方面对多源评价的特点及内在机制进行了探讨与分析。从评价过程看, 其评价目的具有多重性, 评价形式注重匿名性, 且评价结果的合理应用非常重要; 从评价源看, 不同评价源间的评价一致性较低, 且易造成晕轮效应和宽大效应; 从被评价者来看, 个体对多源评价结果的反应, 受到个性特征、反馈信号及自我-他人评价间差距等因素影响。研究也发现, 多源评价所带来的绩效改进结果具有不稳定性。基于此, 如何提高多源评价过程的有效性与准确性, 改善评价者对评价结果的反应, 以及如何对多源评价结果进行有效汇总等是未来值得研究的重要内容。     

Rater Personality,Rating Format,and Social Context: Implications for Performance Appraisal Ratings

This study investigates the effects of rater personality (Conscientiousness and Agreeableness), rating format (graphic rating scale vs. behavioral checklist), and the rating social context (face‐to‐face feedback vs. no face‐to‐face feedback) on rating elevation of performance ratings. As predicted, raters high on Agreeableness showed more elevated ratings than those low on Agreeableness when they expected to have the face‐to‐face feedback meeting. Furthermore, rating format moderated the relationship between Agreeableness and rating elevation, such that raters high on Agreeableness provided less elevated ratings when using the behavioral checklist than the graphic rating scale, whereas raters low on Agreeableness showed little difference in elevation across different rating formats. Results also suggest that the interactive effects of rater personality, rating format, and social context may depend on the performance level of the ratee. The implications of these findings will be discussed.     

Gender-based influences: The promotion recommendation

This study used a between-subjects analysis of variance design (2 × 2 × 2 × 2) to estimate the effects of rater gender, ratee gender, the gender stereotype of the job, and the gender stereotype of the ratee's personal characteristics on a promotion decision. The results indicated that female employees with masculine characteristics were evaluated as most promotable regardless of the gender stereotype of the job or the gender of the rater. We also investigated the personal characteristics raters reported using in making their decisions. We asked raters to select ratee personal characteristics that most influenced their promotion decisions. Discriminant analyses showed that male and female raters approached the rating process differently and that it is predominantly the gender stereotype of the ratee's personal characteristics rather than the ratee's gender that influences the promotion process.     

Computing interrater reliability on the Apple Macintosh computer

A program is described for computing interrater reliability by averaging, for each rater, the correlations between one rater’s ratings and every other rater’s ratings. For situations in which raters rate more than one ratee, raters’ reliabilities can be computed for either each item or each ratee. The program reads data from a text file and puts the reliability coefficients in a text file. The standard Macintosh interface is implemented. The Quick-BASIC program is distributed both as a listing and in compiled form; it can be run with advantage with math coprocessors.     

Reducing the performance-cue bias in work behavior ratings: can groups help?

The authors examined whether the performance-cue bias can be reduced by relying on groups as raters. Study participants (N = 333) were provided with feedback regarding the performance of a workgroup and, after observing the group, assigned to an individual or group rater condition to complete a behavioral rating instrument. Results revealed that when provided with positive (vs. negative) feedback, individuals attributed more effective and fewer ineffective behaviors to the workgroup; however, group ratings were unaffected by the feedback. In addition, feedback biased the decision criteria and false alarm rates of individuals but not of groups. Discussion of when groups may attenuate versus amplify bias in performance appraisal judgments emphasizes 2 key elements-bias magnitude and task perception.     

The Effects of Ratee Prototypicality on Rater Observation and Accuracy1

Social cognition theory asserts that perceivers (raters) assign stimulus persons (ratees) to social categories. These categories help the raters encode, store, and recall information. In a longitudinal design that represented a performance appraisal situation, this study examined the effects of information about a ratee's category membership on the amount of information that raters collected about the ratee prior to rating. One hundred fourteen subjects participated in three separate experimental sessions which spanned a 3-week time period. Among other tasks, subjects were required to rate a subordinate who was described in a manner which made it either difficult or easy to assign the subordinate to a social category. It was predicted and found that raters of ratees who were easily categoriezed spent less time observing the ratees' performance than raters of ratees who were less easily classified. Furthermore, results indicated that it was the effect of rater categorization on observation time that was critical to rating accuracy.     

The impact of rater agreeableness and rating context on the evaluation of poor performance

We tested the effects of rater agreeableness on the rating of others’ poor performance in performance appraisal (PA). We also examined the interactions between rater agreeableness and two aspects of the rating context: ratee self‐ratings and the prospect of future collaboration with the ratee. Participants (n= 230) were allocated to one of six experimental groups (a 3 × 2 between‐groups design) or a control group (n= 20). Participants received accurate, low‐deviated, or high‐deviated self‐ratings from the ratee. Half were notified they would collaborate with the ratee in a future task. High rater agreeableness, positive deviations in self‐rating, and the prospect of future collaboration were all independent predictors of higher PA ratings. The interactions between rater agreeableness and rating context were very small. We argue that conflict avoidance is an important motivation in the PA process.     

SELF-APPRAISAL BASED UPON SUPERVISORY FEEDBACK

Self-appraisals of job performance have historically proven to be weakly related to appraisals conducted by supervisory personnel. Two studies sought to facilitate rater agreement by invoking supervisory performance feedback as a frame of reference for subordinate raters. In the first study, a sample of 401 branch managers from a large lending institution performed a conventional self-appraisal and an appraisal containing instructions referencing supervisory feedback (Feedback Based Self-Appraisal). Feedback Based Self-Appraisals exhibited significantly smaller leniency error, greater total rating variance, and more agreement with superior ratings than conventional self-appraisals. FBSAs produced small improvements over conventional self-appraisals in predicting objective criteria of managerial job performance. Study 2 examined FBSA-supervisory appraisal agreement for two samples of military organization personnel. The relative degree of feedback available in a work environment was found to moderate FBSA-superior agreement.     

Effects of rater conscientiousness on evaluations of task and contextual performance of older and younger co-workers

The possibility for age discrimination and stereotypes to affect performance evaluations is rising. Although careful evaluations might be expected from conscientious raters, little is known about whether they might show more or less bias towards certain age groups. Therefore, in our study using a time-lagged design, we investigated the effects of rater conscientiousness on the performance evaluations of younger and older actual co-worker (N = 242). We found that raters who were more conscientious provided higher ratings for older workers than for younger workers on task performance and organizational citizenship behaviours. Specifically, we tested the model of mediated moderation, in which the relation between rater conscientiousness and ratee age predicts ratee-perceived conscientiousness, which in turn predicts performance ratings. The model was significant for older ratees, but not for younger ratees. We discuss our results in terms of the “similar to me” effects and implications for organizational practices.     

Does Conscientiousness Moderate the Relationship Between Attitudes and Beliefs Regarding Performance Appraisal and Rating Behavior?

Attitudes, beliefs, and orientations of 253 managers toward performance appraisal, along with the personality dimension conscientiousness, were used to predict rating behavior (i.e., mean ratings and measures of performance dimensions discrimination and ratee discrimination). As predicted, relationships between rater attitudes and rating behavior were moderated by conscientiousness; correlations between the attitudes and rating behavior were strongest for raters who were relatively low on conscientiousness.     

Why convene rater teams: An investigation of the benefits of anticipated discussion,consensus, and rater motivation

This study explores the importance of anticipated group discussion, the consensus decision rule, and rater motivation in determining how well rater teams identify ratee behaviors, i.e., behavioral accuracy. Results, based on 382 raters in 111 teams, suggest that the anticipation of group discussion can improve behavioral accuracy, but it appears that the benefits of discussion-only teams are limited to this anticipation effect. Furthermore, it also appears that rater motivation plays an important role in this type of team. Rater teams required to reach consensus, however, appear to show improved behavioral accuracy, regardless of whether raters can anticipate the consensus discussion and regardless of rater motivation levels. Implications, especially for assessment centers, are discussed.     

Rater agreement,accuracy, and experienced cognitive load: Comparison of distributional and traditional assessment approaches to rating performance

Error in performance ratings is typically believed to be due to the cognitive complexity of the rating task. Distributional assessment (DA) is proposed to improve rater accuracy by reducing cognitive load. In two laboratory studies, raters reported perceptions of cognitive effort and difficulty while assessing rating targets using DA or the traditional assessment approach. Across both studies, DA raters showed greater interrater agreement, and Study 2 findings provide some support for DA being associated with greater true score rating accuracy. However, DA raters also reported experiencing greater cognitive load during the rating task, and cognitive load did not mediate the relationship between rating format and rater accuracy. These findings have important implications regarding our understanding of cognitive load in the rating process.     

Interpersonal perception and personality disorders: Utilization of a thin slice approach

Study 1 expands upon previous research by looking at the ability of untrained raters to detect pathological traits within a normal population of college students. In Study 1, 30-s video clips of 81 target persons were shown to 42 raters. Ratings of traits of personality disorders made by thin slice raters reliably predicted scores on the personality pathology measures obtained from the targets themselves and from close peers. Study 2 is a preliminary examination of how pathological rater traits impacts thin slice accuracy. In Study 2, peer and self-report data were examined regarding 87 thin slice raters. Raters who exhibited traits of narcissistic personality disorder were significantly less accurate in making personality predictions regarding targets. Three clusters of personality items were identified based on rater characteristics related to accuracy in predicting behavior.     

A Meta-Analysis of the Relationship Between Rater Liking and Performance Ratings

This meta-analysis reviewed the magnitude and moderators of the relationship between rater liking and performance ratings. The results revealed substantial overlap between rater liking and performance ratings (ρ = .77). Although this relationship is often interpreted as indicative of bias, we review studies that indicate that to some extent the relationship between liking and performance ratings potentially reflects “true” differences in ratee performance. Moderator analyses indicated that the relationship between liking and performance ratings was weaker for ratings of organizational citizenship behaviors, ratings made by peer raters, ratings in nonsales jobs, and ratings made for development; however, the relationship was strong across moderator levels, underscoring the robustness of this relationship. Implications for the interpretation of performance ratings are discussed.     

The Effect of Giving Informal Performance Feedback on Subsequent Formal Memory-Based Performance Evaluations1

Two experiments tested the hypothesis that raters' formal memory-based performance evaluations can be significantly influenced by their having previously given the ratee informal performance feedback. In Experiment 1 subjects either did or did not give informal feedback to another person who performed either well or poorly on an interviewing task. In Experiment 2 subjects role played giving informal feedback about behavior relevant to only one of the two performance dimensions subsequently evaluated. In both experiments subjects later ratcd the interpersonal and task performance of the feedback recipient. The results of both studies support the hypothesis for ratings of interpersonal performance. Giving informal feedback to a ratee exhibiting good interpersonal performance led to more positive interpersonal performance ratings, whereas giving informal feedback to a ratee exhibiting poor interpersonal performance led to more negative interpersonal performance ratings. Task performance ratings, on the other hand, were not affected. Conditions likely to have mitigated the impact of giving informal feedback on the task performance ratings are discussed, as are the implications of the results for practical strategies to improve the quality of formal memory-based performance evaluations.     

The Effects of Source Credibility and Performance Rating Discrepancy on Reactions to Multiple Raters1

Using multiple feedback sources, the present study investigated the effects of source credibility and performance rating discrepancy on recipients' reactions. Individuals performed an ambiguous group task, rated their own performance on the task, and were later provided bogus feedback ostensibly from their peers and an expert rater. Individuals reacted toward the feedback and the source of the feedback as a function of the rating discrepancy and credibility of the feedback source. Generally, more credible sources and their feedback were evaluated more favorably. However, as predicted, this effect was overcome by performance rating discrepancy in the predicted conditions. The results show the importance of studying the interactive effects of message and source characteristics on individuals' reactions.     

Predictive Criterion-Related Validity of Observer Ratings of Personality and Job-Related Competencies Using Multiple Raters and Multiple Performance Criteria

The purpose of this study was to assess the predictive validity of observer ratings of personality and job-related competencies in a selection setting. Based on ratings from multiple raters of both the predictors and the criteria in a sample of MBA students, results indicated that observer ratings of Conscientiousness, Emotional Stability, leadership, and interpersonal skills predicted work performance, team performance, and academic performance. For work performance and team performance, a composite of the four predictors had incremental predictive validity over general mental ability, even after controlling for how well the rater knew the ratee.