Reducing the Effects of Performance Expectations on Behavioral Ratings

In this study, we develop and test two strategies for reducing the effects of performance expectations on behavioral ratings. A 3 × 3 experimental design (N = 169), manipulating preobservation performance cues (positive, negative, or none) and the type of intervention (halo error training, structured recall memory, or none), was conducted. The results of this study indicate that both interventions reduced the effects of performance expectations on behavioral ratings. However, analyses of rating accuracy and measures from both recognition memory and recall memory tests suggest that the structured recall memory intervention has distinct advantages. These analyses indicate that the structured recall intervention can reduce raters' reliance on heuristics and increase the correspondence between raters' memory and their subsequent ratings.     

Effects of Performance Context on Processing Speed and Performance Ratings

The present study demonstrated that the presence of evaluatively polarized context performances not only produces contrast and halo effects on judgments of a target performance, but also causes judgments to be made much faster. Processing speed and positive halo were highly correlated, supporting the notion that halo in performance ratings results from raters' recall and use of a single, general impression. Furthermore, regression analyses demonstrated that processing speed mediates the relationship between context and halo. The relationship between these findings, halo, processing speed, and general impressions, as well as implications for performance appraisals, are discussed.     

Cultural Values and Performance Appraisal: Assessing the Effects of Rater Self-Construal on Performance Ratings

Much of the prior research investigating the influence of cultural values on performance ratings has focused either on conducting cross-national comparisons among raters or using cultural level individualism/collectivism scales to measure the effects of cultural values on performance ratings. Recent research has shown that there is considerable within country variation in cultural values, i.e. people in one country can be more individualistic or collectivistic in nature. Taking the latter perspective, the present study used Markus and Kitayama's (1991) conceptualization of independent and interdependent self-construals as measures of individual variations in cultural values to investigate within culture variations in performance ratings. Results suggest that rater self-construal has a significant influence on overall performance evaluations; specifically, raters with a highly interdependent self-construal tend to show a preference for interdependent ratees, whereas raters high on independent self-construal do not show a preference for specific type of ratees when making overall performance evaluations. Although rater self-construal significantly influenced overall performance evaluations, no such effects were observed for specific dimension ratings. Implications of these results for performance appraisal research and practice are discussed.     

The Effects of Team Context on Peer Ratings of Task and Citizenship Performance

Recent trends indicate that organizations will continue their strategic pursuit of teamwork for the foreseeable future, which will create a need for accurate assessments of individuals’ performance in teams. Although individual behaviors can be perceived and assessed by fellow team members (i.e., peers), the extent to which the team shapes perceivers’ judgments versus the target’s behavior is unclear. We conducted two studies to understand how and why team context influences peer ratings of individual performance. In study 1, we conducted cross-classified modeling on a sample of 7160 performance observations of 568 targets made by 567 perceivers, who were each members of four separate teams. Results indicated that team membership accounted for a substantially higher proportion of perceiver, relative to target, variance. In study 2, we conducted social relations modeling with a sample of 679 performance observations collected from 217 individuals nested in 46 teams to test the effects of psychological safety on perceiver, target, and team variance components. Perceptions of psychological safety accounted for proportionally larger perceiver, relative to target, variance in OCB, and task performance ratings. Altogether, team context appears to affect perceivers’ judgments of behavior more than the target’s behavior itself, implying that peer ratings sourced from different teams may not be comparable. We consider the implications for the collection and interpretation of peer performance ratings in teams and the potential implications for social cognitive theory, such that certain aspects of the team context, including psychological safety, may act as a cognitive heuristic by molding perceiver judgments of targets.

     

Contrast Effects in Performance Ratings: Another Look Across Time

Récemment, Murphy, Balzer, Lockart et Eisenman (1985) rapportèrent que les effets de contraste dans une comparaison intra-personnelle disparaissent subséquemment à un court délai d'évaluation d'un stimulus-cible. Les deux expérimentations que nous décrivons indiquent que les effets de contraste dans une comparaison inter-personnelle résistérent même après un délai de trois semaines entre l'évaluation d'un stimulus de référence et d'un stimulus-cible. Dans l'expérience 1, 270 étudiants furent répartis en neuf groupes expérimentaux ( n = 30) qui représentent toutes les combinaisons possibles de trois indices de performance (élevé, bas, moyen) en deux temps différents. Un effet de contraste significatif fut observé chez ceux des participants qui visèrent et obtinrent une performance moyenne au Temps 2 (omega au carré= 0.16). Dans l'expérimentation 2, 270 étudiants MBA (avec un minimum d'expérience de deux années d'assistanat) répétèrent les résultats de l'expérimentation 1 (omega au carré= 0.18). Ces résultats groupés indiquent que les effets de contraste résistent au temps et valorisent les projets de recherche dans ce domaine.     

The Performance Appraisal Milieu: A Multilevel Analysis of Context Effects in Performance Ratings

Purpose

The purpose of this study was to take an inductive approach in examining the extent to which organizational contexts represent significant sources of variance in supervisor performance ratings, and to explore various factors that may explain contextual rating variability.

Design/Methodology/Approach

Using archival field performance rating data from a large state law enforcement organization, we used a multilevel modeling approach to partition the variance in ratings due to ratees, raters, as well as rating contexts.

Findings

Results suggest that much of what may often be interpreted as idiosyncratic rater variance, may actually reflect systematic rating variability across contexts. In addition, performance-related and non-performance factors including contextual rating tendencies accounted for significant rating variability.

Implications

Supervisor ratings represent the most common approach for measuring job performance, and understanding the nature and sources of rating variability is important for research and practice. Given the many uses of performance rating data, our findings suggest that continuing to identify contextual sources of variability is particularly important for addressing criterion problems, and improving ratings as a form of performance measurement.

Originality/Value

Numerous performance appraisal models suggest the importance of context; however, previous research had not partitioned the variance in supervisor ratings due to omnibus context effects in organizational settings. The use of a multilevel modeling approach allowed the examination of contextual influences, while controlling for ratee and rater characteristics.

     

Effects of Self-Appraisal Information, Appraisal Purpose, and Feedback Target on Performance Appraisal Ratings

This experiment investigated the effects of three factors on performance appraisal ratings: self-appraisal information, appraisal purpose, and feedback target. Two hundred and three subjects rated a subordinate's performance on a clerical task subsequent to receiving either a high or low self-assessment. They were told they would provide performance feedback either to the experimenter (organizational agent) or their subordinate, and their ratings would be used either for an administrative decision or developmental feedback. Performance ratings were significantly higher when subjects received a favorable subordinate self-assessment than when self-assessments were unfavorable. A significant interaction was found between feedback target and the appraisal purpose. Implications for the use of self-appraisals in organizations were discussed.     

Factors Influencing the Inflation of Task Performance Ratings for Workers With Disabilities and Contextual Performance Ratings for Their Coworkers

Contrary to many myths regarding their potential performance, workers with disabilities generally receive performance ratings similar to their nondisabled coworkers. However, their ratings possibly may be inflated above their actual performance levels. In a laboratory experiment, we constrained the performance of a worker with a disability to an extremely low level and assessed the effects of helping behavior, presence and type of disability in the worker, and the perceived attributions of controllability of the disability on task performance ratings for the workers with disabilities and on contextual performance ratings for their coworkers. We found that the perception that a worker is not responsible for the onset of a disability and that having any disability, regardless of type, artificially inflates task performance ratings. We also found inflated contextual performance ratings for coworkers who work with persons who are perceived as being responsible for the onset of their own disability.     

Managerial Experience and the Measurement Equivalence of Performance Ratings

ABSTRACT: Establishing the measurement equivalence of instruments is a prerequisite to making meaningful comparisons between individuals or within individuals over time. Whereas previous research has investigated the effects of rater characteristics on the measurement equivalence of performance ratings, the current study investigated a ratee characteristic—ratee job experience. Using confirmatory factor analysis and item response theory methods with replication, the measurement equivalence of supervisor ratings of 7,200 managers with differing levels of managerial experience was assessed. Overall, results indicated a high degree of measurement equivalence suggesting that meaningful comparison may be made across ratees with different levels of job experience.     

Explaining the Weak Relationship Between Job Performance and Ratings of Job Performance

Ratings of job performance are widely viewed as poor measures of job performance. Three models of the performance–performance rating relationship offer very different explanations and solutions for this seemingly weak relationship. One-factor models suggest that measurement error is the main difference between performance and performance ratings and they offer a simple solution—that is, the correction for attenuation. Multifactor models suggest that the effects of job performance on performance ratings are often masked by a range of systematic nonperformance factors that also influence these ratings. These models suggest isolating and dampening the effects of these nonperformance factors. Mediated models suggest that intentional distortions are a key reason that ratings often fail to reflect ratee performance. These models suggest that raters must be given both the tools and the incentive to perform well as measurement instruments and that systematic efforts to remove the negative consequences of giving honest performance ratings are needed if we hope to use performance ratings as serious measures of job performance.     

The Influences of Performance Cues and Observational Focus on Performance Rating Accuracy1

The present study examined the effects of observational focus and performance cues on rating accuracy. It was hypothesized that these two factors would affect ratings independently: a) subjects given a good performance cue would rate the target more positively than subjects given a poor performance cue and, b) subjects using an event focus would rate the target person more accurately than subjects using a person focus. One hundred twenty undergraduates viewed the same videotape and subsequently stated which of a set of 48 behaviors were exhibited by the target person. The results supported the hypothesized performance cue effect. Observational focus did not have the hypothesized main effect on rating accuracy but was involved in a four-way interaction. The interaction indicated that, as the focus of the subject's attention broadened to the entire event, the errors associated with the performance cue effect were lessened. The subjects using an event focus were likely to generate ratings characterized by a positivity bias, whereas those using a person focus were more likely to generate ratings biased by performance cues.     

Factors Affecting the Convergence of Self-Peer Ratings on Contextual and Task Performance

This study examines factors that predict the extent to which 408 operating-level workers rated themselves higher, lower, or the same as their coworkers rated them, for both task and contextual performance. On ratings of contextual performance, underestimators tended to be distinguished by significantly higher levels of both self-monitoring and social desirability. This trend operated similarly, though not significantly for task performance. Additionally, ratings of quantity of work obtained the highest degree of self-peer rating convergence as compared to ratings of quality of work and contextual performance. These results are discussed in terms of the practical implications for multirater systems.     

A Developmental Analysis of Performance Ratings

In this study, we examined the performance appraisal process from a developmental perspective. Components of cognitive structure, measured in terms of differentiation and hierarchic integration, were obtained from 45 American students representing three different academic levels at three times during a 15-week semester. These students' performance evaluations (ratings) of their psychology instructors were collected during the first and final sessions. The results indicated that although both differentiation and hierarchic integration seemed to change systematically, even over a short period of time, there was no corresponding change in the halo inherent in the raters' ratings. We present possible explanations for the failure of developmental changes in cognitive structure to be reflected in the halo inherent in the performance ratings, as would be predicted by Schneier's (1977) cognitive compatibility hypothesis. Increased emphasis on developmental aspects of the performance appraisal process is recommended.     

The Influence of Indirect Knowledge of Previous Performance on Ratings of Present Performance: The Effects of Job Familiarity and Rater Training

In a 2 × 2 × 2 crossed factorial design, trained or untrained subjects viewed a videotape and evaluated performance on either a familiar (college lecturer) or unfamiliar (salesperson) job. Prior to viewing the videotape, some subjects reviewed positive information about the ratee's prior performance, whereas other subjects did not review any prior performance information. To determine whether assimilation or contrast effects occurred, we compared ratings provided by subjects who reviewed positive information about prior performance with ratings provided by subjects who did not review any prior performance information. A three-way interaction was obtained. Ratings of performance on the familiar job by untrained or trained subjects revealed only a small assimilation effect. However, when rating performance on the unfamiliar job, a large assimilation effect was observed among untrained subjects, whereas a large contrast effect was observed among trained subjects. The results indicate that rater error training may reverse, rather than reduce or eliminate, rating errors that arise from knowledge of a ratee's prior performance. Implications for further understanding and reducing assimilation and contrast effects are discussed.     

线索提示对物理问题表征的影响

考察了复杂物理问题解决中图形线索提示和参数线索提示的作用,结果表明,在难度较大的物理问题中,单一的图形线索提示未给学生带来太大的帮助,而参数线索提示有助于学生搜索到有效算子.讨论了专家在物理问题解决过程中的特点,认为专家的优势之一在于比新手更善于发现关键的线索,能够根据关键线索找到合适的原理来完整地表征问题.