A Developmental Analysis of Performance Ratings

In this study, we examined the performance appraisal process from a developmental perspective. Components of cognitive structure, measured in terms of differentiation and hierarchic integration, were obtained from 45 American students representing three different academic levels at three times during a 15-week semester. These students' performance evaluations (ratings) of their psychology instructors were collected during the first and final sessions. The results indicated that although both differentiation and hierarchic integration seemed to change systematically, even over a short period of time, there was no corresponding change in the halo inherent in the raters' ratings. We present possible explanations for the failure of developmental changes in cognitive structure to be reflected in the halo inherent in the performance ratings, as would be predicted by Schneier's (1977) cognitive compatibility hypothesis. Increased emphasis on developmental aspects of the performance appraisal process is recommended.     

The Effects of Performance Extremities on Ratings of Dynamic Performance

The impact of performance extremities (peaks and troughs) on performance ratings was unexamined. Based on judgment and decision-making theories, we hypothesized that performance extremities would exert an incremental (beyond performance mean and trend) impact on ratings and that the impact of troughs would exceed that of peaks. We also hypothesized that extremities would exert a greater impact when performance trends were in the opposite direction and when performance information was presented in a graphical rather than tabular display format. We tested these hypotheses via a policy-capturing study in which participants rated employee performance profiles across which extremities, trends, and mean levels were manipulated. The results consistently indicated that performance troughs, but not performance peaks, influenced performance ratings in expected ways.     

Effects of Performance Context on Processing Speed and Performance Ratings

The present study demonstrated that the presence of evaluatively polarized context performances not only produces contrast and halo effects on judgments of a target performance, but also causes judgments to be made much faster. Processing speed and positive halo were highly correlated, supporting the notion that halo in performance ratings results from raters' recall and use of a single, general impression. Furthermore, regression analyses demonstrated that processing speed mediates the relationship between context and halo. The relationship between these findings, halo, processing speed, and general impressions, as well as implications for performance appraisals, are discussed.     

The influence of Ratee Gender on Ratings of Organizational Citizenship Behavior1

Two studies were conducted to examine discrepancies in the evaluation of men and women regarding the performance of organizational citizenship behavior (OCB). In Study 1, base‐rate differences in the perceived frequency and value of citizenship behaviors performed by males and females were investigated. A gender by job type interaction was found indicating that women were perceived to engage in OCB more frequently than were men in gender‐neutral and male‐typed jobs. No gender differences were found regarding the value associated with citizenship behaviors. In Study 2, undergraduates rated videotaped male and female instructors who exhibited different levels of OCB. Results revealed a gender by OCB interaction such that more accurate behavioral observations were made when observing males exhibiting OCB and females exhibiting no OCB than when observing males who did not exhibit OCB and females who did exhibit OCB. No gender by OCB interactions were found with regard to ratings of overall performance evaluation or reward recommendations.     

Consideration Set Overvaluation: When Impossibly Favorable Ratings of a Set of Brands Are Observed

Consumers often focus on the characteristics of one brand and consider insufficiently the characteristics of other brands. Singular evaluation processes often result in judgments of a focal brand that are more extreme than warranted. This research investigates the extent to which this extremity effect generalizes to a consideration set consisting of multiple brands. Three experiments show that a set of favorably regarded brands is perceived as more favorable than would be possible if the salience manipulation had no effect. In Experiment 1, participants overestimated the likelihood that the best brand in the market was included in the presented set. In Experiment 2, the average rank assigned to the brands in the presented set was inflated. Extreme judgments were observed regardless of whether the focal set of brands was offered by a specialty store with a high‐quality reputation or by a general merchandise store with a low‐quality reputation. In Experiment 3, extreme intentions to buy now from the focal store were reduced when a comparative (vs. singular) judgment task was performed, but only when the need for cognitive closure was low as opposed to high. The results suggest that singular evaluation processes lead to consideration set overvaluation regardless of which particular favorably regarded brands happen to be included in the set.     

Factors Influencing the Inflation of Task Performance Ratings for Workers With Disabilities and Contextual Performance Ratings for Their Coworkers

Contrary to many myths regarding their potential performance, workers with disabilities generally receive performance ratings similar to their nondisabled coworkers. However, their ratings possibly may be inflated above their actual performance levels. In a laboratory experiment, we constrained the performance of a worker with a disability to an extremely low level and assessed the effects of helping behavior, presence and type of disability in the worker, and the perceived attributions of controllability of the disability on task performance ratings for the workers with disabilities and on contextual performance ratings for their coworkers. We found that the perception that a worker is not responsible for the onset of a disability and that having any disability, regardless of type, artificially inflates task performance ratings. We also found inflated contextual performance ratings for coworkers who work with persons who are perceived as being responsible for the onset of their own disability.     

A Social Relations Analysis of Team Performance Ratings

The current study used a Social Relations Model to analyze self and peer ratings to explore the dynamics of team member perceptions and performance ratings. The results from 29 organizational teams who completed performance ratings of themselves and team members indicated that the most rating variance was attributed to the relationship component, followed by the ratee component, followed by the rater component. Among other findings, the results indicated that self‐ratings were related to how one rates, and is rated by, others; that there were high levels of reciprocity between peers for dimensions that were interpersonal in nature; and that raters tended to evaluate others within, but not necessarily across, dimensions similarly.     

Explaining the Weak Relationship Between Job Performance and Ratings of Job Performance

Ratings of job performance are widely viewed as poor measures of job performance. Three models of the performance–performance rating relationship offer very different explanations and solutions for this seemingly weak relationship. One-factor models suggest that measurement error is the main difference between performance and performance ratings and they offer a simple solution—that is, the correction for attenuation. Multifactor models suggest that the effects of job performance on performance ratings are often masked by a range of systematic nonperformance factors that also influence these ratings. These models suggest isolating and dampening the effects of these nonperformance factors. Mediated models suggest that intentional distortions are a key reason that ratings often fail to reflect ratee performance. These models suggest that raters must be given both the tools and the incentive to perform well as measurement instruments and that systematic efforts to remove the negative consequences of giving honest performance ratings are needed if we hope to use performance ratings as serious measures of job performance.     

When Intertemporal Rewards Are Hedonic,Larger Units of Wait Time Boost Patience

In intertemporal choices between smaller‐sooner (SS) and larger‐later (LL) rewards, five studies reveal that patience for the LL option is influenced by an interactive effect of the measurement units used to express wait time (large vs. small) and the type of the reward (hedonic vs. utilitarian). Specifically, larger time units boost patience, but more so when rewards are hedonic rather than utilitarian. In line with the numerosity heuristic, the effect of time units on patience is mediated by larger time units shrinking wait time perception. This effect arises because hedonic (vs. utilitarian) rewards promote a reliance on heuristics rather than systematic calculations. Therefore, a more calculative mindset diminishes the effect of units even for hedonic rewards and eliminates the hedonic‐utilitarian asymmetry. These results contribute to research on numerosity, intertemporal choice, and hedonic‐utilitarian differences, and offer a simple tool for practitioners to influence patience.     

Managerial Experience and the Measurement Equivalence of Performance Ratings

ABSTRACT: Establishing the measurement equivalence of instruments is a prerequisite to making meaningful comparisons between individuals or within individuals over time. Whereas previous research has investigated the effects of rater characteristics on the measurement equivalence of performance ratings, the current study investigated a ratee characteristic—ratee job experience. Using confirmatory factor analysis and item response theory methods with replication, the measurement equivalence of supervisor ratings of 7,200 managers with differing levels of managerial experience was assessed. Overall, results indicated a high degree of measurement equivalence suggesting that meaningful comparison may be made across ratees with different levels of job experience.     

Reducing the Effects of Performance Expectations on Behavioral Ratings

In this study, we develop and test two strategies for reducing the effects of performance expectations on behavioral ratings. A 3 × 3 experimental design (N = 169), manipulating preobservation performance cues (positive, negative, or none) and the type of intervention (halo error training, structured recall memory, or none), was conducted. The results of this study indicate that both interventions reduced the effects of performance expectations on behavioral ratings. However, analyses of rating accuracy and measures from both recognition memory and recall memory tests suggest that the structured recall memory intervention has distinct advantages. These analyses indicate that the structured recall intervention can reduce raters' reliance on heuristics and increase the correspondence between raters' memory and their subsequent ratings.     

Replication and Extension of Models of Supervisory Job Performance Ratings

This study tested causal models of job knowledge, job proficiency, and supervisor ratings in a sample (N = 838) of U.S. Air Force enlisted personnel. Results indicated that (a) effects of ability and experience were linear, not interactive; (b) different conceptualizations of "experience" play somewhat different causal roles in the determination of job knowledge, job proficiency, and supervisory ratings; (c) general support for the mediational roles of job knowledge and job proficiency; and (d) supervisory ratings reflect both technical (task-based) and motivational (contextual) aspects of performance.     

The Performance Appraisal Milieu: A Multilevel Analysis of Context Effects in Performance Ratings

Purpose

The purpose of this study was to take an inductive approach in examining the extent to which organizational contexts represent significant sources of variance in supervisor performance ratings, and to explore various factors that may explain contextual rating variability.

Design/Methodology/Approach

Using archival field performance rating data from a large state law enforcement organization, we used a multilevel modeling approach to partition the variance in ratings due to ratees, raters, as well as rating contexts.

Findings

Results suggest that much of what may often be interpreted as idiosyncratic rater variance, may actually reflect systematic rating variability across contexts. In addition, performance-related and non-performance factors including contextual rating tendencies accounted for significant rating variability.

Implications

Supervisor ratings represent the most common approach for measuring job performance, and understanding the nature and sources of rating variability is important for research and practice. Given the many uses of performance rating data, our findings suggest that continuing to identify contextual sources of variability is particularly important for addressing criterion problems, and improving ratings as a form of performance measurement.

Originality/Value

Numerous performance appraisal models suggest the importance of context; however, previous research had not partitioned the variance in supervisor ratings due to omnibus context effects in organizational settings. The use of a multilevel modeling approach allowed the examination of contextual influences, while controlling for ratee and rater characteristics.

     

The Influence of Self-Ratings versus Peer Ratings on Supervisors'' Performance Judgments

This study investigated the extent to which supervisors use feedback from outside sources in making performance judgments. A simulation was conducted in which participants with organizational supervisory experience made an initial performance judgment about a profiled employee. Participants then received additional information that was discrepant from their initial judgment (positive or negative) from one of two sources (the profiled employee himself or one of his peers). The direction of the discrepant information and its source interacted in determining final ratings, such that, participants were more likely to use discrepant information to alter their performance judgments in a consistent direction when the source was a peer than when the source was the employee himself. Furthermore, participants' opinions about the usefulness of peer information for performance judgments moderated this interaction. Specifically, participants who believed that information from an employee's peers was useful, were more likely to use discrepant information provided by a peer when making final performance judgments than were participants who did not believe that information from an employee's peers was useful.     

Understanding and Removing the Effects of Performance Cues on Behavioral Ratings

Previous research has demonstrated that performance information (e.g., prior supervisor ratings) can bias behavioral ratings. However, research has not fully explored the effects of performance cues on raters' memory. In addition, no studies have attempted to eliminate this performance cue effect. This study addressed these deficiencies by collecting both free recall and recognition memory measures while testing an unstructured free recall intervention. Results indicate that performance cues do affect the recall of performance relevant behaviors from memory. Contrary to expectations, free recall did not prove to be an effective intervention. Implications of these findings for future attempts to remove the performance cue effect are discussed.     

Predicting Supervisor Ratings of Job Performance Using the NEO Personality Inventory

The resurging interest in identifying personality predictors of job performance has led researchers to use the five-factor model of personality as an important conceptual tool. Two recent meta-analyses illustrated the potential predictive power of the model within an employment context. In this study, we evaluated the relations between a well-validated self-report measure of the five-factor model and supervisor ratings of performance. Fifty-two women and 159 men completed the NEO Personality Inventory (Costa & McCrae, 1985) and were independently rated on various performance dimensions by their supervisors. Scores on the Conscientiousness scales correlated with all the performance ratings over this diverse number of occupational groups. Low Neuroticism and high Extraversion scores also predicted high performance. Low straightforwardness (a facet of agreeableness) was associated with ability to accomplish work-related goals and to adapt to changing work conditions. The results support the utility of the five-factor model as a predictor of job performance.     

A Meta-Analysis of the Relationship Between Rater Liking and Performance Ratings

This meta-analysis reviewed the magnitude and moderators of the relationship between rater liking and performance ratings. The results revealed substantial overlap between rater liking and performance ratings (ρ = .77). Although this relationship is often interpreted as indicative of bias, we review studies that indicate that to some extent the relationship between liking and performance ratings potentially reflects “true” differences in ratee performance. Moderator analyses indicated that the relationship between liking and performance ratings was weaker for ratings of organizational citizenship behaviors, ratings made by peer raters, ratings in nonsales jobs, and ratings made for development; however, the relationship was strong across moderator levels, underscoring the robustness of this relationship. Implications for the interpretation of performance ratings are discussed.