A Developmental Analysis of Performance Ratings

In this study, we examined the performance appraisal process from a developmental perspective. Components of cognitive structure, measured in terms of differentiation and hierarchic integration, were obtained from 45 American students representing three different academic levels at three times during a 15-week semester. These students' performance evaluations (ratings) of their psychology instructors were collected during the first and final sessions. The results indicated that although both differentiation and hierarchic integration seemed to change systematically, even over a short period of time, there was no corresponding change in the halo inherent in the raters' ratings. We present possible explanations for the failure of developmental changes in cognitive structure to be reflected in the halo inherent in the performance ratings, as would be predicted by Schneier's (1977) cognitive compatibility hypothesis. Increased emphasis on developmental aspects of the performance appraisal process is recommended.     

Effects of Performance Context on Processing Speed and Performance Ratings

The present study demonstrated that the presence of evaluatively polarized context performances not only produces contrast and halo effects on judgments of a target performance, but also causes judgments to be made much faster. Processing speed and positive halo were highly correlated, supporting the notion that halo in performance ratings results from raters' recall and use of a single, general impression. Furthermore, regression analyses demonstrated that processing speed mediates the relationship between context and halo. The relationship between these findings, halo, processing speed, and general impressions, as well as implications for performance appraisals, are discussed.     

Factors Influencing the Inflation of Task Performance Ratings for Workers With Disabilities and Contextual Performance Ratings for Their Coworkers

Contrary to many myths regarding their potential performance, workers with disabilities generally receive performance ratings similar to their nondisabled coworkers. However, their ratings possibly may be inflated above their actual performance levels. In a laboratory experiment, we constrained the performance of a worker with a disability to an extremely low level and assessed the effects of helping behavior, presence and type of disability in the worker, and the perceived attributions of controllability of the disability on task performance ratings for the workers with disabilities and on contextual performance ratings for their coworkers. We found that the perception that a worker is not responsible for the onset of a disability and that having any disability, regardless of type, artificially inflates task performance ratings. We also found inflated contextual performance ratings for coworkers who work with persons who are perceived as being responsible for the onset of their own disability.     

Consideration Set Overvaluation: When Impossibly Favorable Ratings of a Set of Brands Are Observed

Consumers often focus on the characteristics of one brand and consider insufficiently the characteristics of other brands. Singular evaluation processes often result in judgments of a focal brand that are more extreme than warranted. This research investigates the extent to which this extremity effect generalizes to a consideration set consisting of multiple brands. Three experiments show that a set of favorably regarded brands is perceived as more favorable than would be possible if the salience manipulation had no effect. In Experiment 1, participants overestimated the likelihood that the best brand in the market was included in the presented set. In Experiment 2, the average rank assigned to the brands in the presented set was inflated. Extreme judgments were observed regardless of whether the focal set of brands was offered by a specialty store with a high‐quality reputation or by a general merchandise store with a low‐quality reputation. In Experiment 3, extreme intentions to buy now from the focal store were reduced when a comparative (vs. singular) judgment task was performed, but only when the need for cognitive closure was low as opposed to high. The results suggest that singular evaluation processes lead to consideration set overvaluation regardless of which particular favorably regarded brands happen to be included in the set.     

Explaining the Weak Relationship Between Job Performance and Ratings of Job Performance

Ratings of job performance are widely viewed as poor measures of job performance. Three models of the performance–performance rating relationship offer very different explanations and solutions for this seemingly weak relationship. One-factor models suggest that measurement error is the main difference between performance and performance ratings and they offer a simple solution—that is, the correction for attenuation. Multifactor models suggest that the effects of job performance on performance ratings are often masked by a range of systematic nonperformance factors that also influence these ratings. These models suggest isolating and dampening the effects of these nonperformance factors. Mediated models suggest that intentional distortions are a key reason that ratings often fail to reflect ratee performance. These models suggest that raters must be given both the tools and the incentive to perform well as measurement instruments and that systematic efforts to remove the negative consequences of giving honest performance ratings are needed if we hope to use performance ratings as serious measures of job performance.     

Reducing the Effects of Performance Expectations on Behavioral Ratings

In this study, we develop and test two strategies for reducing the effects of performance expectations on behavioral ratings. A 3 × 3 experimental design (N = 169), manipulating preobservation performance cues (positive, negative, or none) and the type of intervention (halo error training, structured recall memory, or none), was conducted. The results of this study indicate that both interventions reduced the effects of performance expectations on behavioral ratings. However, analyses of rating accuracy and measures from both recognition memory and recall memory tests suggest that the structured recall memory intervention has distinct advantages. These analyses indicate that the structured recall intervention can reduce raters' reliance on heuristics and increase the correspondence between raters' memory and their subsequent ratings.     

Managerial Experience and the Measurement Equivalence of Performance Ratings

ABSTRACT: Establishing the measurement equivalence of instruments is a prerequisite to making meaningful comparisons between individuals or within individuals over time. Whereas previous research has investigated the effects of rater characteristics on the measurement equivalence of performance ratings, the current study investigated a ratee characteristic—ratee job experience. Using confirmatory factor analysis and item response theory methods with replication, the measurement equivalence of supervisor ratings of 7,200 managers with differing levels of managerial experience was assessed. Overall, results indicated a high degree of measurement equivalence suggesting that meaningful comparison may be made across ratees with different levels of job experience.     

Replication and Extension of Models of Supervisory Job Performance Ratings

This study tested causal models of job knowledge, job proficiency, and supervisor ratings in a sample (N = 838) of U.S. Air Force enlisted personnel. Results indicated that (a) effects of ability and experience were linear, not interactive; (b) different conceptualizations of "experience" play somewhat different causal roles in the determination of job knowledge, job proficiency, and supervisory ratings; (c) general support for the mediational roles of job knowledge and job proficiency; and (d) supervisory ratings reflect both technical (task-based) and motivational (contextual) aspects of performance.     

The Performance Appraisal Milieu: A Multilevel Analysis of Context Effects in Performance Ratings

Purpose

The purpose of this study was to take an inductive approach in examining the extent to which organizational contexts represent significant sources of variance in supervisor performance ratings, and to explore various factors that may explain contextual rating variability.

Design/Methodology/Approach

Using archival field performance rating data from a large state law enforcement organization, we used a multilevel modeling approach to partition the variance in ratings due to ratees, raters, as well as rating contexts.

Findings

Results suggest that much of what may often be interpreted as idiosyncratic rater variance, may actually reflect systematic rating variability across contexts. In addition, performance-related and non-performance factors including contextual rating tendencies accounted for significant rating variability.

Implications

Supervisor ratings represent the most common approach for measuring job performance, and understanding the nature and sources of rating variability is important for research and practice. Given the many uses of performance rating data, our findings suggest that continuing to identify contextual sources of variability is particularly important for addressing criterion problems, and improving ratings as a form of performance measurement.

Originality/Value

Numerous performance appraisal models suggest the importance of context; however, previous research had not partitioned the variance in supervisor ratings due to omnibus context effects in organizational settings. The use of a multilevel modeling approach allowed the examination of contextual influences, while controlling for ratee and rater characteristics.

     

Predicting Supervisor Ratings of Job Performance Using the NEO Personality Inventory

The resurging interest in identifying personality predictors of job performance has led researchers to use the five-factor model of personality as an important conceptual tool. Two recent meta-analyses illustrated the potential predictive power of the model within an employment context. In this study, we evaluated the relations between a well-validated self-report measure of the five-factor model and supervisor ratings of performance. Fifty-two women and 159 men completed the NEO Personality Inventory (Costa & McCrae, 1985) and were independently rated on various performance dimensions by their supervisors. Scores on the Conscientiousness scales correlated with all the performance ratings over this diverse number of occupational groups. Low Neuroticism and high Extraversion scores also predicted high performance. Low straightforwardness (a facet of agreeableness) was associated with ability to accomplish work-related goals and to adapt to changing work conditions. The results support the utility of the five-factor model as a predictor of job performance.     

Understanding and Removing the Effects of Performance Cues on Behavioral Ratings

Previous research has demonstrated that performance information (e.g., prior supervisor ratings) can bias behavioral ratings. However, research has not fully explored the effects of performance cues on raters' memory. In addition, no studies have attempted to eliminate this performance cue effect. This study addressed these deficiencies by collecting both free recall and recognition memory measures while testing an unstructured free recall intervention. Results indicate that performance cues do affect the recall of performance relevant behaviors from memory. Contrary to expectations, free recall did not prove to be an effective intervention. Implications of these findings for future attempts to remove the performance cue effect are discussed.     

Cultural Values and Performance Appraisal: Assessing the Effects of Rater Self-Construal on Performance Ratings

Much of the prior research investigating the influence of cultural values on performance ratings has focused either on conducting cross-national comparisons among raters or using cultural level individualism/collectivism scales to measure the effects of cultural values on performance ratings. Recent research has shown that there is considerable within country variation in cultural values, i.e. people in one country can be more individualistic or collectivistic in nature. Taking the latter perspective, the present study used Markus and Kitayama's (1991) conceptualization of independent and interdependent self-construals as measures of individual variations in cultural values to investigate within culture variations in performance ratings. Results suggest that rater self-construal has a significant influence on overall performance evaluations; specifically, raters with a highly interdependent self-construal tend to show a preference for interdependent ratees, whereas raters high on independent self-construal do not show a preference for specific type of ratees when making overall performance evaluations. Although rater self-construal significantly influenced overall performance evaluations, no such effects were observed for specific dimension ratings. Implications of these results for performance appraisal research and practice are discussed.     

A Meta-Analysis of the Relationship Between Rater Liking and Performance Ratings

This meta-analysis reviewed the magnitude and moderators of the relationship between rater liking and performance ratings. The results revealed substantial overlap between rater liking and performance ratings (ρ = .77). Although this relationship is often interpreted as indicative of bias, we review studies that indicate that to some extent the relationship between liking and performance ratings potentially reflects “true” differences in ratee performance. Moderator analyses indicated that the relationship between liking and performance ratings was weaker for ratings of organizational citizenship behaviors, ratings made by peer raters, ratings in nonsales jobs, and ratings made for development; however, the relationship was strong across moderator levels, underscoring the robustness of this relationship. Implications for the interpretation of performance ratings are discussed.     

绩效评价的若干问题

通过对绩效的内涵、绩效评价标准的确定、评量指标体系的构建以及多样性评量指标的整合意义与方法等四个绩效评价的最基本问题进行反思 ,提出如下观点 :(1 )对绩效进行考量时必须同时考虑时间、方式和结果三个维度。 (2 )绩效评价标准的确定应先于具体评量指标的选择 ,且应保持一定的稳定性 ,不可随意增删 ;(3 )评量指标体系的构建应本着贵精不贵多、贵明确不贵模糊、贵敏感不贵迟钝 ,贵关键不贵空泛的原则进行 ;(4)多样性评量指标是否整合需根据具体的需要而定 ,整合时要考虑评量指标本身的权重差异以及不同评价者由于对标准尺度掌握的松紧不一和评判角度不同所带来的对最终评价结果的影响。     

Inaccurate self-knowledge formation as a result of automatic behavior

Four studies tested a post-priming misattribution process whereby a primed goal automatically influences people's behavior, but because people are unaware of that influence, they misattribute their behavior to some other internal state. People who were primed with a goal were more likely to choose an activity that was relevant to that goal, but did not recognize that the prime had influenced their choices. Instead, people used more accessible and plausible reasons to explain their behavior. The goals were seeking romantic interaction (Studies 1 and 2), helping (Study 3) and earning money (Study 4). People made choices related to these goals but misattributed the choices to temporary preferences (Studies 1 and 3) and more permanent dispositions (Studies 2 and 4). The misattribution had downstream effects, leading to choice behavior consistent with the erroneous self-knowledge. We suggest that automatic behavior can lead to a confabulated self-knowledge with behavioral consequences.     

Generalizability Theory as Construct-Related Evidence of the Validity of Job Performance Ratings

Investigations of the construct-related evidence of the validity of performance ratings have been rare, perhaps because researchers are dissuaded by the con- siderable amount of evidence needed to show construct validity (Landy, 1986). It is argued that generalizability (G) theory (Cronbach, Gleser, Nanda, & Rajaratnam, 1972) is well-suited to investigations of construct-related evi- dence of validity because a single generalizability investigation may provide multiple inferences of validity. G theory permits the researcher to partition observed score variance into universe (true) score variance and multiple, distinct estimates of error variance. G theory was illustrated through the anal- ysis of proficiency ratings of 256 Air Force jet engine mechanics. Mechanics were rated on three different rating forms by themselves, peers, and supervi- sors. Interpretation of G study variance components revealed suitable evi- dence of construct validity. Ratings within sources were reliable. Proficiency ratings showed strong convergence over rating forms, though not over rating sources. Raters showed adequate discriminant validity across rating dimen- sions. The expectation of convergence over sources was further questioned.     

Validity of Selection/ Classification Variables for Performance in Three Navy Ratings

This study examined speed of advancement and score on a job knowledge test as job performance criteria in the validation of selection and classification variables for three Navy ratings. Longitudinal samples spanning a 4-year period were drawn for men in radioman, machinist's mate, and hospital corpsman ratings who, within their first enlistment, advanced to E-4 by means of the Navy competitive advancement system. Validity coefficients for Navy enlisted selection and classification variables (Armed Services Vocational Aptitude Battery [ASVAB] and education) were computed for personnel in each rating, subgrouped by entry pay grade. Validity coefficients for the job knowledge test were high by historical standards, and the effects were highest for the subgroups with the highest selection standards. Validity coefficients for the advancement criterion were much lower. Advancement, the criterion that appeared to have the greater face validity, was not as good a measure of job performance as the job knowledge test. The criteria were predicted much better by mental ability than by level of education variables. Personnel who enlisted at E-2 and E-3 pay grades spent substantially more time working at the journeyman performance level during their first enlistment than did personnel who enlisted at E-1.