ON THE INTERCHANGEABILITY OF OBJECTIVE AND SUBJECTIVE MEASURES OF EMPLOYEE PERFORMANCE: A META-ANALYSIS

A meta-analysis of studies containing both objective and subjective ratings of employee performance resulted in a corrected mean correlation of .389. This value, although significantly greater than zero, indicates that objective and subjective performance measures should not be used interchangeably. Moreover, in no moderator subgroup examined did the correlation suggest convergent validity. After discussing issues related to resolving the previous anomalies of primary and meta-analytic results, a secondary analysis suggested that objective and subjective measures of the same construct at the same level may be used inter-changeably. The secondary analysis, however, was based on a very limited sample. Future research should address the appropriate dimensionality of employee performance.     

SELF- VERSUS OTHERS' RATINGS AS PREDICTORS OF ASSESSMENT CENTER RATINGS: VALIDATION EVIDENCE FOR 360-DEGREE FEEDBACK PROGRAMS

Although 360-degree feedback programs are rapidly increasing in popularity, few studies have examined how well ratings from these programs predict an independent criterion. This study had 2 main aims: First, to examine the validity of ratings from a 360-degree feedback program using assessment center ratings as an independent criterion and to determine which source (i.e., self, supervisor, peers, or subordinates) provided the most valid predictor of the criterion measure of competency. Second, to better understand the relationship between self-observer discrepancies and an independent criterion. The average of supervisor, peer, and subordinate ratings predicted performance on the assessment center, as did the supervisor ratings alone. The self-ratings were negatively and nonlinearly related to performance with some of those who gave themselves the highest ratings having the lowest performance on the assessment center. Supervisor ratings successfully discriminated between overestimators but were not as successful at discriminating underestimators, suggesting that more modest feedback recipients might be underrated by their supervisors. Peers overestimated performance for poor performers. Explanations of the results and the implications for the use of self-ratings in evaluations, the design of feedback reports, and the use of 360-degree feedback programs for involving and empowering staff are discussed.     

THE RELATIONSHIP BETWEEN SUPERVISORY RATINGS AND RESULTS-ORIENTED MEASURES OF PERFORMANCE: A META-ANALYSIS

A meta-analysis was conducted with 23 studies to assess the relationship between supervisory ratings and results-oriented measures of performance. As hypothesized, the corrected mean correlation was higher when a relative (versus absolute) rating format was used and when a composite (versus overall) rating method was used. These differences did not, however, account for all of the remaining variance around the relationship between ratings and results. Suggestions are offered for the direction of future research and practice.     

A COMPARISON OF CRITERIA FOR TEST VALIDATION: A META-ANALYTIC INVESTIGATION

Meta-analyses of validity coefficients from tests of clerical abilities for five criteria—supervisor ratings, supervisor rankings, work samples, production quantity, and production quality—were conducted, and the resulting expected true validities were compared. Ratings, rankings, work samples, and production quantity all resulted in high test validities. Validities resulting from ratings and quantity-of-production criteria were highly similar across tests. Validities resulting from rankings and work samples were on the average higher than those from ratings and quantity of production. The fifth criterion, quality of production, had low predictability and did not generalize across situations.     

A FIELD STUDY OF PERFORMANCE APPRAISAL PURPOSE: RESEARCH- VERSUS ADMINISTRATIVE- BASED RATINGS

Many researchers have discussed the theoretical and practical importance of rating purpose. Nevertheless, the body of empirical studies, the majority of which were conducted in a laboratory setting, focus on leniency. There has been little research on other effects of rating purpose. The present study examines 223 ratees in a field setting for whom there were both administrative-based performance appraisal ratings (which were actually used for personnel decisions) and research-based performance appraisal ratings (obtained for a validation study). Two of the hypotheses were supported; administrative ratings were more lenient than research-based ratings. The administrative-based ratings demonstrated a statistically significant relationship with ratee seniority, while the research-based ratings did not. There was mixed support for a third hypothesis: Research ratings were significantly correlated with a predictor, while the administrative ratings were not. The difference between the validity coefficients, however, was not significant. Contrary to the hypothesis, the rank order between administrative-based and research-based ratings was relatively high ( r = 33).     

PREDICTORS OF OBJECTIVE AND SUBJECTIVE CAREER SUCCESS: A META-ANALYSIS

Using the contest- and sponsored-mobility perspectives as theoretical guides, this meta-analysis reviewed 4 categories of predictors of objective and subjective career success: human capital, organizational sponsorship, sociodemographic status, and stable individual differences. Salary level and promotion served as dependent measures of objective career success, and subjective career success was represented by career satisfaction. Results demonstrated that both objective and subjective career success were related to a wide range of predictors. As a group, human capital and sociodemographic predictors generally displayed stronger relationships with objective career success, and organizational sponsorship and stable individual differences were generally more strongly related to subjective career success. Gender and time (date of the study) moderated several of the relationships examined.     

ON THE VALIDITY OF SUBJECTIVE MEASURES OF COMPANY PERFORMANCE

Subjective measures of company performance are widely used in research and typically are interpreted as equivalent to objective measures. Yet, the assumption of equivalence is open to challenge. We compared the use of both types of measure in 3 separate samples. Findings were consistent in showing that: (a) subjective and objective measures of company performance were positively associated (convergent validity); (b) those relationships were stronger than those between measures of differing aspects of performance using the same method (discriminant validity); and (c) the relationships of subjective and objective company performance measures with a range of independent variables were equivalent (construct validity).     

OBJECTIVE AND SUBJECTIVE PERFORMANCE AS INFLUENCED BY DRUG-INDUCED VARIATIONS IN ACTIVATION LEVEL

Performance was measured by seven sensorimotor and perceptual tests given to 30 students in six successive trials, spread over five hours, before and after the intake of either 15 mg dexamphetamine, zoo mg Pentobarbitone, or a placebo. Subjective performance, level of aspiration, wakefulness, and mood were measured by a scaling method. A close agreement was found between the amount of objective and subjective improvement following dexamphetamine, whereas the objective impairment induced by Pentobarbitone was not reflected in the subjective measures. These results were examined in relation to the other indices of activation.     

THE INFLUENCE OF UPWARD FEEDBACK ON SELF- AND FOLLOWER RATINGS OF LEADERSHIP

The impact of upward feedback (followers' perceptions of leadership provided to leaders) on leaders' self-evaluations and followers' subsequent ratings of leadership was assessed in a field setting. Subjects were 978 student leaders and their 1,232 followers. Results indicated that overall, leaders' behaviors as rated by followers improved after feedback. Leaders' self-evaluations following feedback became more similar to the evaluations provided by followers. Interestingly, when leaders were grouped according to whether feedback was positive, neutral, or negative based on agreement between self- and follower ratings, differences between groups in post-feedback self-evaluations and follower ratings emerged. Self-evaluations for leaders receiving negative feedback (high self- relative to followers' ratings) went down, while self-evaluations for leaders receiving positive feedback (low self- relative to follower's ratings) went up. Follower ratings of leaders who received negative feedback improved following feedback to leaders, while there was no change in follower ratings for those receiving positive feedback. Implications for using upward feedback in organizations are discussed.     

EFFECTS OF NITROUS OXIDE ON SUBJECTIVE AND OBJECTIVE VARIABLES

Effects of nitrous oxide in different subanaesthetic concentrations were studied in twelve subjects. Subjective scales of intoxication and of eight other mood variables were obtained by the method of magnitude estimation. Objective performance was examined by tests on memory and arithmetic. For most variables the dose-response curves obtained showed progressively larger effects with increasing dose-levels. At concentrations below 20 per cent the changes were, on the whole, very slight. The results are compared with similar data from an alcohol experiment.     

TIME RELATIONS OF OBJECTIVE AND SUBJECTIVE REACTIONS TO D-AMPHETAMINE AND PENTOBARBITONE

Successive measurements of pulse rate, blood pressure, objective and subjective reaction speed, and six mood variables were made on 12 healthy subjects during 135 min. periods following the intake of either 15 mg d-amphetamine, 200 mg Pentobarbitone, or a placebo. It was shown that (1) the two drugs had essentially opposite effects on all objective and most subjective variables; (2) after d-amphetamine subjective effects appeared earlier and reached their maximum more rapidly than objective effects; (3) both types of reaction were less pronounced after Pentobarbitone than after d-amphetamine under the present dose conditions; and that (4) after Pentobarbitone maximum objective effects were attained at an earlier point in time.     

THE EFFECTS OF PERCEIVED SYSTEM KNOWLEDGE ON THE AGREEMENT BETWEEN SELF-RATINGS AND SUPERVISOR RATINGS

The viability of self-assessment in the performance appraisal process has been the focus of recent work. Although some contend that self-appraisals can be beneficial, the consistent finding of lenient and inaccurate self-ratings has led others to discount and doubt their usefulness. Recent research (Farh & Dobbins, 1989a,b) has investigated the influence of a variety of factors in improving the accuracy of self-ratings. The present study was conducted to examine the effect of perceived system knowledge (PSK) on the discrepancy between self- and supervisor ratings. Seventy-three employees of two banking institutions participated by completing a questionnaire on their perceived knowledge of the performance appraisal system. The results of moderated regression analyses suggest that the level of knowledge did in fact influence the agreement among subordinates and supervisors. Self-ratings were more congruent with supervisors'ratings when the subordinates reported high levels of perceived system knowledge. Implications of these findings for organizations and areas for future research are discussed.     

A LONGITUDINAL EXAMINATION OF RATER AND RATEE EFFECTS IN PERFORMANCE RATINGS

The consistency and loci of leniency, halo, and range restriction effects in performance ratings were investigated in a longitudinal study. Ratings were provided by approximately 90 supervisors in a metropolitan police department, who rated approximately 350 police-rank subordinates on five occasions over a three and one-half year period. Rating effects were computed separately as rater-and ratee-based statistics, and intercorrelated among the five rating periods. The nature of the data set made it possible to hold either raters or ratees constant for each analysis, thus permitting inferences regarding the sources of reliable variance in effects as due to raters or ratees. It was concluded that reliable variance in mean ratings is partly attributable to ratees, but mainly introduced by raters. Reliable halo variance is attributable to raters, and range restriction is a product of stable group performance variability within intact ratee groups. Implications of these results for future rating process research are discussed.     

PERFORMANCE APPRAISAL BEHAVIORS: SUPERVISOR PERCEPTIONS AND SUBORDINATE REACTIONS

This study examined supervisor perceptions and subordinate reactions to formal performance-appraisal reviews. The performance-appraisal behaviors of supervisors and the reactions of their subordinates were studied in a sample of university employees. A factor analysis revealed that there were three dimensions of formal performance appraisals: two developmental dimensions (being supportive; emphasizing performance improvement) and one administrative dimension (discussing pay and advancement). Regression analyses suggested that supervisors supported highly rated individuals and stressed improvement efforts for poor performers. After controlling for the level of previous performance ratings, results indicated that support in the appraisal review was associated with higher levels of employee motivation, while discussing pay and advancement was associated with higher levels of employee satisfaction. Unfortunately, improvement efforts by the supervisors did not influence job performance one year later.     

STRESS AND PERFORMANCE: EFFECTS OF SUBJECTIVE WORK LOAD AND TIME URGENCY1

Subjective work load, time urgency, and other stress/motivation variables were measured for management personnel taking a demanding problem-solving exam at the end of a two-week training course. Comparing measures of precourse ability and final exam performance, the primary findings were that the corrected performance scores had strong negative linear (not inverted-U) relations with both subjective work load and time urgency. General state anxiety and task involvement did not substantially relate to performance. The results are discussed in terms of the nature of this particular task and the predictions of various stress/performance theories. In problem solving or other tasks requiring novel responses, these data suggest that increases in psychological stresses like subjectively high work load and time urgency uniformly impair performance across the whole range of these variables.     

THE EFFECTS OF SELF-MONITORING AND SUPERVISOR FEEDBACK ON STAFF PERFORMANCE IN A RESIDENTIAL SETTING

We evaluated the effects of a self-monitoring procedure to increase staff on-task behavior and adherence to scheduled activities. Self-monitoring involved the use of activity cards that staff members completed and carried with them to assist in determining the activities for which they were responsible at any given time. Increases in both on-schedule and on-task behavior resulted. Supervisor feedback was subsequently added because some staff members did not maintain consistently high levels of performance. Generalization data indicated that staff members implemented the procedure during evening hours without specific programming. The advantages and limitations of using a self-monitoring procedure for improving performance of staff members in residential settings are discussed.     

DYNAMIC CRITERIA REVISITED: A LONGITUDINAL STUDY OF PERFORMANCE STABILITY AND PREDICTIVE VALIDITY

The concept of dynamic criteria has been the subject of a recent debate regarding both the definition and prevalence of the phenomenon (Austin, Humphreys, & Hulin, 1989; Barrett & Alexander, 1989; Barrett, Caldwell, & Alexander, 1985). The present paper questions the adequacy of the conceptual framework underlying the debate and provides data supporting a refined concept of dynamic criteria. The incidence and possible causes of change in relative performance were investigated using weekly performance data from 509 sewing machine operators. Analyses were conducted to determine the degree of performance consistency, potential moderators of consistency, and the stability of predictor-criteria relationships using multiple predictors and criteria. Results revealed a steady decline in performance stability coefficients as the interval between measures increased. This decay was evident regardless of employees' prior job experience, cognitive ability, or psychomotor ability. Analyses of predictive validity coefficients revealed temporal changes in validity for both objective and subjective criteria, but not in the expected direction. The validity of cognitive ability increased, the validity of psychomotor ability was stable, and that of prior job experience decreased over time. Implications for theory and research are discussed.     

RELIABILITY AND VALIDITY OF IN-BASKET PERFORMANCE MEASURES

The in-basket test is a frequently used assessment tool for predicting job performance and, to a lesser extent, as a technique for research and development in various human resource applications. While the in-basket test has obvious attractive features for application in the business setting, there is a great degree of uniqueness in each application and relatively little research focusing on critical issues involved in construction or evaluation. In this paper, the literature focusing on the psychometric properties of in-basket measures of performance are reviewed and evaluated. The compiled evidence provides only modest support of the usefulness of the in-basket test as a measurement tool.