COMPARISONS OF MANAGERIAL AND EMPLOYEE SATISFACTION WITH A PERFORMANCE APPRAISAL SYSTEM

Questionnaires were administered to managers and employees in a large multinational corporation to assess satisfaction with various aspects of a performance appraisal system. Comparisons of managers' and employees' satisfaction were made by conducting factor analyses for each sample. The results indicate moderate similarity between the two groups. However, two significant differences were revealed by the analysis. First, there is evidence that employees perceive certain aspects of the appraisal system in a global way, whereas managers differentiate among various components and see them as distinct entities. Second, the relative importance of the factors differs between the two groups. The largest portion of variance accounted for in the employee sample pertained to general satisfaction with the system whereas for managers it pertained to the types of ratings made on the appraisal form. The results are discussed in terms of the different perspectives managers and employees have in the appraisal process.     

A FIELD STUDY OF PERFORMANCE APPRAISAL PURPOSE: RESEARCH- VERSUS ADMINISTRATIVE- BASED RATINGS

Many researchers have discussed the theoretical and practical importance of rating purpose. Nevertheless, the body of empirical studies, the majority of which were conducted in a laboratory setting, focus on leniency. There has been little research on other effects of rating purpose. The present study examines 223 ratees in a field setting for whom there were both administrative-based performance appraisal ratings (which were actually used for personnel decisions) and research-based performance appraisal ratings (obtained for a validation study). Two of the hypotheses were supported; administrative ratings were more lenient than research-based ratings. The administrative-based ratings demonstrated a statistically significant relationship with ratee seniority, while the research-based ratings did not. There was mixed support for a third hypothesis: Research ratings were significantly correlated with a predictor, while the administrative ratings were not. The difference between the validity coefficients, however, was not significant. Contrary to the hypothesis, the rank order between administrative-based and research-based ratings was relatively high ( r = 33).     

REACTION OF EMPLOYEES TO PERFORMANCE APPRAISAL INTERVIEWS AS A FUNCTION OF THEIR PARTICIPATION IN RATING SCALE DEVELOPMENT

A substantial amount of research has been conducted on the performance appraisal feedback interview. The purpose of the present field study was to see whether employee involvement in the development of rating scales used in the feedback interview affected ratees' perceptions of the interview. Behaviorally anchored rating scales (BARS) were developed for five job families in a midwestem hospital. A participation group consisted of employees from each of the job families who were involved in all phases of BARS construction. A nonparticipation group involved an equal number of employees from each of the job families who were not involved in the BARS development. The results showed that participation in BARS construction led to favorable perceptions regarding the performance appraisal interview process as well as positive outcomes.     

PERFORMANCE APPRAISAL OF BEHAVIOR-BASED COMPETENCIES: A RELIABLE AND VALID PROCEDURE

A new performance appraisal system, developed for promotions in the Royal Canadian Mounted Police non-commissioned officer ranks, fairly differentiated among candidates. Members ( N = 6,571) illustrated their performance on core competencies with behavioral examples. Supervisors and then review boards used a BARS procedure to reliably rate performance. Both candidates and supervisors supported the system. The performance appraisal scores predicted career advancement in the organization.     

BEHAVIORAL OBSERVATION SCALES AND THE EVALUATION OF PERFORMANCE APPRAISAL EFFECTIVENESS

An earlier article by the present authors (Bernardin and Kane, 1980) pointed out a serious flaw in the conceptual basis of Behavioral Observation Scales (BOS). The present article explains this flaw in more detail and shows that its solution would make BOS indistinguishable from other methods already in existence. The focus then shifts from the conception of appraisal methods to their evaluation with the presentation of a discussion of the special problems of error capitalization that arise in the use of item analysis in the development of multi-item rating scales. This discussion proceeds to describe the correct approach to removing the effects of error capitalization in the evaluation of the psychometric properties of such rating scales.     

EXPECTATIONS OF BEHAVIORALLY ANCHORED RATING SCALES

Behaviorally anchored rating scales (BARS) are reviewed from two perspectives. First, the particular BARS methodology is assessed on the basis of its fulfilling three broad categories of "criteria for criteria:" (1) utilization criteria, (2) qualitative criteria, and (3) quantitative or psychometric criteria. These three broad categories are composed of 14 specific criteria by which performance evaluation methodologies can be assessed. The second perspective involves an evaluation of BARS in terms of psychometric criteria but with the literature broken down into studies concerned with (1) development of BARS, (2) utilization of BARS, (3) comparison of BARS to other methods, and (4) rater training in BARS use. Conclusions from these reviews indicate that BARS is no better or worse than other methods when assessed on a quantitative basis whereas it has greater potential when assessed on the utilization and qualitative criteria. Suggestions are offered for extending BARS research to process questions and to domains other than performance appraisal perse     

INTERPERSONAL AFFECT AND PERFORMANCE APPRAISAL: A FIELD STUDY

Interpersonal affect has been found, in prior laboratory research, to be related to ratings of job performance. Such findings have been taken to mean that affect creates bias in ratings. The present study was conducted to determine if this relationship would hold up in a field setting. The present study was also designed to examine how structured diary-keeping, and the nature of the appraisal instrument, might be related to affect-appraisal relationships. The results for 85 raters, and 404 ratees, suggested that affect was significantly related to all ratings, but more strongly related to trait-like ratings than task/outcome-like ratings, and that having raters keep performance diaries actually increased the strength of the relationship between affect and ratings. We concluded that affect may not be a biasing influence on ratings, but may be a result of better subordinate performance. Results from an analysis of the diary content supported this conclusion. Implications for the role of affect on ratings and the nature of the relationship between past performance and interpersonal affect in field settings are discussed.     

A METHODOLOGICAL APPROACH TO JOB CLASSIFICATION FOR PERFORMANCE APPRAISAL PURPOSES

This paper illustrates a methodological approach to classifying jobs for the purpose of developing performance evaluation instruments. The approach was to administer a worker-oriented job inventory to 2,023 incumbents across several jobs and several levels of responsibility (rank) in the U. S. Coast Guard. The data from the job inventory were then analyzed using Tucker's Three-Mode Factor Analysis. Output from the three-mode factor analysis was used to identify combinations of jobs and ranks for which separate appraisal instruments could be developed. In addition, output from the three-mode factor analysis was used to suggest the content of the various appraisal instruments. Advantages, applications, and limitations of this approach are discussed.     

TRUST AND THE COMMUNICATION OF PERFORMANCE APPRAISAL INFORMATION: THE EFFECT OF FEEDBACK ON PERFORMANCE AND JOB SATISFACTION

The feedback of performance appraisal information is a critical communication activity in organizations. Research on the topic, however, has generally been limited to (1) treating feedback as an undifferentiated construct, and (2) not examining the important moderating impact of trust of the receiver in the sender. Using a sample of 100 managers, this study explores the dimensionality of the feedback construct, the relationships of feedback to individual performance and satisfaction, and the moderating effects of trust on these associations. Results indicate that various aspects of feedback are differentially related to performance and satisfaction under conditions of high and low trut. Importantly, the communication of performance appraisal information is found to be a more important correlate of satisfaction and performance for subordinates who express low trust in their superiors.     

DEVELOPMENT OF MULTIPLE JOB PERFORMANCE MEASURES IN A REPRESENTATIVE SAMPLE OF JOBS

The goal of criterion development in Project A was to construct multiple measures of the major components of job performance such that the total performance domain for a representative sample of the population of entry-level enlisted positions in the U.S. Army was covered. These measures were to be used as criteria against which to validate both experimental and existing predictors of job performance. The initial model specified that performance is multidimensional within two major categories of dimensions designated as organization-wide and job specific. The development strategy involved describing the total domain of job content via extensive task analyses and critical incident analyses, generating the critical performance dimensions that constitute it, constructing measures for each dimension, and evaluating each measure using expert judgment and field test data. The specific measures developed consisted of rating scales, tests of job knowledge, hands-on job samples, and archival records. The major steps in the job analyses, content sampling, instrument construction, and instrument evaluation are described, and the final array of criterion measures is presented.     

BEHAVIORAL OBSERVATION SCALES FOR PERFORMANCE APPRAISAL PURPOSES

Behavioral items ( N = 78) critical to the job success of logging supervisors were developed from 1204 critical incidents, the frequency with which a supervisor ( N = 300) engaged in each behavior was rated on a 5-point Likert type scale by two sets of observers. A factor analysis reduced the items to 38 and 33, respectively, for the two sets of observers which in turn constituted 10 and 11 factors or criteria for performance evaluation purposes. Multiple regression equations based on composite scores were used to predict cost-related measures of logging crew effectiveness. The shrinkage in Rs after double cross-validation was moderately small. Moreover, the behavioral observation scales (BOS) that were developed by factor analyzing the observation ratings had moderately high reliability and accounted for more variance in the cost-related measures than did the BOS developed by traditional judgmental clustering techniques. The similarities and differences between BOS and BES procedures are discussed.