The Influences of Performance Cues and Observational Focus on Performance Rating Accuracy1

The present study examined the effects of observational focus and performance cues on rating accuracy. It was hypothesized that these two factors would affect ratings independently: a) subjects given a good performance cue would rate the target more positively than subjects given a poor performance cue and, b) subjects using an event focus would rate the target person more accurately than subjects using a person focus. One hundred twenty undergraduates viewed the same videotape and subsequently stated which of a set of 48 behaviors were exhibited by the target person. The results supported the hypothesized performance cue effect. Observational focus did not have the hypothesized main effect on rating accuracy but was involved in a four-way interaction. The interaction indicated that, as the focus of the subject's attention broadened to the entire event, the errors associated with the performance cue effect were lessened. The subjects using an event focus were likely to generate ratings characterized by a positivity bias, whereas those using a person focus were more likely to generate ratings biased by performance cues.     

The effect of dimension content on observation and ratings of job performance

Recent theory and research in the performance appraisal area (e.g., Denisi, Cafferty, & Meglino, 1984; Feldman, 1986; Ilgen & Feldman, 1983; Williams, Denisi, & Blencoe, 1985) have suggested that providing information regarding the performance dimension to be rated will cause raters to select appropriate observational schemata and, as a result, produce higher quality ratings. A study was conducted to determine if giving raters dimension-relevant information prior to performance observation would affect their attention processes and rating quality. Prior to watching a videotape of an instructor giving a lecture, 156 subjects were given either: 1) correct information, 2) incorrect information, or 3) no information regarding dimensions of performance they would subsequently be asked to rate. The results indicated that giving prior information regarding dimension content affected subjects' attention processes. Further, raters receiving no information and those receiving misinformation prior to performance observation produced less accurate ratings compared to expert raters. Ratings produced by subjects receiving correct information did not differ significantly from experts' ratings. These results are discussed in terms of both their practical and theoretical implications.     

Effects of analyst training and amount of available job related information on job analysis ratings

The present study investigated the effects of job analysis training and the amount of information available to the job analyst on job analysis ratings. Forty subjects were asked to complete the Position Analysis Questionnaire (PAQ) and the Job Components Inventory (JCI) for the position of College Resident Assistant. Results indicated that when using the PAQ as the job analysis instrument, there was no significant effect for either training or the amount of information received by the job analyst. For the JCI, the results indicated that trained analysts were more reliable and accurate than untrained analysts. Also, it was found that the more information received by the job analysts, the more reliable and accurate were their JCI ratings.     

Accuracy in Detecting Potential Underage Drinkers: The Role of Contextual Effects in Proofing1

Three studies investigated the prevalence and influence of contextual effects in social judgments of age as they concern the purchase of alcohol. In Experiment 1, prior to rating a target individual, college students rated a series of photographs of persons considerably older or younger than the legal drinking age. Contrary to previous research on contrast effects, a cognitive assimilation effect was obtained for perceptions of age. Subjects rated a target person older when the prior stimuli were older, and younger when exposed to youthful stimuli. In Experiment 2, subjects again were exposed to older or younger stimuli or control stimuli (pictures of a university campus) and asked to rate a target individual. In an attempt to make salient the relevant perceptual category, subjects were asked specifically to make age ratings of the priming stimuli. Results again indicated an assimilation effect for age and the decision to proof. Attraction and liking data tended to demonstrate a contrast effect. Experiment 3 replicated the assimilation effects found in Experiments 1 and 2 using actual store clerks. These data are considered in light of the social factors that affect judgments in policies that seek to restrict access to alcohol among minors.     

Representation-saving effects of prior knowledge in memory for simple technical prose

Prior knowledge of the content of a passage should reduce the effort required to encode the passage and facilitate its recall. This paper presents such effects of prior knowledge upon comprehension and memory of simple technical prose. The basic procedure was to collect ratings of the amount of prior knowledge for individual passage sentences and individual subjects, and then to determine whether these familiarity ratings predicted study time and recall. In order to validate the rating method, and also to obtain data on prior knowledge of individual facts, subjects also completed an objective test of prior knowledge of the passage facts. Three different encoding-task conditions were used: a self-paced study task, a forced-pace study task, and an incidental-learning task. A cued-recall test followed each condition. In the self-paced task, readers studied unfamiliar material longer than familiar material, but recalled at the same level regardless of familiarity. In contrast, familiarity did predict recall in the forced-pace and incidental tasks. This task specificity is explained in terms of the subjects’ encoding strategies. The basic effect of prior knowledge can be explained not only by the elaboration principle, but also by a representation-saving principle, which is presented in a simulation model that can account for the effects of familiarity on study time.     

Verbal and behavioral responses to familiar and unfamiliar music

Thirty adult subjects completed an Operant Music Listening Recorder (OMLR) task with two unfamiliar and two familiar music channels (one “popular” and one “classical” in each case), then rated the music on each channel on seven-point scales for liking, quality, and familiarity. The two familiar music channels received significantly higher ratings than the two unfamiliar channels on all three rating scales, and there were no significant main effects or interactions involving musical style. The predictions (a) that subjects would spend more time listening to the familiar channels on the OMLR task, and (b) that measures of listening time would display higher correlations with ratings of liking than with those of quality both received weak support from the direction of the results, but neither was confirmed by statistical significance tests. The results are discussed in terms of the fragmentation between the affective and evaluative components of attitudes toward music.     

Gender-based influences: The promotion recommendation

This study used a between-subjects analysis of variance design (2 × 2 × 2 × 2) to estimate the effects of rater gender, ratee gender, the gender stereotype of the job, and the gender stereotype of the ratee's personal characteristics on a promotion decision. The results indicated that female employees with masculine characteristics were evaluated as most promotable regardless of the gender stereotype of the job or the gender of the rater. We also investigated the personal characteristics raters reported using in making their decisions. We asked raters to select ratee personal characteristics that most influenced their promotion decisions. Discriminant analyses showed that male and female raters approached the rating process differently and that it is predominantly the gender stereotype of the ratee's personal characteristics rather than the ratee's gender that influences the promotion process.     

Understanding the latent structure of job performance ratings

This study quantified the effects of 5 factors postulated to influence performance ratings: the ratee's general level of performance, the ratee's performance on a specific dimension, the rater's idiosyncratic rating tendencies, the rater's organizational perspective, and random measurement error. Two large data sets, consisting of managers (n = 2,350 and n = 2,142) who received developmental ratings on 3 performance dimensions from 7 raters (2 bosses, 2 peers, 2 subordinates, and self) were used. Results indicated that idiosyncratic rater effects (62% and 53%) accounted for over half of the rating variance in both data sets. The combined effects of general and dimensional ratee performance (21% and 25%) were less than half the size of the idiosyncratic rater effects. Small perspective-related effects were found in boss and subordinate ratings but not in peer ratings. Average random error effects in the 2 data sets were 11% and 18%.     

A COMPARISON OF HOLISTIC VERSUS DECOMPOSED RATING OF POSITION ANALYSIS QUESTIONNAIRE WORK DIMENSIONS

Recent studies have attempted to reduce the cost and intrusiveness of the Position Analysis Questionnaire (PAQ) by limiting the amount of information provided to the analyst, with consistently negative results. We examined an alternative technique for improving the cost-effectiveness of the PAQ that avoids the need to rate the hundreds of items that constitute the instrument. Three groups of raters (professional job analysts, graduate students in industrial psychology who were familiar with the PAQ, and PAQ-unfamiliar undergraduates) made direct holistic ratings of the PAQ dimensions for four familiar jobs. The holistic ratings were compared with decomposed PAQ dimension profiles obtained from the item-level ratings of the professional analysts. Cronbach accuracy analyses indicated near-zero convergence between the holistic and decomposed dimension ratings, even for the professional PAQ job analysts. We conclude that holistic rating of dimensions is not an effective means of reducing the cost of a PAQ job analyses and that it is likely to be similarly ineffective with task- or ability-based instruments.     

The job satisfaction-job performance relationship: a qualitative and quantitative review

A qualitative and quantitative review of the relationship between job satisfaction and job performance is provided. The qualitative review is organized around 7 models that characterize past research on the relationship between job satisfaction and job performance. Although some models have received more support than have others, research has not provided conclusive confirmation or disconfirmation of any model, partly because of a lack of assimilation and integration in the literature. Research devoted to testing these models waned following 2 meta-analyses of the job satisfaction-job performance relationship. Because of limitations in these prior analyses and the misinterpretation of their findings, a new meta-analysis was conducted on 312 samples with a combined N of 54,417. The mean true correlation between overall job satisfaction and job performance was estimated to be .30. In light of these results and the qualitative review, an agenda for future research on the satisfaction-performance relationship is provided.     

任职者任务绩效水平对其工作分析结果的影响

在控制了个体人口统计学变量和排除组织水平变量的影响下,探讨了电厂设计和编辑两个职位任职者的任务绩效水平对于他们Occupational Information Network (以下缩写为O*NET)工作分析问卷评价结果的影响。层次回归结果发现,控制了相关因素后,发电厂设计人员的任务绩效水平能显著影响其对技术性技能的水平评价,编辑的任务绩效水平能显著影响信息处理的重要性评价和水平评价,在一定程度上证实了“工作分析结果差异来源于真实差异”的理论     

Explaining the Weak Relationship Between Job Performance and Ratings of Job Performance

Ratings of job performance are widely viewed as poor measures of job performance. Three models of the performance–performance rating relationship offer very different explanations and solutions for this seemingly weak relationship. One-factor models suggest that measurement error is the main difference between performance and performance ratings and they offer a simple solution—that is, the correction for attenuation. Multifactor models suggest that the effects of job performance on performance ratings are often masked by a range of systematic nonperformance factors that also influence these ratings. These models suggest isolating and dampening the effects of these nonperformance factors. Mediated models suggest that intentional distortions are a key reason that ratings often fail to reflect ratee performance. These models suggest that raters must be given both the tools and the incentive to perform well as measurement instruments and that systematic efforts to remove the negative consequences of giving honest performance ratings are needed if we hope to use performance ratings as serious measures of job performance.     

The Impact of Computerized Performance Monitoring and Prior Performance Knowledge on Performance Evaluation1

Despite claims that computerized performance monitoring (CPM) systems provide objective performance data and thus foster accurate employee evaluations, few research studies have examined the impact of CPM data on the performance appraisal process. A laboratory experiment was conducted to assess the impact of prior performance level on requests for computerized performance information, and the influence of both prior performance level and requested information on performance evaluation. Forty-four male and female undergraduates electronically monitored four simulated employees whose prior performance was either high or low and whose performance during the monitoring period was either high or low. Results indicated that prior performance level and employee performance during the monitoring period independently influenced both current and future performance ratings. Further, when the simulated employee performed at a level incongruent with her prior performance, subjects requested more data about the employee's performance, were less certain about their ratings of the employee's current and future performance, and rated the employee's current performance as more variable than when the employee performed at a level congruent with prior performance. These results indicate that use of CPM systems that allow on-line access to employee performance and that record requested information may increase performance evaluation accuracy by facilitating the implementation of search strategies best suited to specific appraisal tasks and minimizing memory-related biases. However, such systems may not eliminate the effects of attention/encoding biases.     

Familiarisation with faces selectively enhances sensitivity to changes made to the eyes

Two experiments were designed to isolate the facial information utilised in the learning of new faces. In experiments 1 and 2, two groups of subjects were each trained on different groups of faces by means of a dynamic video presentation. They were then shown both trained and novel faces in a same-different decision task, where 'different' trials included manipulations of internal and external facial features, and the task was to decide whether two images were identical or had a difference in one or more features. Both experiments showed that hair change was most easily detected in untrained (unfamiliar) faces. When faces had been trained (familiar), detection of eye changes was selectively enhanced and sensitivity to hair changes was maintained. While previous studies have suggested that familiar face representations are weighted towards their internal features, our experimental results show that this is due to selective enhancement of sensitivity to changes made to the eyes alone, with no reduction in the salience of the hair. Moreover, within the limits of the familiarisation used here, there was no enhancement of the representation of the other internal face features examined.     

Selective Responsiveness of Chronically Ill Children to Assessments of Depression

Many investigators have noted that depression is a common symptom among pediatric cancer patients. However, prevalence rates vary widely across studies This variation in prevalence rates may be due, in part, to selective reporting of patients based on measure, used and environmental cues. In this study, we evaluated 50 chronically ill pediatric patients (19 cancer and 31 diabetic patients) for their use of selective reporting of depression. Factors in the 2 x 2 design were Intervention (disclosure videotape and cartoon videotape) and Examiner (familiar examiner and unfamiliar examiner). In the intervention manipulation, subjects were shown either a videotape prompting the child that self-disclosure was appropriate or a tape of a cartoon (control condition). In the Examiner manipulation, subjects were administered the experimental measures by either a familiar (parent) or unfamiliar (research assistant) examiner. Dependent variables were the Children's Depression inventory (CDI; Kovacs, 1981), the Depression scale of the Roberts Apperception Test for Children (RATC; McArthur & Roberts, 1982), and a depression measure taken from the Child Behavior Checktist (CBCL; Achenbach & Edelbrock, 1983). As hypothesized, the Examiner x intervention interaction revealed that children who did not view the disclosure videotape and who were tested by an unfamiliar examiner gave significantly lower self-reports of depression on the CDI than children in the other conditions. However, parent and child projective reports of depression did not vary as a function of experimental condition. The results are interpreted as selective responding on the part of pediatric patients. Limitations of assessing internal psychological states in children are discussed.     

Mothers' Perceptions of Infants Labelled ‘Depressed’

Forty-four depressed and non-depressed mothers participated in a videotaped interaction with their own infant and then rated the videotape using the Infant Stereotyping Scale and the Interaction Rating Scale. In addition, one half of the mothers rated a videotape of an unfamiliar infant who was labelled psychologically ‘depressed’ and the other half rated a videotape of the same infant with no label given. Both the depressed and non-depressed mothers rated the ‘depressed’ labelled infant more negatively than the non-labelled infant on the attributes of physical potency, cognitive competence, sociability and difficult behaviour. Physical appearance was the only rating that was not biased by the ‘depressed’ label. Mothers' ratings of their own infants were more positive than their ratings of the non-labelled stimulus infant. Depressed mothers did not see their infants more negatively except on one rating. They rated the physical appearance of their own infant more negatively than non-depressed mothers.     

Effects of biased psychological reports on two types of teachers' ratings

This study investigated the effects of biased psychological reports on teachers' behavioral and expectations ratings of a kindergarten child seen in a videotape. Biases were produced by randomly distributing to one third of the subjects a fictitious favorable psychological report, while one third read a fictitious unfavorable version. The remaining third read a short fable and acted as a control group. Subjects rated the test performance of the child in the videotape and then attributed their expectations for his possessing certain cognitive and effective traits. A significant difference was noted in ratings of expected characteristics, but not in performance ratings, suggesting that the biased psychological report influenced one type of rating but not the other. Furthermore, the two dependent variables were not significantly correlated.     

Unfamiliar Untrained Observers’ Ratings of Adolescent Safety Behaviors Within Social Interactions with Unfamiliar Peer Confederates

Adolescents experiencing social anxiety often engage in safety behaviors―covert avoidance strategies for managing distress (e.g., avoiding eye contact)―that factor into the development and maintenance of their concerns. Prior work supports the psychometric properties of the Subtle Avoidance Frequency Examination (SAFE), a self-report survey of safety behaviors. Yet, we need complementary methods for assessing these behaviors within contexts where adolescents often experience concerns, namely, interactions with unfamiliar peers. Recent work indicates that, based on short, direct social interactions with adolescents, individuals posing as unfamiliar peers (i.e., peer confederates) and without assessment training can capably report about adolescent social anxiety. We built on prior work by testing whether we could gather valid SAFE reports from unfamiliar untrained observers (UUOs), who observed adolescents within archived recordings of these short social interactions. A mixed clinical/community sample of 105 adolescents self-reported on their functioning and participated in a series of social interaction tasks with peer confederates, who also provided social anxiety reports about the adolescent. Based on video recordings of these tasks, trained independent observers rated adolescents’ observed social skills, and an additional set of UUOs completed SAFE reports of these same adolescents. Unfamiliar untrained observers’ SAFE reports (a) related to adolescents’ SAFE self-reports, (b) distinguished adolescents on clinically elevated social anxiety concerns, (c) related to trained independent observers’ ratings of adolescent social skills within interactions with peer confederates, and (d) related to adolescents’ self-reported arousal within these same interactions. Our findings support use of unfamiliar observers’ perspectives to understand socially anxious adolescents’ interpersonal functioning.