Reducing the performance-cue bias in work behavior ratings: can groups help?

The authors examined whether the performance-cue bias can be reduced by relying on groups as raters. Study participants (N = 333) were provided with feedback regarding the performance of a workgroup and, after observing the group, assigned to an individual or group rater condition to complete a behavioral rating instrument. Results revealed that when provided with positive (vs. negative) feedback, individuals attributed more effective and fewer ineffective behaviors to the workgroup; however, group ratings were unaffected by the feedback. In addition, feedback biased the decision criteria and false alarm rates of individuals but not of groups. Discussion of when groups may attenuate versus amplify bias in performance appraisal judgments emphasizes 2 key elements-bias magnitude and task perception.     

A new look at recognition in the Brown-Peterson distractor paradigm: Toward the application of new methodology to unsolved problems of recognition memory

Data from a recognition version of the classic Brown-Peterson short-term memory paradigm was analyzed using a modified version of the conjoint recognition model (Brainerd, Reyna, & Mojardin, 1999), which assumes that recognition is based on either a verbatim comparison of the recognition probe and the target item or a gist comparison of the items. Separate groups of participants were instructed to judge whether the recognition probe was an item from the current trial (exclusion condition), a previous trial (prior-only condition), or either the current or the previous trial (inclusion condition). The concept of gist is commonly thought of as meaning based. Our interpretation of the results suggests that the concept of gist need also emphasize similarity of environmental context. In addition, the results show that priming the recognition probe affects a participant's decision bias but does not enhance or impair the memory traces on which the recognition judgments are based. An additional analysis using Batchelder and Riefer's (1990) source-monitoring model supports the observation that priming affects only decision bias.     

Bending perception to desire: Effects of task demands,motivation, and cognitive resources

Determinants of motivated judgments were examined in this research. Three experiments investigated how dominant motivation, biasing difficulty and mental resources combine to produce motivationally congruent judgments. Studies 1 and 2 showed that where a biasing motivation is dominant the presence of resources can augment a motivational bias in judgment. Study 3 replicated that result and showed that resources contribute to the formation of biased judgments only where biasing is difficult to accomplish, but not where it is relatively easy to accomplish. In addition, Study 3 showed that where the accuracy motivation is dominant and biasing is the easy default, unbiased judgments will occur only in the presence (vs. absence) of resources. In contrast, where unbiased judgments are easy to come by, such judgments occur irrespective of resources.     

Safety behaviors and judgmental biases in social anxiety disorder

Two experiments were conducted to examine the link between safety behaviors and social judgments in social anxiety disorder (SAD). Safety behaviors were manipulated in the context of a controlled laboratory-based social interaction, and subsequent effects of the manipulation on the social judgments of socially anxious participants (N = 50, Study 1) and individuals meeting diagnostic criteria for generalized SAD (N = 80, Study 2) were examined. Participants were randomly assigned to either a safety behavior reduction plus exposure condition (SB + EXP) or a graduated exposure (EXP) control condition, and then took part in a conversation with a trained experimental confederate. Results revealed across both studies that participants in the SB + EXP group were less negative and more accurate in judgments of their performance following safety behavior reduction relative to EXP participants. Study 2 also demonstrated that participants in the SB + EXP group displayed lower judgments about the likelihood of negative outcomes in a subsequent social event compared to controls. Moreover, reduction in safety behaviors mediated change in participant self-judgments and future social predictions. The current findings are consistent with cognitive theories of anxiety, and support the causal role of safety behaviors in the persistence of negative social judgments in SAD.     

Effects of sex on raters' accountability

The effects of sex (rater and ratee) on raters' accountability in the context of performance appraisal were investigated. The 130 participating undergraduates (men and women) rated a fictitious male or female's performance on a clerical task subsequent to receiving self-assessment information. As expected, raters' knowledge of a high self-assessment was followed by significantly higher performance ratings than after knowledge of a low self-assessment. Contrary to expectations, no differences were found for either raters' or ratees' sex. The results suggest that the sex of the rater or ratee is not associated with raters' accountability.     

Measuring the duration of perception

An analysis is presented of ways in which the total duration of perception of transient visual stimuli may be determined by means of psychophysical judgments of the simultaneity (or relative precedence) of two sensory events. This analysis yields a new method for measuring the duration of perception that only requires judgments of the simultaneity of the offset of one visual target with the onset of another (“offset-onset” judgments), and is thus free of differential biases between onset-onset and offset-onset judgments of simultaneity which could be involved in previous measurements. When three or more perceived durations need to be determined, the new method is more efficient than earlier methods; it requires measurement of only one PSE in order to evaluate one response duration as compared to two PSEs per response duration for previous methods. We also describe ways of determining the presence of some kinds of biases and quantitatively evaluating the magnitude of bias in the new method, as well as bias in onset-onset or offset-offset judgments of simultaneity alone; such evaluations of differential bias were not possible for the earlier methods. An experimental example of a bias analysis is described. No significant biasing effects were detected in the measures of perceived duration that were extracted as either retinal location or background luminance was changed, although background luminance itself markedly influenced the values of perceived duration.     

Why so confident? The influence of outcome desirability on selective exposure and likelihood judgment

Previous studies that have directly manipulated outcome desirability have often found little effect on likelihood judgments (i.e., no desirability bias or wishful thinking). The present studies tested whether selections of new information about outcomes would be impacted by outcome desirability, thereby biasing likelihood judgments. In Study 1, participants made predictions about novel outcomes and then selected additional information to read from a buffet. They favored information supporting their prediction, and this fueled an increase in confidence. Studies 2 and 3 directly manipulated outcome desirability through monetary means. If a target outcome (randomly preselected) was made especially desirable, then participants tended to select information that supported the outcome. If made undesirable, less supporting information was selected. Selection bias was again linked to subsequent likelihood judgments. These results constitute novel evidence for the role of selective exposure in cases of overconfidence and desirability bias in likelihood judgments.     

The Paris-Peoria Solution: Innovations in Appraising Regional and International Sales Personnel

Abstract

How can salesmen in different locations be fairly evaluated? Traditional appraisal methods are deficient because they assume equal performance distributions among different regions or countries. Further, individual supervisory bias may make even within-region performance measures unfair, not comparable, and indefensible. A new Team Evaluation Census (TEC) appraisal is described in terms of four innovations—natural, direct comparisons to anchor appraisal judgments and minimize leniency; benchmarks to improve comparability and reduce performer competition; multiple and linking raters to improve appraisal reliability and validity and to improve across- group comparability; and safeguards to ensure individual fairness and allow rater validation.     

Reducing bias in frequency judgment by improving source monitoring

A common finding in judgment and decision making is that people's frequency judgments often fail to map onto objective frequencies. The present research examined the possibility that one source of bias in frequency judgment is attributable to people's inability to screen out irrelevant memory traces. We used a two-list source-monitoring paradigm to investigate whether frequency judgments are influenced by "extra-experimental" experiences and whether enhancing source monitoring improves judgment accuracy. Across four experiments we found: (1) frequency judgments regarding one list were biased by the second, (2) manipulating encoding between lists improved source monitoring and resulted in more accurate judgments, (3) manipulating item context between lists improved source monitoring and resulted in more accurate judgments, but only when the context was item specific, and (4) manipulating simple-background context between lists was ineffective at improving source monitoring.     

Distinguishing Context Effects From Context Errors in Judgments of Behavior1

Context effects (assimilation and contrast) are examined in relation to accuracy in judgments of stimuli. Context effects are distinguished from context errors. This is shown to depend on one's definition of true scores, rater tendencies (leniency-severity) relative to true scores, and the direction and magnitude of observed context effects. The framework is illustrated empirically in a study of contrast effects involving performance judgments. Implications for reliability, validity, and agreement of behavior judgments in practice are explored.     

Overconfidence in Initial Self-Efficacy Judgments: Effects on Decision Processes and Performance

Judgments of task-specific, expected performance (i.e., self-efficacy) can affect the activities one chooses to pursue and the extent of effort devoted to these activities. However, relatively little is known about the accuracy of self-efficacy judgments or their effects on behavior, performance, and perceptions of performance in complex cognitive tasks. The results of a pilot study and experiment indicate that initial, "first-impression" self-efficacy judgments made in cognitively complex tasks are biased toward overestimates of personal ability (i.e., "overconfidence"). The experiment manipulated performance expectations to illuminate how overestimates of initial self-efficacy affect decision making. Inducing positive expectations produced overconfidence in choice accuracy, but did not increase effort, attention to strategy, or performance relative to mildly negative and strongly negative expectations. In contrast, inducing mildly negative expectations increased effort, attention to strategy, and performance relative to strongly negative expectations. The results suggest that the demotivational effects of initial negative expectations are more robust than the motivational effects of initial positive expectations. ln addition, inducing mildly negative expectations may improve performance more than positive expectations in at least some tasks and settings.     

A theory of bidirectional judgments

This paper is concerned with a scaling theory for “bidirectional” judgments, for which the order of judgment is reversible, as in fractional and multiple ratio estimation judgments. With the assumption that judgments are mediated by perceived relations of pairs of stimuli, the theory is developed for judgments of comparison stimuli in relation to standards, taking explicit account of the location of the comparison stimulus relative to the standard. The theory of bidirectional judgments, based on a theory of relative judgment by Fagot (1978, 1979), entails a partial nesting of models characterized by a progressive weakening of the constraints placed on the structure of the data. The weakest model, the relative bias/directional standard (RBDS) model, allows each standard to have two biasing effects, depending on the location of the standard above or below the comparison stimulus. Tests of the theory were carried out on the ratio estimation of brightness and weight data of Engen and Levy (1955) and the part-sum estimation data of Goude (1962). Only the RBDS model was found acceptable for all three data sets     

The Effect of Rater Expectations on the Evaluation of a Hypothetical Subordinate

In a laboratory study of the effect of rater expectations on the evaluation of a hypothetical subordinate's performance, 159 undergraduates role played the part of a sales manager who had been given high, low, or no expectations about a subordinates's potential along with hypothetical concrete or abstract performance data. Hypotheses regarding the interaction of performance data and expectations on evaluations were not supported. Although evaluations of the subordinate's performance did not vary across conditions, the student raters recommended larger raises for the subordinate for whom they had low expectations than for the subordinate for whom they had high expectations. The key predictor of a recommended raise appeared to be the extent to which the subordinate met or exceeded rater expectations. These results were reconfirmed with an independent cross-validation sample of 168 subjects.     

Judging psychopathology: Expectation and contrast

An experiment was performed to assess the role of expectations in psychosocial contrast effects. Subjects were initially presented with a series of word definitions that were either high or low pathology; their task was to indicate those definitions that had been produced by schizophrenic (vs nonschizophrenic) patients. After this “induction” series, subjects were presented with a common “test” series that included both definitions and handwriting samples; they evaluated the degree of psychopathology that was implied by each item. Some subjects were required to state their expectations in connection with the test items. First they attempted to guess the percentage of schizophrenics in the overall test series; second, before being presented with each test item, they guessed whether the patient in question was (or was not) schizophrenic. Subjects' expectations were consistent with the induction series to which they were assigned; those assigned to the high-pathology induction expected the test series to include more schizophrenic patients than those in the low-pathology group. Nonetheless, despite these expectations, subjects showed a contrast effect in their evaluations of the test definitions (but not in evaluating the handwriting samples). That is, subjects from the low-pathology induction group rated the test definitions as being more pathological than did those in the high-pathology condition. Within-group correlations indicated that the subjects' expectations were directly related to subsequent judgments. A two-path model is discussed, relating (a) the respondents' induction group assignment, (b) his/her expectations, and (c) subsequent test judgments.     

Modeling Rater Disagreement for ADHD: Are Parents or Teachers Biased?

The present study is the first to utilize twin modeling to examine whether parent-teacher disagreement for ADHD ratings is due to parent or teacher bias, or due to raters observing different but valid ADHD behaviors. A joint analysis was conducted with 106 twin pairs, including twins selected for ADHD and control twin pairs. Total ADHD scores were analyzed using multiple rater models that estimate genetic and environmental contributions common to both raters and unique to each rater. Results suggest that 1) disagreement in ADHD ratings is strongly due to parents and teachers observing different ADHD behaviors, some of which is valid and some of which is due to bias, and 2) parents may be more biased than teachers in their ADHD ratings.     

Sex differences in identifying the facial affect of normal and mirror-reversed faces

The influences of sex and lateralized visual hemispace bias in the judgment of the emotional valence of faces during a free viewing condition are evaluated. 73 subjects (aged 18 to 52 yr.) viewed videotaped facial expressions of emotion in normal and mirror-reversed orientation and classified each face as a positive, negative, or neutral expression. There was a significant interaction between the sex of the rater and the orientation of the face that influenced the proportion of correct classifications. Male and female perceivers did not differ in the accuracy of their affect judgments for faces viewed in normal orientation, whereas reversal of the orientation of the faces resulted in a significant enhancement of accuracy judgments for the males but not the females. The results suggest greater cerebral lateralization of perceptual processes in males.     

Triangulating Political Responsibility: The Motivated Formation of Responsibility Judgments

This article analyzes the process through which partisan bias arises during the formation of citizens' judgments of political responsibility. Informed by theories of motivated political reasoning, it argues that exposure to partisan cues motivates partisans to pursue directional goals, goals which bias the cognitive processing of information and, in turn, overall judgments of responsibility. It further argues that the nature of this biased processing will be such that partisans devalue information inconsistent with their partisan affect. Using a pair of experiments, I test these hypotheses by manipulating both objective evidence concerning gubernatorial responsibility for a state's fiscal imbalance and the presence of partisan cues. Findings support both sets of expectations. The results also suggest that the effects of partisan bias are greater in judgments tied to institutional actions than in those tied to institutional roles and expectations.     

On the Practical Significance of Hindsight Bias: The Case of the Expectancy-Disconfirmation Model of Consumer Satisfaction

The implications of the phenomenon of hindsight bias for the expectancy-disconfirmation model of consumer satisfaction are investigated, and the moderating effects of choice (i.e., whether or not a product is selected on the basis of one′s expectations regarding the product′s likely performance) on the incidence of hindsight bias and on the relationships between expectations, performance, disconfirmation, and satisfaction are considered. A study dealing with consumers′ satisfaction with personalized envelopes shows that perceived performance biases people′s recall of their foresight expectations in a systematic way (hindsight bias), that hindsight, rather than foresight, expectations are the more potent influence on disconfirmation and satisfaction, and that choice moderates the degree of hindsight bias and the relationships between satisfaction and its antecedents.     

Rating leniency and halo in multisource feedback ratings: testing cultural assumptions of power distance and individualism-collectivism

This study extends multisource feedback research by assessing the effects of rater source and raters' cultural value orientations on rating bias (leniency and halo). Using a motivational perspective of performance appraisal, the authors posit that subordinate raters followed by peers will exhibit more rating bias than superiors. More important, given that multisource feedback systems were premised on low power distance and individualistic cultural assumptions, the authors expect raters' power distance and individualism-collectivism orientations to moderate the effects of rater source on rating bias. Hierarchical linear modeling on data collected from 1,447 superiors, peers, and subordinates who provided developmental feedback to 172 military officers show that (a) subordinates exhibit the most rating leniency, followed by peers and superiors; (b) subordinates demonstrate more halo than superiors and peers, whereas superiors and peers do not differ; (c) the effects of power distance on leniency and halo are strongest for subordinates than for peers and superiors; (d) the effects of collectivism on leniency were stronger for subordinates and peers than for superiors; effects on halo were stronger for subordinates than superiors, but these effects did not differ for subordinates and peers. The present findings highlight the role of raters' cultural values in multisource feedback ratings.     

The world may not be just for you but you'd better not say it: On the social value of expressing personal belief in a just world

The expression of personal belief in a just world (PBJW) has been discussed as a criterion of excellence in social judgments. In four experimental studies we hypothesized and found that targets who express high versus low PBJW are judged as more: (i) deserving of success and (ii) suited to socio-organizational expectations. The four studies show that suitability to socio-organizational expectations mediates the relation between PBJW expressed and success deservingness, even after controlling for judgments of likability, status, rationality, optimism, and targets as victims. Studies 2 and 3 show this pattern occurs regardless of target performance appraisal. Study 4 indicates that expressing low PBJW decreases the social value of individuals, but expressing high PBJW does not increase it. We discuss the impact of PBJW expression on people's lives, namely on upward social mobility of members of low-status groups, and the influence of the negativity bias on judgments caused by PBJW expression.