The disutility of the hard-easy effect in choice confidence

A common finding in confidence research is the hard-easy effect, in which judges exhibit greater overconfidence for more difficult sets of questions. Many explanations have been advanced for the hard-easy effect, including systematic cognitive mechanisms, experimenter bias, random error, and statistical artifact. In this article, I mathematically derive necessary and sufficient conditions for observing a hard-easy effect, and I relate these conditions to previous explanations for the effect. I conclude that all types of judges exhibit the hard-easy effect in almost all realistic situations. Thus, the effect’s presence cannot be used to distinguish between judges or to draw support for specific models of confidence elicitation.     

过度自信的研究展望

过度自信是个体高估自身判断精确度的一种认知偏差。过高估计和过高定位作为过度自信的两种主要类型被认为是个体在评价其绝对能力和相对能力时的表现。一般认为信息加工的偏差与判断误差的无偏性是造成过高估计的主要原因。自我提升动机、权重差异与信息的差异被认为是产生过高定位的原因。但是过度自信的这两种类型却在不同难度的任务巾出现了分离现象。最近,研究者提出了贝叶斯过度自信,用贝叶斯推理对不同任务中二者的分离进行了整合。过度自信产生的原因和内在心理机制、过度自信对决策的影响以及过度自信中的个体差异研究将会成为该领域日后研究的趋势。     

True Overconfidence in Interval Estimates: Evidence Based on a New Measure of Miscalibration

Overconfidence is often regarded as one of the most prevalent judgment biases. Several studies show that overconfidence can lead to suboptimal decisions of investors, managers, or politicians. Recent research, however, questions whether overconfidence should be regarded as a bias and shows that standard “overconfidence” findings can easily be explained by different degrees of knowledge of agents plus a random error in predictions. We contribute to the current literature and ongoing research by extensively analyzing interval estimates for knowledge questions, for real financial time series, and for artificially generated charts. We thereby suggest a new method to measure overconfidence in interval estimates, which is based on the implied probability mass behind a stated prediction interval. We document overconfidence patterns, which are difficult to reconcile with rationality of agents and which cannot be explained by differences in knowledge as differences in knowledge do not exist in our task. Furthermore, we show that overconfidence measures are reliable in the sense that there exist stable individual differences in the degree of overconfidence in interval estimates, thereby testing an important assumption of behavioral economics and behavioral finance models: stable individual differences in the degree of overconfidence across people. We do this in a “field experiment,” for different levels of expertise of subjects (students on the one hand and professional traders and investment bankers on the other hand), over time, by using different miscalibration metrics, and for tasks that avoid common weaknesses such as a non‐representative selection of trick questions. Copyright © 2012 John Wiley & Sons, Ltd.     

Caution! Warnings as a Useless Countermeasure to Reduce Overconfidence? An Experimental Evaluation in Light of Enhanced and Dynamic Warning Designs

People often display excessive overconfidence when providing interval estimates, which biases decision‐making. Research has investigated the various measures to effectively reduce overconfidence, and the use of warnings has subsequently been considered to have a negligible reduction effect. We demonstrate with two separate experiments that the impact of warnings has to be reviewed in light of dynamic warning designs and cognitive warning process models. In experiment 1, in contrast to previous studies that only used unstructured warnings, we implement a warning incorporating some core elements of a structured warning design based on research in the fields of human factors and ergonomics. Furthermore, accounting for recent developments in the warning literature, we distinguish between static and dynamic warning design. In experiment 2, we examine the effectiveness of different elements of dynamic warnings. We show that a significantly higher reduction of overconfidence can be achieved by combining a structured warning content with a dynamic stimulus change to increase the warning's noticeability. Copyright © 2016 John Wiley & Sons, Ltd.     

Social comparison and confidence: When thinking you’re better than average predicts overconfidence (and when it does not)

A common social comparison bias—the better-than-average-effect—is frequently described as psychologically equivalent to the individual-level judgment bias known as overconfidence. However, research has found “Hard–easy” effects for each bias that yield a seemingly paradoxical reversal: Hard tasks tend to produce overconfidence but worse-than-average perceptions, whereas easy tasks tend to produce underconfidence and better-than-average effects. We argue that the two biases are in fact positively related because they share a common psychological basis in subjective feelings of competence, but that the “hard–easy” reversal is both empirically possible and logically necessary under specifiable conditions. Two studies are presented to support these arguments. We find little support for personality differences in these biases, and conclude that domain-specific feelings of competence account best for their relationship to each other.     

Cognitive processes in implicit attitude tasks: An experimental validation of the Trip Model

Implicit attitude tasks have become very popular in various areas of psychology. However, little is known about the cognitive processes they involve. To address this issue, we investigated the underlying processes of the Go/No‐go Association Task (GNAT), a go/no‐go variant of the well‐known Implicit Association Test (IAT). More precisely, we tested two alternative multinomial processing tree (MPT) models of GNAT performance, the Trip Model and the generalized Quad Model. Both models assume that GNAT performance is influenced not only by automatic associations but also by response biases and a controlled discrimination process. However, the two models differ with respect to an additional overcoming bias process. In contrast to the Quad Model, the Trip Model assumes that overcoming bias does not play a major role in GNAT performance. Instead, the Trip Model emphasizes the role of response biases. We report three experiments that demonstrate the validity of the Trip Model for GNAT data. Copyright © 2010 John Wiley & Sons, Ltd.     

Option fixation: A cognitive contributor to overconfidence

The ASC model of choice and confidence in general knowledge proposes that respondents first Assess the familiarity of presented options, and then use the high-familiarity option as a retrieval cue to Search memory for the purposes of Constructing an explanation about why that high-familiarity option is true. The ASC process implies that overconfidence results in part from a tendency to fixate on the high-familiarity option, to the neglect of the other option. If this implication is true, then judgment tasks requiring respondents to evaluate each option independently should result in reduced overconfidence as compared with standard judgment tasks. Two experiments tested this implication, and found that confidence and overconfidence were reduced when respondents evaluated options independently. The findings support the proposal that option fixation contributes to overconfidence, and also clarify the limitations of random error explanations of overconfidence.     

Overconfidence: It Depends on How, What, and Whom You Ask

Many studies have reported that the confidence people have in their judgments exceeds their accuracy and that overconfidence increases with the difficulty of the task. However, some common analyses confound systematic psychological effects with statistical effects that are inevitable if judgments are imperfect. We present three experiments using new methods to separate systematic effects from the statistically inevitable. We still find systematic differences between confidence and accuracy, including an overall bias toward overconfidence. However, these effects vary greatly with the type of judgment. There is little general overconfidence with two-choice questions and pronounced overconfidence with subjective confidence intervals. Over- and underconfidence also vary systematically with the domain of questions asked, but not as a function of difficulty. We also find stable individual differences. Determining why some people, some domains, and some types of judgments are more prone to overconfidence will be important to understanding how confidence judgments are made.     

Naive empiricism and dogmatism in confidence research: a critical examination of the hard-easy effect

Two robust phenomena in research on confidence in one's general knowledge are the overconfidence phenomenon and the hard-easy effect. In this article, the authors propose that the hard-easy effect has been interpreted with insufficient attention to the scale-end effects, the linear dependency, and the regression effects in data and that the continued adherence to the idea of a "cognitive overconfidence bias" is mediated by selective attention to particular data sets. A quantitative review of studies with 2-alternative general knowledge items demonstrates that, contrary to widespread belief, there is (a) very little support for a cognitive-processing bias in these data; (b) a difference between representative and selected item samples that is not reducible to the difference in difficulty; and (c) near elimination of the hard-easy effect when there is control for scale-end effects and linear dependency.     

Confidence in judgment

Experiments have shown that, generally, people are overconfident about the correctness of their answers to questions. Cognitive psychologists have attributed this to biases in the way people generate and handle evidence for and against their views. The overconfidence phenomenon and cognitive psychologists' accounts of its origins have recently given rise to three debates. Firstly, ecological psychologists have proposed that overconfidence is an artefact that has arisen because experimenters have used question material not representative of the natural environment. However, it now appears that some overconfidence remains even after this problem has been remedied. Secondly, it has been proposed that overconfidence is an artefactual regression effect that arises because judgments contain an inherently random component. However, those claiming this appear to use the term overconfidence to refer to a phenomenon quite different from the one that the cognitive psychologists set out to explain. Finally, a debate has arisen about the status of perceptual judgments. Some claim that these evince only underconfidence and must, therefore, depend on mechanisms fundamentally different from those subserving other types of judgment. Others have obtained overconfidence with perceptual judgments and argue that a unitary theory is more appropriate. At present, however, no single theory provides an adequate account of the many diverse factors that influence confidence in judgment.     

Born to be biased? Unrealistic optimism and error management theory

When individuals display cognitive biases, they are prone to developing systematically false beliefs. Evolutionary psychologists have argued that rather than being a flaw in human cognition, biases may actually be design features. In my paper, I assess the claim that unrealistic optimism is such a design feature because it is a form of error management. Proponents of this theory say that when individuals make decisions under uncertainty, it can be advantageous to err on the side of overconfidence if the potential gains through success are high and the costs of failure are low. I argue that there are a number of conceptual problems in matching the theory with the existing data. I also show that there is empirical evidence against the error management hypothesis.     

The Role of Individual Differences in the Accuracy of Confidence Judgments

Generally, self-assessment of accuracy in the cognitive domain produces overconfidence, whereas self-assessment of visual perceptual judgments results in under-confidence. Despite contrary empirical evidence, in models attempting to explain those phenomena, individual differences have often been disregarded. The authors report on 2 studies in which that shortcoming was addressed. In Experiment 1, participants (N = 520) completed a large number of cognitive-ability tests. Results indicated that individual differences provide a meaningful source of overconfidence and that a metacognitive trait might mediate that effect. In further analysis, there was only a relatively small correlation between test accuracy and confidence bias. In Experiment 2 (N = 107 participants), both perceptual and cognitive ability tests were included, along with measures of personality. Results again indicated the presence of a confidence factor that transcended the nature of the testing vehicle. Furthermore, a small relationship was found between that factor and some self-reported personality measures. Thus, personality traits and cognitive ability appeared to play only a small role in determining the accuracy of self-assessment. Collectively, the present results suggest that there are multiple causes of miscalibration, which current models of over- and underconfidence fail to encompass.     

The role of individual differences in the accuracy of confidence judgments

Generally, self-assessment of accuracy in the cognitive domain produces overconfidence, whereas self-assessment of visual perceptual judgments results in underconfidence. Despite contrary empirical evidence, in models attempting to explain those phenomena, individual differences have often been disregarded. The authors report on 2 studies in which that shortcoming was addressed. In Experiment 1, participants (N= 520) completed a large number of cognitive-ability tests. Results indicated that individual differences provide a meaningful source of overconfidence and that a metacognitive trait might mediate that effect. In further analysis, there was only a relatively small correlation between test accuracy and confidence bias. In Experiment 2 (N = 107 participants), both perceptual and cognitive ability tests were included, along with measures of personality. Results again indicated the presence of a confidence factor that transcended the nature of the testing vehicle. Furthermore, a small relationship was found between that factor and some self-reported personality measures. Thus, personality traits and cognitive ability appeared to play only a small role in determining the accuracy of self-assessment. Collectively, the present results suggest that there are multiple causes of miscalibration, which current models of over- and underconfidence fail to encompass.     

Overconfidence in an objective anticipatory motor task

Overconfidence can place humans in hazardous situations, and yet it has been observed in a variety of cognitive tasks in which participants have to rate their own performance. We demonstrate here that overconfidence can be revealed in a natural and objective visuo-motor task. Participants were asked to press a key in synchrony with a predictable visual event and were rewarded if they succeeded and sometimes penalized if they were too quick or too slow. If they had used their own motor uncertainty in anticipating the timing of the visual stimulus, they would have maximized their gain. However, they instead displayed an overconfidence in the sense that they underestimated the magnitude of their uncertainty and the cost of their error. Therefore, overconfidence is not limited to subjective ratings in cognitive tasks, but rather appears to be a general characteristic of human decision making.     

A status-enhancement account of overconfidence

In explaining the prevalence of the overconfident belief that one is better than others, prior work has focused on the motive to maintain high self-esteem, abetted by biases in attention, memory, and cognition. An additional possibility is that overconfidence enhances the person's social status. We tested this status-enhancing account of overconfidence in 6 studies. Studies 1-3 found that overconfidence leads to higher social status in both short- and longer-term groups, using naturalistic and experimental designs. Study 4 applied a Brunswikian lens analysis (Brunswik, 1956) and found that overconfidence leads to a behavioral signature that makes the individual appear competent to others. Studies 5 and 6 measured and experimentally manipulated the desire for status and found that the status motive promotes overconfidence. Together, these studies suggest that people might so often believe they are better than others because it helps them achieve higher social status. (PsycINFO Database Record (c) 2012 APA, all rights reserved).     

Overconfidence effects in category learning: a comparison of connectionist and exemplar memory models

Exemplar and connectionist models were compared on their ability to predict overconfidence effects in category learning data. In the standard task, participants learned to classify hypothetical patients with particular symptom patterns into disease categories and reported confidence judgments in the form of probabilities. The connectionist model asserts that classifications and confidence are based on the strength of learned associations between symptoms and diseases. The exemplar retrieval model (ERM) proposes that people learn by storing examples and that their judgments are often based on the first example they happen to retrieve. Experiments 1 and 2 established that overconfidence increases when the classification step of the process is bypassed. Experiments 2 and 3 showed that a direct instruction to retrieve many exemplars reduces overconfidence. Only the ERM predicted the major qualitative phenomena exhibited in these experiments.     

Facial affect processing and depression susceptibility: cognitive biases and cognitive neuroscience

Facial affect processing is essential to social development and functioning and is particularly relevant to models of depression. Although cognitive and interpersonal theories have long described different pathways to depression, cognitive-interpersonal and evolutionary social risk models of depression focus on the interrelation of interpersonal experience, cognition, and social behavior. We therefore review the burgeoning depressive facial affect processing literature and examine its potential for integrating disciplines, theories, and research. In particular, we evaluate studies in which information processing or cognitive neuroscience paradigms were used to assess facial affect processing in depressed and depression-susceptible populations. Most studies have assessed and supported cognitive models. This research suggests that depressed and depression-vulnerable groups show abnormal facial affect interpretation, attention, and memory, although findings vary based on depression severity, comorbid anxiety, or length of time faces are viewed. Facial affect processing biases appear to correspond with distinct neural activity patterns and increased depressive emotion and thought. Biases typically emerge in depressed moods but are occasionally found in the absence of such moods. Indirect evidence suggests that childhood neglect might cultivate abnormal facial affect processing, which can impede social functioning in ways consistent with cognitive-interpersonal and interpersonal models. However, reviewed studies provide mixed support for the social risk model prediction that depressive states prompt cognitive hypervigilance to social threat information. We recommend prospective interdisciplinary research examining whether facial affect processing abnormalities promote-or are promoted by-depressogenic attachment experiences, negative thinking, and social dysfunction.     

Information processing biases and panic disorder: relationships among cognitive and symptom measures

To test cognitive models of panic disorder, a range of information processing biases were examined among persons with panic disorder (N=43) and healthy control participants (N=38). Evidence for automatic associations in memory was assessed using the Implicit Association Test, interference effects related to attention biases were assessed using a modified supraliminal Stroop task, and interpretation biases were assessed using the Brief Body Sensations Interpretation Questionnaire. In addition, the relationship between information processing biases and clinical markers of panic (including affective, behavioral, and cognitive symptom measures) was investigated, along with the relationships among biases. Results indicated more threat biases among the panic (relative to control) group on each of the information processing measures, providing some of the first evidence for an implicit measure of panic associations. Further, structural equation modeling indicated that the information processing bias measures were each unique predictors of panic symptoms, but that the bias indicators did not relate to one another. These findings suggest that cognitive factors may independently predict panic symptoms, but not covary. Results are discussed in terms of their support for cognitive models of panic and the potential for automatic versus strategic processing differences across the tasks to explain the low relationships across the biases.     

Realism of confidence in sensory discrimination

Is there a common and general basis for confidence in human judgment? Recently, we found that the properties of confidence judgments in the sensory domain mirror those previously established in the cognitive domain; notably, we found underconfidence on easy sensory judgments and overconfidence on hard sensory judgments. In contrast, data from the Uppsala laboratory in Sweden suggest that sensory judgments are unique; they found a pervasive underconfidence bias, with overconfidence being evident only on very hard sensory judgments. Olsson and Winman (1996) attempted to resolve the debate on the basis of methodological issues related to features of the stimulus display in a visual discrimination task. A reanalysis of the data reported in Baranski and Petrusic (1994), together with the findings of a new experiment that controlled stimulus display characteristics, supports the position that the difference between the Canadian and the Swedish data is real and, thus, may reflect cross-national differences in confidence in sensory discrimination.     

Context Matters,but How Much? Latent state–trait analysis of cognitive ability assessments

Cognitive abilities are thought to represent temporally stable constructs, however, accumulating evidence suggests that effects of the measurement situation could affect its measurement (e.g., effects of test motivation, stress level). The present study modeled these effects explicitly in a latent variables approach. In contrast to previous studies, we investigated participants (job candidates) in repeated high‐stakes settings (N = 188). We found that cognitive ability measurements in high‐stakes settings not only reflect a stable latent trait and random measurement error, but also systematic effects of the test setting. Our results support the application of cognitive ability tests in organizational contexts but have implications for its use in applied settings such as personnel selection.