The domain specificity and generality of overconfidence: Individual differences in performance estimation bias

One hundred twenty-three college students performed a knowledge assessment task and a game of motor skill in which they had to predict their performance before each block of trials. There was a bias in the direction of overconfidence on both tasks, even though the latter involved the motor domain, did not require the use of numeric probabilities, and allowed predictions to be made by using an aggregate judgment made in a frequentist mode. An analysis of individual differences indicated that there was considerable domain specificity in confidence judgments. However, participants who persevered in showing overconfidence in the motor task—despite previous feedback revealing their overconfident performance predictions—were significantly more overconfident in the knowledge calibration task than were participants who moderated their motor performance predictions so as to remove their bias toward overconfidence. The latter finding is consistent with explanations of overconfidence effects that implicate mechanisms with some degree of domain generality.     

Why people fail to recognize their own incompetence

Successful negotiation of everyday life would seem to require people to possess insight about deficiencies in their intellectual and social skills. However, people tend to be blissfully unaware of their incompetence. This lack of awareness arises because poor performers are doubly cursed: Their lack of skill deprives them not only of the ability to produce correct responses, but also of the expertise necessary to surmise that they are not producing them. People base their perceptions of performance, in part, on their preconceived notions about their skills. Because these notions often do not correlate with objective performance, they can lead people to make judgments about their performance that have little to do with actual accomplishment.     

Persistence of overconfidence in young children: Factors that lead to more accurate predictions of memory performance

Young children are typically overconfident regarding both cognitive abilities. This overconfidence may be due to development underpinnings. Previous research has demonstrated that children exhibit robust and persistent overconfidence in a simple memory-recall task. Two experiments investigated this overconfidence in 1st–4th and 4th–6th grade students. In the first experiment, we explored both the development of accurate predictions of recall and young students’ confidence in their memory performance predictions. It was found that not until 4th grade did students’ overconfidence begin to wane. In the second experiment, we investigated a condition under which 4th–6th graders might make more accurate predictions of their ability to recall simple stimuli, specifically, when the items to be remembered were unfamiliar to the students. The results confirmed our overconfidence in familiarity hypothesis. We discuss these findings in the context of metacognition.     

Calibration of sensory and cognitive judgments: Two different accounts

The purpose of this study was to test predictions of two recent theories of realism of confidence. Ecological approaches to realism of confidence in one's general knowledge (Gigerenzer et al. , 1991; Juslin, in press a ; Björkman, in press) predict good calibration or, in the case of poor cognitive adjustment, overconfidence, within the cognitive domain. The subjective distance theory of confidence in sensory discriminations (Björkman et al. , 1992) predicts a pervasive underconfidence bias for sensory discriminations. Empirical data are reported showing that: (a) Calibration for sensory judgments is considerably poorer than calibration for well adapted cognitive judgements, a difference that can be entirely traced to underconfidence in the sensory domain. (b) While an initial overconfidence bias in the cognitive domain is removed by outcome feedback, the bias observed in sensory discriminations is unaffected even by a prolonged feedback session. It is suggested that the nature of confidence in sensory discriminations is different from the nature of confidence in cognitive judgments.     

Probabilistic mental models: a Brunswikian theory of confidence

Research on people's confidence in their general knowledge has to date produced two fairly stable effects, many inconsistent results, and no comprehensive theory. We propose such a comprehensive framework, the theory of probabilistic mental models (PMM theory). The theory (a) explains both the overconfidence effect (mean confidence is higher than percentage of answers correct) and the hard-easy effect (overconfidence increases with item difficulty) reported in the literature and (b) predicts conditions under which both effects appear, disappear, or invert. In addition, (c) it predicts a new phenomenon, the confidence-frequency effect, a systematic difference between a judgment of confidence in a single event (i.e., that any given answer is correct) and a judgment of the frequency of correct answers in the long run. Two experiments are reported that support PMM theory by confirming these predictions, and several apparent anomalies reported in the literature are explained and integrated into the present framework.     

The misunderstood limits of folk science: an illusion of explanatory depth

People feel they understand complex phenomena with far greater precision, coherence, and depth than they really do; they are subject to an illusion-an illusion of explanatory depth. The illusion is far stronger for explanatory knowledge than many other kinds of knowledge, such as that for facts, procedures or narratives. The illusion for explanatory knowledge is most robust where the environment supports real-time explanations with visible mechanisms. We demonstrate the illusion of depth with explanatory knowledge in Studies 1-6. Then we show differences in overconfidence about knowledge across different knowledge domains in Studies 7-10. Finally, we explore the mechanisms behind the initial confidence and behind overconfidence in Studies 11 and 12. Implications for the roles of intuitive theories in models of concepts and cognition are discussed.     

Narcissism,confidence, and risk attitude

The present research addresses whether narcissists are more overconfident than others and whether this overconfidence leads to deficits in decision making. In Study 1, narcissism predicted overconfidence. This was attributable to narcissists' greater confidence despite no greater accuracy. In Study 2, participants were offered fair bets on their answers. Narcissists lost significantly more points in this betting task than non‐narcissists, due both to their greater overconfidence and greater willingness to bet. Finally, in Study 3, narcissists' predictions of future performance were based on performance expectations rather than actual performance. This research extends the literature on betting on knowledge to the important personality dimension of narcissism. Copyright © 2004 John Wiley & Sons, Ltd.     

Influences of metamemory on performance predictions for text

When predicting future performance on tests over text material, do individuals estimate retention in addition to assessing comprehension? In Experiment 1, participants either rated their comprehension or predicted performance for each text, with lower ratings indicating lower confidence either in comprehension or in eventual performance. Judgement magnitude was significantly lower for performance predictions than for comprehension ratings, suggesting that predictions were based partly on retention estimates. In Experiment 2, predictions varied with anticipated test delay (15 min or 2 weeks) whereas comprehension ratings did not, providing further evidence that individuals estimate retention when predicting performance. Analyses of individual differences suggest that both good and poor performers incorporate retention estimates when predicting performance, but better performers do so in a more discriminative manner. Implications for theory of metacognitive judgements are discussed.     

True Overconfidence in Interval Estimates: Evidence Based on a New Measure of Miscalibration

Overconfidence is often regarded as one of the most prevalent judgment biases. Several studies show that overconfidence can lead to suboptimal decisions of investors, managers, or politicians. Recent research, however, questions whether overconfidence should be regarded as a bias and shows that standard “overconfidence” findings can easily be explained by different degrees of knowledge of agents plus a random error in predictions. We contribute to the current literature and ongoing research by extensively analyzing interval estimates for knowledge questions, for real financial time series, and for artificially generated charts. We thereby suggest a new method to measure overconfidence in interval estimates, which is based on the implied probability mass behind a stated prediction interval. We document overconfidence patterns, which are difficult to reconcile with rationality of agents and which cannot be explained by differences in knowledge as differences in knowledge do not exist in our task. Furthermore, we show that overconfidence measures are reliable in the sense that there exist stable individual differences in the degree of overconfidence in interval estimates, thereby testing an important assumption of behavioral economics and behavioral finance models: stable individual differences in the degree of overconfidence across people. We do this in a “field experiment,” for different levels of expertise of subjects (students on the one hand and professional traders and investment bankers on the other hand), over time, by using different miscalibration metrics, and for tasks that avoid common weaknesses such as a non‐representative selection of trick questions. Copyright © 2012 John Wiley & Sons, Ltd.     

The psychology of being a beginner

Beginners are plagued with overconfidence. They perform the worst on tests of knowledge or skill and yet are the most overconfident. One would expect that learning would better calibrate beginners and lead to better quality decisions. However, learning can instead produce overconfidence. In this article I discuss how being beginner can lead to errors in self-assessments and riskier decisions. I review differences in beginner overconfidence in several places in the literature. First, the Dunning–Kruger Effect, which finds those with the least knowledge are the most overconfident. To be sure, beginners can range from rank beginners, those who have acquired no knowledge, to experienced beginners, beginners who having gained some but not vast knowledge. Next, I discuss the beginner's bubble that follows the trajectory of confidence and overconfidence as people transition from rank to experienced beginners. The beginner's bubble pattern finds rank beginners have insight into their poor abilities. However, with some learning there is a surge of confidence and overconfidence. Lastly, I explore differences in confidence and overconfidence as learners transition from rank to experienced beginners in other places in the literature.     

Child witnesses' metamemory realism

This study investigated the degree of realism in the confidence judgments of 11 to 12-year-olds (41 girls and 40 boys) of their answers to questions relating to a short film clip showing a kidnapping event. Four different confidence scales were used: a numeric scale, a picture scale, a line scale and a written scale. The results demonstrated that the children showed a high level of overconfidence in their memories. However, no significant differences between the four confidence scales were found. Weak gender differences were found in that the girls were slightly, but significantly, better calibrated than the boys. In addition, although both boys and girls overestimated the total number of memory questions they had answered correctly, the boys gave higher estimates compared with the girls. In brief, the results indicate that, at least in the context investigated, 11-12 year-old children's confidence in and estimations of their own event memory show poor realism (overconfidence and overestimation). A comparison with previous research on adults indicates that 11 to 12-year-old children show noticeably poorer realism.     

Unskilled and optimistic: Overconfident predictions despite calibrated knowledge of relative skill

Those who are less skilled tend to overestimate their abilities more than do those who are more skilled—the so-called Dunning–Kruger effect. Less-skilled performers presumably have less of the knowledge needed to make informed guesses about their relative performance. If so, the Dunning–Kruger effect should vanish when participants do have access to information about their relative ability and performance. Competitive bridge players predicted their results for bridge sessions before playing and received feedback about their actual performance following each session. Despite knowing their own relative skill and showing unbiased memory for their performance, they made overconfident predictions consistent with a Dunning–Kruger effect. This bias persisted even though players received accurate feedback about their predictions after each session. The finding of a Dunning–Kruger effect despite knowledge of relative ability suggests that differential self-knowledge is not a necessary precondition for the Dunning–Kruger effect. At least in some cases, the effect might reflect a different form of irrational optimism.     

The Overconfidence Phenomenon as a Consequence of Informal Experimenter-Guided Selection of Almanac Items

The paper argues for an ecological approach to realism of confidence in general knowledge. It is stressed that choice of answer to almanac items and confidence judgments derive from knowledge structures formed by adaptation to a natural environment. People are regarded as well-calibrated to their natural environments and the overconfidence phenomenon is seen as a consequence of the procedures involved in the creation of "traditional" general knowledge items, rather than as the result of a cognitive bias. An experiment is reported showing that when items are informally selected by human "selectors," instructed to select items that differentiate between more and less knowledgeable subjects, we observe poor calibration and the usual overconfidence phenomenon. However, when the item selection is "debiased" and the objects to be compared in the almanac items are selected randomly from a natural environment the overconfidence phenomenon disappears and we observe good calibration, and good resolution. These results support an ecological approach to realism of confidence and suggest that general procedures for debiasing overconfidence may be unwarranted.     

Realism in confidence judgements of performance based on implicit learning

In two experiments we employed calibration methods to investigate the realism of participants' confidence ratings of their own classification performance based on knowledge acquired after training on an artificial grammar. In Experiment 1 participants showed good realism (but overconfidence) for grammatical strings but very poor realism for non-grammatical strings. Method of training (string repetition in writing or mere exposure) did not affect the realism. Furthermore, the participants underestimated their overall performance. In Experiment 2, using a more complex grammar and controlling for two types of associative chunk-strength, participants showed good realism (but still overconfidence) for both letter and symbol strings, irrespective of grammaticality. Together, these experiments show that implicit learning can give rise to knowledge products that are associated with fairly realistic meta-knowledge. It is argued that both the zero-correlation criterion and the guessing criterion are misplaced when used to define implicit knowledge; two reasons being that confidence judgements may be affected both by implicit knowledge and by inferences.     

Improving college students’ evaluation of text learning using idea-unit standards

When recalling key definitions from class materials, college students are often overconfident in the quality of their responses. Even with commission errors, they often judge that their response is entirely or partially correct. To further understand this overconfidence, we investigated whether idea-unit judgements would reduce overconfidence (Experiments 1 and 2) and whether students inflated their scores because they believed that they knew answers but just responded incorrectly (Experiment 2). College students studied key-term definitions and later attempted to recall each definition when given the key term (e.g., What is the availability heuristic?). All students judged the quality of their recall, but some were given a full-definition standard to use, whereas other students first judged whether their response included each of the individual ideas within the corresponding correct answer. In Experiment 1, making these idea-unit judgements reduced overconfidence for commission errors. In Experiment 2, some students were given the correct definitions and graded other students’ responses, and some students generated idea units themselves before judging their responses. Students were overconfident even when they graded other students’ responses, and, as important, self-generated idea units for each definition also reduced overconfidence in commission errors. Thus, overconfidence appears to result from difficulties in evaluating the quality of recall responses, and such overconfidence can be reduced by using idea-unit judgements.     

Determinants of Overconfidence and Miscalibration: The Roles of Random Error and Ecological Structure

Previous authors have attributed findings of overconfidence to psychological bias or to experimental designs unrepresentative of the environment. This paper provides evidence for an alternative explanation. A model is presented in which reported confidence is a function of the validity of information used by the subject, and a random error component. The model predicts greater overconfidence for question sets in which informational cues are less valid. This result corresponds to the well-known hard/easy effect. The model also predicts that unrepresentative design (Gigerenzer, Hoffrage, & Kleinbölting, 1991) is sufficient but not necessary for overconfidence to occur. These predictions are tested, and results provide support for the model. Subjects are overconfident according to usual measures such as calibration, even though on average they report the diagnosticity of informational cues correctly. Furthermore, overconfidence is greater for harder sets of questions, even when those sets are representative of the environment. A post hoc analysis reveals some intriguing individual differences among subjects. Some people appear to have a true psychological bias toward reporting high levelsof confidence, whereas others have a psychological bias in the direction of underconfidence.     

Why the unskilled are unaware: Further explorations of (absent) self-insight among the incompetent

People are typically overly optimistic when evaluating the quality of their performance on social and intellectual tasks. In particular, poor performers grossly overestimate their performances because their incompetence deprives them of the skills needed to recognize their deficits. Five studies demonstrated that poor performers lack insight into their shortcomings even in real world settings and when given incentives to be accurate. An additional meta-analysis showed that it was lack of insight into their own errors (and not mistaken assessments of their peers) that led to overly optimistic estimates among poor performers. Along the way, these studies ruled out recent alternative accounts that have been proposed to explain why poor performers hold such positive impressions of their performance.     

Actor-observer differences in realism in confidence and frequency judgments

Taking a social psychological approach to metacognitive judgments, this study analyzed the difference in realism (validity) in confidence and frequency judgments (i.e., estimates of overall accuracy) between one's own and another person's answers to general knowledge questions. Experiment 1 showed that when judging their own answers, compared with another's answers, the participants exhibited higher overconfidence, better ability to discriminate correct from incorrect answers, lower accuracy, and lower confidence. However, the overconfidence effect could be attributable to the lowest level of confidence. Furthermore, when heeding additional information about another's answers the participants showed higher confidence and better discrimination ability. The overconfidence effect of Experiment 1 was not found in Experiment 2. However, the results of Experiment 2 were consistent with Experiment 1 in terms of discrimination ability, confidence, and accuracy. Finally, in both experiments the participants gave lower frequency judgments of their own overall accuracy compared with their frequency judgments of another person's overall accuracy.     

过度自信的跨文化差异

过度自信是指个体过高地估计自身判断的精确度,其经典的研究范式是常识问题及常识问题的变式现实情境问题。使用常识问题的研究一致发现,东方人显著比西方人更过度自信;使用现实情景问题的研究则发现,西方人比东方人更过度自信或两者无显著性差异。对“东方人比西方人过度自信”的解释包括认知习惯和教育系统两种主流观点。最后对出现不同研究结果的原因进行讨论,并进一步探讨解释的有效性和“西方人比东方人过度自信”的原因。     

Overconfident prediction of future actions and outcomes by self and others

In a follow-up study to Dunning, Griffin, Milojkovic, and L. Ross (1990), which had investigated the phenomenon of overconfidence in social prediction, two samples of first-year undergraduates were invited to make predictions about their own future responses (and, in the case of Sample 2, also those of their roommates) over the months ahead. These predictions were accompanied by confidence estimates and were evaluated in the light of actual responses reported later by the subjects in question. The primary finding was that self-predictions, like social predictions, proved to be consistently overconfident. As in Dunning et al., moreover, overconfidence could be traced to two sources. First, expressions of particularly high confidence rarely proved to be warranted; as confidence increased, the gap between accuracy and confidence widened. Second, predictions that went against relevant base rates yielded very low accuracy in the face of relatively unattenuated confidence levels. The implications of these results are discussed, and one potentially important underlying mechanism--the failure to make adequate inferential allowance for the uncertainties of situational construal--is proposed for further research.