The overconfidence effect in social prediction

In five studies with overlapping designs and intents, subjects predicted a specific peer's responses to a variety of stimulus situations, each of which offered a pair of mutually exclusive and exhaustive response alternatives. Each prediction was accompanied by a subjective probability estimate reflecting the subjects' confidence in its accuracy--a measure validated in Study 5 by having subjects choose whether to "gamble" on the accuracy of their prediction or on the outcome of a simple aleatory event. Our primary finding was that in social prediction, as in other judgmental domains, subjects consistently proved to be highly overconfident. That is, regardless of the type of prediction item (e.g., responses to hypothetical dilemmas, responses to contrived laboratory situations, or self-reports of everyday behaviors) and regardless of the type of information available about the person whose responses they were predicting (e.g., predictions about roommates or predictions based on prior interviews), the levels of accuracy subjects achieved fell considerably below the levels required to justify their confidence levels. Further analysis revealed two specific sources of overconfidence. First, subjects generally were overconfident to the extent they were highly confident. Second, subjects were most likely to be overconfident when they knowingly or unknowingly made predictions that ran counter to the relevant response base rates and, as a consequence, achieved low accuracy rates that their confidence estimates failed to anticipate. Theoretical and normative implications are discussed and proposals for subsequent research offered.     

Actor-observer differences in realism in confidence and frequency judgments

Taking a social psychological approach to metacognitive judgments, this study analyzed the difference in realism (validity) in confidence and frequency judgments (i.e., estimates of overall accuracy) between one's own and another person's answers to general knowledge questions. Experiment 1 showed that when judging their own answers, compared with another's answers, the participants exhibited higher overconfidence, better ability to discriminate correct from incorrect answers, lower accuracy, and lower confidence. However, the overconfidence effect could be attributable to the lowest level of confidence. Furthermore, when heeding additional information about another's answers the participants showed higher confidence and better discrimination ability. The overconfidence effect of Experiment 1 was not found in Experiment 2. However, the results of Experiment 2 were consistent with Experiment 1 in terms of discrimination ability, confidence, and accuracy. Finally, in both experiments the participants gave lower frequency judgments of their own overall accuracy compared with their frequency judgments of another person's overall accuracy.     

Persistence of overconfidence in young children: Factors that lead to more accurate predictions of memory performance

Young children are typically overconfident regarding both cognitive abilities. This overconfidence may be due to development underpinnings. Previous research has demonstrated that children exhibit robust and persistent overconfidence in a simple memory-recall task. Two experiments investigated this overconfidence in 1st–4th and 4th–6th grade students. In the first experiment, we explored both the development of accurate predictions of recall and young students’ confidence in their memory performance predictions. It was found that not until 4th grade did students’ overconfidence begin to wane. In the second experiment, we investigated a condition under which 4th–6th graders might make more accurate predictions of their ability to recall simple stimuli, specifically, when the items to be remembered were unfamiliar to the students. The results confirmed our overconfidence in familiarity hypothesis. We discuss these findings in the context of metacognition.     

Effects of Role Assignment and Verbal Interaction on Accuracy and Overconfidence in Interpersonal Judgment1

In an experiment designed to test the effects of role assignment and verbal interaction on accuracy and overconfidence in interpersonal judgment, subjects estimated the questionnaire responses of a randomly assigned partner and assessed their confidence in each estimate. Ninety-five subjects were assigned to one of four conditions in a 2 (role vs. no-role) by 2 (verbal interaction vs. no-interaction) design. Results indicated that accuracy was positively related to the weight subjects gave to their own responses in estimating the other person's responses, and both role assignment and verbal interaction caused subjects to give less weight to their own responses. Because they gave less weight to their own responses, subjects in the role conditions were less accurate than those in the no-role conditions, and this reduction in accuracy resulted in greater overconfidence and worse calibration. Practical implications of these results are discussed.     

The domain specificity and generality of overconfidence: Individual differences in performance estimation bias

One hundred twenty-three college students performed a knowledge assessment task and a game of motor skill in which they had to predict their performance before each block of trials. There was a bias in the direction of overconfidence on both tasks, even though the latter involved the motor domain, did not require the use of numeric probabilities, and allowed predictions to be made by using an aggregate judgment made in a frequentist mode. An analysis of individual differences indicated that there was considerable domain specificity in confidence judgments. However, participants who persevered in showing overconfidence in the motor task—despite previous feedback revealing their overconfident performance predictions—were significantly more overconfident in the knowledge calibration task than were participants who moderated their motor performance predictions so as to remove their bias toward overconfidence. The latter finding is consistent with explanations of overconfidence effects that implicate mechanisms with some degree of domain generality.     

The psychology of being a beginner

Beginners are plagued with overconfidence. They perform the worst on tests of knowledge or skill and yet are the most overconfident. One would expect that learning would better calibrate beginners and lead to better quality decisions. However, learning can instead produce overconfidence. In this article I discuss how being beginner can lead to errors in self-assessments and riskier decisions. I review differences in beginner overconfidence in several places in the literature. First, the Dunning–Kruger Effect, which finds those with the least knowledge are the most overconfident. To be sure, beginners can range from rank beginners, those who have acquired no knowledge, to experienced beginners, beginners who having gained some but not vast knowledge. Next, I discuss the beginner's bubble that follows the trajectory of confidence and overconfidence as people transition from rank to experienced beginners. The beginner's bubble pattern finds rank beginners have insight into their poor abilities. However, with some learning there is a surge of confidence and overconfidence. Lastly, I explore differences in confidence and overconfidence as learners transition from rank to experienced beginners in other places in the literature.     

Depression, realism, and the overconfidence effect: are the sadder wiser when predicting future actions and events?

Do depressed individuals make more realistic judgments than their nondepressed peers in real world settings? Depressed and nondepressed Ss in 2 studies were asked to make predictions about future actions and outcomes that might occur in their personal academic and social worlds. Both groups of Ss displayed overconfidence, that is, they overestimated the likelihood that their predictions would prove to be accurate. Of key importance, depressed Ss were less accurate in their predictions, and thus more overconfident, than their nondepressed counterparts. These differences arose because depressed Ss (a) were more likely to predict the occurrence of low base-rate events and (b) were less likely to be correct when they made optimistic predictions (i.e., stated that positive events would occur or that aversive outcomes would not). Discussion focuses on implications of these findings for the depressive realism hypothesis.     

Unskilled but aware: reinterpreting overconfidence in low-performing students

People are generally overconfident in their self-assessments and this overconfidence effect is greatest for people of poorer abilities. For example, poor students predict that they will perform much better on exams than they do. One explanation for this result is that poor performers in general are doubly cursed: They lack knowledge of the material, and they lack awareness of the knowledge that they do and do not possess. The current studies examined whether poor performers in the classroom are truly unaware of their deficits by examining the relationship between students' exam predictions and their confidence in these predictions. Relative to high-performing students, the poorer students showed a greater overconfidence effect (i.e., their predictions were greater than their performance), but they also reported lower confidence in these predictions. Together, these results suggest that poor students are indeed unskilled but that they may have some awareness of their lack of metacognitive knowledge.     

Narcissism,confidence, and risk attitude

The present research addresses whether narcissists are more overconfident than others and whether this overconfidence leads to deficits in decision making. In Study 1, narcissism predicted overconfidence. This was attributable to narcissists' greater confidence despite no greater accuracy. In Study 2, participants were offered fair bets on their answers. Narcissists lost significantly more points in this betting task than non‐narcissists, due both to their greater overconfidence and greater willingness to bet. Finally, in Study 3, narcissists' predictions of future performance were based on performance expectations rather than actual performance. This research extends the literature on betting on knowledge to the important personality dimension of narcissism. Copyright © 2004 John Wiley & Sons, Ltd.     

Overconfidence: It Depends on How, What, and Whom You Ask

Many studies have reported that the confidence people have in their judgments exceeds their accuracy and that overconfidence increases with the difficulty of the task. However, some common analyses confound systematic psychological effects with statistical effects that are inevitable if judgments are imperfect. We present three experiments using new methods to separate systematic effects from the statistically inevitable. We still find systematic differences between confidence and accuracy, including an overall bias toward overconfidence. However, these effects vary greatly with the type of judgment. There is little general overconfidence with two-choice questions and pronounced overconfidence with subjective confidence intervals. Over- and underconfidence also vary systematically with the domain of questions asked, but not as a function of difficulty. We also find stable individual differences. Determining why some people, some domains, and some types of judgments are more prone to overconfidence will be important to understanding how confidence judgments are made.     

Adult age differences in the realism of confidence judgments: overconfidence, format dependence, and cognitive predictors

Realistic confidence judgments are essential to everyday functioning, but few studies have addressed the issue of age differences in overconfidence. Therefore, the authors examined this issue with probability judgment and intuitive confidence intervals in a sample of 122 healthy adults (ages: 35-40, 55-60, 70-75 years). In line with predictions based on the na?ve sampling model (P. Juslin, A. Winman, & P. Hansson, 2007), substantial format dependence was observed, with extreme overconfidence when confidence was expressed as an intuitive confidence interval but not when confidence was expressed as a probability judgment. Moreover, an age-related increase in overconfidence was selectively observed when confidence was expressed as intuitive confidence intervals. Structural equation modeling indicated that the age-related increases in overconfidence were mediated by a general cognitive ability factor that may reflect executive processes. Finally, the results indicated that part of the negative influence of increased age on general ability may be compensated for by an age-related increase in domain-relevant knowledge.     

Overconfidence and the Big Five

It has been consistently observed that people are generally overconfident when assessing their performance. In the current study, participants completed Goldberg's Big Five personality inventory and then completed a cognitive task designed to assess overconfidence (defined as the difference between confidence and accuracy). Extraversion significantly predicted overconfidence (with the other Big Five factors controlled statistically). In addition, openness/intellectance significantly predicted confidence and accuracy but not overconfidence (again, with the other Big Five factors controlled statistically). Theoretical implications and implications for future research are discussed.     

Confidence judgments in studies of individual differences

This paper examines psychometric properties of scores derived from calibration curves (overconfidence, calibration, resolution, and slope) and an analogue of overconfidence that is based on a posttest estimate of the proportion of correctly solved items. Four tests from the theory of fluid and crystallized intelligence were used, and two of these tests employed both sequential and simultaneous methods of item presentation. The results indicate that the overconfidence score not only has the highest reliability, but is the only score with a reliability normally considered adequate for use in individual differences research. There is some, albeit weak, difference in subjects' level of overconfidence between sequential and simultaneous methods of item presentation. Correlational evidence confirms our previous findings that overconfidence scores from perceptual and ‘knowledge’ tasks define the same factor. In agreement with the results of Gigerenzer, Hoffrage and Kleinbolting (1991), subjects' post-test estimates of their performance showed lower levels of overconfidence than did the traditional measures based on subjects' confidence judgment responses to individual items. Also, after controlling for the actual test performances, the post-test performance estimates and average confidence ratings were only slightly positively correlated, suggesting that different psychological processes may underlie these two measures. Finally, our results suggest that average confidence over all items in the test may be a more useful measure in individual differences research than scores derived from calibration curves.     

Calibration of sensory and cognitive judgments: Two different accounts

The purpose of this study was to test predictions of two recent theories of realism of confidence. Ecological approaches to realism of confidence in one's general knowledge (Gigerenzer et al. , 1991; Juslin, in press a ; Björkman, in press) predict good calibration or, in the case of poor cognitive adjustment, overconfidence, within the cognitive domain. The subjective distance theory of confidence in sensory discriminations (Björkman et al. , 1992) predicts a pervasive underconfidence bias for sensory discriminations. Empirical data are reported showing that: (a) Calibration for sensory judgments is considerably poorer than calibration for well adapted cognitive judgements, a difference that can be entirely traced to underconfidence in the sensory domain. (b) While an initial overconfidence bias in the cognitive domain is removed by outcome feedback, the bias observed in sensory discriminations is unaffected even by a prolonged feedback session. It is suggested that the nature of confidence in sensory discriminations is different from the nature of confidence in cognitive judgments.     

The role of construal processes in overconfident predictions about the self and others

Overconfident behavioral predictions and trait inferences may occur because people make inadequate allowance for the uncertainties of situational construal. In Studies 1-3, Ss estimated how much time or money they would spend in various hypothetical, incompletely specified situations. Ss then offered associated "confidence limits" under different "construal conditions". In Study 4, Ss made trait inferences about someone they believed had responded "deviantly"--again with situational details unspecified and construal conditions manipulated. In all 4 studies, Ss who made predictions or trait inferences without being able to assume the accuracy of their situational construals offered confidence limits no broader than those of Ss who made their responses contingent on such accuracy. Only in conditions where Ss were obliged to offer alternative construals did they appropriately broaden their confidence limits or weaken their trait inferences.     

Unskilled and optimistic: Overconfident predictions despite calibrated knowledge of relative skill

Those who are less skilled tend to overestimate their abilities more than do those who are more skilled—the so-called Dunning–Kruger effect. Less-skilled performers presumably have less of the knowledge needed to make informed guesses about their relative performance. If so, the Dunning–Kruger effect should vanish when participants do have access to information about their relative ability and performance. Competitive bridge players predicted their results for bridge sessions before playing and received feedback about their actual performance following each session. Despite knowing their own relative skill and showing unbiased memory for their performance, they made overconfident predictions consistent with a Dunning–Kruger effect. This bias persisted even though players received accurate feedback about their predictions after each session. The finding of a Dunning–Kruger effect despite knowledge of relative ability suggests that differential self-knowledge is not a necessary precondition for the Dunning–Kruger effect. At least in some cases, the effect might reflect a different form of irrational optimism.     

Regulation of memory accuracy with multiple answers: the plurality option

We report two experiments that investigated the regulation of memory accuracy with a new regulatory mechanism: the plurality option. This mechanism is closely related to the grain-size option but involves control over the number of alternatives contained in an answer rather than the quantitative boundaries of a single answer. Participants were presented with a slideshow depicting a robbery (Experiment 1) or a murder (Experiment 2), and their memory was tested with five-alternative multiple-choice questions. For each question, participants were asked to generate two answers: a single answer consisting of one alternative and a plural answer consisting of the single answer and two other alternatives. Each answer was rated for confidence (Experiment 1) or for the likelihood of being correct (Experiment 2), and one of the answers was selected for reporting. Results showed that participants used the plurality option to regulate accuracy, selecting single answers when their accuracy and confidence were high, but opting for plural answers when they were low. Although accuracy was higher for selected plural than for selected single answers, the opposite pattern was evident for confidence or likelihood ratings. This dissociation between confidence and accuracy for selected answers was the result of marked overconfidence in single answers coupled with underconfidence in plural answers. We hypothesize that these results can be attributed to overly dichotomous metacognitive beliefs about personal knowledge states that cause subjective confidence to be extreme.     

Probabilistic mental models: a Brunswikian theory of confidence

Research on people's confidence in their general knowledge has to date produced two fairly stable effects, many inconsistent results, and no comprehensive theory. We propose such a comprehensive framework, the theory of probabilistic mental models (PMM theory). The theory (a) explains both the overconfidence effect (mean confidence is higher than percentage of answers correct) and the hard-easy effect (overconfidence increases with item difficulty) reported in the literature and (b) predicts conditions under which both effects appear, disappear, or invert. In addition, (c) it predicts a new phenomenon, the confidence-frequency effect, a systematic difference between a judgment of confidence in a single event (i.e., that any given answer is correct) and a judgment of the frequency of correct answers in the long run. Two experiments are reported that support PMM theory by confirming these predictions, and several apparent anomalies reported in the literature are explained and integrated into the present framework.     

Improving metacognitive accuracy: How failing to retrieve practice items reduces overconfidence

People often exhibit inaccurate metacognitive monitoring. For example, overconfidence occurs when people judge that they will remember more information on a future test then they actually do. The present experiments examined whether a small number of retrieval practice opportunities would improve participants’ metacognitive accuracy by reducing overconfidence. Participants studied Lithuanian–English paired associates and predicted their performance on an upcoming memory test. Then they attempted to retrieve one or more practice items (or none in the control condition) and made a second prediction. Experiment 1 showed that failing to retrieve a single practice item lead to improved subsequent performance predictions – participants became less overconfident. Experiment 2 directly manipulated retrieval failure and showed that again failure to retrieve a single practice item significantly improved subsequent predictions, relative to when participants successfully retrieved the practice item. Finally, Experiment 3 showed that additional retrieval practice opportunities reduced overconfidence and improved prediction accuracy.     

Counselor Confidence and Accuracy of Prognoses of Success or Failure

This study assessed the accuracy of predictions of freshman and overall college scholastic performance made by groups of high school counselors, college advisors, and counseling psychologists from a university counseling center in relation to the confidence of these judges that their prognoses were accurate. Predictions were made from three sets of case information. The results revealed that: (1) the degree of confidence counselors indicated in their freshman and overall college “pass” predictions was appropriately related to accuracy; (2) counselor confidence in freshman “fail” predictions was not related to accuracy although the “fail” judgments tended to be more accurate than the “pass” prognoses; (3) counselor confidence in their overall “fail” predictions was not significantly related to accuracy and, unlike the results for the freshman judgments, the overall “fail” predictions were not more accurate than the “pass” predictions; (4) the amount of case data available was not related to counselor predictive accuracy.