Adult age differences in the realism of confidence judgments: overconfidence, format dependence, and cognitive predictors

Realistic confidence judgments are essential to everyday functioning, but few studies have addressed the issue of age differences in overconfidence. Therefore, the authors examined this issue with probability judgment and intuitive confidence intervals in a sample of 122 healthy adults (ages: 35-40, 55-60, 70-75 years). In line with predictions based on the na?ve sampling model (P. Juslin, A. Winman, & P. Hansson, 2007), substantial format dependence was observed, with extreme overconfidence when confidence was expressed as an intuitive confidence interval but not when confidence was expressed as a probability judgment. Moreover, an age-related increase in overconfidence was selectively observed when confidence was expressed as intuitive confidence intervals. Structural equation modeling indicated that the age-related increases in overconfidence were mediated by a general cognitive ability factor that may reflect executive processes. Finally, the results indicated that part of the negative influence of increased age on general ability may be compensated for by an age-related increase in domain-relevant knowledge.     

Overconfidence: It Depends on How, What, and Whom You Ask

Many studies have reported that the confidence people have in their judgments exceeds their accuracy and that overconfidence increases with the difficulty of the task. However, some common analyses confound systematic psychological effects with statistical effects that are inevitable if judgments are imperfect. We present three experiments using new methods to separate systematic effects from the statistically inevitable. We still find systematic differences between confidence and accuracy, including an overall bias toward overconfidence. However, these effects vary greatly with the type of judgment. There is little general overconfidence with two-choice questions and pronounced overconfidence with subjective confidence intervals. Over- and underconfidence also vary systematically with the domain of questions asked, but not as a function of difficulty. We also find stable individual differences. Determining why some people, some domains, and some types of judgments are more prone to overconfidence will be important to understanding how confidence judgments are made.     

Option fixation: A cognitive contributor to overconfidence

The ASC model of choice and confidence in general knowledge proposes that respondents first Assess the familiarity of presented options, and then use the high-familiarity option as a retrieval cue to Search memory for the purposes of Constructing an explanation about why that high-familiarity option is true. The ASC process implies that overconfidence results in part from a tendency to fixate on the high-familiarity option, to the neglect of the other option. If this implication is true, then judgment tasks requiring respondents to evaluate each option independently should result in reduced overconfidence as compared with standard judgment tasks. Two experiments tested this implication, and found that confidence and overconfidence were reduced when respondents evaluated options independently. The findings support the proposal that option fixation contributes to overconfidence, and also clarify the limitations of random error explanations of overconfidence.     

Aging and conditional probability judgments: a global matching approach

Age differences in bias in conditional probability judgments were investigated based on predictions derived from the Minerva-Decision Making model (M. R. P. Dougherty, C. F. Gettys, & E. E. Ogden, 1999), a global matching model of likelihood judgment. In this study, 248 younger and older adults completed frequency judgment and conditional probability judgment tasks. Age differences in the frequency judgment task are interpreted as an age-related deficit in memory encoding. Older adults' stronger biases in the probability judgment task point to age differences in criterion setting. Age-related biases were eliminated when age groups were equated on memory encoding by means of study time manipulation. The authors conclude that older adults' stronger judgment biases are a function of memory impairment.     

Probability judgment and subadditivity: The role of working memory capacity and constraining retrieval

In this research, we examined the role that individual differences in working memory (WM) capacity, the strength of alternatives, and time constraints play in probability judgment and subadditivity. With a laboratory-based learning task, Experiment 1 revealed that the degree to which participants’ probability judgments were subadditive was negatively correlated with a measure of WM capacity, even when variance due to short-term memory capacity was removed. In addition, participants were more subadditive when the viable alternatives were all rather weak. Experiment 2 extended the WM-capacity-subadditivity correlation to a population judgment task and revealed that subadditivity increases when the judgment task is performed under time constraints. Results support a model that assumes that people make probability judgments by comparing the focal hypothesis with relevant alternatives retrieved from long-term memory and that people high in WM span include more alternatives in the comparison process. Time constraints are assumed to truncate the alternative generation process, leading to fewer alternatives being recalled from long-term memory.     

The domain specificity and generality of overconfidence: Individual differences in performance estimation bias

One hundred twenty-three college students performed a knowledge assessment task and a game of motor skill in which they had to predict their performance before each block of trials. There was a bias in the direction of overconfidence on both tasks, even though the latter involved the motor domain, did not require the use of numeric probabilities, and allowed predictions to be made by using an aggregate judgment made in a frequentist mode. An analysis of individual differences indicated that there was considerable domain specificity in confidence judgments. However, participants who persevered in showing overconfidence in the motor task—despite previous feedback revealing their overconfident performance predictions—were significantly more overconfident in the knowledge calibration task than were participants who moderated their motor performance predictions so as to remove their bias toward overconfidence. The latter finding is consistent with explanations of overconfidence effects that implicate mechanisms with some degree of domain generality.     

The naïve intuitive statistician: a naïve sampling model of intuitive confidence intervals

The perspective of the na?ve intuitive statistician is outlined and applied to explain overconfidence when people produce intuitive confidence intervals and why this format leads to more overconfidence than other formally equivalent formats. The na?ve sampling model implies that people accurately describe the sample information they have but are na?ve in the sense that they uncritically take sample properties as estimates of population properties. A review demonstrates that the na?ve sampling model accounts for the robust and important findings in previous research as well as provides novel predictions that are confirmed, including a way to minimize the overconfidence with interval production. The authors discuss the na?ve sampling model as a representative of models inspired by the na?ve intuitive statistician.     

The recalcitrance of overconfidence and its contribution to decision aid neglect

Three experiments tested the hypothesis that people's overconfidence in the quality of their intuitive judgment strategies contributes to their reluctance to use helpful actuarial judgment aids. Participants engaged in a judgment task that required them to use five cues to decide whether a prospective juror favored physician‐assisted suicide. Participants had the opportunity to examine the judgments of a statistical equation that correctly classified 77% of the prospective jurors. In all experiments, participants infrequently examined the equation, performed worse than the equation, and were highly overconfident. In Experiments 1 and 2, outcome feedback and calibration feedback failed to reduce overconfidence. In Experiment 3, enhanced calibration feedback reduced overconfidence and increased reliance on the equation, thus leading to improved judgment performance. Copyright © 2005 John Wiley & Sons, Ltd.     

Subjective probability intervals: how to reduce overconfidence by interval evaluation

Format dependence implies that assessment of the same subjective probability distribution produces different conclusions about over- or underconfidence depending on the assessment format. In 2 experiments, the authors demonstrate that the overconfidence bias that occurs when participants produce intervals for an uncertain quantity is almost abolished when they evaluate the probability that the same intervals include the quantity. The authors successfully apply a method for adaptive adjustment of probability intervals as a debiasing tool and discuss a tentative explanation in terms of a naive sampling model. According to this view, people report their experiences accurately, but they are naive in that they treat both sample proportion and sample dispersion as unbiased estimators, yielding small bias in probability evaluation but strong bias in interval production.     

Realism of confidence in sensory discrimination

Is there a common and general basis for confidence in human judgment? Recently, we found that the properties of confidence judgments in the sensory domain mirror those previously established in the cognitive domain; notably, we found underconfidence on easy sensory judgments and overconfidence on hard sensory judgments. In contrast, data from the Uppsala laboratory in Sweden suggest that sensory judgments are unique; they found a pervasive underconfidence bias, with overconfidence being evident only on very hard sensory judgments. Olsson and Winman (1996) attempted to resolve the debate on the basis of methodological issues related to features of the stimulus display in a visual discrimination task. A reanalysis of the data reported in Baranski and Petrusic (1994), together with the findings of a new experiment that controlled stimulus display characteristics, supports the position that the difference between the Canadian and the Swedish data is real and, thus, may reflect cross-national differences in confidence in sensory discrimination.     

Culture and Probability Judgment Accuracy: The Influence of Holistic Reasoning

A well-established phenomenon in the judgment and decision-making tradition is the overconfidence one places in the amount of knowledge that one possesses. Overconfidence or probability judgment accuracy varies not only individually but also across cultures. However, research efforts to explain cross-cultural variations in the overconfidence phenomenon have seldom been made. In Study 1, the authors compared the probability judgment accuracy of U.S. Americans (N = 108) and Mexican participants (N = 100). In Study 2, they experimentally primed culture by randomly assigning English/Spanish bilingual Mexican Americans (N = 195) to response language. Results of both studies replicated the cross-cultural variation of probability judgment accuracy previously observed in other cultural groups. U.S. Americans displayed less overconfidence when compared to Mexicans. These results were then replicated in bilingual participants, when culture was experimentally manipulated with language priming. Holistic reasoning did not account for the cross-cultural variation of overconfidence. Suggestions for future studies are discussed.     

Confidence depends on level of aggregation

The credible intervals that people set around their point estimates are typically too narrow (cf. Lichtenstein, Fischhoff, & Phillips, 1982). That is, a set of many such intervals does not contain the actual values of the criterion variables as often as it should given the probability assigned to this event for each estimate. The typical interpretation of such data is that people are overconfident about the accuracy of their judgments. This paper presents data from two studies showing the typical levels of overconfidence for individual estimates of unknown quantities. However, data from the same subjects on a different measure of confidence for the same items, their own global assessment for the set of multiple estimates as a whole, showed significantly lower levels of confidence and overconfidence than their average individual assessment for items in the set. It is argued that the event and global assessments of judgment quality are fundamentally different and are affected by unique psychological processes. Finally, we discuss the implications of a difference between confidence in single and multiple estimates for confidence research and theory.     

Memory storage and retrieval processes in category learning

The detailed course of learning is studied for categorization tasks defined by independent or contingent probability distributions over the features of category exemplars. College-age subjects viewed sequences of bar charts that simulated symptom patterns and responded to each chart with a recognition and a categorization judgment. Fuzzy, probabilistically defined categories were learned relatively rapidly when individual features were correlated with category assignment, more slowly when only patterns carried category information. Limits of performance were suboptimal, evidently because of capacity limitations on judgmental processes as well as limitations on memory. Categorization proved systematically related to feature and exemplar probabilities, under different circumstances, and to similarity among exemplars of categories. Unique retrieval cues for exemplar patterns facilitated recognition but entered into categorization only at retention intervals within the range of short-term memory. The findings are interpreted within the framework of a general array model that yields both exemplar-similarity and feature-frequency models as special cases and provides quantitative accounts of the course of learning in each of the categorization tasks studied.     

Calibration of sensory and cognitive judgments: A single model for both

In a recent issue of this journal, Winman and Juslin (34 , 135–148, 1993) present a model of the calibration of subjective probability judgments for sensory discrimination tasks. They claim that the model predicts a pervasive underconfidence bias observed in such tasks, and present evidence from a training experiment that they interpret as supporting the notion that different models are needed to describe judgment of confidence in sensory and in cognitive tasks. The model is actually part of the more comprehensive decision variable partition model of subjective probability calibration that was originally proposed in Ferrell and McGoey (Organizational Behavior and Human Performance, 26 , 32–53, 1980). The characteristics of the model are described and it is demonstrated that the model does not predict underconfidence, that it is fully compatible with the overconfidence frequently found in calibration studies with cognitive tasks, and that it well represents experimental results from such studies. It is concluded that only a single model is needed for both types of task.     

Integration of the ecological and error models of overconfidence using a multiple-trace memory model.

A memory processes account of the calibration of probability judgments was examined. A multiple-trace memory model, Minerva-Decision Making (MDM; M. R. P. Dougherty, C. F. Gettys, & E. E. Ogden, 1999), used to integrate the ecological (Brunswikian) and the error (Thurstonian) models of overconfidence, is described. The model predicts that overconfidence should decrease both as a function of experience and as a function of encoding quality. Both increased experience and improved encoding quality result in lower variance in the output of the model, which in turn leads to improved calibration. Three experiments confirmed these predictions. Implications of MDM's account of overconfidence are discussed.     

Cross-task and cross-manipulation stability in shifting the decision criterion

In recognition memory experiments participants must discriminate between old and new items, a judgment influenced by response bias. Research has shown substantial individual differences in the extent to which people will strategically adjust their response bias to diagnostic cues such as the prior probability of an old item. Despite this significant between subject variability, shifts in bias have been found to be relatively predictive within individuals across memory tests. Experiment 1 sought to determine whether this predictability extends beyond memory. Results revealed that the amount a subject shifted response bias in a recognition memory task was significantly predictive of shifting in a visual perception task, suggesting that shifting can generalise outside of a specific testing domain. Experiment 2 sought to determine how predictive shifting would be across two manipulations well known to induce shifts in bias: a probability manipulation and a response payoff manipulation. A modest positive relationship between these two methods was observed, suggesting that shifting behaviour is relatively predictive across different manipulations of shifting. Overall, results from both experiments suggest that response bias shifting, like response bias setting, is a relatively stable behaviour within individuals despite changes in test domain and test manipulation.     

直觉和分析在不同信息获取与呈现方式下对多特征目标判断质量的影响

采用多特征目标判断任务,考察了不同信息获取方式与信息呈现方式条件下,直觉和分析思维模式对判断质量的影响。结果发现：（1）在多特征目标判断任务中,直觉思维比分析思维更有效;（2）直觉型决策者运用直觉思维的判断质量比感觉型决策者更高;（3）归类组块的信息呈现方式能够有效提升决策者运用直觉思维的判断质量;（4）思维模式、信息获取方式与信息呈现方式对多特征目标的判断质量存在三阶交互作用,归类组块的信息呈现方式使直觉型决策者运用直觉思维的判断质量最高,而运用分析思维的判断质量低于感觉型决策者;归类组块排列信息时,直觉型决策者运用分析思维的判断质量低于信息随机排列时的水平。     

Effects of reiteration,hindsight bias,and memory on realism in eyewitness confidence

This study investigated the effects of repetition, memory, feedback, and hindsight bias on the realism in confidence in answers to questions on a filmed kidnapping. In Experiment 1 the participants showed overconfidence in all conditions. In the Repeat condition (‘how confident are you now that your previous answers are correct’) overconfidence was reduced as a consequence of the decrease in confidence in both correct and incorrect answers compared with the Repeat condition when the participants received feedback on their answers and were asked to remember their initial confidence, the confidence level was higher for correct and lower for incorrect answers. In Experiment 2, recalled confidence (the Memory condition) increased compared with the original confidence both for correct and incorrect answers; the effect of this was increased overconfidence. The Hindsight condition showed a decrease in confidence in incorrect answers. The results suggest that a unique hindsight effect may be more clearly present for incorrect than for correct answers. Our study gives further evidence for the malleability of the realism in eyewitness confidence and we discuss both the theoretical and forensic implications of our findings. Copyright © 2000 John Wiley & Sons, Ltd.     

True Overconfidence in Interval Estimates: Evidence Based on a New Measure of Miscalibration

Overconfidence is often regarded as one of the most prevalent judgment biases. Several studies show that overconfidence can lead to suboptimal decisions of investors, managers, or politicians. Recent research, however, questions whether overconfidence should be regarded as a bias and shows that standard “overconfidence” findings can easily be explained by different degrees of knowledge of agents plus a random error in predictions. We contribute to the current literature and ongoing research by extensively analyzing interval estimates for knowledge questions, for real financial time series, and for artificially generated charts. We thereby suggest a new method to measure overconfidence in interval estimates, which is based on the implied probability mass behind a stated prediction interval. We document overconfidence patterns, which are difficult to reconcile with rationality of agents and which cannot be explained by differences in knowledge as differences in knowledge do not exist in our task. Furthermore, we show that overconfidence measures are reliable in the sense that there exist stable individual differences in the degree of overconfidence in interval estimates, thereby testing an important assumption of behavioral economics and behavioral finance models: stable individual differences in the degree of overconfidence across people. We do this in a “field experiment,” for different levels of expertise of subjects (students on the one hand and professional traders and investment bankers on the other hand), over time, by using different miscalibration metrics, and for tasks that avoid common weaknesses such as a non‐representative selection of trick questions. Copyright © 2012 John Wiley & Sons, Ltd.     

Persistence of overconfidence in young children: Factors that lead to more accurate predictions of memory performance

Young children are typically overconfident regarding both cognitive abilities. This overconfidence may be due to development underpinnings. Previous research has demonstrated that children exhibit robust and persistent overconfidence in a simple memory-recall task. Two experiments investigated this overconfidence in 1st–4th and 4th–6th grade students. In the first experiment, we explored both the development of accurate predictions of recall and young students’ confidence in their memory performance predictions. It was found that not until 4th grade did students’ overconfidence begin to wane. In the second experiment, we investigated a condition under which 4th–6th graders might make more accurate predictions of their ability to recall simple stimuli, specifically, when the items to be remembered were unfamiliar to the students. The results confirmed our overconfidence in familiarity hypothesis. We discuss these findings in the context of metacognition.