Confidence depends on level of aggregation

The credible intervals that people set around their point estimates are typically too narrow (cf. Lichtenstein, Fischhoff, & Phillips, 1982). That is, a set of many such intervals does not contain the actual values of the criterion variables as often as it should given the probability assigned to this event for each estimate. The typical interpretation of such data is that people are overconfident about the accuracy of their judgments. This paper presents data from two studies showing the typical levels of overconfidence for individual estimates of unknown quantities. However, data from the same subjects on a different measure of confidence for the same items, their own global assessment for the set of multiple estimates as a whole, showed significantly lower levels of confidence and overconfidence than their average individual assessment for items in the set. It is argued that the event and global assessments of judgment quality are fundamentally different and are affected by unique psychological processes. Finally, we discuss the implications of a difference between confidence in single and multiple estimates for confidence research and theory.     

The overconfidence effect in social prediction

In five studies with overlapping designs and intents, subjects predicted a specific peer's responses to a variety of stimulus situations, each of which offered a pair of mutually exclusive and exhaustive response alternatives. Each prediction was accompanied by a subjective probability estimate reflecting the subjects' confidence in its accuracy--a measure validated in Study 5 by having subjects choose whether to "gamble" on the accuracy of their prediction or on the outcome of a simple aleatory event. Our primary finding was that in social prediction, as in other judgmental domains, subjects consistently proved to be highly overconfident. That is, regardless of the type of prediction item (e.g., responses to hypothetical dilemmas, responses to contrived laboratory situations, or self-reports of everyday behaviors) and regardless of the type of information available about the person whose responses they were predicting (e.g., predictions about roommates or predictions based on prior interviews), the levels of accuracy subjects achieved fell considerably below the levels required to justify their confidence levels. Further analysis revealed two specific sources of overconfidence. First, subjects generally were overconfident to the extent they were highly confident. Second, subjects were most likely to be overconfident when they knowingly or unknowingly made predictions that ran counter to the relevant response base rates and, as a consequence, achieved low accuracy rates that their confidence estimates failed to anticipate. Theoretical and normative implications are discussed and proposals for subsequent research offered.     

The accuracy of beliefs about retrieval cues

We investigated the accuracy of predictions about semantic, environmental, and phonological cues for remembering. Subjects rated the pleasantness of 10 words in each of four rooms, predicted the number of words that they would recall with and without one of the three types of cues, and then were tested for free or cued recall. Consistent with their predictions, subjects who received semantic cues recalled more words than did subjects in the free-recall group. The subjects in the other cuing conditions did not benefit from the cues; furthermore, they overestimated the value of phonological cues, and they believed that environmental cues were ineffective. Finally, confidence ratings for cued-recall predictions did not reflect the pattern of cued-recall performance. Subjects were least confident about their predictions for semantic cuing and most confident about their predictions for recall to be cued phonologically.     

Creating new memories that are quickly accessed and confidently held

In two experiments involving a total of 542 subjects, a series of slides depicting a burglary was shown. After the initial event, subjects were exposed to one or more narratives about the event that contained some misinformation or neutral information about four critical details. Finally, subjects were tested on their memories of what they saw, and their reaction times and confidence levels were measured. When subjects took a standard test in which the misinformation item was a possible response option, they responded very quickly and confidently when making this incorrect choice. Misled subjects responded as quickly and confidently to these "unreal" memories as they did to their genuine memories. It does not seem, then, that the misinformation effect arises from a large proportion of subjects who must resolve a conflict between two memories when they are tested, a conflict that would be expected to take time. When subjects took a modified test in which the misinformation item was not a possible response, misled subjects were as accurate as were controls, but they responded more slowly, regardless of whether they ultimately chose the right or wrong option. These findings indicate that misinformation does introduce some form of interference not detected by a simple test of accuracy.     

Remembering September 11th: the role of retention interval and rehearsal on flashbulb and event memory

Retention interval and rehearsal effects on flashbulb and event memory for 11th September 2001 (9/11) were examined. In Experiment 1, college students were assessed three times (Groups 1 and 2) or once (Group 3) over 11 weeks. In Experiment 2, three new groups assessed initially at 23 weeks (Group 4), 1 year (Group 5), or 2 years (Group 6) were compared at 1 year and at 2 years with subsamples of those assessed previously. No effects of retention interval length or rehearsal were found for flashbulb memory, which contained details at each assessment. Event memory, but not consistency, was detrimentally affected by long retention intervals, but improved with rehearsal. Recall was higher for the reception event than for the main events. Also, consistency from 1 day to 11 weeks, but not 1 year to 2 years, was higher for flashbulb memory than for event memory. Event recall was enhanced when respondents conceived of their memory as vivid, frozen, and encompassing a longer period of time. Positive correlations were found for event memory with confidence in accuracy and with rehearsal through discussion at 2 years.     

Imagination equally influences false memories of high and low plausibility events

To examine the effects of event plausibility on people's false beliefs and memories for imagined childhood events, subjects took part in a three‐stage procedure. First, subjects rated how confident they were that they had experienced certain childhood events. They also rated their memories of the events. Second, 1 week later, subjects imagined one high, one moderate and one low plausibility event. Third, 1 week later (and 2 weeks after their initial ratings), subjects rated their confidence and memory a second time. Imagining the events made subjects more confident that they were genuine experiences and gave subjects clearer and more complete memories. Plausibility did not affect subjects' confidence but it did affect their memories. Subjects developed clearer and more complete memories for high, followed by moderate, followed by low plausibility events regardless of whether those events were imagined. We use a nested model of plausibility, belief and memory to discuss our findings. Copyright © 2008 John Wiley & Sons, Ltd.     

Different confidence-accuracy relationships for feature-based and familiarity-based memories

Participants studied naturalistic pictures presented for varying brief durations and then received a recognition test on which they indicated whether each picture was old or new and rated their confidence. In 1 experiment they indicated whether each "old"/"new" response was based on memory for a specific feature in the picture or instead on the picture's general familiarity; in another experiment, we defined pictures that tended to elicit feature versus familiarity responses. Thus, feature/familiarity was a dependent variable in 1 experiment and an independent variable in the other. In both experiments feature-based responses were more accurate than those that were familiarity based, and confidence and accuracy increased with duration for both response types. However, when confidence was controlled for, mean accuracy was higher for familiarity-based than for feature-based responses. The theoretical implication is that confidence and accuracy arise from different underlying information. The applied implication is that confidence differences should not be taken as implying accuracy differences when the phenomenal basis of the memory reports differ.     

(Non)Complementary updating of belief in two hypotheses

Past research has led to the conclusion that two competing hypotheses are represented dependently, and confidence in them is updated in a complementary manner. It is argued here that confidence in two hypotheses can be represented either dependently or independently. Changes in confidence in the former case are always complementary, but changes in the latter case are complementary only under certain conditions. In three simulated medical diagnosis experiments, subjects learned about two illnesses in a manner expected to lead to either dependent or independent confidence. They were then presented with two symptoms sequentially (for each of several patients), updating confidence after each. Experiment 1 demonstrated that changes in confidence in the two illnesses were largely complementary for subjects with dependent, but not independent, confidence. Experiment 2 showed that encouraging consideration of the alternative led to more complementary changes for subjects with independent confidence. Experiment 3 succeeded in producing complementary changes from these subjects. Thus, complementarity does not imply dependent confidence, nor does independent confidence imply non-complementarity.     

Confidence in Personal Preferences

The first aim of this study was to test the self‐consistency model (SCM) of subjective confidence as it applies to personal preferences. According to SCM, participants presented with a two‐alternative forced‐choice (2AFC) item draw a small sample of representations of the item. Their confidence reflects the extent to which the choice is representative of the population of representations associated with the item, and the likelihood of making that choice on subsequent occasions. The second aim was to use confidence judgment as a clue to the dynamics of online preference construction. Participants were presented with 2AFC items measuring everyday personal preferences. The task was presented five times. In line with SCM, (i) when participants changed their preferences across presentations, they were systematically more confident when they made their more frequent choice; (ii) confidence in a choice in the item's first presentation predicted the likelihood of repeating that choice in subsequent presentations; (iii) despite the idiosyncratic nature of personal preferences, confidence was higher for consensual than for nonconsensual preferences; (iv) when participants predicted the preferences of others, they were also more confident when their predictions agreed with those of others; and (v) the confidence/accuracy correlation for predictions was positive for consensually correct but negative for consensually wrong predictions. These results suggest that confidence in preferences can help separate between the stable and variable contributions to preference construction in terms of the population of representations available in memory and the representations that are accessible at the time of preference solicitation, respectively. Copyright © 2012 John Wiley & Sons, Ltd.     

The behavioral theory of timing: Reinforcer rate determines pacemaker rate

In the behavioral theory of timing, pulses from a hypothetical Poisson pacemaker produce transitions between states that are correlated with adjunctive behavior. The adjunctive behavior serves as a discriminative stimulus for temporal discriminations. The present experiments tested the assumption that the average interpulse time of the pacemaker is proportional to interreinforcer interval. Responses on a left key were reinforced at variable intervals for the first 25 s since the beginning of a 50-s trial, and right-key responses were reinforced at variable intervals during the second 25 s. Psychometric functions relating proportion of right-key responses to time since trial onset, in 5-s intervals across the 50-s trial, were sigmoidal in form. Average interpulse times derived by fitting quantitative predictions from the behavioral theory of timing to obtained psychometric functions decreased when the interreinforcer interval was decreased and increased when the interreinforcer interval was increased, as predicted by the theory. In a second experiment, average interpulse times estimated from trials without reinforcement followed global changes in interreinforcer interval, as predicted by the theory. Changes in temporal discrimination as a function of interreinforcer interval were therefore not influenced by the discrimination of reinforcer occurrence. The present data support the assumption of the behavioral theory of timing that interpulse time is determined by interreinforcer interval.     

Twelve‐ to 14‐month‐old infants can predict single‐event probability with large set sizes

Previous research has revealed that infants can reason correctly about single‐event probabilities with small but not large set sizes ( Bonatti, 2008 ; Teglas et al., 2007 ). The current study asks whether infants can make predictions regarding single‐event probability with large set sizes using a novel procedure. Infants completed two trials: A preference trial to determine whether they preferred pink or black lollipops and a test trial where infants saw two jars, one containing mostly pink lollipops and another containing mostly black lollipops. The experimenter removed one occluded lollipop from each jar and placed them in two separate opaque cups. Seventy‐eight percent of infants searched in the cup that contained a lollipop from the jar with a higher proportion of their preferred color object, significantly better than chance. Thus infants can reason about single‐event probabilities with large set sizes in a choice paradigm, and contrary to most findings in the infant literature, the prediction task used here appears a more sensitive measure than the standard looking‐time task.     

Accountability and judgment processes in a personality prediction task

In this experiment, we investigated the impact of accountability--social pressures to justify one's views to others--on cognitive processing in a personality-prediction task. Subjects were presented with the responses of actual test-takers to 16 items from Jackson's Personality Research Form (PRF) and asked to predict how these individuals responded to an additional set of 16 items from the same test. Subjects were assigned to a no-accountability condition (they learned that all of their responses would be anonymous), a preexposure-accountability condition (they learned of the need to justify their responses before seeing the test-takers' PRF responses), and a postexposure-accountability condition (they learned of the need to justify their responses after seeing the test-takers' PRF responses). Preexposure-accountability subjects reported more integratively complex impressions of test-takers, made more accurate behavioral predictions, and reported more appropriate levels of confidence in their predictions than did either no-accountability or postexposure-accountability subjects. We conclude by considering possible psychological mediators of these effects as well as the broader theoretical implications of the findings for the development of contingency models of judgment and choice.     

Self-doubt: Uncertainty as a motivating factor on effort in an exercise endurance task

ObjectivesResearch on the relationship between self-efficacy beliefs and subsequent performance on a task has typically supported a positive linear model. However, these results typically excluded a moderate level of self-efficacy as an independent variable nor used quadratic regression analyses to test for a curvilinear relationship. There are reasons to believe that a more accurate relationship between self-efficacy and performance is curvilinear (i.e., that some self-doubt of self-efficacy may predict optimal effort) under certain circumstances.DesignThe current study examined this relationship with the muscular endurance task of a plank exercise.MethodsSeventy-five participants participated in two trials of the exercise. Self-efficacy was recorded prior to each trial and performance in the plank exercise was used as an indicator of motivational effort.ResultsThere was a significant curvilinear relationship between efficacy and performance on the first trial and a significant linear relationship between the two on the second trial. Further analyses showed that individuals who substantially over or underestimated their abilities on the first trial did not significantly alter their effort on the second trial.ConclusionsThe current study provides some support for the possibility that some self-doubt can be a motivating factor for individuals to exert maximal effort when initially attempting an exercise endurance task.     

The threshold for successiveness

In Experiment 1, three stimuli were presented on each trial; the second and third were simultaneous. Ss said which of the three they saw first and then made a different second guess. Second guesses were correct more often than chance when first guesses were wrong, contradicting the predictions of a psychological moment theory. Psychometric functions are inconsistent with a theory which holds that Ss can order their perceptions and that errors result from lack of correspondence between this order and that of the stimuli. In Experiment 2, two pairs of stimuli, each simultaneous or successive, were presented on each trial, the interval between their onsets varying, and Ss guessed whether each was successive. A moment-like cyclic process affecting the perception of successive stimuli should raise the correlation between responses when both pairs were successive and beginning at the same time; this was not found.     

A photo,a suggestion,a false memory

We examined the effect of photographs on children's memories for events that did and did not happen. Over three interviews, 10‐year olds saw three true photos and one false photo. Half the children saw a doctored photo of themselves and other family members in a hot air balloon, while the remaining half saw only the hot air balloon. At each interview, children reported what they could remember about each event, rated their confidence that the events really happened and rated how much they could remember. Children who saw the photo showing themselves in the balloon developed more false memories than those who saw only the balloon, but when children in either condition developed false memories, they were equally confident that the event was real. These data highlight a potential problem with the use of photographs as tools in therapy. Copyright © 2007 John Wiley & Sons, Ltd.     

Curiosity and exploratory behaviour towards possible and impossible events in children and adults

In four experiments with 4‐, 6‐, and 9‐year‐old children and adults, the hypothesis was tested that, all other conditions being equal, a novel and unusual event elicits stronger curiosity and exploratory behaviour if its suggested explanation involves an element of the supernatural than if it does not (the impossible over possible effect – the I/P effect). Participants were shown an unusual phenomenon (a spontaneous disintegration of a physical object in an apparently empty box) framed in the context of either a magical (the impossible event) or scientific (the possible event) explanation. In the verbal trial, participants showed a clear understanding of the difference between the effect of genuine magic and the effect of a trick. In the behavioural trial, both children and adults showed the I/P effect. They were more likely to run the risk of losing their valuable objects in order to explore the impossible event than the possible event. Follow‐up experiments showed that the I/P effect couldn't be explained as an artifact of the different degrees of cost of exploratory behaviour in the possible and impossible conditions or as a result of misinterpreting magic as tricks. The I/P effect emerged when the cost of exploratory behaviour was moderate and disappeared when the cost was perceived as too high or too low.     

Theory Testing Using Quantitative Predictions of Effect Size

Traditional Null Hypothesis Testing procedures are poorly adapted to theory testing. The methodology can mislead researchers in several ways, including: (a) a lack of power can result in an erroneous rejection of the theory; (b) the focus on directionality (ordinal tests) rather than more precise quantitative predictions limits the information gained; and (c) the misuse of probability values to indicate effect size. An alternative approach is proposed which involves employing the theory to generate explicit effect size predictions that are compared to the effect size estimates and related confidence intervals to test the theoretical predictions. This procedure is illustrated employing the Transtheoretical Model. Data from a sample ( N = 3,967) of smokers from a large New England HMO system were used to test the model. There were a total of 15 predictions evaluated, each involving the relation between Stage of Change and one of the other 15 Transtheoretical Model variables. For each variable, omega-squared and the related confidence interval were calculated and compared to the predicted effect sizes. Eleven of the 15 predictions were confirmed, providing support for the theoretical model. Quantitative predictions represent a much more direct, informative, and strong test of a theory than the traditional test of significance.     

Self-discrepancies and emotional vulnerability: how magnitude, accessibility, and type of discrepancy influence affect

Two studies examined whether the type of emotional change experienced by individuals is influenced by the magnitude and accessibility of the different types of self-discrepancies they possess. In both studies, subjects filled out a measure of self-discrepancy a few weeks prior to the experimental session. Subjects were asked to list up to 10 attributes each for different self-states--their actual self, their ideal self (their own or others' hopes and goals for them), and their ought self (their own or others' beliefs about their duty and obligations). Magnitude of self-discrepancy was calculated by comparing the attributes in the actual self to the attributes in either the ideal self or the ought self, with the total number of attribute pairs that matched being subtracted from the total number of attribute pairs that mismatched. In Study 1, subjects were asked to imagine either a positive event or a negative event and were then given a mood measure and a writing-speed task. Subjects with a predominant actual:ideal discrepancy felt more dejected (e.g., sad) and wrote more slowly in the negative event condition than in the positive event condition, whereas subjects with a predominant actual:ought discrepancy, if anything, felt more agitated (e.g., afraid) and wrote more quickly in the negative event condition. In Study 2, subjects were selected who were either high in both kinds of discrepancies or low in both. Half of the subjects in each group were asked to discuss their own and their parents' hopes and goals for them (ideal priming), and the other half were asked to discuss their own and their parents' beliefs concerning their duty and obligations (ought priming). For high-discrepancy subjects, but not low-discrepancy subjects, ideal priming increased their dejection whereas ought priming increased their agitation. The implications of these findings for identifying cognitive-motivational factors that may serve as vulnerability markers for emotional problems is discussed.     

Persistent overconfidence despite practice: The role of task experience in preschoolers’ recall predictions

In three experiments, preschoolers’ ability to predict their picture recall was examined. Children studied 10 pictures, predicted how many they would recall, and then attempted to recall them. This study-prediction-recall trial was repeated multiple times with new pictures on each trial. In Experiment 1, children were overconfident on the initial trial, and this overconfidence persisted across three trials. In Experiment 2, children predicted either their own performance or another child’s performance. Their predictions were overconfident across all trials regardless of whether they made predictions for themselves or for another child, suggesting that wishful thinking cannot fully account for their overconfidence. In Experiment 3, some children postdicted their previous recall performance prior to making each prediction. Although their postdictions were quite accurate, their predictions were still overconfident across five trials. Preschoolers’ overconfidence was remarkably resistant to the repeated experience of recalling fewer pictures than the children had predicted. Even asking them to report the number that they recalled on a previous trial, which they could do accurately, did not cause them to lower their predictions across trials.     

On the evaluation of one-sided evidence

We examine predictions and judgments of confidence based on one-sided evidence. Some subjects saw arguments for only one side of a legal dispute while other subjects (called ‘jurors’) saw arguments for both sides. Subjects predicted the number of jurors who favored the plaintiff in each case. Subjects who saw only one side made predictions that were biased in favor of that side. Furthermore, they were more confident but generally less accurate than subjects who saw both sides. The results indicate that people do not compensate sufficiently for missing information even when it is painfully obvious that the information available to them is incomplete. A simple manipulation that required subjects to evaluate the relative strength of the opponent's side greatly reduced the tendency to underweigh missing evidence.