The Confidence-Accuracy Relationship: The Effects of Confidence Assessment and Choosing1

Male and female college undergraduates were exposed to a staged theft. For half of the subjects, confidence judgments were assessed both before and after viewing a photo lineup. For the other half, confidence judgments were assessed only after viewing the lineup. Subjects in both conditions viewed a target-present or target-absent lineup under negativey biased, unbiased, or positively biased instructions. Across all subjects, confidence and accuracy were significantly correlated (r= .30). There was a significantly stronger relationship between confidence and accuracy among choosers (r= 50) than among nonchoosers (r= .14). Choosing and confidence did not correlate significantly with each other. Identification accuracy was significantly poorer when witnesses had been asked before viewing the lineup to state their confidence that they would make an accurate identification than when confidence was measured only after an identification had been attempted. However, the before-after manipulation did not affect the magnitude of the confidence-accuracy relationship. The present results offer some support for the general proposition that choosing and the timing of confidence assessments should be viewed as moderating variables in the interpretation of the confidence-accuracy relationship. These data offer little support for predictions based upon self-perception theory and are in direct disagreement with the widely held assertion that witnesses are confident in whatever choice they make, regardless of its correctness.     

Demonstrating and eliminating the aggregation effect in physical skill tasks

Confidence judgments can be elicited in multiple ways. One of these procedures is to provide confidence judgments regarding each of a number of cases (individual judgments). A second procedure is to provide confidence judgments about a set of items (an aggregate judgment). Much research has demonstrated an aggregation effect—that individual judgments are more confident than aggregate judgments—within the cognitive knowledge domain. However, this effect has not previously been investigated with physical performance skill tasks. In three experiments, participants gave individual and aggregate judgments regarding the number of successful tosses they would make in either a ring toss, ping‐pong toss, or basketball toss task. In keeping with the aggregation effect, individual judgments were more confident than were aggregate judgments of success. Additionally, we eliminated the aggregation effect in Experiments 2 and 3 by employing a case‐specific base rate manipulation. Consistent with previous research with cognitive tasks, these results suggest that individual confidence judgments for physical skill tasks are determined primarily by characteristics associated with the individual case to be judged, whereas aggregate confidence judgments are determined by a more general evaluation of one's ability in the domain. Copyright © 2009 John Wiley & Sons, Ltd.     

Effects of alternating set for speed or accuracy on response time,accuracy and confidence in a unidimensional discrimination task

Three approaches to a theoretical analysis of confidence judgments are considered: one linking confidence to the number of sensory observations, one based on a distinction between ‘state’ and ‘process’ factors, and a ‘balance of evidence’ hypothesis developed from an accumulator model of discrimination. An experiment is described in which observers were asked to decide which of two parallel lines was the longer, and to rate their confidence in each judgment. Each observer's set for speed or accuracy was manipulated over successive blocks of trials, and effects on response time, accuracy, and confidence were examined. Under an accuracy set, observers produced a higher percentage of correct responses, had longer response times, and made more confident judgments than under a set for speed. Within both speed and accuracy blocks, however, confidence ratings were inversely related to response time. The data on response accuracy, time, and confidence indicate certain deficiencies in either of the first two approaches, but were well accounted for by the third.     

Accuracy of confidence judgments: Stability and generality in the learning process and predictive validity for learning outcome

One aspect of metacognition is the monitoring of memory or comprehension measured with retrospective confidence judgments after test taking. The research questions of the present study were whether different measures for the accuracy of such confidence judgments are stable over learning time, whether they generalize over two different tests, and whether they predict the learning outcome. In order to answer these questions, a study was conducted in which university students (N = 113) learned about the basic concepts of operant conditioning for 30 min. Knowledge tests with confidence judgments presented after each item were obtained before learning, after 10 min, and at the end of the learning session. Bias and absolute bias were calculated as absolute measures of accuracy, and gamma, Pearson’s r, andd _a were calculated as relative measures of accuracy. The results showed that the absolute measures were stable, but that the relative measures were not. Furthermore, absolute bias, gamma, and Pearson’s r obtained 10 min after the beginning of the learning process predicted the learning outcome. The results are discussed with regard to research on different measures of accuracy for confidence judgments.     

Confidence and Eyewitness Identifications: The Cross‐Race Effect,Decision Time and Accuracy

Participants encountered same‐race and cross‐race faces at encoding, completed a series of line‐up identification tests and provided confidence ratings by using one of nine different confidence scales. Confidence was less well calibrated with identification accuracy when participants selected a cross‐race than a same‐race face because of overconfidence. By contrast, there was no cross‐race effect on confidence–accuracy calibration when participants responded ‘not present’. Whereas confidence was a very strong predictor of accuracy for fast identifications of a line‐up face, this was much less the case for slower decisions. Highly confident identifications showed a dramatic drop in accuracy from faster decisions to slower decisions, whereas there was little change in accuracy between faster and slower decisions for moderately confident or weakly confident identifications. Finally, we observed little influence of the format of the nine different confidence scales: numerical and verbal scales produced comparable calibration scores, as did scales with few or many points. Copyright © 2015 John Wiley & Sons, Ltd.     

Relations between linear models: Implications for the lens model

Brunswik's lens model has been widely used in the modeling and analysis of judgment tasks and the lens model equation has been an important part of most analyses. In such analyses, researchers often have based arguments and interpretations upon the magnitude of various components of the lens model equation. One component is G, an index of agreement between a linear model of the subject's judgments and a linear model of the task environment. This paper addresses the index G and shows that while in principle −1 G 1, the distribution of G is often skewed toward high values so that caution must be used in the interpretation of its magnitude. Moreover, the results reported here apply to the relation between linear models in general.     

Exposure duration: Effects on eyewitness accuracy and confidence

The current study examined the relationship between the length of exposure to a face in an eyewitness setting and identification accuracy and confidence. A sample of 164 young (ages 17–25) and older (ages 59–81) adults viewed a simulated crime in which they saw the culprit's face for a short (12 s) or long (45 s) duration. They were then tested with a target absent (a line‐up not containing the culprit) or target present line‐up. Identification accuracy rates for both young and older participants were significantly higher under the long exposure condition. In the short exposure condition, witnesses who had made a correct identification of the target were more confident than incorrect witnesses. In the long exposure condition the confidence ratings of accurate and inaccurate witnesses did not differ. Discussion focuses on the extent to which extended exposure may inflate confidence judgments and variables that may moderate the relationship between exposure duration and face recognition accuracy.     

Maximum Likelihood Analysis of Nonlinear Structural Equation Models With Dichotomous Variables

In this article, a maximum likelihood approach is developed to analyze structural equation models with dichotomous variables that are common in behavioral, psychological and social research. To assess nonlinear causal effects among the latent variables, the structural equation in the model is defined by a nonlinear function. The basic idea of the development is to augment the observed dichotomous data with the hypothetical missing data that involve the latent underlying continuous measurements and the latent variables in the model. An EM algorithm is implemented. The conditional expectation in the E-step is approximated via observations simulated from the appropriate conditional distributions by a Metropolis-Hastings algorithm within the Gibbs sampler, whilst the M-step is completed by conditional maximization. Convergence is monitored by bridge sampling. Standard errors are also obtained. Results from a simulation study and a real example are presented to illustrate the methodology.     

Children's numerical skill and judgments of confidence in estimation

This study investigated how children's fluent use of a basic numerical skill is related to their perceptions of confidence on a task of quantitative estimation. One hundred four 10th graders (age 15.7 years) were categorized according to level of numerical fluency as assessed by timed tasks of counting in various nonunity increments. They then made estimates of large numbers of dots and rated how confident they were on each trial. The estimation task varied according to two within-subject factors of task difficulty. It was found that counting skill was related not only to accuracy in estimating, but also to the appropriateness of the children's confidence in their responses. Skillful children made judgments that corresponded with actual task difficulty. A derived calibration score corroborated ANOVA findings of the interaction of skill level and task difficulty on confidence ratings. It also revealed that while girls were less confident than boys, they were actually more realistic in their judgments. Findings are discussed in terms of (a) the effect of stimulus knowledge on awareness of task difficulty and (b) sex differences in achievement-related expectancies.     

The contribution of metamemory deficits to schizophrenia

A number of recent studies have demonstrated that individuals with schizophrenia display knowledge corruption; that is, they hold false information with strong conviction. This aberration in metamemory is thought to stem from poor memory accuracy in conjunction with impaired discrimination of correct and incorrect judgments in terms of confidence. Thirty-one participants with schizophrenia, along with 61 healthy control participants and 48 control participants with other psychiatric conditions, participated in a computerized source memory task. Whereas no differences in memory accuracy were observed between the group with schizophrenia and the group with other psychiatric diagnoses, knowledge corruption was specifically impaired in those with schizophrenia. Schizophrenia participants showed a significantly decreased confidence gap: They were more confident in errors and less confident in correct responses relative to those in the control groups. Knowledge corruption is theorized to be a potential risk factor for the emergence of delusions.     

Order effects and memory for evidence in individual versus group decision making in auditing

Sequential processing of evidence may lead to recency effect, a potential bias in judgment. The present research seeks to extend the literature on recency effects by assessing the potential moderating influence of team work: whether group decision making moderates the severity of recency effects predicted by Hogarth and Einhorn (1992), and whether group processing influences the accuracy of, and confidence in memory for evidence. Experienced auditors from a Big‐6 accounting firm made audit judgments, either individually or as groups. They were randomly assigned to one of two levels of evidence presentation order. After performing the judgment task, participants completed two evidence recognition tests. Consistent with prior findings, recency effects on judgments were observed, but only for individuals. Group judgments or audit reports were not affected by recency. Order effects, however, did not translate into different choices of audit reports, and did not persist in memories of either individuals or groups. As expected, group memory was more accurate than individual memory and groups were more confident than individuals. Overall, confidence in accurate memories was greater than in inaccurate ones. Copyright © 1999 John Wiley & Sons, Ltd.     

Peer Input and Revised Judgment: Exploring the Effects of (Un) Biased Confidence1

An experiment was conducted to test the effects of biased versus unbiased peer input on the revised judgments of others. After completing a set of knowledge items and assessing their confidence in each answer, subjects were: (a) given written input (in the form of answers and confidence assessments) from a peer who had completed the same set of items, and (b) allowed to revise their earlier answers and confidence assessments. Peer input was either overconfident, underconfident, or appropriately confident. Relative to appropriately confident input, both overconfident and underconfident input caused subjects’ accuracy in judgment to suffer. Overconfident input was particularly harmful because it led to more extreme overconfidence without any increase in accuracy. Practical implications of these results are discussed.     

Recollection-based prospective metamemory judgments are more accurate than those based on confidence: judgments of remembering and knowing (JORKS)

Five experiments were conducted to examine whether the nature of the information that is monitored during prospective metamemory judgments affected the relative accuracy of those judgments. We compared item-by-item judgments of learning (JOLs), which involved participants determining how confident they were that they would remember studied items, with judgments of remembering and knowing (JORKs), which involved participants determining whether studied items would later be accompanied by contextual details (i.e., remembering) or would not (i.e., knowing). JORKs were more accurate than JOLs when remember-know or confidence judgments were made at test and when cued recall was the outcome measure, but not for yes-no recognition. We conclude that the accuracy of metamemory judgments depends on the nature of the information monitored during study and test and that metamemory monitoring can be improved if participants are asked to base their judgments on contextual details rather than on confidence. These data support the contention that metamemory decisions can be based on qualitatively distinct cues, rather than an overall memory strength signal.     

Mixture Factor Analysis for Approximating a Nonnormally Distributed Continuous Latent Factor With Continuous and Dichotomous Observed Variables

Mixture factor analysis is examined as a means of flexibly estimating nonnormally distributed continuous latent factors in the presence of both continuous and dichotomous observed variables. A simulation study compares mixture factor analysis with normal maximum likelihood (ML) latent factor modeling. Different results emerge for continuous versus dichotomous outcomes. For dichotomous outcomes, normal ML path estimates have bias that worsens as latent factor skew/kurtosis increases and does not diminish as sample size increases, whereas the mixture factor analysis model produces nearly unbiased estimators as sample sizes increase (500 and greater) and offers near nominal coverage probability. For continuous outcome variables, both methods produce factor loading estimates with minimal bias regardless of latent factor skew, but the mixture factor analysis is more efficient. The method is demonstrated using data motivated by a study on youth with cystic fibrosis examining predictors of treatment adherence. In summary, mixture factor analysis provides improvements over normal ML estimation in the presence of skewed/kurtotic latent factors, but due to variability in the estimator relating the latent factor to dichotomous outcomes and computational issues, the improvements were only fully realized, in this study, at larger sample sizes (500 and greater).     

A Comparison of Imputation Strategies for Ordinal Missing Data on Likert Scale Variables

This article compares a variety of imputation strategies for ordinal missing data on Likert scale variables (number of categories = 2, 3, 5, or 7) in recovering reliability coefficients, mean scale scores, and regression coefficients of predicting one scale score from another. The examined strategies include imputing using normal data models with naïve rounding/without rounding, using latent variable models, and using categorical data models such as discriminant analysis and binary logistic regression (for dichotomous data only), multinomial and proportional odds logistic regression (for polytomous data only). The result suggests that both the normal model approach without rounding and the latent variable model approach perform well for either dichotomous or polytomous data regardless of sample size, missing data proportion, and asymmetry of item distributions. The discriminant analysis approach also performs well for dichotomous data. Naïvely rounding normal imputations or using logistic regression models to impute ordinal data are not recommended as they can potentially lead to substantial bias in all or some of the parameters.     

The benefits of knowing what you know (and what you don’t): How calibration affects credibility

People tend to believe, and take advice from, informants who are highly confident. However, people use more than a mere “confidence heuristic.” We believe that confidence is influential because—in the absence of other information—people assume it is a valid cue to an informant’s likelihood of being correct. However, when people get evidence about an informant’s calibration (i.e., her confidence-accuracy relationship) they override reliance on confidence or accuracy alone. Two experiments in which participants choose between two opposing witnesses to a car accident show that neither confidence nor accuracy alone explains judgments of credibility; rather, whether a person is seen as credible ultimately depends on whether the person demonstrates good calibration. Credibility depends on whether sources were justified in believing what they believed.     

Accuracy of Social Perception: An Integration and Review of Meta‐Analyses

This review examines the overall accuracy of social perception across several research topics and identifies factors that influence the accuracy of social perception. Findings from 14 meta‐analyses examining topics such as social/personality judgments, health judgments, legal judgments, and academic/vocational judgments were obtained. Social perception accuracy was generally moderate, yielding an average effect size (r) of .32. However, individual meta‐analytic effects varied widely, with some topics yielding small effects (e.g., lie detection, eyewitness identification) and other topics yielding large effects (e.g., educational judgments, health judgments). Several moderators of social perception accuracy were identified, including the nature of the information source, familiarity of the target, type of personality trait, and severity of the outcome being judged. These findings provide a comprehensive summary and novel integration of disparate findings on the accuracy of social perception. Concluding remarks highlight avenues for future research and call for cross‐disciplinary collaborations that would enhance our understanding of social perception.     

Evidence of metacognitive awareness in young children who have experienced a repeated event

Two studies examined children's confidence judgments in the accuracy of their memories after repeated experience of an event. Children aged 5 to 6 years took part in an event once or four times, were provided with misinformation either shortly after (Study 1) or a while after (Study 2), and interviewed with yes/no recognition questions 3 months later. Children in the repeated‐experience conditions were highly confident of their accurate responses to questions about items that were identical rather than variable across occurrences, and this discrimination was best at the shorter delay. The results show that children were able to metacognitively monitor the accuracy of their responses to qualitatively different kinds of details, and indicate that age is not the only determinant of metacognitive awareness after being misled. Rather, the nature of event representations must also be considered. Copyright © 2005 John Wiley & Sons, Ltd.     

On-the-Job Experience and Skill at Detecting Deception1

The role of on-the-job experience in fostering skill at detecting deception was examined. A deception-detection test was administered to three samples of more than 100 subjects each: a group of undergraduates with no special experiences at detecting deceit; a group of new recruits to a federal law enforcement training program, who had some limited on-the-job experience at detecting deceit; and a group of advanced federal law enforcement officers, with years of experience working at jobs in which the detection of deception is very important. Although the officer samples were more confident about their judgments of deceptiveness than were the students, they were no more accurate than the students. None of the three groups showed a significant improvement in deception-detection success from the first half to the second half of the test; however, the advanced officers felt increasingly confident about their performance as they progressed through the test. Correlational analyses of the relationship between accuracy and confidence provided further evidence that experience does not improve people's awareness oi the accuracy or inaccuracy of their judgments. The findings from this research are compared to the results of research on other kinds of professional decision-makers (e.g., clinical psychologists), and several theoretical perspectives on the role of experience in decision making are discussed.     

Memory and metamemory for semantic information in obsessive-compulsive disorder

Several studies have been conducted on OCD patients' memory and metamemory performance in episodic tasks. However, there is a clear lack of research addressing these issues for semantic memory (i.e., retrieval of information from long-term memory). Although findings regarding a memory deficit is somewhat equivocal, the empirical evidence clearly demonstrates that OCD patients with primarily checking compulsions show reduced confidence in their memory performance. The purpose of the present study was to investigate memory and metamemory performance of checkers in semantic memory domain. We compared checker OCD patients, non-checker OCD patients and normal controls on their ability to retrieve answers to general knowledge questions with a recall as well as a recognition test. We also investigated prospective (feeling-of-knowing (FOK)) and retrospective (confidence) metamemory judgments. Checker OCs were not poorer in retrieving semantic information from long-term memory. Neither were they less confident about their ability to remember currently unrecallable information in the future (FOK judgments) or about the accuracy of retrieved information (confidence judgments). Moreover, accuracy of metamemory judgments were comparable across groups. Overall, our results revealed that checker OCs do not show a memory or metamemory deficit when semantic memory was concerned, suggesting that any memory and metamemory deficit may be special to recently experienced materials.