The confidence-accuracy relationship in eyewitness identification: effects of lineup instructions, foil similarity, and target-absent base rates

Discriminating accurate from mistaken eyewitness identifications is a major issue facing criminal justice systems. This study examined whether eyewitness confidence assists such decisions under a variety of conditions using a confidence-accuracy (CA) calibration approach. Participants (N = 1,200) viewed a simulated crime and attempted 2 separate identifications from 8-person target-present or target-absent lineups. Confidence and accuracy were calibrated for choosers (but not nonchoosers) for both targets under all conditions. Lower overconfidence was associated with higher diagnosticity, lower target-absent base rates, and shorter identification latencies. Although researchers agree that courtroom expressions of confidence are uninformative, our findings indicate that confidence assessments obtained immediately after a positive identification can provide a useful guide for investigators about the likely accuracy of an identification.     

Gender-related traits in gay men, lesbian women, and heterosexual men and women: the virtual identify of homosexual-heterosexual diagnosticity and gender diagnosticity

Three studies investigated the relationship between gender-related traits and sexual orientation. Study 1 showed that gay men and lesbians in an unselected sample of 721 college students differed from same-sex heterosexuals most strongly on gender diagnosticity (GD) measures, which assess male- versus female-typicality of interests (effect sizes of 2.70 for men and .96 for women) and least strongly on measures of instrumentality (I) and expressiveness (E). In Study 2, GD measures showed large differences between 95 gay and 136 heterosexual men (effect sizes of 1.61 and 1.83) and between 46 lesbian and 225 heterosexual women (effect sizes of .98 and 1.28), whereas I and E showed much smaller differences. In Study 3, GD showed large differences between 90 gay and 81 heterosexual men (effect sizes of 1.76 and 1.97) and between 82 lesbians and 108 heterosexual women (effect sizes 1.67 and 1.70). whereas I and E showed much smaller differences. Using data from Studies 2 and 3, "gay-heterosexual diagnosticity" measures were computed for men and "lesbian-heterosexual diagnosticity" measures for women, based on occupational and hobby preferences. These measures correlated very strongly with GD measures.     

Determinants of Overconfidence and Miscalibration: The Roles of Random Error and Ecological Structure

Previous authors have attributed findings of overconfidence to psychological bias or to experimental designs unrepresentative of the environment. This paper provides evidence for an alternative explanation. A model is presented in which reported confidence is a function of the validity of information used by the subject, and a random error component. The model predicts greater overconfidence for question sets in which informational cues are less valid. This result corresponds to the well-known hard/easy effect. The model also predicts that unrepresentative design (Gigerenzer, Hoffrage, & Kleinbölting, 1991) is sufficient but not necessary for overconfidence to occur. These predictions are tested, and results provide support for the model. Subjects are overconfident according to usual measures such as calibration, even though on average they report the diagnosticity of informational cues correctly. Furthermore, overconfidence is greater for harder sets of questions, even when those sets are representative of the environment. A post hoc analysis reveals some intriguing individual differences among subjects. Some people appear to have a true psychological bias toward reporting high levelsof confidence, whereas others have a psychological bias in the direction of underconfidence.     

Can self-reported encoding strategy and recognition skill be diagnostic of performance in eyewitness identifications?

The relationship between 3 witness factors and identification accuracy, as well as calibration and diagnosticity of confidence, was investigated. A total of 384 participants in an eyewitness experiment rated their facial recognition skill, general memory skill, and self-reported encoding strategy on a questionnaire presented after the photo-confrontation. Participants who rated themselves to be good face recognizers showed a slightly higher overall accuracy with a more diagnostic confidence-accuracy relation. Participants who reported that they relied on a holistic encoding strategy were associated with more accurate identifications and a stronger confidence-accuracy relation than those who reported an analytic encoding strategy. Degree of self-reported general memory skill was not diagnostic of identification performance.     

Diagnosticity,confidence, and the need for information

Recent research has suggested that people prefer to use the most diagnostic available information as the basis for their choices and decisions, and are most confident in those decisions when information is highly diagnostic. However, the effect of information diagnosticity on the need for additional information has yet to be investigated; that is, in an optional stopping task, will the amount of information requested depend upon information diagnosticity? Three models of the role of diagnosticity in information use were examined; expected value, a confidence criterion, and information cost. Subjects attempted to categorize stimuli with the aid of information of varying costs and diagnosticity levels. They requested more information when it was obtained at a low cost. More importantly, across cost conditions, subjects consistently requested greater amounts of information when that information was of a low diagnosticity. These data seem most consistent with use of a confidence criterion that is adjusted for information costs.     

Positive versus negative face recognition decisions: confidence,accuracy, and response latency

A large amount of eyewitness identification and face recognition research has investigated the confidence–accuracy (CA) relationship. One consistent finding is that positive recognition decisions (or choosers) demonstrate superior CA calibration to negative recognition decisions (or non‐choosers). This experiment tested whether an explanation of this difference, based on the information available for confidence judgements, accounted for the pattern of CA calibration in positive and negative face recognition decisions. CA calibration for positive and negative decisions was compared for both item and associative recognition judgements. Significantly greater resolution was observed for positive decisions in both the item and associative conditions. Similarly, for both judgement types, positive decisions evidenced a stronger response latency–accuracy relationship than negative decisions. Implications for diagnosing the accuracy of eyewitness identification are discussed. Copyright © 2005 John Wiley & Sons, Ltd.     

Some psychometric characteristics of gender diagnosticity measures: reliability, validity, consistency across domains, and relationship to the big five.

Preferences for various occupations, school subjects, everyday activities, and hobbies and amusements were rated by 119 male and 145 female Ss. Discriminant analyses were conducted to compute gender diagnostic probabilities. Ss also rated themselves on Big Five traits and completed the Bem Sex-Role Inventory (BSRI) and the Personal Attributes Questionnaire (PAQ). Results indicated that (a) gender diagnosticity measures showed high reliability, (b) gender diagnosticity predicted sex of S and self-ascribed masculinity (M) and femininity (F) better than contrasted-groups M-F scales, (c) gender diagnosticity measures displayed substantial consistency across domains both within and across the sexes, and (d) gender diagnosticity measures were independent of the Big Five and PAQ and BSRI scales both within and across the sexes, whereas PAQ and BSRI scales loaded highly on Big Five dimensions.     

Understanding spatial diagram structure: An analysis of hierarchies,matrices, and networks

Abstract diagrams are powerful tools for comprehension and problem solving in diverse contexts. Two studies examined the structural properties of (i.e., applicability conditions for) three interrelated spatial diagrams—hierarchies, matrices, and networks. College students from two groups with distinct educational backgrounds and learning histories—advanced computer science students and representative undergraduates—rated the diagnosticity of the hypothesized applicability conditions for each of the 3 diagrams. The results validated 24–26 of the 30 hypothesized applicability conditions and provided evidence regarding the relative importance, or diagnosticity, of the validated properties for each type of diagram. A different set of properties was identified as most highly diagnostic for each type of diagram, indicating that the three spatial diagrams are optimized to serve different representational functions: The matrix stores static information about the kind of relation that exists between pairs of items in different sets, the network conveys dynamic information by showing the local connections and global routes connecting the items being represented, and the hierarchy depicts a rigid structure of power or precedence relations among items. The quantitative and qualitative differences in representational knowledge due to educational background are discussed.     

Understanding spatial diagram structure: an analysis of hierarchies, matrices, and networks

diagrams are powerful tools for comprehension and problem solving in diverse contexts. Two studies examined the structural properties of (i.e., applicability conditions for) three interrelated spatial diagrams—hierarchies, matrices, and networks. College students from two groups with distinct educational backgrounds and learning histories—advanced computer science students and representative undergraduates—rated the diagnosticity of the hypothesized applicability conditions for each of the 3 diagrams. The results validated 24-26 of the 30 hypothesized applicability conditions and provided evidence regarding the relative importance, or diagnosticity, of the validated properties for each type of diagram. A different set of properties was identified as most highly diagnostic for each type of diagram, indicating that the three spatial diagrams are optimized to serve different representational functions: The matrix stores static information about the kind of relation that exists between pairs of items in different sets, the network conveys dynamic information by showing the local connections and global routes connecting the items being represented, and the hierarchy depicts a rigid structure of power or precedence relations among items. The quantitative and qualitative differences in representational knowledge due to educational background are discussed.     

Modeling patterns of probability calibration with random support theory: Diagnosing case-based judgment

We describe four broad characterizations of subjective probability calibration (overconfidence, conservatism, ecologically perfect calibration, and case-based judgment) and show how Random Support Theory (RST) can serve as a tool for representing, evaluating, and discriminating between these perspectives. We present five studies of probability judgment in a simulated stock market setting and analyse the calibration data in terms of RST parameters. The observed pattern of calibration varies with the outcome base rate and cue value diagnosticity, as predicted by case-based judgment. A similar pattern of calibration is found in real-world judgments of experts in various domains. Case-based RST—defined as RST with stable parameter values—provides a parsimonious account of the substantial changes in calibration performance observed across different judgment environments.     

Implicit diagnosticity in an information-buying task. How do we use the information that we bring with us to a problem?

Past research suggests that people may make use of diagnosticity information when explicit data regarding P(D | H) and P(D | ? H) are given to them. However, people fall victim to pseudodiagnosticity biases and ignore P(D | ? H) when such data must be actively sought. This series of four experiments utilized judgment problems in which subjects have knowledge of P(D | ? H) but must recognize the relevance of that knowledge for the judgment at hand. It was hypothesized that subjects who genuinely understood the role of P(D | ? H) in hypothesis testing would respond to this manipulation of implicit diagnosticity by exhibiting greater confidence and lesser information buying when given evidence of relatively high diagnosticity. In the first three studies, subjects attempted to judge the guilt or innocence of suspects in several fictional crimes. In the fourth experiment, subjects attempted to judge the club membership status of students at their own university. Greater amounts of information were bought when the only available information was of low diagnosticity. Subjects also expressed greater confidence in judgments made using highly diagnostic information. However, within the legal scenario, sensitivity to diagnosticity was dependent upon the implication of the cues received. Results are discussed with respect to Bayesian probability, expected value theory, and a confidence criterion model of information purchasing.     

类别使用的特征诊断效应

运用预备实验评定过的鱼轮廓图,考察了类别判断和特征预测这两种类别使用方式下的特征诊断效应。被试是48名大学生,学习材料是两类鱼的类别成员,在测验阶段,实验1判断新项目类别标签,实验2预测新项目的缺失特征。结果表明,类别判断条件下,单个特征诊断力加强和诊断性特征数量增加都对类别判断有促进作用;特征预测条件下,只有单个特征诊断力加强可以促进特征预测,诊断性特征数量增加无助于特征预测     

Confidence in own and others' knowledge

A person's confidence judgement of a statement reflects his/her degree of belief in the correctness of that statement. Deficient ability to assess the correctness of statements (or beliefs) can have serious consequences in many situations. This study compares the realism (calibration) of subjects' confidence ratings in two situations ( n = 64). The first situation was when the subjects confidence rated their own answers to general knowledge questions. The second was when the subjects gave confidence ratings of another person's answers to general knowledge questions. The results show that subjects were more poorly calibrated and were more overconfident in the second situation, i.e. when they gave confidence ratings of answers given by another person, compared with when they rated their own answers. The data further indicates that the results can not be explained in terms of the amount of cognitive processes invested when making the confidence judgements. For example, the subjects rated the other person's answers to questions they had answered themselves, and to questions they had not seen before. No differences in confidence or in calibration and other measures of judgmental realism were found between these two categories of questions. Nor did instructions to imagine the thought process of the other person improve any of these measures. The subjects disagreed with the other person's answer on 23% of all occasions. Significantly poorer calibration was shown where subjects disagreed with the other person than where they agreed. Contents of a social nature attended to by the subjects may have affected the results. The results, when related to previous research in the area, give rise to the question of how the social situation can be arranged to achieve the best calibration.     

A probability-based measure of effect size: robustness to base rates and other factors

Calculating and reporting appropriate measures of effect size are becoming standard practice in psychological research. One of the most common scenarios encountered involves the comparison of 2 groups, which includes research designs that are experimental (e.g., random assignment to treatment vs. placebo conditions) and nonexperimental (e.g., testing for gender differences). Familiar measures such as the standardized mean difference (d) or the point-biserial correlation (rpb) characterize the magnitude of the difference between groups, but these effect size measures are sensitive to a number of additional influences. For example, R. E. McGrath and G. J. Meyer (2006) showed that rpb is sensitive to sample base rates, and extending their analysis to situations of unequal variances reveals that d is, too. The probability-based measure A, the nonparametric generalization of what K. O. McGraw and S. P. Wong (1992) called the common language effect size statistic, is insensitive to base rates and more robust to several other factors (e.g., extreme scores, nonlinear transformations). In addition to its excellent generalizability across contexts, A is easy to understand and can be obtained from standard computer output or through simple hand calculations.     

The Effects of Subject-Defined Categories on Judgmental Accuracy in Confidence Assessment Tasks

The accuracy of confidence judgments can be determined using measures of discrimination and calibration. The present paper utilizes a new assessment methodology that decomposes the confidence assessment task, allowing us to investigate discrimination and calibration skills in greater depth than has been done in previous studies. Researchers investigating the goodness of confidence judgments have typically grouped forecasters' assessments into experimenter-defined categories, generally in equal widths of .10. In the present research, subjects created their own categories and later assigned confidence judgments to the categories, separating the tasks of discriminating categories (discrimination) and assigning numbers to categories (calibration). Further, the typical assessment procedure assumes that subjects are able to discriminate equally across the confidence scale. Since subjects in the present study defined their own assessment categories, they could locate those categories at any point on the scale. A final issue of interest was whether subjects were able to determine accurately the number of categories into which they could discriminate. Sixty subjects performed 1 of 2 tasks, general knowledge or forecasting, in both relatively easy and relatively hard conditions. Results showed a trade-off in performance: Calibration generally became worse as the number of categories increased, while discrimination generally improved. Overall accuracy was not affected by the number of categories used. Further, subjects partitioned categories more at the high end of the scale. Finally, measures showed that subjects were not accurate in their beliefs about their own discrimination ability.     

Determinants of information-seeking behavior

Two studies examined situational determinants of choice among anagram tests that varied both in difficulty and in diagnosticity (the information they provided about one's own ability). In both studies, subjects worked on a preliminary anagram test before making their choices. Study 1 manipulated level of performance on the preliminary test. Results showed that high performance led to preferring more difficult and more diagnostic tests. In Study 2, subjects were either paid or not paid for their performance on the preliminary test. Results showed that pay led to a preference for more diagnostic tests. Unexpectedly, results of both studies showed that although difficulty and diagnosticity were defined independently of one another, they were not perceived as such. Thus, high diagnostic tests were perceived as more difficult; more difficult tests were perceived as more diagnostic; and the difference between high and low diagnostic tests in perceived diagnosticity and choice of items (high diagnostic tests had higher scores on both measures) were more pronounced among more difficult tests. Motivational as well as cognitive interpretations of the results were discussed.     

Children's metacognitive judgments in an eyewitness identification task

Two experiments examined children's metacognitive monitoring of recognition judgments within an eyewitness identification paradigm. A confidence-accuracy (CA) calibration approach was used to examine patterns of calibration, over-/underconfidence, and resolution. In Experiment 1, children (n=619, mean age=11 years 10 months) and adults (n=600) viewed a simulated crime and attempted two separate identifications from 8-person target-present or target-absent lineups given lineup instructions that manipulated witnesses choosing patterns by varying the degree of social pressure. For choosers, but not nonchoosers, meaningful CA relations were observed for adults but not for children. Experiment 2 tested a guided hypothesis disconfirmation manipulation designed to improve the realism of children's metacognitive judgments. Children (N=796, mean age=11 years 11 months) in experimental and control conditions viewed a crime and attempted two separate identifications. The manipulation had minimal impact on the CA relation for choosers and nonchoosers. In contrast to adults, children's identification confidence provides no useful guide for investigators about the likely guilt or innocence of a suspect. These experiments revealed limitations in children's metacognitive monitoring processes that have not been apparent in previous research on recall and recognition with younger children.     

CONFIANCE EXCESSIVE: CONNAISSANCES ET CROYANCES

It had already been found (Lamarche 1988; Oskamp 1965) that the overconfidence shown in answering a question increased with the number of new informations given. The objective of our research was to verify if there was a relation between that phenomenon, obtained with knowledge questions, and the evolution of confidence, in the same circumstances, when belief questions are involved. On the one hand, we presented to our subjects knowledge questions which were repeated with a growing number of clues; on the other hand, the same subjects were asked to form an impression of a person based on a list of traits presented sequentially. Using a calibration index, we were able to observe anew the growing overconfidence with knowledge questions. However, in the case of beliefs, the confidence level did not grow in the same way. We explain that result by our choice of traits. Nevertheless, there was a relation between the increase of confidence and the decrease of calibration. We think this could be the result of a lack of involvement with the belief questionnaire which is less so when answered first.