Using ecphoric confidence ratings to discriminate seen from unseen faces: The effects of retention interval and distinctiveness

Theories of confidence processing for recognition judgments suggest that confidence indexes the degree of match between a presented stimulus and an image in memory (ecphoric similarity). Recent research has demonstrated that having participants rate their confidence that a face had been previously seen provides an equivalent or a better index of the stimulus's status than does eliciting a simple binary response (Sauer, Brewer, & Weber, Journal of Experimental Psychology: General 137: 528-547, 2008). Using a face recognition paradigm, we manipulated retention interval and stimulus distinctiveness to directly test the suggestion that confidence indexes ecphoric similarity and to probe the boundary conditions for using confidence ratings to discriminate seen from unseen faces. Consistent with the proposed ecphoric basis for confidence ratings, mean confidence was higher for previously seen than for unseen faces, and conditions conducive to the formation of strong memories improved confidence-based discrimination. In all conditions, after the application of a classification algorithm, confidence ratings provided a more sensitive index of face status (i.e., seen or unseen) than did binary responses.     

Decision criteria do not shift: Commentary on Mueller and Weidemann (2008)

The effects of base rates and payoffs on the shapes of rating receiver operating characteristic curves are inconsistent with the basic assumptions of signal detection theory (SDT), in particular the notion of a shifting decision criterion. Mueller and Weidemann (2008) propose that these unexpected phenomena are not due to problems with the decision- criterion construct but are instead due to two compounded effects: instability of the decision criterion across trials, and even greater instability in the flanking criteria that determine which confidence rating will be reported. There are several problems with the authors’ decision-noise hypothesis. First, even if their hypothesis about decision noise were taken for granted, the key feature of the ratings data that rejects the SDT model would remain a mystery. Second, the same violations of SDT that are exhibited in the ratings paradigm are also exhibited in the yes-no detection task when response time is substituted for confidence as a basis for analysis. Finally, the decision-noise hypothesis predicts that sensitivity will increase when one source of this variation-the response on a previous trial-is controlled for. This prediction was consistently violated in both the yes-no and ratings conditions of Mueller and Weidemann’s experiment. In an Addendum, we respond to Weidemann and Mueller’s (2008) reply to this Comment.     

A pattern recognition account of decision making

In the domain of pattern recognition, experiments have shown that perceivers integrate multiple sources of information in an optimal manner. In contrast, other research has been interpreted to mean that decision making is nonoptimal. As an example, Tversky and Kahneman (1983) have shown that subjects commit a conjunction fallacy because they judge it more likely that a fictitious person named Linda is a bank teller and a feminist than just a bank teller. This judgment supposedly violates probability theory, because the probability of two events can never be greater than the probability of either event alone. The present research tests the hypothesis that subjects interpret this judgment task as a pattern recognition task. If this hypothesis is correct, subjects’ judgments should be described accurately by the fuzzy logical model of perception (FLMP)—a successful model of pattern recognition. In the first experiment, the Linda task was extended to an expanded factorial design with five vocations and five avocations. The probability ratings were described well by the FLMP and described poorly by a simple probability model. The second experiment included (1) two fictitious people, Linda and Joan, as response alternatives and (2) both ratings and categorization judgments. Although the ratings were accurately described by both the FLMP and an averaging of the sources of information, the categorization judgments were described better by the FLMP. These results reveal important similarities in recognizing patterns and in decision making. Given that the FLMP is an optimal method for combining multiple sources of information, the probability judgments appear to be optimal in the same manner as pattern-recognition judgments.     

Response selection strategies and realism of confidence judgments

Two studies were conducted to examine how response selection strategy is related to confidence ratings and to performance on general knowledge questions. In both studies subjects were asked to answer 80 general knowledge questions and to rate their confidence in the correctness of the answer selected. A pilot study, in which subjects thought aloud while answering general knowledge questions, was carried out to identify different response selection strategies. In the first study, 40 subjects were asked to indicate which of four strategies (immediate recognition, inference, intuition, or guessing) they used for selecting an answer. In Study 2, think aloud reports from 20 subjects were coded into the same four strategies. The distribution of strategies differed between the studies, but there were very similar relations among strategy, confidence, and correctness of answer in the two studies. Response selection strategy was related to correctness of answer when confidence was partialed out. More specifically, immediate recognition was associated with higher proportion correct than with the other strategies. It was also found that ratings of how difficult the knowledge questions were to fellow students of the subjects were on a much more realistic level than the confidence ratings were. It is concluded that people could improve their confidence judgments by taking into account (a) how difficult a question is to other people, and (b) the response selection strategy used for answering the question.     

Social influence on metacognitive evaluations: The power of nonverbal cues

Metacognitive evaluations refer to the processes by which people assess their own cognitive operations with respect to their current goal. Little is known about whether this process is susceptible to social influence. Here we investigate whether nonverbal social signals spontaneously influence metacognitive evaluations. Participants performed a two-alternative forced-choice task, which was followed by a face randomly gazing towards or away from the response chosen by the participant. Participants then provided a metacognitive evaluation of their response by rating their confidence in their answer. In Experiment 1, the participants were told that the gaze direction was irrelevant to the task purpose and were advised to ignore it. The results revealed an effect of implicit social information on confidence ratings even though the gaze direction was random and therefore unreliable for task purposes. In addition, nonsocial cues (car) did not elicit this effect. In Experiment 2, the participants were led to believe that cue direction (face or car) reflected a previous participant's response to the same question—that is, the social information provided by the cue was made explicit, yet still objectively unreliable for the task. The results showed a similar social influence on confidence ratings, observed with both cues (car and face) but with an increased magnitude relative to Experiment 1. We additionally showed in Experiment 2 that social information impaired metacognitive accuracy. Together our results strongly suggest an involuntary susceptibility of metacognitive evaluations to nonverbal social information, even when it is implicit (Experiment 1) and unreliable (Experiments 1 and 2).     

Eyewitness accuracy and confidence: within- versus between-subjects correlations

Previous researchers using between-subjects comparisons have found eyewitness confidence and accuracy to be only negligibly correlated. In this study, we examined the predictive power of confidence in within-subject terms. Ninety-six subjects answered, and made confidence ratings for, a series of questions about a crime they witnessed. The average between-subjects and within-subject accuracy-confidence correlations were comparably low: r = .14 (p less than .001) and r = .17 (p less than .001), respectively. Confidence is neither a useful predictor of the accuracy of a particular witness nor of the accuracy of particular statements made by the same witness. Another possible predictor of accuracy, response latency, correlated only negligibly with accuracy (r = -.09 within subjects), but more strongly with confidence (r = -.27 within subjects). This pattern was obtained for both between-subjects and within-subject comparisons. The theoretical and practical implications of these results are discussed.     

Response latency as a predictor of the accuracy of children's reports

Researchers have explored various diagnostic cues to the accuracy of information provided by child eyewitnesses. Previous studies indicated that children's confidence in their reports predicts the relative accuracy of these reports, and that the confidence-accuracy relationship generally improves as children grow older. In this study, we examined the added contribution of response latency to the prediction of children's accuracy over and above that of confidence ratings. In Experiments 1 and 2, 2nd and 5th graders studied picture-event pairs and were tested using forced-choice, 2-alternative, or 5-alternative questions. In Experiment 3, children watched a slideshow depicting a story and were tested by 5-alternative questions about story details. The children indicated their confidence in each response, and response latency was measured. The results of all experiments suggested that children in both age groups relied on response latency as a cue for confidence, and this reliance contributed to the success with which they monitored the accuracy of their reports. When the test format was easy (Experiment 1), 2nd graders were as accurate as 5th graders in monitoring the accuracy of their answers, and the latency of their responses was no less predictive of accuracy. When the task was more difficult, age differences emerged. Nevertheless, in all experiments and for both age groups, response latency was found to have added value for predicting accuracy over and above that of confidence. Theoretical and practical implications of these findings for predicting the accuracy of children's reports are discussed.     

Another look at the relationship between phonetic symbolism and the feeling of knowing

Koriat (1975), using a word-matching phonetic symbolism task, found a significant relationship between translation accuracy and the degree of subjective confidence associated with the response. This study examined the hypothesis that since responses on which subjects are in consensus are more likely correct than incorrect, confidence ratings may actually be related to the degree of consensus, regardless of the correctness of the response. The results strongly supported this hypothesis. For items with a consensually correct response, translation accuracy increased with degree of subjective confidence, while for items with a consensually incorrect response it decreased. Consensual responses were judged subjectively more compelling than nonconsensual responses regardless of their accuracy. Some implications of these findings are outlined.     

Being confident without seeing: What subjective measures of visual consciousness are about

Can observers be confident about the accuracy of a discrimination response without a visual experience of the stimulus? In a series of five experiments, observers performed a masked orientation discrimination task, a masked shape discrimination task, or a random-dot motion discrimination task, followed by two subjective ratings after each trial, in which participants reported either their visual experience of the stimulus or their confidence in being correct. We observed that the threshold for ratings of the perception of the stimulus was above the threshold for ratings of a decision, that decision ratings outperformed stimulus ratings in predicting trial accuracy, and that different decision-related scales were more strongly associated with other decision-related scales than with ratings of stimulus clarity. We propose a taxonomy of subjective measures of consciousness that differentiates between subjective measures relating to the percept of the stimulus and measures relating to a discrimination decision and discuss the relation to type II blindsight.     

Induced forgetting and reduced confidence in our personal past? The consequences of selectively retrieving emotional autobiographical memories

People build their sense of self, in part, through their memories of their personal past. What is striking about these personal memories is that, in many instances, they are inaccurate, yet confidently held. Most researchers assume that confidence ratings are based, in large part, on the memory's mnemonic features. That is, the more vivid or detailed the memory, the higher the confidence people have in its accuracy. However, we explore a heretofore underappreciated source on which confidence ratings may be based: the accessibility of memories as a result of selective retrieval. To explore this possibility, we use Anderson, Bjork, and Bjork's retrieval-induced forgetting (RIF) paradigm with emotional (positive and negative) autobiographical memories. We found the standard RIF effect for memory recall across emotional valence. That is, selective retrieval of emotional autobiographical memories induced forgetting of related, but not retrieved emotional autobiographical memories compared to the baseline. More interestingly, we found that the confidence ratings for positive memories mirrored the RIF pattern: decreased confidence for related, unpracticed autobiographical memories relative to the baseline. For negative memories, we found the opposite pattern: increased confidence for both practiced autobiographical memories and related, unpracticed autobiographical memories. We discuss these results in terms of accessibility, the diverging mnemonic consequences of selectively retrieving positive and negative autobiographical memories and personal identity.     

The assessment of emotional clarity via response times to emotion items: shedding light on the response process and its relation to emotion regulation strategies

Researchers have begun to use response times (RTs) to emotion items as an indirect measure of emotional clarity. Our first aim was to scrutinise the properties of this RT measure in more detail than previously. To be able to provide recommendations as to whether (and how) emotional intensity – as a possible confound – should be controlled for, we investigated the specific form of the relation between emotional intensity and RTs to emotion items. In particular, we assumed an inverted U-shaped relation at the item level. Moreover, we analysed the RT measure’s convergent validity with respect to individuals’ confidence in their emotion ratings. As a second aim, we compared the predictive validity of emotional clarity measures (RT measure, self-report) with respect to daily emotion regulation. The results of three experience sampling studies showed that the association between emotional intensity and RT followed an inverted U shape. RT was in part related to confidence. Emotional clarity measures were unrelated to reappraisal. There was some evidence that lower emotional clarity was related to a greater use of suppression. The findings highlight that emotional intensity and squared emotional intensity should be controlled for when using the RT measure of emotional clarity in future research.     

Measuring any conscious content versus measuring the relevant conscious content: Comment on Sandberg et al.

Sandberg et al. show that the Perceptual Awareness Scale (PAS) scale is sensitive compared to confidence ratings and wagering in detecting accurate perception. They go on to argue that the PAS scale is hence a sensitive measure of conscious perception compared to confidence ratings, a claim disputed here. The fact that some visual content is conscious does not entail that the visual content relevant to making a discrimination is conscious. For example, if one saw a square but was only aware of seeing a flash of something, then one has not consciously seen a square. When PAS and confidence ratings come in conflict, we suggest that it is confidence ratings that more reliably indicate the conscious status of contents allowing discrimination.     

Probability assessment with response times and confidence in perception and knowledge

In both a perceptual and a general knowledge comparison task, participants categorized the time they took to decide, selecting one of six categories ordered from “Slow” to Fast”. Subsequently, they rated confidence on a six-category scale ranging from “50%” to “100%”. Participants were able to accurately scale their response times thus enabling the treatment of the response time (RT) categories as potential confidence categories. Probability assessment analyses of RTs revealed indices of over/underconfidence, calibration, and resolution, each subject to the “hard-easy” effect, comparable to those obtained with the actual confidence ratings. However, in both the perceptual and knowledge domains, resolution (i.e., the ability to use the confidence categories to distinguish correct from incorrect decisions) was significantly better with confidence ratings than with RT categorization. Generally, comparable results were obtained with scaling of the objective RTs, although subjective categorization of RTs provided probability assessment indices superior to those obtained from objective RTs. Taken together, the findings do not support the view that confidence arises from a scaling of decision time.     

Response styles in affect ratings: making a mountain out of a molehill

Ratings of affect words are the most commonly used method to assess pleasant affect (PA) and unpleasant affect (UA). The reliance on self-reports would be problematic if affect ratings were heavily influenced by response styles. Several recent publications have indeed suggested (a) that the influence of response styles on affect ratings is pervasive, (b) that this influence can be controlled by variations of the response format using multitrait-multimethod models, and (c) the discriminant validity of PA and UA is spurious. In this article, we examined the evidence for these claims. We demonstrate that (a) response styles have a negligible effect on affect ratings, (b) multiple response formats produce the same results as a single response format, and (c) the discriminant validity of PA and UA is not a method artifact. Rather, evidence against discriminant validity is due to the use of inappropriate response formats that respondents interpreted as bipolar scales.     

Rorschach Administration: A Comparison of the Effect of Two Instructions

Ratings of affect words are the most commonly used method to assess pleasant affect (PA) and unpleasant affect (UA). The reliance on self-reports would be problematic if affect ratings were heavily influenced by response styles. Several recent publications have indeed suggested (a) that the influence of response styles on affect ratings is pervasive, (b) that this influence can be controlled by variations of the response format using multitrait-multimethod models, and (c) the discriminant validity of PA and UA is spurious. In this article, we examined the evidence for these claims. We demonstrate that (a) response styles have a negligible effect on affect ratings, (b) multiple response formats produce the same results as a single response format, and (c) the discriminant validity of PA and UA is not a method artifact. Rather, evidence against discriminant validity is due to the use of inappropriate response formats that respondents interpreted as bipolar scales.     

The role of self‐rated ability in the accuracy of confidence judgements in eyewitness memory and general knowledge

It is argued that confidence stems in part from self‐rated ability in a domain of knowledge and that in eyewitness memory such perceptions are erroneous. Two experiments tested these hypotheses. In both experiments participants rated their relative ability in the domains of eyewitness memory and general knowledge and subsequently took tests of each, giving confidence ratings for each item attempted. In both studies, self‐rated ability predicted performance for general knowledge, but not eyewitness memory. Across participants confidence ratings were significant predictors of accuracy for general knowledge, but not for eyewitness memory. In Experiment 1 self‐rated ability was predictive of confidence ratings for both domains, although this effect was weaker in Experiment 2. The argument that the accuracy of confidence judgements in eyewitness memory is undermined by a lack of insight into relative expertise is therefore supported. Copyright © 2004 John Wiley & Sons, Ltd.     

Subjective measures of awareness and implicit cognition

In this article, we examine whether artificial grammar learning is implicit according to a subjective criterion of awareness based on confidence ratings. In four experiments, participants discriminated between grammatical and ungrammatical sequences in both the same (Experiment 1) and a novel (Experiments 2-4) vocabulary and indicated their confidence in each decision. Replicating earlier studies, confidence judgments reported on a continuous scale (50%-100%) were only weakly related to accuracy, suggesting that learning was implicit. In contrast, confidence judgments reported on a binary scale (high vs. low) revealed that confidence was related to accuracy. We show that participants are better able to place their phenomenal states on a binary scale, as compared with a continuous scale.     

Inducing blindsight in normal observers

We attempted to induce blindsight in normal observers, in an effort to replicate and extend the findings of Kolb and Braun (1995). In that demonstration, observers were able to localize a target in the absence of visual awareness, indicated by the lack of a correlation between localization accuracy and confidence ratings. Replication of this work seemed essential, given the failed attempt by Morgan, Mason, and Solomon (1997). A key aspect of the present work was the use of a pointing response, which is believed to have access to the unconscious representations subserving blindsight. In the critical rivalrous condition, the display consisted of Gabor patterns presented dichoptically with orthogonal orientation in each eye. Binocular summation of left and right images combined to give the appearance of a uniform plaid, camouflaging the texture-defined target. Our attempt to demonstrate blindsight in normal observers was unsuccessful, in that the localization accuracy of the texture-defined target and the observers’ confidence ratings were positively correlated. Although the replication was unsuccessful, the results are valuable in that they provide closure to this widely publicized but fruitless line of inquiry.     

Algebra Mal-Rules and Cognitive Accounts of Error

We report an empirical study of elementary algebra errors, conducted in three separate schools. The errors are diagnosed using mal-rules, as proposed by Sleeman (1984, 1,985). Our analysis uncovers the following properties of algebra mal-rules: The frequency of mal-rules is severely skewed, there are many infrequent mal-rules and few frequent ones; mal-rules are very unstable, students typically use mal-rules very irregularly; different mal-rules have explanatory power in different schools (many of our most powerful mal-rules are previously unreported); mal-rule diagnosis Is more successful with more skilled students; students' confidence ratings do not partition the total set of mal-rules, every mal-rule we find is associated with high confidence ratings by at least one student. The Implications of our data for cognitive theories of error generation are discussed. Contrary to commonplace assumptions, we argue that It is impossible to make a clear distinction between slips and mistakes; most errors depend on properties of the knowledge base and the cognitive architecture. Errors In a procedural skill cannot be assumed to be purely syntactic In orgin.     

Examining the reliability and predictive validity of performance assessments by soccer coaches and scouts: The influence of structured collection and mechanical combination of information

Soccer coaches and scouts typically assess in-game soccer performance to predict players’ future performance. However, there is hardly any research on the reliability and predictive validity of coaches’ and scouts’ performance assessments, or on strategies they can use to optimize their predictions. In the current study, we examined whether robust principles from psychological research on selection – namely structured information collection and mechanical combination of predictor information through a decision-rule – improve soccer coaches’ and scouts’ performance assessments. A total of n = 96 soccer coaches and scouts participated in an elaborate within-subjects experiment. Participants watched soccer players’ performance on video, rated their performance in both a structured and unstructured manner, and combined their ratings in a holistic and mechanical way. We examined the inter-rater reliability of the ratings and assessed the predictive validity by relating the ratings to players’ future market values. Contrary to our expectations, we did not find that ratings based on structured assessment paired with mechanical combination of the ratings showed higher inter-rater reliability and predictive validity. In contrast, unstructured-holistic ratings yielded the highest reliability and predictive validity, although differences were marginal. Overall, reliability was poor and predictive validities small-to-moderate, regardless of the approach used to rate players’ performance. The findings provide insights into the difficulty of predicting future performance in soccer.