Realism of confidence in sensory discrimination

Is there a common and general basis for confidence in human judgment? Recently, we found that the properties of confidence judgments in the sensory domain mirror those previously established in the cognitive domain; notably, we found underconfidence on easy sensory judgments and overconfidence on hard sensory judgments. In contrast, data from the Uppsala laboratory in Sweden suggest that sensory judgments are unique; they found a pervasive underconfidence bias, with overconfidence being evident only on very hard sensory judgments. Olsson and Winman (1996) attempted to resolve the debate on the basis of methodological issues related to features of the stimulus display in a visual discrimination task. A reanalysis of the data reported in Baranski and Petrusic (1994), together with the findings of a new experiment that controlled stimulus display characteristics, supports the position that the difference between the Canadian and the Swedish data is real and, thus, may reflect cross-national differences in confidence in sensory discrimination.     

A model for realism of confidence judgments: Implications for underconfidence in sensory discrimination

In a recent issue of this journal, Björkman, Juslin, and Winman (1993) presented a model of the calibration of subjective confidence judgments for sensory discrimination which they called “subjective distance theory.” They proposed that there was a robust underconfidence bias in such judgments, that the model predicted such a bias, and that two different models were needed for the calibration of subjective confidence for cognitive judgments and for sensory ones. This paper addresses issues they raised. It points out that they have not presented a new model, but rather a portion of a more general one, the “decision-variable partition model” originally proposed in Ferrell and McGoey (1980). This paper explores properties of the model and shows, contrary to Björkman, Juslin, and Winman’s hypotheses, that the model does not predict under-confidence, that the “hard-easy effect” can be observed with sensory discriminations, and that the model fits not only sensory, but also cognitive judgments.     

Realism of confidence in obsessive-compulsive checkers

The study examined whether obsessive-compulsive (OC) checkers have reduced confidence in their knowledge. OC checkers were compared with panic disorder (PD) patients and nonpatient controls using a calibration-of-knowledge procedure. Participants completed a general knowledge questionnaire, rated their confidence in each answer, and estimated the total number of correct answers. These responses were converted to 2 measures of confidence relative to performance--over/underconfidence and over/underestimation. OC checkers had lower scores than nonpatients did on both measures, whereas the PD patients did not differ from either group. For the OC checkers, relative confidence was inversely related to the severity of obsessions. The authors speculate that confidence may depend on a confirmation bias in testing hypotheses and that the reduced confidence in OC checkers may reflect a disconfirmation bias in this population.     

Realism in confidence judgements of performance based on implicit learning

In two experiments we employed calibration methods to investigate the realism of participants' confidence ratings of their own classification performance based on knowledge acquired after training on an artificial grammar. In Experiment 1 participants showed good realism (but overconfidence) for grammatical strings but very poor realism for non-grammatical strings. Method of training (string repetition in writing or mere exposure) did not affect the realism. Furthermore, the participants underestimated their overall performance. In Experiment 2, using a more complex grammar and controlling for two types of associative chunk-strength, participants showed good realism (but still overconfidence) for both letter and symbol strings, irrespective of grammaticality. Together, these experiments show that implicit learning can give rise to knowledge products that are associated with fairly realistic meta-knowledge. It is argued that both the zero-correlation criterion and the guessing criterion are misplaced when used to define implicit knowledge; two reasons being that confidence judgements may be affected both by implicit knowledge and by inferences.     

Underconfidence in sensory discrimination: The interaction between experimental setting and response strategies

In a recent issue of this journal, Baranski and Petrusic (1994) presented empirical data revealing overconfidence in sensory discrimination. In this paper, we propose an explanation of Baranski and Petrusic’s results, based on an idiosyncrasy in the experimental setting that misleads subjects who are using an unwarranted symmetry assumption. Experiment 1 showed that when this hypothesis is controlled for, a large underconfidence bias is obtained with Baranski and Petrusic’s procedure. The results of Experiment 2 confirmed that overconfidence is difficult to obtain in subject-controlled sensory discrimination tasks, even for a very low proportion of correct responses. The different results obtained in sensory and cognitive tasks suggest that one should not uncritically draw parallels between confidence in sensory and cognitive judgments.     

The role of the decision system in sensory and memory experiments using confidence judgments

This paper uses an artificial “sensory task” (a dice game) to investigate whether it is possible to distinguish between two-state and multistate sensory processes by examining confidence ratings of a group of unpracticed Os. The results indicate that Os use a decision rule that is deterministic so that both operating characteristics and a posteriori probability functions reflect the number of states used by the decision system. Thus, it would seem to be possible to use confidence ratings of Os to evaluate theoretical issues.     

The fallacy of placing confidence in confidence intervals

Interval estimates – estimates of parameters that include an allowance for sampling uncertainty – have long been touted as a key component of statistical analyses. There are several kinds of interval estimates, but the most popular are confidence intervals (CIs): intervals that contain the true parameter value in some known proportion of repeated samples, on average. The width of confidence intervals is thought to index the precision of an estimate; CIs are thought to be a guide to which parameter values are plausible or reasonable; and the confidence coefficient of the interval (e.g., 95 %) is thought to index the plausibility that the true parameter is included in the interval. We show in a number of examples that CIs do not necessarily have any of these properties, and can lead to unjustified or arbitrary inferences. For this reason, we caution against relying upon confidence interval theory to justify interval estimates, and suggest that other theories of interval estimation should be used instead.     

The role of memory for past test in the underconfidence with practice effect

According to the Memory for Past Test (MPT) heuristic, judgments of learning (JOLs) may be based, in part, on memory for the correctness of answers on a previous test. The authors explored MPT as the source of the underconfidence with practice effect (UWP; A. Koriat, L. Sheffer, & H. Ma'ayan, 2002), whereby Trial 1 overconfidence switches to underconfidence by Trial 2. Immediate and delayed JOLs were contrasted because only immediate JOLs demonstrate UWP. Consistent with MPT for immediate JOLs, Trial 1 test performance better predicted Trial 2 JOLs than did Trial 2 test performance. Delayed JOLs showed the reverse. Furthermore, items forgotten on Trial 1 but remembered on Trial 2 contributed disproportionately to UWP, but only with immediate JOLs.     

The Truth about Realism: Natural Realism,Many Worlds,and Global M-Realism

Philosophia - An attempt was made to show how we can plausibly commit to mathematical realism. For the purpose of illustration, a defence of natural realism for arithmetic was developed that draws...     

The contribution of judgment scale to the unskilled-and-unaware phenomenon: How evaluating others can exaggerate over- (and under-) confidence

The unskilled-and-unaware phenomenon occurs when low performers tend to overestimate their performance on a task, whereas high performers judge their performance more accurately (and sometimes underestimate it). In previous research, this phenomenon has been observed for a variety of cognitive tasks and judgment scales. However, the role of judgment scale in producing the unskilled-and-unaware phenomenon has not been systematically investigated. Thus, we present four studies in which all participants judged their performance on both a relative scale (percentile rank) and an absolute scale (number correct). The studies included a variety of performance tasks (general knowledge questions, math problems, introductory psychology questions, and logic questions) and test formats (multiple-choice, recall). Across all tasks and formats, the percentile-rank judgments were less accurate than the absolute judgments, particularly for low and high performers. Furthermore, in Studies 1–3, the absolute judgments were highly accurate, even when the percentile-rank judgments were not. Thus, differences in the accuracy of percentile-rank judgments across skill levels do not always represent differences in self-awareness, but rather they may arise from difficulties that performers have at evaluating how well others are performing. Most importantly, the unskilled-and-unaware phenomenon on a relative scale does not guarantee inaccurate self-evaluations of absolute performance.     

The Caveman's Conscience: Evolution and Moral Realism

An increasingly popular moral argument has it that the story of human evolution shows that we can explain the human disposition to make moral judgments without relying on a realm of moral facts. Such facts can thus be dispensed with. But this argument is a threat to moral realism only if there is no realist position that can explain, in the context of human evolution, the relationship between our particular moral sense and a realm of moral facts. I sketch a plausible evolutionary story that illuminates this relationship. First, the sorts of adaptive pressures facing early humans would have produced more than just potent prosocial emotions, as evolutionary antirealists like to claim; it would have produced judgments—often situated within emotions—to the effect that others could reasonably disapprove of some bit of conduct, for an early human who cared deeply about how others might respond to her action enjoyed the benefits of more cooperative exchanges than those early humans who did not. Second, according to objectivist versions of moral constructivism, moral facts just are facts about how others, ideally situated, would respond to one's conduct. Thus if any objectivist moral constructivism story is true, then we can intelligibly assert that a) our capacity for moral judgment is the product of adaptive pressures acting on early humans and b) some moral judgments are objectively true.     

The Significance of Semantic Realism

This paper is concerned with the relationship between the metaphysical doctrine of realism about the external world and semantic realism, as characterised by Michael Dummett. I argue that Dummett's conception of the relationship is flawed, and that Crispin Wright's account of the relationship, although designed to avoid the problems which beset Dummett's, nevertheless fails for similar reasons. I then aim to show that despite the fact that Dummett and Wright both fail to give a plausible account of the relationship between semantic realism and the metaphysical doctrine of realism, the semantic issue and the metaphysical issue are importantly related. I outline the precise sense in which the evaluation of semantic realism is relevant to the evaluation of realism about the external world, a sense overlooked by opponents of Dummett, such as Simon Blackburn and Michael Devitt. I finish with some brief remarks on metaphysics, semantics, and the nature of philosophy, and suggest that Dummett's arguments against semantic realism can retain their relevance to metaphysical debate even if we reject Dummett's idea that the theory of meaning is thefoundation of all philosophy.     

The Price of Non-reductive Moral Realism

Non-reductive moral realism is the view that there are moral properties which cannot be reduced to natural properties. If moral properties exist, it is plausible that they strongly supervene on non-moral properties- more specifically, on mental, social, and biological properties. There may also be good reasons for thinking that moral properties are irreducible. However, strong supervenience and irreducibility seem incompatible. Strong supervenience entails that there is an enormous number of modal truths (specifically, truths about exactly which non-moral properties necessitate which moral properties); and all these modal truths must be explained. If these modal truths can all be explained, then it must be a fundamental truth about the essence of each moral property that the moral property is necessarily equivalent to some property that can be specified purely in mental, social and biological terms; and this fundamental truth appears to be a reduction of the moral property in question. The best way to resist this argument is by resorting to the claim that mental and social properties are not, strictly speaking, natural properties, but are instead properties that can only be analysed in partly normative terms. Acceptance of that claim is the price of non-reductive moral realism.     

Comparing objective and subjective learning curves: judgments of learning exhibit increased underconfidence with practice

When participants studied a list of paired associates for several study-test cycles, their judgments of learning (JOLs) exhibited relatively good calibration on the 1st cycle, with a slight overconfidence. However, a shift toward marked underconfidence occurred from the 2nd cycle on. This underconfidence-with-practice (UWP) effect was very robust across several experimental manipulations, such as feedback or no feedback regarding the correctness of the answer, self-paced versus fixed-rate presentation, different incentives for correct performance, magnitude and direction of associative relationships, and conditions producing different degrees of knowing. It was also observed both in item-by-item JOLs and in aggregate JOLs. The UWP effect also occurred for list learning and for the memory of action events. Several theoretical explanations for this counterintuitive effect are discussed.     

Residual sensory capacities of the deaf: a signal detection analysis of a visual discrimination task.

This experiment compared the visual sensory sensitivity of deaf and hearing subjects in a signal detection paradigm. Subjects (ns = 6) were required to give forced-choice responses to a brightness discrimination task under three stimulus probability conditions (0.25, 0.50, and 0.75). A total of 1,800 trials were given to each subject and utilized to construct isosensitivity functions and d' and Beta, indices for sensory sensitivity and response bias, respectively. The results showed that no enhanced sensory sensitivity is present for these deaf children and questions the classical sensory compensation hypothesis. Furthermore, the deaf subjects responded in a relatively bias-free manner to variations in stimulus probability.