Subjective confidence in one's answers: the consensuality principle

In answering general-information questions, a within-person confidence-accuracy (C-A) correlation is typically observed, suggesting that people can monitor the correctness of their knowledge. However, because the correct answer is generally the consensual answer--the one endorsed by most participants--confidence judgment may actually monitor the consensuality of the answer rather than its correctness. Indeed, the C-A correlation was positive for items with a consensually correct answer but negative for items with a consensually wrong answer. Results suggest that the consensuality-confidence correlation may be mediated by 2 internal mnemonic cues that are correlated with consensuality: Consensual answers are reached faster and are selected more consistently by the same person on different occasions than nonconsensual answers. The results argue against a direct-access view of confidence judgments and suggest that such judgments will be accurate only as long as people's responses are by and large correct across the sampled items, thus stressing the criticality of a representative design.     

Falsifying mental models: Testing the predictions of theories of syllogistic reasoning

Four experiments are reported that tested the claim, drawn from mental models theory, that reasoners attempt to construct alternative representations of problems that might falsify preliminary conclusions they have drawn. In Experiment 1, participants were asked to indicate which alternative conclusion(s) they had considered in a syllogistic reasoning task. In Experiments 2-4, participants were asked to draw diagrams consistent with the premises, on the assumption that these diagrams would provide insights into the mental representation being used. In none of the experiments was there any evidence that people constructed more models for multiple-model than for single-model syllogisms, nor was there any correlation between number of models constructed and overall accuracy. The results are interpreted as showing that falsification of the kind proposed by mental models theory may not routinely occur in reasoning.     

Response demands and the recruitment of heuristic strategies in syllogistic reasoning

Two experiments investigated whether dealing with a homogeneous subset of syllogisms with time-constrained responses encouraged participants to develop and use heuristics for abstract (Experiment 1) and thematic (Experiment 2) syllogisms. An atmosphere-based heuristic accounted for most responses with both abstract and thematic syllogisms. With thematic syllogisms, a weaker effect of a belief heuristic was also observed, mainly where the correct response was inconsistent with the atmosphere of the premises. Analytic processes appear to have played little role in the time-constrained condition, whereas their involvement increased in a self-paced, unconstrained condition. From a dual-process perspective, the results further specify how task demands affect the recruitment of heuristic and analytic systems of reasoning. Because the syllogisms and experimental procedure were the same as those used in a previous neuroimaging study by Goel, Buchel, Frith, and Dolan (2000), the result also deepen our understanding of the cognitive processes investigated by that study.     

Response demands and the recruitment of heuristic strategies in syllogistic reasoning

Two experiments investigated whether dealing with a homogeneous subset of syllogisms with time-constrained responses encouraged participants to develop and use heuristics for abstract (Experiment 1) and thematic (Experiment 2) syllogisms. An atmosphere-based heuristic accounted for most responses with both abstract and thematic syllogisms. With thematic syllogisms, a weaker effect of a belief heuristic was also observed, mainly where the correct response was inconsistent with the atmosphere of the premises. Analytic processes appear to have played little role in the time-constrained condition, whereas their involvement increased in a self-paced, unconstrained condition. From a dual-process perspective, the results further specify how task demands affect the recruitment of heuristic and analytic systems of reasoning. Because the syllogisms and experimental procedure were the same as those used in a previous neuroimaging study by Goel, Buchel, Frith, and Dolan (2000), the result also deepen our understanding of the cognitive processes investigated by that study.     

Individual differences in working memory capacity and resistance to belief bias in syllogistic reasoning

In two experiments, we investigated the possibility that individual differences in working memory capacity (WMC) would provide resistance to belief bias in syllogistic reasoning. In Experiment 1 (N?=?157), participants showed a belief bias effect in that they had longer response times and decreased accuracy on syllogisms with conflict between the validity and believability of the conclusion than on syllogisms with no such conflict. However, this effect did not differ as a function of individual differences in WMC. Experiment 2 (N?=?122) replicated this effect with the addition of decontextualized (i.e., nonsense) syllogisms as a different means of measuring the magnitude of the belief bias effect. Although individual differences in WMC and fluid intelligence were related to better reasoning overall, the magnitude of the belief bias effect was not smaller for participants with greater WMC. The present study offers two novel findings: (a) The belief bias effect is independent of individual differences in WMC and fluid intelligence, and (b) resolving conflict in verbal reasoning is not a type of conflict resolution that correlates with individual differences in WMC, further establishing boundary conditions for the role of WMC in human cognitive processes.     

Content and strategy in syllogistic reasoning.

Syllogistic reasoning has been investigated as a general deductive process (Johnson-Laird & Byrne, 1991; Revlis, 1975; Rips, 1994). However, several studies have demonstrated the role of cognitive strategies in this type of reasoning. These strategies focus on the method used by the participants (Ford, 1995; Gilhooly, Logie, Wetherick, & Wynn, 1993) and strategies related to different interpretations of the quantified premises (Roberts, Newstead, & Griggs, 2001). In this paper, we propose that content (as well as individual cognitive differences) is an important factor in inducing a certain strategy or method for syllogistic resolution. Specifically, we suggest that syllogisms with a causal conditional premise that can be extended by an agency premise induce the use of a conditional method. To demonstrate this, we carried out two experiments. Experiment 1 provided evidence that this type of syllogism leads participants to draw the predicted conditional conclusions, in contrast with control content syllogisms. In Experiment 2, we demonstrated that the drawing of conditional conclusions is based on a causal conditional to an agent representation of the syllogism premises. These results support the role of content as inducing a particular strategy for syllogistic resolution. The implications of these results are discussed.     

Subjective measures of awareness and implicit cognition

In this article, we examine whether artificial grammar learning is implicit according to a subjective criterion of awareness based on confidence ratings. In four experiments, participants discriminated between grammatical and ungrammatical sequences in both the same (Experiment 1) and a novel (Experiments 2-4) vocabulary and indicated their confidence in each decision. Replicating earlier studies, confidence judgments reported on a continuous scale (50%-100%) were only weakly related to accuracy, suggesting that learning was implicit. In contrast, confidence judgments reported on a binary scale (high vs. low) revealed that confidence was related to accuracy. We show that participants are better able to place their phenomenal states on a binary scale, as compared with a continuous scale.     

Confidence and accuracy in deductive reasoning

In two experiments, we investigated the relationship between confidence and accuracy in syllogistic reasoning. Participants judged the validity of conclusions and provided confidence ratings twice for each problem: once quickly and again after further deliberation. Correlations between confidence and accuracy were small or nonexistent. In addition, confidence and accuracy were mediated by different variables. Confidence judgments appeared to reflect external cues, so that confidence was greater when the participants were allowed additional time to think about the problem, as well as when the conclusion was either believable or unbelievable, rather than neutral. In contrast, accuracy changed little as a function of the amount of time available and did not differ for believable and neutral problems. These data support a model in which initial decisions are made quickly, on the basis of heuristic cues, and analytic processes are used to justify or rationalize the earlier decision.     

逻辑－信念冲突与推理难度对逻辑和信念判断的影响

双加工理论框架下的平行竞争模型和默认干涉模型对推理过程中启发式和分析式加工的启动次序和相互作用的性质提出了不同假设。本研究通过考察被试在信念判断和逻辑判断两种指导语下解决不同难度和不同类型推理问题的正确率和反应时,对两个模型的预测进行了检验。实验结果表明,解决简单问题时,问题类型对信念判断有显著影响,对逻辑判断无显著影响;解决复杂问题时,问题类型对逻辑判断的影响显著大于对信念判断的影响。实验结果支持平行竞争模型的假设,即两类加工在推理过程中同时启动,共同竞争最终的推理反应。     

Logic feels so good-I like it! Evidence for intuitive detection of logicality in syllogistic reasoning

When people evaluate syllogisms, their judgments of validity are often biased by the believability of the conclusions of the problems. Thus, it has been suggested that syllogistic reasoning performance is based on an interplay between a conscious and effortful evaluation of logicality and an intuitive appreciation of the believability of the conclusions (e.g., Evans, Newstead, Allen, & Pollard, 1994). However, logic effects in syllogistic reasoning emerge even when participants are unlikely to carry out a full logical analysis of the problems (e.g., Shynkaruk & Thompson, 2006). There is also evidence that people can implicitly detect the conflict between their beliefs and the validity of the problems, even if they are unable to consciously produce a logical response (e.g., De Neys, Moyens, & Vansteenwegen, 2010). In 4 experiments we demonstrate that people intuitively detect the logicality of syllogisms, and this effect emerges independently of participants' conscious mindset and their cognitive capacity. This logic effect is also unrelated to the superficial structure of the problems. Additionally, we provide evidence that the logicality of the syllogisms is detected through slight changes in participants' affective states. In fact, subliminal affective priming had an effect on participants' subjective evaluations of the problems. Finally, when participants misattributed their emotional reactions to background music, this significantly reduced the logic effect.     

Actor-observer differences in realism in confidence and frequency judgments

Taking a social psychological approach to metacognitive judgments, this study analyzed the difference in realism (validity) in confidence and frequency judgments (i.e., estimates of overall accuracy) between one's own and another person's answers to general knowledge questions. Experiment 1 showed that when judging their own answers, compared with another's answers, the participants exhibited higher overconfidence, better ability to discriminate correct from incorrect answers, lower accuracy, and lower confidence. However, the overconfidence effect could be attributable to the lowest level of confidence. Furthermore, when heeding additional information about another's answers the participants showed higher confidence and better discrimination ability. The overconfidence effect of Experiment 1 was not found in Experiment 2. However, the results of Experiment 2 were consistent with Experiment 1 in terms of discrimination ability, confidence, and accuracy. Finally, in both experiments the participants gave lower frequency judgments of their own overall accuracy compared with their frequency judgments of another person's overall accuracy.     

Ideological belief bias with political syllogisms

Abstract

The belief bias in reasoning occurs when individuals are more willing to accept conclusions that are consistent with their beliefs than conclusions that are inconsistent. The present study examined a belief bias in syllogisms containing political content. In two experiments, participants judged whether conclusions were valid, completed political ideology measures, and completed a cognitive reflection test. The conclusions varied in validity and in their political ideology (conservative or liberal). Participants were sensitive to syllogisms’ validity and conservatism. Overall, they showed a liberal bias, accepting more liberal than conservative conclusions. Furthermore, conservative participants accepted more conservative conclusions than liberal conclusions, whereas liberal participants showed the opposite pattern. Cognitive reflection did not magnify this effect as predicted by a motivated system 2 reasoning account of motivated ideological reasoning. These results suggest that people with different ideologies may accept different conclusions from the same evidence.     

The influence of rule availability and item similarity on metacognitive monitoring during categorisation

In four experiments, we examined the sources of metacognitive judgments during the categorisation of new items after the learning of old items was completed. In the rule condition, the categorisation rule was explicitly given to the participants during learning, while in the no-rule condition participants relied on feedback to infer category membership. In the transfer phase, two types of novel items were used: good and bad transfer items. Transfer items also differed in the level of similarity to their training pairs. In the rule condition, all types of transfer items were classified with high accuracy and confidence. In the no-rule condition, a dissociation between accuracy and confidence was revealed. Good transfer items were classified more accurately than bad transfer items, whereas similar items were classified with higher confidence than dissimilar items. The obtained results suggest the utilisation of two potential cues for metacognitive judgments: declarative knowledge if the correct rule is explicitly available, and item similarity if it is difficult to infer the correct rule from feedback.     

Subjective confidence in perceptual judgments: a test of the self-consistency model

Two questions about subjective confidence in perceptual judgments are examined: the bases for these judgments and the reasons for their accuracy. Confidence in perceptual judgments has been claimed to rest on qualitatively different processes than confidence in memory tasks. However, predictions from a self-consistency model (SCM), which had been confirmed for general-information questions (Koriat, 2010) and social attitudes (Koriat & Adiv, 2010), are shown to hold true also for perceptual judgments. In SCM, confidence is modeled by the procedure for assessment of statistical level of confidence: For a 2-alternative, forced-choice item, confidence is based on the consistency with which the choice is favored across a sample of representations of the item, and acts as a monitor of the likelihood that a new sample will yield the same choice. Assuming that these representations are drawn from commonly shared populations of representations associated with each item, predictions regarding the basis of confidence were confirmed by results concerning the functions relating confidence and choice latency to interparticipant consensus and to intraparticipant consistency for majority and minority choices. With regard to the confidence-accuracy (C/A) relationship, the consensuality principle, documented for general-knowledge tasks (Koriat, 2008a), was replicated for perceptual judgments: Confidence correlated with the consensuality of the choice rather than with its correctness, suggesting that the C/A correlation is due to the relationship between confidence and self-consistency and is positive only as long as the correct choices are the consistently made choices. SCM provides a general model for the basis and accuracy of confidence judgments across different domains.     

Strategies in Syllogistic Reasoning

This paper is about syllogistic reasoning, i.e., reasoning from such pairs of premises as, All the chefs are musicians; some of the musicians are painters. We present a computer model that implements the latest account of syllogisms, which is based on the theory of mental models. We also report four experiments that were designed to test this account. Experiments 1 and 2 examined the strategies revealed by the participants' use of paper and pencil as aids to reasoning. Experiment 3 used a new technique to externalize thinking. The participants had to refute, if possible, putative conclusions by constructing external models that were examples of the premises but counterexamples of the conclusions. Experiment 4 used the same techniques to examine the participants' strategies as they drew their own conclusions from syllogistic premises. The results of the experiments showed that individuals not trained in logic can construct counterexamples, that they use similar operations to those implemented in the computer model, but that they rely on a much greater variety of interpretations of premises and of search strategies than the computer model does. We re-evaluates current theories of syllogistic reasoning in the light of these results.     

Distinguishing states of awareness from confidence during retrieval: evidence from amnesia

Two experiments were conducted to determine whether recollective experience is distinguishable from confidence. In Experiment 1, we tested college participants in a within-subjects design and replicated Gardiner and Java's (1990) findings from a between-subjects design. We observed higher remember judgments for words than for nonwords, but higher know judgments for nonwords than for words. For confidence judgments, we obtained greater sure than unsure responses for both words and nonwords. In Experiment 2, we tested amnesic participants and matched control participants. Control participants produced the same pattern of results as college participants, but the results of amnesic participants diverged in an important way. For confidence judgments, the amnesic participants, like the control and college participants, made more sure than unsure judgments to both words and nonwords. But for recollective judgments, amnesic participants did not produce the crossover interaction for words and nonwords. This striking difference between the performance of memory-intact and amnesic participants demonstrates that recollective judgments and confidence that accompany retrieval are not isomorphic psychological experiences.     

The accuracy of meta-metacognitive judgments: regulating the realism of confidence

Can people improve the realism of their confidence judgments about the correctness of their episodic memory reports by deselecting the least realistic judgments? An assumption of Koriat and Goldsmith's (Psychol Rev 103:490-517, 1996) model is that confidence judgments regulate the reporting of memory reports. We tested whether this assumption generalizes to the regulation of the realism (accuracy) of confidence judgments. In two experiments, 270 adults in separate conditions answered 50 recognition and recall questions about the contents of a just-seen video. After each answer, they made confidence judgments about the answer's correctness. In Experiment 1, the participants in the recognition conditions significantly increased their absolute bias when they excluded 15 questions. In Experiment 2, the participants in the recall condition significantly improved their calibration. The results indicate that recall, more than recognition, offers valid cues for participants to increase the realism of their report. However, the effects were small with only weak support for the conclusion that people have some ability to regulate the realism in their confidence judgments.     

Phonological and visual distinctiveness effects in syllogistic reasoning: Implications for mental models theory

Two experiments are reported in which the representational distinctiveness of terms within categorical syllogisms was manipulated in order to examine the assumption of mental models theory that abstract, spatially based representations underpin deduction. In Experiment 1, participants evaluated conclusion validity for syllogisms containing either phonologically distinctive terms (e.g., harks, paps, and fids) or phonologically nondistinctive terms (e.g., fuds, fods, and feds). Logical performance was enhanced with the distinctive contents, suggesting that the phonological properties of syllogism terms can play an important role in deduction. In Experiment 2, participants received either the phonological materials from Experiment 1 or syllogisms involving distinctive or nondistinctive visual contents. Logical inference was again enhanced for the distinctive contents, whether phonological or visual in nature. Our findings suggest a broad involvement of multimodal information in syllogistic reasoning and question the assumed primacy of abstract, spatially organized representations in deduction, as is claimed by mental models theorists.     

Logic, beliefs, and instruction: a test of the default interventionist account of belief bias

According to dual-process accounts of thinking, belief-based responses on reasoning tasks are generated as default but can be intervened upon in favor of logical responding, given sufficient time, effort, or cognitive resource. In this article, we present the results of 5 experiments in which participants were instructed to evaluate the conclusions of logical arguments on the basis of either their logical validity or their believability. Contrary to the predictions arising from these accounts, the logical status of the presented conclusion had a greater impact on judgments concerning its believability than did the believability of the conclusion on judgments about whether it followed logically. This finding was observed when instructional set was presented as a between-participants factor (Experiment 1), when instruction was indicated prior to problem presentation by a cue (Experiment 2), and when the cue appeared simultaneously with conclusion presentation (Experiments 3 and 4). The finding also extended to a range of simple and more complex argument forms (Experiment 5). In these latter experiments, belief-based judgments took significantly longer than those made under logical instructions. We discuss the implications of these findings for default interventionist accounts of belief bias.     

Decision Strategies and the Confidence–Accuracy Relationship in Face Recognition

The decision strategy used to select a choice set from an array of alternative options is known to affect the composition of the final choice set. Specifically, individuals incorporate more answers into their choice set when it is created by eliminating implausible items than when the set is created through the inclusion of plausible options. This difference is accounted for in a decision framework that posits a general reluctance to change the status quo (i.e., actively include or exclude an item). We extended this work to investigate, not only the decisions themselves, but also metacognitive judgments (i.e., confidence in the accuracy of the choice set). In two face recognition experiments, we tested the impact of decision strategy (Experiment 1) and confidence judgment strategy (Experiment 2) on the confidence–accuracy relationship. In Experiment 1, participants completed two blocks of recognition trials, one under inclusion (marking previously seen faces) and one under elimination (marking previously unseen faces) instructions. We observed superior resolution (i.e., discrimination between correct and incorrect) for inclusion trials, but only when they were completed prior to use of the elimination strategy. In Experiment 2, all participants completed face recognition trials under inclusion instructions, but we manipulated the strategy used to assess confidence. Again, we observed a significant impact of strategy on confidence–accuracy resolution. Thus, we observed that both the strategy employed to reach a decision and that employed to assess confidence affected the confidence–accuracy relationship. We discuss theoretical and applied (particularly for eyewitness identification and multiple‐choice testing) implications. Copyright © 2012 John Wiley & Sons, Ltd.