On belief bias in syllogistic reasoning

A multinomial model is used to disentangle the respective contributions of reasoning processes and response bias in conclusion-acceptance data that exhibit belief bias. A model-based meta-analysis of 22 studies reveals that such data are structurally too sparse to allow discrimination of different accounts of belief bias. Four experiments are conducted to obtain richer data, allowing deeper tests through the use of the multinomial model. None of the current accounts of belief bias is consistent with the complex pattern of results. A new theory of belief bias is proposed that assumes that most reasoners construct only one mental model representing the premises as well as the conclusion or, in the case of an unbelievable conclusion, its logical negation. New predictions derived from the theory are confirmed in 4 additional studies.     

Working memory, metacognitive uncertainty, and belief bias in syllogistic reasoning

Studies of syllogistic reasoning have shown that the size of the belief bias effect varies with manipulations of logical validity and problem form. This paper presents a mental models-based account, which explains these findings in terms of variations in the working-memory demands of different problem types. We propose that belief bias may reflect the use of a heuristic that is applied when a threshold of uncertainty in one's processing-attributable to working-memory overload-is exceeded during reasoning. Three experiments are reported, which tested predictions deriving from this account. In Experiment 1, conclusions of neutral believability were presented for evaluation, and a predicted dissociation was observed in confidence ratings for responses to valid and invalid arguments, with participants being more confident in the former. In Experiment 2, an attempt to manipulate working-memory loads indirectly by varying syllogistic figure failed to produce predicted effects upon the size of the belief bias effect. It is argued that the employment of a conclusion evaluation methodology minimized the effect of the figural manipulation in this experiment. In Experiment 3, participants' articulatory and spatial recall capacities were calibrated as a direct test of working-memory involvement in belief bias. Predicted differences in the pattern of belief bias observed between highand lowspatial recall groups supported the view that limited working memory plays a key role in belief bias.     

The source of belief bias effects in syllogistic reasoning.

In studies of the belief bias effect in syllogistic reasoning, an interaction between logical validity and the believability of the conclusion has been found; in essence, logic has a larger effect on unbelievable than on believable conclusions. Two main explanations have been proposed for this finding. The selective scrutiny account claims that people focus on the conclusion and only engage in logical processing if this is found to be unbelievable; while the misinterpreted necessity account claims that subjects misunderstand what is meant by logical necessity and respond on the basis of believability when indeterminate syllogisms are presented. Experiments 1 and 2 compared the predictions of these two theories by examining whether the interaction would disappear if only determinate syllogisms were used. It did, thus providing strong support for the misinterpreted necessity explanation. However, the results are also consistent with a version of the mental models theory, and so Experiment 3 was carried out to compare these two explanations. The mental models theory received strong support, as it did also in the follow-up Experiments 4 and 5. It is concluded that people try to construct a mental model of the premises but, if there is a believable conclusion consistent with the first model they produce, then they fail to construct alternative models.     

On the conflict between logic and belief in syllogistic reasoning

Three experiments are reported that investigate the weighting attached to logic and belief in syllogistic reasoning. Substantial belief biases were observed despite controls for possible conversions of the premises. Equally substantial effects of logic were observed despite controls for two possible response biases. A consistent interaction between belief and logic was also recorded; belief bias was more marked on invalid than on valid syllogisms. In all experiments, verbal protocols were recorded and analyzed. These protocols are interpreted in some cases as providing rationalizations for prejudiced decisions and, in other cases, as reflecting a genuine process of premise to conclusion reasoning. In the latter cases, belief bias was minimal but still present. Similarly, even subjects who focus primarily on the conclusion are influenced to an extent by the logic. Thus a conflict between logic and belief is observed throughout, but at several levels of extent.     

Using forced choice to test belief bias in syllogistic reasoning

In deductive reasoning, believable conclusions are more likely to be accepted regardless of their validity. Although many theories argue that this belief bias reflects a change in the quality of reasoning, distinguishing qualitative changes from simple response biases can be difficult (Dube, Rotello, & Heit, 2010). We introduced a novel procedure that controls for response bias. In Experiments 1 and 2, the task required judging which of two simultaneously presented syllogisms was valid. Surprisingly, there was no evidence for belief bias with this forced choice procedure. In Experiment 3, the procedure was modified so that only one set of premises was viewable at a time. An effect of beliefs emerged: unbelievable conclusions were judged more accurately, supporting the claim that beliefs affect the quality of reasoning. Experiments 4 and 5 replicated and extended this finding, showing that the effect was mediated by individual differences in cognitive ability and analytic cognitive style. Although the positive findings of Experiments 3–5 are most relevant to the debate about the mechanisms underlying belief bias, the null findings of Experiments 1 and 2 offer insight into how the presentation of an argument influences the manner in which people reason.     

Characterizing belief bias in syllogistic reasoning: A hierarchical Bayesian meta-analysis of ROC data

The belief-bias effect is one of the most-studied biases in reasoning. A recent study of the phenomenon using the signal detection theory (SDT) model called into question all theoretical accounts of belief bias by demonstrating that belief-based differences in the ability to discriminate between valid and invalid syllogisms may be an artifact stemming from the use of inappropriate linear measurement models such as analysis of variance (Dube et al., Psychological Review, 117(3), 831–863, 2010). The discrepancy between Dube et al.’s, Psychological Review, 117(3), 831–863 (2010) results and the previous three decades of work, together with former’s methodological criticisms suggests the need to revisit earlier results, this time collecting confidence-rating responses. Using a hierarchical Bayesian meta-analysis, we reanalyzed a corpus of 22 confidence-rating studies (N =?993). The results indicated that extensive replications using confidence-rating data are unnecessary as the observed receiver operating characteristic functions are not systematically asymmetric. These results were subsequently corroborated by a novel experimental design based on SDT’s generalized area theorem. Although the meta-analysis confirms that believability does not influence discriminability unconditionally, it also confirmed previous results that factors such as individual differences mediate the effect. The main point is that data from previous and future studies can be safely analyzed using appropriate hierarchical methods that do not require confidence ratings. More generally, our results set a new standard for analyzing data and evaluating theories in reasoning. Important methodological and theoretical considerations for future work on belief bias and related domains are discussed.     

Logic and belief across the lifespan: the rise and fall of belief inhibition during syllogistic reasoning

Popular reasoning theories postulate that the ability to inhibit inappropriate beliefs lies at the heart of the human reasoning engine. Given that people's inhibitory capacities are known to rise and fall across the lifespan, we predicted that people's deductive reasoning performance would show similar curvilinear age trends. A group of children (12‐year‐olds), young adults (20‐year‐olds), and older adults (65+‐year‐olds) were presented with a classic syllogistic reasoning task and a decision‐making questionnaire. Results indicated that on syllogisms where beliefs and logic conflicted, reasoning performance showed the expected curvilinear age trend: Reasoning performance initially increased from childhood to early adulthood but declined again in later life. On syllogisms where beliefs and logic were consistent and sound reasoning did not require belief inhibition, however, age did not affect performance. Furthermore, across the lifespan we observed that the better people were at resisting intuitive temptations in the decision‐making task, the less they were biased by their beliefs on the conflict syllogisms. As with the effect of age, one's ability to override intuitions in the decision‐making task did not mediate reasoning performance on the no‐conflict syllogisms. Results lend credence to the postulated central role of inhibitory processing in those situations where beliefs and logic conflict.     

A transitive-chain theory of syllogistic reasoning

A new theory of syllogistic reasoning, called the transitive-chain theory, is presented. The transitive-chain theory proposes that information about set relations is represented in memory by pairs of informational components. The theory further proposes that information about set relations is integrated by applying a small set of rules to transitive chains of these set relations. The rules that are applied to these chains are specified in detail. The theory is cast in terms of information-processing models for variants of the syllogistic reasoning task, and then mathematical models that quantify each of these information-processing models are presented. In a series of five experiments, the transitive-chain theory provides a good account of the response-choice data for syllogisms with various types of content, quantifiers, and logical relations (categorical and conditional). The results of these experiments offer tentative answers to four issues in the theory of syllogistic reasoning: (a) representation of premise information; (b) combination of premise information; (c) sources of difficulty in syllogistic reasoning; and (d) generality of the processes used in syllogistic reasoning.     

Effects of belief and logic on syllogistic reasoning: Eye-movement evidence for selective processing models

Studies of syllogistic reasoning have demonstrated a nonlogical tendency for people to endorse more believable conclusions than unbelievable ones. This belief bias effect is more dominant on invalid syllogisms than valid ones, giving rise to a logic by belief interaction. We report an experiment in which participants' eye movements were recorded in order to provide insights into the nature and time course of the reasoning processes associated with manipulations of conclusion validity and believability. Our main dependent measure was people's inspection times for syllogistic premises, and we tested predictions deriving from three contemporary mental-models accounts of the logic by belief interaction. Results supported recent "selective processing" theories of belief bias (e.g., Evans, 2000; Klauer, Musch, & Naumer, 2000), which assume that the believability of a conclusion biases model construction processes, rather than biasing the search for falsifying models (e.g., Oakhill & Johnson-Laird, 1985) or a response stage of reasoning arising from subjective uncertainty (e.g., Quayle & Ball, 2000). We conclude by suggesting that the eye-movement analyses in reasoning research may provide a useful adjunct to other process-tracing techniques such as verbal protocol analysis.     

Believability and syllogistic reasoning

In this paper we investigate the locus of believability effects in syllogistic reasoning. We identify three points in the reasoning process at which such effects could occur: the initial interpretation of premises, the examination of alternative representations of them (in all of which any valid conclusion must be true), and the "filtering" of putative conclusions. The effect of beliefs at the first of these loci is well established. In this paper we report three experiments that examine whether beliefs have an effect at the other two loci. In experiments 1 and 2 subjects drew their own conclusions from syllogisms that suggested believable or unbelievable ones. In the third experiment they evaluated conclusions that were presented to them. The data show that beliefs both affect the examination of alternative models and act as a filter on putative conclusions. We conclude by showing how some types of problem and some problem contents make the existence of alternative models more obvious than others.     

A three-factor model of syllogistic reasoning: The study of isolable stages

Computer models of the syllogistic reasoning process are constructed. The models are used to determine the influence of three factors—the misinterpretation of the premises, the limited capacity of working memory, and the operation of the deductive strategy—on subjects’ behavior. Evidence from Experiments 1, 2, and 3 suggests that all three factors play important roles in the production of errors when “possibly true” and “necessarily false” are the two response categories. This conclusion does not agree with earlier analyses that had singled out one particular factor as crucial. Evidence from Experiment 4 suggests that the influence of the first two factors remains strong when “necessarily true” is used as an additional response category. However, the third factor appears to interact with task demands. Some concluding analyses suggest that the models offer alternative explanations for certain well established results.     

Working memory and syllogistic reasoning

The purpose of this study was to examine the relation between working memory span and syllogistic reasoning performance. In addition, performance for the reasoning task was compared to predictions made by mental model theory and the probability heuristics model. According to mental model theory, syllogisms that require the use of more mental models are more difficult. According to the probability heuristics model difficulty is related to the number of probabilistic heuristics that must be applied, or (for invalid syllogisms) inconsistencies between the derived and correct conclusion. The predictions of these theories were examined across two experiments. In general, people with larger working memory capacities reasoned better. Also, the responses made by people with larger capacities were more likely to correspond to the predictions made by both mental model theory and the probability heuristics model. Relations between working memory span and performance were also consistent with both theories.     

Source credibility and syllogistic reasoning

Two experiments examined whether a source credibility effect would be observed for a syllogistic reasoning task. In the experiments, people were given two statements, presented as the results from a survey, followed by a conclusion that was supposedly made by one of two sources. In Experiment 1, one of the sources was described as honest and the other as dishonest, and in Experiment 2, one of the sources was described as an expert and the other as a non-expert. Because a pilot experiment showed that credibility can be overridden by people’s experience with a source, all conclusions in Experiments 1 and 2 were ones that were likely to be accepted (i.e., necessary and possible strong conclusions). Both experiments showed a clear source credibility effect, particularly for the invalid conclusions. These results, along with the belief bias effect and previous research with conditional reasoning, suggest that people can be influenced by extraneous context, such as the honesty or expertise of a source, in a syllogistic reasoning task.     

Components of syllogistic reasoning

The present research sought to understand the components of syllogistic reasoning used in a syllogistic evaluation task. In this task, subjects must indicate whether a conclusion such as “Some Yale professors are humbugs” is definitely true, or never true of a set of premises such as “Some humbugs study syllogistic reasoning; some Yale professors study syllogistic reasoning”. A modified form of componential analysis (Sternberg 1977, 1978) was used to decompose the syllogistic evaluation task with abstract content into encoding and encoding plus combination subtasks. The response-choice data from these subtasks were used to provide (a) direct tests of a proposed theory of syllogistic reasoning, and in particular, of its assumptions about sources of error in syllogistic reasoning; and (b) direct inferences regarding the representation of relations between the subject and predicate of the premises as encoded and combined. The results supported a proposed transitive-chain model of syllogistic reasoning.     

A successive-conditionalization approach to disjunctive and syllogistic reasoning

With p and q each standing for a familiar event, a disjunctive statement, “either p or q”, seems quite different from its material conditional, “if not p then q”. The notions of sufficiency and necessity seem specific to conditional statements. It is surprising, however, to find that perceived sufficiency and necessity affect disjunctive reasoning in the way they affect conditional reasoning. With B and C each standing for a category name, a universal statement, “all B are C”, seems stronger than its logically equivalent conditional statement, “if B then C”. However, the effects of perceived sufficiency or necessity were found to be as pronounced in conditional reasoning as in syllogistic reasoning. Furthermore, two experiments also showed that (a) MP (modus ponens)-comparable disjunctive reasoning was as difficult as MT (modus tollens)-comparable disjunctive reasoning, and that (b) MT-comparable syllogisms were easier to solve than MT problems in conditional reasoning.     

The development of linear syllogistic reasoning

Between 24 and 26 children in each of grades 3, 5, 7, 9, and 11 (with mean ages of 8.5, 10.2, 13.0, 15.0, and 16.6 years, respectively) were tested in their ability to solve linear syllogisms, problems such as “John is taller than Mary. Mary is taller than Pete. Who is tallest? John, Mary, Pete.” Response latencies and error rates decreased across grade levels and sessions. Component latencies also generally decreased with increasing age. Four alternative information-processing models were fit to the group latency data at each grade level. These models were (a) a spatial model, according to which people represent the terms of a linear syllogism in the form of a linear spatial array; (b) a linguistic model, according to which linear syllogisms are solved via inferences on functional relations represented by linguistic deep structures; (c) an algorithmic model, according to which people solve linear syllogisms by applying a series of simple, essentially mechanical steps, and (d) a mixed model, which combines selected features of the spatial and linguistic models, and adds new features of its own. The latency data supported the mixed model at each grade level, although in grade 9 the model was not the preferred one until the second session of testing.     

Linguistic marking,strategy, and affect in syllogistic reasoning

It has repeatedly been shown that three-term-series problems with unmarked comparatives (e.g., taller, higher) are solved more quickly than otherwise identical problems using their marked opposites (e.g., shorter, lower). Clark's principle of lexical marking accounts for these results in terms of a simpler semantic featural coding of the unmarked comparative with respect to its marked counterpart. Huttenlocher's theory of spatial imagery accounts for these same results via the subjects' mental ordering of the three terms as instructed by the problem statements. The present research demonstrated that while the lower latency of the unmarked adjective is a reliable effect, congruent ordering strategies are necessary for significant results. Subjects who order terms from unmarked to marked produce significant results for Clark's principle of lexical marking. Those who order terms in opposite direction do not. Further, it is shown that the choice of direction of ordering is itself significantly influenced by the affective value of the adjectives in context and to the individual subject.This research is based on a thesis submitted in partial fulfillment of the requirements for the doctoral degree to the University of Illinois.     

Activation of end-terms in syllogistic reasoning

We report five experiments showing that the activation of the end-terms of a syllogism is determined by their position in the composite model of the premises. We show that it is not determined by the position of the terms in the rule being applied (Ford, 1994), by the syntactic role of the terms in the premises (Polk & Newell, 1995; Wetherick & Gilhooly, 1990), by the type of conclusion (Chater & Oaksford, 1999), or by the terms from the source premise (Stenning & Yule, 1997). In our first experiment we found that after reading a categorical premise, the most active term is the last term in the premise. In Experiments 2, 3, and 4 we demonstrated that this pattern of activity is due to the position of the concepts in the model of the premises, regardless of the delay after reading the premises (150 or 2000 msec) or the quantity of the quantifiers (universal or existential). The fifth experiment showed that the pattern switches around after participants evaluate a conclusion. We propose that the last element in the model maintains a higher level of activity during the comprehension process because it is generally used to attach the incoming information. After this process, the first term becomes more active because it is the concept to which the whole representation is referred. These results are predicted by the mental model theory (Johnson-Laird & Byrne, 1991), but not by the verbal reasoning theory (Polk & Newell, 1995), the graphical methods theory (Yule & Stenning, 1992), the attachment-heuristic theory (Chater & Oaksford, 1999), or the mental rules theory (Ford, 1994).     

The effect of figure on syllogistic reasoning

Several investigators of reasoning with abstract categorical syllogisms have noted that the four figures of the classical syllogism, which vary in the order in which terms occur in the major and minor premises, resemble the four three-stage mediation paradigms in paired associate learning. On the basis of the analogy, figure differences have been predicted and some support has been obtained for these predictions. The present paper proposes an alternative information processing explanation for figure effects based upon contradictions between the correct conclusions that follow from forward and backward processing of the premises. This explanation, in contrast to the associative explanation, successfully predicts which premise combinations will show figure effects as well as the nature of the specific errors which will occur.