Inspection times and the selection task: What do eye-movements reveal about relevance effects?

Three experiments are reported that used eye-movement tracking to investigate the inspection-time effect predicted by Evans' (1996) heuristic-analytic account of the Wason selection task. Evans' account proposes that card selections are based on the operation of relevance-determining heuristics, whilst analytic processing only rationalizes selections. As such, longer inspection times should be associated with selected cards (which are subjected to rationalization) than with rejected cards. Evidence for this effect has been provided by Evans (1996) using computer- presented selection tasks and instructions for participants to indicate (with a mouse pointer) cards under consideration. Roberts (1998b) has argued that mouse pointing gives rise to artefactual support for Evans' predictions because of biases associated with the task format and the use of mouse pointing. We eradicated all sources of artefact by combining careful task constructions with eye-movement tracking to measure directly on-line attentional processing. All three experiments produced good evidence for the robustness of the inspection-time effect, supporting the predictions of the heuristic-analytic account.     

Inspection times, the change task, and the rapid-response selection task

Three experiments are reported, which are based upon the Wason four-card selection task inspection time paradigm, in which subjects solve computer-presented trials while using a mouse to indicate the card currently under consideration. Evans (1996) had shown that selected cards were inspected for longer than non-selected cards, and this was taken as support for the existence of pre-conscious heuristic processes that direct attention towards relevant aspects of a problem. However, Roberts (1998b) suggested that this inspection time effect is artefactual, due to task format induced biases. Experiment 1 utilized a "change" task: Cards were presented either as selected or not selected, and subjects changed these where necessary. This demonstrated an association between card selection and inspection time independently of one between the act of response and inspection time. Experiment 2 utilized a standard selection task, but subjects either responded within 2 s of each card presentation, or made selections with no time pressure. The curtailment of thinking time increased matching behaviour—more cards matching the terms in the rules were selected—and was replicated in Experiment 3 using a within-subjects design. Overall, the data support Evans' heuristic-analytic framework albeit with some caveats.     

Bayesian Rationality for the Wason Selection Task? A Test of Optimal Data Selection Theory

Oaksford and Chater (1994) proposed to analyse the Wason selection task as an inductive instead of a deductive task. Applying Bayesian statistics, they concluded that the cards that participants tend to select are those with the highest expected information gain. Therefore, their choices seem rational from the perspective of optimal data selection. We tested a central prediction from the theory in three experiments: card selection frequencies should be sensitive to the subjective probability of occurrence for individual cards. In Experiment 1, expected frequencies of the p- and the q-card were manipulated independently by concepts referring to large vs. small sets. Although the manipulation had an effect on card selection frequencies, there was only a weak correlation between the predicted and the observed patterns. In the second experiment, relative frequencies of individual cards were manipulated more directly by explicit frequency information. In addition, participants estimated probabilities for the four logical cases and of the conditional statement itself. The experimental manipulations strongly affected the probability estimates, but were completely unrelated to card selections. This result was replicated in a third experiment. We conclude that our data provide little support for optimal data selection theory.     

The suppression of q card selections: Evidence for deductive inference in Wason's Selection Task

The results of three experiments investigating the role of deductive inference in Wason's selection task are reported. In Experiment 1, participants received either a standard one-rule problem or a task containing a second rule, which specified an alternative antecedent. Both groups of participants were asked to select those cards that they considered were necessary to test whether the rule common to both problems was true or false. The results showed a significant suppression of q card selections in the two-rule condition. In addition there was weak evidence for both decreased p selection and increased not-q selection. In Experiment 2 we again manipulated number of rules and found suppression of q card selections only. Finally, in Experiment 3 we compared one- and two-rule conditions with a two-rule condition where the second rule specified two alternative antecedents in the form of a disjunction. The q card selections were suppressed in both of the two-rule conditions but there was no effect of whether the second rule contained one or two alternative antecedents. We argue that our results support the claimthat people make inferences about the unseen side of the cards when engaging with the indicative selection task.     

Temporal distinctiveness and repetition benefits in task switching: Disentangling stimulus-related and response-related contributions

Despite the popularity of the Wason selection task in the psychology of reasoning, doubt remains as to whether card choices actually reflect a process of reasoning. One view is that while participants reason about the cards and their hidden sides—as indicated by protocol analysis—this reasoning merely confabulates explanations for cards that were preconsciously cued. This hypothesis has apparently been supported by studies that show that participants predominantly inspect cards which they end up selecting. In this paper, we reanalyse the data of one such study, which used eye-movement tracking to record card inspection times (Ball, Lucas, Miles, & Gale, 2003). We show that while cards favoured by matching bias are inspected for roughly equal lengths of times, their selection rates are strongly affected by their logical status. These findings strongly support a two-stage account in which attention is necessary but not sufficient for card selections. Hence, reasoning does indeed affect participants' choices on this task.     

Temporal delays can facilitate causal attribution: Towards a general timeframe bias in causal induction

Does reasoning occur on the Wason selection task, or are card selections determined purely on the basis of heuristic processes? To answer this question two relevance-based theories of reasoning are compared: (1) the theory of Evans (1984, 1989; Evans &; Over, 1996), which takes the heuristic viewpoint, and (2) the theory of Sperber, Cara, and Girotto (1995), which takes the reasoning viewpoint. In three experiments, the effect of removing matching cards from the selection task array is examined. It is argued that the Sperber et al. theory makes clearer predictions about the results of these manipulations, which are confirmed, and that the Evans theory can only accommodate them if it allows the operation of reasoning processes. The results are also discussed in relation to Roth's (1979) account of the selection task, mental models theory, and information gain theory.     

Alternative antecedents, probabilities, and the suppression of fallacies in Wason's selection task

Three experiments examined the influence of a second rule on the pattern of card selections on Wason's selection task. In Experiment 1 participants received a version of the task with a single test rule or one of two versions of the task with the same original test rule together with a second rule. The probability of q was manipulated in the two-rules conditions by varying the size of the antecedent set in the second rule. The results showed a significant suppression of q card and not-p card selections in the alternative-rule conditions, but no difference as a function of antecedent set size. In Experiment 2 the size of the antecendent set in the two-rules conditions was manipulated using the context of a computer printing double-sided cards. The results showed a significant reduction of q card selections in the two-rules conditions, but no effect of p set size. In Experiment 3 the scenario accompanying the rule was manipulated, and it specified a single alternative antecedent or a number of alternative antecedents. The q card selection rates were not affected by the scenario manipulation but again were suppressed by the presence of a second rule. Our results suggest that people make inferences about the unseen side of the cards when engaging with the task and that these inferences are systematically influenced by the presence of a second rule, but are not influenced by the probabilistic characteristics of this rule. These findings are discussed in the context of decision theoretic views of selection task performance (Oaksford & Chater, 1994).     

Facilitation in the abstract selection task: The effects of attentional and instructional factors

The present study examined performance on Wason's four-card abstract selection task. Baseline performance is very poor, usually less than 10% correct; and this task has a long record of resistance to facilitation. It was hypothesized that the two primary sources of difficulty are selective encoding of the problem information and the lack of satisfactory analytic processing. Three experiments were conducted to test this hypothesis. In Experiment 1, performance was improved by explicating the implication rule. The majority of subjects, however, still failed to make the correct selection. Subjects were required in Experiment 2 to provide reasons for their selection or non-selection of each of the cards. This response procedure, paired with an explicated rule, led to further improvements in performance (over 50% correct selections). In Experiment 3, the influence of the type of selection instruction (true-false vs. violation) was examined. Paired with an explicated rule and the reasons response format, violation instructions led to one of the highest correct selection rates ever observed for any version of the selection task: over 80% correct. Because of the importance of this result, it was replicated twice. The results of these three experiments are discussed in terms of Johnson-Laird and Byrne's mental models theory and Evans's two-stage model of reasoning.     

Cognitive niches: an ecological model of strategy selection

How do people select among different strategies to accomplish a given task? Across disciplines, the strategy selection problem represents a major challenge. We propose a quantitative model that predicts how selection emerges through the interplay among strategies, cognitive capacities, and the environment. This interplay carves out for each strategy a cognitive niche, that is, a limited number of situations in which the strategy can be applied, simplifying strategy selection. To illustrate our proposal, we consider selection in the context of 2 theories: the simple heuristics framework and the ACT-R (adaptive control of thought-rational) architecture of cognition. From the heuristics framework, we adopt the thesis that people make decisions by selecting from a repertoire of simple decision strategies that exploit regularities in the environment and draw on cognitive capacities, such as memory and time perception. ACT-R provides a quantitative theory of how these capacities adapt to the environment. In 14 simulations and 10 experiments, we consider the choice between strategies that operate on the accessibility of memories and those that depend on elaborate knowledge about the world. Based on Internet statistics, our model quantitatively predicts people's familiarity with and knowledge of real-world objects, the distributional characteristics of the associated speed of memory retrieval, and the cognitive niches of classic decision strategies, including those of the fluency, recognition, integration, lexicographic, and sequential-sampling heuristics. In doing so, the model specifies when people will be able to apply different strategies and how accurate, fast, and effortless people's decisions will be.     

Inspection Times and the Selection Task: Are They Relevant?

Five experiments are reported based upon Evans' (1996) inspection time paradigm in which subjects are required to solve computer-presented Wason Selection Task problems while simultaneously using a mouse to indicate which card is currently under consideration. It had previously been found that selected cards were inspected for considerably longer than were non-selected cards, and this was taken as support for the existence of pre-conscious heuristics that direct attention towards relevant aspects of a problem. The first experiment reported here fully replicated this effect. However, by systematically varying the task format in subsequent experiments, the effect was found to diminish, disappear, or even reverse. The change in effect size and direction was not accompanied by any systematic variations in the subjects' card choices, indicating that the changes in taskformat had not altered the operation of the relevance-determining heuristics. On balance, it is suggested that the inspection time effect appears to be artefactual, and the inspection time paradigm therefore does not constitute satisfactory evidence for the existence of pre-conscious heuristics.     

A quantitative model of optimal data selection in Wason's selection task.

The optimal data selection model proposed by Oaksford and Chater (1994) successfully formalized Wason's selection task (Wason, 1966). The model, however, involved some questionable assumptions and was also not sufficient as a model of the task because it could not provide quantitative predictions of the card selection frequencies. In this paper, the model was revised to provide quantitative fits to the data. The model can predict the selection frequencies of cards based on a selection tendency function (STF), or conversely, it enables the estimation of subjective probabilities from data. Past experimental data were first re-analysed based on the model. In Experiment 1, the superiority of the revised model was shown. However, when the relationship between antecedent and consequent was forced to deviate from the biconditional form, the model was not supported. In Experiment 2, it was shown that sufficient emphasis on probabilistic information can affect participants' performance. A detailed experimental method to sort participants by probabilistic strategies was introduced. Here, the model was supported by a subgroup of participants who used the probabilistic strategy. Finally, the results were discussed from the viewpoint of adaptive rationality.     

The elusive pragmatic reasoning schemas effect

The present study examined the pragmatic reasoning schema theory of deductive reasoning—specifically, its explanation of performance on the selection task. Experiment 1 replicated a result crucial to the theory, the finding of facilitation on abstract versions of the selection task based on pragmatic reasoning schemas. However, Experiments 2, 3, and 4 established that this facilitation was dependent upon two presentation factors: (1) the presence of explicit negatives on the NOT P and NOT Q cards and (2) the inclusion of a checking context in the problem statement. These results are discussed in terms of Evans's two-stage (heuristic/analytic processing) model of reasoning.     

Working memory and syllogistic reasoning

The purpose of this study was to examine the relation between working memory span and syllogistic reasoning performance. In addition, performance for the reasoning task was compared to predictions made by mental model theory and the probability heuristics model. According to mental model theory, syllogisms that require the use of more mental models are more difficult. According to the probability heuristics model difficulty is related to the number of probabilistic heuristics that must be applied, or (for invalid syllogisms) inconsistencies between the derived and correct conclusion. The predictions of these theories were examined across two experiments. In general, people with larger working memory capacities reasoned better. Also, the responses made by people with larger capacities were more likely to correspond to the predictions made by both mental model theory and the probability heuristics model. Relations between working memory span and performance were also consistent with both theories.     

Automatic-heuristic and executive-analytic processing during reasoning: Chronometric and dual-task considerations

Human reasoning has been shown to overly rely on intuitive, heuristic processing instead of a more demanding analytic inference process. Four experiments tested the central claim of current dual-process theories that analytic operations involve time-consuming executive processing whereas the heuristic system would operate automatically. Participants solved conjunction fallacy problems and indicative and deontic selection tasks. Experiment 1 established that making correct analytic inferences demanded more processing time than did making heuristic inferences. Experiment 2 showed that burdening the executive resources with an attention-demanding secondary task decreased correct, analytic responding and boosted the rate of conjunction fallacies and indicative matching card selections. Results were replicated in Experiments 3 and 4 with a different secondary-task procedure. Involvement of executive resources for the deontic selection task was less clear. Findings validate basic processing assumptions of the dual-process framework and complete the correlational research programme of K. E. Stanovich and R. F. West (2000).     

A process model for water jug problems

We develop and evaluate a model for the water jug task in which a subject is required to find a sequence of moves (pouring operations) which produce a specified amount of water in each jug. Experiment 1 was designed to evaluate the meansends, move selection heuristics that are assumed by the model. Experiment 2 tested the model's predictions concerning those aspects of the water jug task that determine problem difficulty.A three stage process model incorporating GPS-like, means-ends heuristics and assumptions concerning the utilization of short- and long-term memory was able to account for differences across problems as well as details of the performance of subjects solving a given problem. We conclude that a GPS-like model that only selects one move at a time (no forward planning of move sequences or setting up of subgoals) can provide a good account of solution behavior in the water jug task.     

Does causal knowledge help us be faster and more frugal in our decisions?

One challenge that has to be addressed by the fast and frugal heuristics program is how people manage to select, from the abundance of cues that exist in the environment, those to rely on when making decisions. We hypothesize that causal knowledge helps people target particular cues and estimate their validities. This hypothesis was tested in three experiments. Results show that when causal information about some cues was available (Experiment 1), participants preferred to search for these cues first and to base their decisions on them. When allowed to learn cue validities in addition to causal information (Experiment 2), participants also became more frugal (i.e., they searched fewer of the available cues), made more accurate decisions, and were more precise in estimating cue validities than was a control group that did not receive causal information. These results can be attributed to the causal relation between the cues and the criterion, rather than to greater saliency of the causal cues (Experiment 3). Overall, our results support the hypothesis that causal knowledge aids in the learning of cue validities and is treated as a meta-cue for identifying highly valid cues.     

Knowing when you don't know enough: Children's judgements about ambiguous information

Two experiments were carried out to find out whether young children's failure to realise when messages are ambiguous is specific to verbally transmitted information about a person's intended meaning. In Experiment 1, children judged whether they had been told/shown enough to identify which one of a set of cards the experimenter had chosen. In one game the experimenter gave verbal messages about her chosen cards, and in a second game, she gave visual messages. With ambiguous visual messages, relevant parts of cards were physically covered. We expected that this would make it more obvious that the speaker's intended meaning was not being fully conveyed. No difference was found between verbal and visual conditions: correct judgements about message ambiguity occured with the same frequency in both.In Experiment 2, children judged in one game whether they had been told enough about the experimenter's chosen card, as in Experiment 1. In a second game, visual information about a card was not conveyed by the experimenter, Rather, the child operated a pointer, set in a disc with windows, beneath which lay cards. The child judged whether the window showed enough for him/her to tell which card the pointer indicated. Again, correct judgments about ambiguity occured with the same frequency in both games.The results implied that children's failure to realise when verbal messages are ambiguous is but one aspect of a more general failure to realise when one has insufficient information at one's disposal to guarantee a correct interpretation of what the world is like.     

Intuitive probabilistic inference in capuchin monkeys

The ability to reason about probabilities has ecological relevance for many species. Recent research has shown that both preverbal infants and non-human great apes can make predictions about single-item samples randomly drawn from populations by reasoning about proportions. To further explore the evolutionary origins of this ability, we conducted the first investigation of probabilistic inference in a monkey species (capuchins; Sapajus spp.). Across four experiments, capuchins (N = 19) were presented with two populations of food items that differed in their relative distribution of preferred and non-preferred items, such that one population was more likely to yield a preferred item. In each trial, capuchins had to select between hidden single-item samples randomly drawn from each population. In Experiment 1 each population was homogeneous so reasoning about proportions was not required; Experiments 2–3 replicated previous probabilistic reasoning research with infants and apes; and Experiment 4 was a novel condition untested in other species, providing an important extension to previous work. Results revealed that at least some capuchins were able to make probabilistic inferences via reasoning about proportions as opposed to simpler quantity heuristics. Performance was relatively poor in Experiment 4, so the possibility remains that capuchins may use quantity-based heuristics in some situations, though further work is required to confirm this. Interestingly, performance was not at ceiling in Experiment 1, which did not involve reasoning about proportions, but did involve sampling. This suggests that the sampling task posed demands in addition to reasoning about proportions, possibly related to inhibitory control, working memory, and/or knowledge of object permanence.     

An age-related dissociation between knowing rules and using them

Four experiments examined children's ability to use their knowledge to guide their behavior in a dimensional change (color-shape) card sort. In Experiment 1, 3- and 4-year-olds were told to sort cards first by one dimension (e.g., color: “Yellow ones go here; green ones go there”) and then by the other. The majority of 3-year-olds continued to use the preswitch rules on the postswitch phase, despite expressing knowledge of the postswitch rules by pointing to the appropriate location when asked about each rule. Experiment 2 found that this dissociation between knowledge and its use occurs even after a single preswitch trial. Experiments 3 and 4 demonstrated that the dissociation also occurs when verbal rather than manual responses are required. Together, the findings indicate that knowing rules is sometimes insufficient to permit their use. According to the cognitive complexity and control theory, the growth of reflection between 3 and 5 years of age underlies increases in control over thought and action by allowing children to integrate incompatible pairs of rules into a single rule system.     

Automatic–heuristic and executive–analytic processing during reasoning: Chronometric and dual-task considerations

Human reasoning has been shown to overly rely on intuitive, heuristic processing instead of a more demanding analytic inference process. Four experiments tested the central claim of current dual-process theories that analytic operations involve time-consuming executive processing whereas the heuristic system would operate automatically. Participants solved conjunction fallacy problems and indicative and deontic selection tasks. Experiment 1 established that making correct analytic inferences demanded more processing time than did making heuristic inferences. Experiment 2 showed that burdening the executive resources with an attention-demanding secondary task decreased correct, analytic responding and boosted the rate of conjunction fallacies and indicative matching card selections. Results were replicated in Experiments 3 and 4 with a different secondary-task procedure. Involvement of executive resources for the deontic selection task was less clear. Findings validate basic processing assumptions of the dual-process framework and complete the correlational research programme of K. E. Stanovich and R. F. West (2000).