Watching diagnoses develop: Eye movements reveal symptom processing during diagnostic reasoning

Finding a probable explanation for observed symptoms is a highly complex task that draws on information retrieval from memory. Recent research suggests that observed symptoms are interpreted in a way that maximizes coherence for a single likely explanation. This becomes particularly clear if symptom sequences support more than one explanation. However, there are no existing process data available that allow coherence maximization to be traced in ambiguous diagnostic situations, where critical information has to be retrieved from memory. In this experiment, we applied memory indexing, an eye-tracking method that affords rich time-course information concerning memory-based cognitive processing during higher order thinking, to reveal symptom processing and the preferred interpretation of symptom sequences. Participants first learned information about causes and symptoms presented in spatial frames. Gaze allocation to emptied spatial frames during symptom processing and during the diagnostic response reflected the subjective status of hypotheses held in memory and the preferred interpretation of ambiguous symptoms. Memory indexing traced how the diagnostic decision developed and revealed instances of hypothesis change and biases in symptom processing. Memory indexing thus provided direct online evidence for coherence maximization in processing ambiguous information.     

Memory activation of multiple hypotheses in sequential diagnostic reasoning

In sequential diagnostic reasoning, observed evidence activates hypotheses about possible causes in memory. These memory activations have been previously examined with a probe reaction task for problems with a single correct diagnosis. We applied this process tracing method to ambiguous problems with multiple compatible hypotheses. When participants reasoned about the causes of ambiguous symptom sequences, they were prompted to respond to probes representing hypotheses. The response time to a probe was shorter if the current support for the respective hypothesis was stronger indicating that the processing of compatible hypotheses can be traced. For sequences with two equally supported hypotheses, the initial hypothesis was more often chosen as the final diagnosis (a primacy effect). Probe reaction times suggest that the initial hypothesis has been activated more strongly already early, when it was finally chosen as the diagnosis. Nevertheless, substantial variance in response times limits the task's applicability for process tracing.     

Data acquisition dynamics and hypothesis generation

When formulating explanations for the events we witness in the world, temporal dynamics govern the hypotheses we generate. In our view, temporal dynamics influence beliefs over three stages: data acquisition, hypothesis generation, and hypothesis maintenance and updating. This paper presents experimental and computational evidence for the influence of temporal dynamics on hypothesis generation through dynamic working memory processes during data acquisition in a medical diagnosis task. We show that increasing the presentation rate of a sequence of symptoms leads to a primacy effect, which is predicted by the dynamic competition in working memory that dictates the weights allocated to individual data in the generation process. Individual differences observed in hypothesis generation are explained by differences in working memory capacity. Finally, in a simulation experiment we show that activation dynamics at data acquisition also accounts for results from a related task previously held to support capacity-unlimited diagnostic reasoning.     

Working memory dynamics bias the generation of beliefs: The influence of data presentation rate on hypothesis generation

Although temporal dynamics are inherent aspects of diagnostic tasks, few studies have investigated how various aspects of time course influence hypothesis generation. An experiment is reported that demonstrates that working memory dynamics operating during serial data acquisition bias hypothesis generation. The presentation rate (and order) of a sequence of serially presented symptoms was manipulated to be either fast (180 ms per symptom) or slow (1,500 ms per symptom) in a simulated medical diagnosis task. When the presentation rate was slow, participants chose the disease hypothesis consistent with the symptoms appearing later in the sequence. When the presentation rate was fast, however, participants chose the disease hypothesis consistent with the symptoms appearing earlier in the sequence, therefore representing a novel primacy effect. We predicted and account for this effect through competitive working memory dynamics governing information acquisition and the contribution of maintained information to the retrieval of hypotheses from long-term memory.     

Diagnostic hypothesis generation and human judgment

Diagnostic hypothesis-generation processes are ubiquitous in human reasoning. For example, clinicians generate disease hypotheses to explain symptoms and help guide treatment, auditors generate hypotheses for identifying sources of accounting errors, and laypeople generate hypotheses to explain patterns of information (i.e., data) in the environment. The authors introduce a general model of human judgment aimed at describing how people generate hypotheses from memory and how these hypotheses serve as the basis of probability judgment and hypothesis testing. In 3 simulation studies, the authors illustrate the properties of the model, as well as its applicability to explaining several common findings in judgment and decision making, including how errors and biases in hypothesis generation can cascade into errors and biases in judgment.     

Dual Space Search During Scientific Reasoning

The purpose of the two studies reported here was to develop an integrated model of the scientific reasoning process. Subjects were placed in a simulated scientific discovery context by first teaching them how to use an electronic device and then asking them to discover how a hitherto unencountered function worked. To do this task, subjects had to formulate hypotheses based on their prior knowledge, conduct experiments, and evaluate the results of their experiments. In the first study, using 20 adult subjects, we identified two main strategies that subjects used to generate new hypotheses. One strategy was to search memory and the other was to generalize from the results of previous experiments. We described the former group as searching an hypothesis space, and the latter as searching an experiment space. In a second study, with 10 adults, we investigated how subjects search the hypothesis space by instructing them to state all the hypotheses that they could think of prior to conducting any experiments. Following this phase, subjects were then allowed to conduct experiments. Subjects who could not think of the correct rule in the hypothesis generation phase discovered the correct rule only by generalizing from the results of experiments in the experimental phase.
Both studies provide support for the view that scientific reasoning can be characterized as search in two problem spaces. By extending Simon and Lea's (1974) Generalized Rule Inducer, we present a general model of Scientific Discovery as Dual Search (SDDS) that shows how search in two problem spaces (an hypothesis space and an experiment space) shapes hypothesis generation, experimental design, and the evaluation of hypotheses. The model also shows how these processes interact with each other. Finally, we interpret earlier findings about the psychology of scientific reasoning in terms of the SDDS model.     

Hypothetico-nomological aspects of medical diagnosis Part I: General structure of the diagnostic process and its hypothesis-directed stage

In medical diagnostic examination three main stages may be distinguished: (a) initial exploration, (b) hypothesis-directed investigation, and (c) final diagnosis making. The purpose of this work is to study some methodological problems concerning the second of the above stages of the diagnosis and to prepare a background for a mathematical model [30] of this process. In diagnostic problem solving, the reasoning proceeds along the main lines traced by some initial suggestions and passes through various intermediate elements which are connected with one another forming ramifying chains and nets of inferences and hypotheses. Such a complex mental construction is based on laws which form medical knowledge and reflect various regularities and relations, causal, structural, functional, and others. The main components of diagnostic reasoning may be divided into several classes according to their function and content: leading hypotheses, working hypotheses, main diagnostic hypotheses, statements accepted as certain, intermediary and reserve hypotheses, therapeutic suggestions of immediate consequence. In an example of diagnostic problem solving these types of propositions are defined and analysed. In diagnostic reasoning, as in every other process of rational problem solving, explanation of the observed symptoms and signs and testing of the explaining hypotheses play a predominant role. These procedures form successive, frequently numerous and diversified steps and stages of the reasoning, leading to the construction of a mental model of the patient's state. Some problems relative to the scheme of explanation, especially to that which is based on causal laws, are discussed.     

Processing Illness Information: The Role of Disease Prototypes

Recently Bishop and Converse (1986) proposed that information about physical symptoms is interpreted by relating those symptoms to preexisting disease prototypes. The two present studies further examined this formulation by testing hypotheses concerning the speed of processing symptom information as well as associations made to sets of physical symptoms as a function of prototypicality. As predicted, Experiment 1 showed that response time to highly prototypical symptom sets was significantly shorter than that for symptom sets containing irrelevant symptoms. Also as predicted, the results of Experiment 2 showed significant differences in the associations made by experiment participants to symptom sets as a function of the prototypicality of the symptoms in those sets. Participants made more category-based associations to highly prototypical symptom sets than to those lower in prototypicality but made more associations to individual symptoms for symptom sets low in prototypicality. Implications for the prototype hypothesis and for understanding the processing of illness information are discussed.     

Relevance-driven information search in “pseudodiagnostic” reasoning

When faced with two competing hypotheses, people sometimes prefer to look at multiple sources of information in support of one hypothesis rather than to establish the diagnostic value of a single piece of information for the two hypotheses. This is termed pseudodiagnostic reasoning and has often been understood to reflect, among other things, poor information search strategies. Past research suggests that diagnostic reasoning may be more easily fostered when participants seek data to help in the selection of one of two competing courses of action as opposed to situations where they seek data to help infer which of two competing hypotheses is true. In the experiment reported here, we provide the first empirical evidence demonstrating that manipulating the relevance of the feature for which participants initially receive information determines whether they will make a nominally diagnostic or pseudodiagnostic selection. The discussion of these findings focuses on implications for the ability to engage in diagnostic hypothesis testing.     

Relevance-driven information search in "pseudodiagnostic" reasoning

When faced with two competing hypotheses, people sometimes prefer to look at multiple sources of information in support of one hypothesis rather than to establish the diagnostic value of a single piece of information for the two hypotheses. This is termed pseudodiagnostic reasoning and has often been understood to reflect, among other things, poor information search strategies. Past research suggests that diagnostic reasoning may be more easily fostered when participants seek data to help in the selection of one of two competing courses of action as opposed to situations where they seek data to help infer which of two competing hypotheses is true. In the experiment reported here, we provide the first empirical evidence demonstrating that manipulating the relevance of the feature for which participants initially receive information determines whether they will make a nominally diagnostic or pseudodiagnostic selection. The discussion of these findings focuses on implications for the ability to engage in diagnostic hypothesis testing.     

Decision making and professional experience: an experimental study of generating, evaluating and modifying diagnostic hypotheses

Experimental investigations on generation, evaluation, and modification of diagnostic hypotheses in symptom pattern classification revealed in different domains (e.g. clinical decision making) that reasoning in non-experts and experts differ in the 'content' (e.g. issue-directed substance of concepts) rather than in the 'form' of reasoning (e.g. number, specificity of hypotheses). These results are essentially accounted for by the experts' upper-level flexibility in the interpretation of data. In a two-factorial design (competence x predictive value of data), patients' clinical data with varying predictive value were given singularly in succession to two groups (experts and novices; n = 20) of subjects. The task of the subjects was (a) to name their assumptions and (b) give a summarizing decision on the--probable--diagnostic category of hypothetical cases. Results: Between experts and novices no difference was found in the number and degree of specification of hypotheses; in summarizing decisions, experts considered more categories as plausible than novices; on patient data with low predictive value, experts considered more specific categories. The moment of hypothesis formation depends not on the competence level but on the predictive value of the patient's clinical data.     

The Influence of Activation Level on Belief Bias in Relational Reasoning

A novel explanation of belief bias in relational reasoning is presented based on the role of working memory and retrieval in deductive reasoning, and the influence of prior knowledge on this process. It is proposed that belief bias is caused by the believability of a conclusion in working memory which influences its activation level, determining its likelihood of retrieval and therefore its effect on the reasoning process. This theory explores two main influences of belief on the activation levels of these conclusions. First, believable conclusions have higher activation levels and so are more likely to be recalled during the evaluation of reasoning problems than unbelievable conclusions, and therefore, they have a greater influence on the reasoning process. Secondly, prior beliefs about the conclusion have a base level of activation and may be retrieved when logically irrelevant, influencing the evaluation of the problem. The theory of activation and memory is derived from the Atomic Components of Thought‐Rational (ACT‐R) cognitive architecture and so this account is formalized in an ACT‐R cognitive model. Two experiments were conducted to test predictions of this model. Experiment 1 tested strength of belief and Experiment 2 tested the impact of a concurrent working memory load. Both of these manipulations increased the main effect of belief overall and in particular raised belief‐based responding in indeterminately invalid problems. These effects support the idea that the activation level of conclusions formed during reasoning influences belief bias. This theory adds to current explanations of belief bias by providing a detailed specification of the role of working memory and how it is influenced by prior knowledge.     

Strategies of clinical hypothesis testing

This research examines whether Psychology students, when they test clinical hypotheses, follow either confirmatory or disconfirmatory reasoning strategies. Two hundred and six psychology students, divided in four groups, participated. One group received information about the probability that the hypothesis was correct by means of verbal labels, and another group, by means of numerical expressions. An additional group received the information that getting a precise diagnosis was clinically important. In a last group, diagnostic tests allowed them to increase certainty about the hypothesis. Results show a partial use of confirmatory strategies because, although participants did not seek confirming information, they indeed avoided collecting disconfirming information. When the information increased certainty about the hypothesis, confirmatory strategies became more likely. Neither the increase in the task importance nor the numerical expression of the likelihood that the hypothesis was correct seemed to affect the testing strategy used.     

Memory activation and the availability of explanations in sequential diagnostic reasoning

In the field of diagnostic reasoning, it has been argued that memory activation can provide the reasoner with a subset of possible explanations from memory that are highly adaptive for the task at hand. However, few studies have experimentally tested this assumption. Even less empirical and theoretical work has investigated how newly incoming observations affect the availability of explanations in memory over time. In this article we present the results of 2 experiments in which we address these questions. While participants diagnosed sequentially presented medical symptoms, the availability of potential explanations in memory was measured with an implicit probe reaction time task. The results of the experiments were used to test 4 quantitative cognitive models. The models share the general assumption that observations can activate and inhibit explanations in memory. They vary with respect to how newly incoming observations affect the availability of explanations over time. The data of both experiments were predicted best by a model in which all observations in working memory have the same potential to activate explanations from long-term memory and in which these observations do not decay. The results illustrate the power of memory activation processes and show where additional deliberate reasoning strategies might come into play.     

Diagnostic reasoning with circumstantial evidence

Individuals often evaluate hypotheses about the cause of particular events on the basis of circumstantial evidence. This article describes a quantitative model of belief revision with circumstantial evidence. The model assumes that subjects revise belief on the basis of a cascaded reasoning process that combines beliefs about three premises—the association of a clue and a possible cause, and forward and backward implications of the clue—to revise belief in a causal hypothesis. Subjects in three experiments solved fictional murder mysteries, reporting on each trial a subset of the beliefs specified by the model. Experiment 1 demonstrates that the model provides a good account of the reasoning process, over several contexts in which clues are evaluated. Subjects appear to develop causal models that affect the interpretation of clues. Experiment 2 provides further evidence on the development of causal models and compares the present model with a model based on Bayes' theorem. Experiment 3 is a control experiment which demonstrates that the belief assessments used in Experiments 1 and 2 do not alter the process of belief revision. Tests of subjects' memory for suspect-clue associations provide further support for the hypothesis that subjects develop causal models of the true cause and demonstrate the contrast between reasoning with currently available information and retrospective access to the facts on which belief revision is based. The present view is compared with other theories of causal thinking and belief revision.     

Effects of explaining on children's preference for simpler hypotheses

Research suggests that the process of explaining influences causal reasoning by prompting learners to favor hypotheses that offer “good” explanations. One feature of a good explanation is its simplicity. Here, we investigate whether prompting children to generate explanations for observed effects increases the extent to which they favor causal hypotheses that offer simpler explanations, and whether this changes over the course of development. Children aged 4, 5, and 6 years observed several outcomes that could be explained by appeal to a common cause (the simple hypothesis) or two independent causes (the complex hypothesis). We varied whether children were prompted to explain each observation or, in a control condition, to report it. Children were then asked to make additional inferences for which the competing hypotheses generated different predictions. The results revealed developmental differences in the extent to which children favored simpler hypotheses as a basis for further inference in this task: 4-year-olds did not favor the simpler hypothesis in either condition; 5-year-olds favored the simpler hypothesis only when prompted to explain; and 6-year-olds favored the simpler hypothesis whether or not they explained.     

Hypothesizing about signaling networks

The current knowledge about signaling networks is largely incomplete. Thus biologists constantly need to revise or extend existing knowledge. The revision and/or extension is first formulated as theoretical hypotheses, then verified experimentally. Many computer-aided systems have been developed to assist biologists in undertaking this challenge. The majority of the systems help in finding “patterns” in data and leave the reasoning to biologists. A few systems have tried to automate the reasoning process of hypothesis formation. These systems generate hypotheses from a knowledge base and given observations. A main drawback of these knowledge-based systems is the knowledge representation formalism they use. These formalisms are mostly monotonic and are now known to be not quite suitable for knowledge representation, especially in dealing with the inherently incomplete knowledge about signaling networks. We propose an action language based framework for hypothesis formation for signaling networks. We show that the hypothesis formation problem can be translated into an abduction problem. This translation facilitates the complexity analysis and an efficient implementation of our system. We illustrate the applicability of our system with an example of hypothesis formation in the signaling network of the p53 protein.     

The Effects of Experimentally Induced Rumination Versus Distraction on Analogue Posttraumatic Stress Symptoms

Rumination has been suggested to be an important factor maintaining posttraumatic stress disorder (PTSD). Using an analogue design, this study aimed to experimentally test the hypothesis that trauma-related rumination maintains PTSD symptoms. Fifty-one participants were first asked to give a detailed narrative of a negative life event and were then randomly assigned to a rumination or distraction condition. In line with the hypotheses, rumination about the event resulted in the maintenance of negative mood and intrusive memories immediately after the manipulation whereas distraction resulted in symptom reduction. However, this effect was reversed during a subsequent symptom provocation task, in which distraction led to a greater increase in some of the symptoms than rumination. Results are in line with the idea that rumination is involved in the maintenance of PTSD but may suggest a complex relationship between rumination and posttraumatic stress symptoms.     

From symptoms to causes: Diversity effects in diagnostic reasoning

A single causal agent can often give rise to a cascade of consequences that can be envisioned as a branching pathway in which symptoms are the terminal nodes. In three studies, we investigated whether reasoning about root causes on the basis of such symptoms would conform to a diversity effect analogous to that found in inductive reasoning about properties of hierarchically organized categories. A strong diversity effect was found both for reasoning about medical diseases that drew on existing background knowledge and for reasoning that did not. Specifically, the presence of a root cause was more likely to be induced when the symptoms present were further apart in the branching structure.     

Working memory limitation as a source of confusion in the abstract THOG task

Limitations of working memory are proposed as a major determinant of problem difficulty in the THOG task. This task is a logical reasoning task which uses an exclusive disjunction and requires hypothetico-deductive reasoning. Four experiments with students of mathematics or psychology were used to test the hypotheses that, first, guiding participants' attention facilitates the task and, second, the use of paper and pencil as external problem representation reliefs working memory load. Focusing participants' attention upon a critical aspect of the task does not improve solution rates. Students of mathematics were better than students of psychology, but only if they were allowed to use paper and pencil or to work on the task repeatedly. These results partially support the working memory hypothesis. They point toward the importance of training and practice in relatively simple meta-cognitive skills in logical reasoning. Received: 20 March 2000 / Accepted: 22 January 2001