Concept Discovery in a Scientific Domain

The scientific reasoning strategies used to discover a new concept in a scientific domain were investigated in two studies. An innovative task in which subjects discover new concepts in molecular biology was used. This task was based upon one set of experiments that Jacob and Monod used to discover how genes are controlled, and for which they were awarded the Nobel prize. In the two studies reported in this article, subjects were taught some basic facts and experimental techniques in molecular biology, using a simulated molecular genetics laboratory on a computer. Following their initial training, they were then asked to discover how genes are controlled by other genes. In Study 1, subjects found no evidence that was consistent with their initial hypothesis. Subjects then set one of two goals for conducting experiments and evaluating data. One goal was to search for evidence consistent with the current hypothesis (and they did not attend to the features of discrepant findings); none of the subjects who only had this goal succeeded at discovering how the genes were controlled. Other subjects in Study 1 used a different goal: Upon noticing evidence inconsistent with their current hypothesis, these subjects set a new goal of attempting to explain the cause of the discrepant findings. Using this goal, a subset of these subjects discovered the correct solution to the problem. Study 2 was conducted to test the hypothesis that subjects' goals of finding evidence consistent with their current hypothesis blocks consideration of alternate hypotheses and generation of new goals, it was predicted that if subjects could achieve their initial goal of discovering evidence consistent with their current hypothesis, they would then attend to particular features of discrepant evidence and solve the problem. To test this prediction, an additional mechanism of genetic control that was consistent with subjects' initial goal was added to the genes. Here, subjects had to discover two mechanisms of control: one mechanism consistent with their current hypothesis, and one inconsistent with their hypothesis. Twice as many subjects reached the correct solution in Study 2 than in Study 1. The findings of the two studies indicate that goals provide a powerful constraint on the cognitive processes underlying scientific reasoning and that the types of goals that are represented determine many of the reasoning errors that subjects make.     

Accommodating Surprise in Taxonomic Tasks: The Role of Expertise

This paper reports a psychological study of human categorization that looked at the procedures used by expert scientists when dealing with puzzling items. Five professional botanists were asked to specify a category from a set of positive and negative instances. The target category in the study was defined by a feature that was unusual, hence situations of uncertainty and puzzlement were generated. Subjects were asked to think aloud while solving the tasks, and their verbal reports were analyzed. A number of problem solving strategies were identified, and subsequently integrated in a model of knowledge‐guided inductive categorization. Our model proposes that expert knowledge influences the subjects' reasoning in more complex ways than suggested by earlier investigations of scientific reasoning. As in previous studies, domain knowledge influenced our subjects' hypothesis generation and testing; but, additionally, it played a central role when subjects revised their hypotheses.     

Learning Problem‐Solving Rules as Search Through a Hypothesis Space

Learning to solve a class of problems can be characterized as a search through a space of hypotheses about the rules for solving these problems. A series of four experiments studied how different learning conditions affected the search among hypotheses about the solution rule for a simple computational problem. Experiment 1 showed that a problem property such as computational difficulty of the rules biased the search process and so affected learning. Experiment 2 examined the impact of examples as instructional tools and found that their effectiveness was determined by whether they uniquely pointed to the correct rule. Experiment 3 compared verbal directions with examples and found that both could guide search. The final experiment tried to improve learning by using more explicit verbal directions or by adding scaffolding to the example. While both manipulations improved learning, learning still took the form of a search through a hypothesis space of possible rules. We describe a model that embodies two assumptions: (1) the instruction can bias the rules participants hypothesize rather than directly be encoded into a rule; (2) participants do not have memory for past wrong hypotheses and are likely to retry them. These assumptions are realized in a Markov model that fits all the data by estimating two sets of probabilities. First, the learning condition induced one set of Start probabilities of trying various rules. Second, should this first hypothesis prove wrong, the learning condition induced a second set of Choice probabilities of considering various rules. These findings broaden our understanding of effective instruction and provide implications for instructional design.     

Diagnostic reasoning with circumstantial evidence

Individuals often evaluate hypotheses about the cause of particular events on the basis of circumstantial evidence. This article describes a quantitative model of belief revision with circumstantial evidence. The model assumes that subjects revise belief on the basis of a cascaded reasoning process that combines beliefs about three premises—the association of a clue and a possible cause, and forward and backward implications of the clue—to revise belief in a causal hypothesis. Subjects in three experiments solved fictional murder mysteries, reporting on each trial a subset of the beliefs specified by the model. Experiment 1 demonstrates that the model provides a good account of the reasoning process, over several contexts in which clues are evaluated. Subjects appear to develop causal models that affect the interpretation of clues. Experiment 2 provides further evidence on the development of causal models and compares the present model with a model based on Bayes' theorem. Experiment 3 is a control experiment which demonstrates that the belief assessments used in Experiments 1 and 2 do not alter the process of belief revision. Tests of subjects' memory for suspect-clue associations provide further support for the hypothesis that subjects develop causal models of the true cause and demonstrate the contrast between reasoning with currently available information and retrospective access to the facts on which belief revision is based. The present view is compared with other theories of causal thinking and belief revision.     

Use of current explanations in multicausal abductive reasoning

In multicausal abductive tasks a person must explain some findings by assembling a composite hypothesis that consists of one or more elementary hypotheses. If there are n elementary hypotheses, there can be up to 2ⁿ composite hypotheses. To constrain the search for hypotheses to explain a new observation, people sometimes use their current explanation—the previous evidence and their present composite hypothesis of that evidence; however, it is unclear when and how the current explanation is used. In addition, although a person's current explanation can narrow the search for a hypothesis, it can also blind the problem solver to alternative, possibly better, explanations. This paper describes a model of multicausal abductive reasoning that makes two predictions regarding the use of the current explanation. The first prediction is that the current explanation is not used to explain new evidence if there is a simple (i.e., nondisjunctive, concrete) hypothesis to account for that evidence. The second prediction is that the current explanation is used when attempting to discriminate among several alternative hypotheses for new evidence. These hypotheses were tested in three experiments. The results are consistent with the second prediction: the current explanation is used when discriminating among alternative hypotheses. However, the first prediction—that the current explanation is not used when a simple hypothesis can account for new data—received only limited support. Participants used the current explanation to constrain their interpretation of new data in 46.5% of all trials. This suggests that context-independent strategies compete with context-dependent ones—an interpretation that is consistent with recent work on strategy selection during problem solving.     

Conflict between incompetences and influence of a low-expertise source in hypothesis testing

In a 2–4–4–like reasoning task, 69 subjects tested hypotheses following exposure to a low-expertise source proposing an alternative hypothesis. Subjects compared self- and source's competence either independently or interdependently. Results show that interdependence leads subjects to assert self-validity and the source's invalidity, and to test hypotheses through confirmation. Independence produces a conflict between incompetences, i.e. doubt concerning self- and source's validity, leading to disconfirmatory testing.     

Sources of difficulty in deductive reasoning: The THOG task

The THOG task presents four designs constructed from two shapes and two colours. Subjects are told that the experimenter has written down one of the shapes and one of the colours and are provided the rule that if, and only if, any design has either the shape or the colour, but not both, written down, then it is a THOG. Finally, they are given an exemplar and are asked to classify the remaining designs. Successful solution requires construction of hypotheses, reasoning under each hypothesis, and comparison of the results under each to reach a final conclusion. Few subjects are able to provide adequate responses on the standard version of the task. We present the results of four experiments, with 160 undergraduates each presented with one of eight versions of the task. Experiments 1 and 2 indicate that (1) some subjects think that the properties that are written down are identical to those of the exemplar, although these are not the same subjects who exhibit the modal error pattern, (2) many subjects correctly understand the disjunction of the rule but fail to consider the hypotheses, and (3) poor initial encoding of the problem is not easily corrected. Experiment 3 investigates the sufficiency of the claim of Griggs and Newstead (1982) that appropriate problem solution follows from explicit presentation of all problem information (including use of positive labels for properties that are not written down), and Experiment 4 investigates the necessity of the claim. The results of Experiments 3 and 4 show that presenting positive category labels does increase the frequency of correct solution; however, positive category labels are not necessary for such improvement. Separation of the labels of the THOG rule from those of the exemplar, or informing subjects that only one other design is a THOG, also increases the frequency of successful solution. The results suggest that many people have some fairly sophisticated reasoning skills. but application of these skills is easily discouraged when the features of the task lead to poor initial encoding.     

Heuristics and biases in diagnostic reasoning: II. Congruence,information, and certainty

Six experiments were carried out to examine possible heuristics and biases in the evaluation of yes-or-no questions for the purpose of hypothesis testing. In some experiments, the prior probability of the hypotheses and the conditional probabilities of the answers given each hypothesis were elicited from the subjects; in other experiments, they were provided. We found the following biases (systematic departures from a normative model), and interviews and justifications suggested that each was the result of a corresponding heuristic: Congruence bias. Subjects overvalued questions that have a high probability of a positive result given the most likely hypothesis. This bias was apparently reduced when alternative hypotheses or probabilities of negative results are explicitly stated. Information bias. Subjects evaluated questions as worth asking even when there is no answer that can change the hypothesis that will be accepted as a basis for action. Certainty bias. Subjects overvalued questions that have the potential to establish, or rule out, one or more hypotheses with 100% probability. These heuristics are explained in terms of the idea that people fail to consider certain arguments against the use of questions that seem initially worth asking, specifically, that a question may not distinguish likely hypotheses or that no answer can change the hypothesis accepted as a basis for action.     

Relevance-driven information search in "pseudodiagnostic" reasoning

When faced with two competing hypotheses, people sometimes prefer to look at multiple sources of information in support of one hypothesis rather than to establish the diagnostic value of a single piece of information for the two hypotheses. This is termed pseudodiagnostic reasoning and has often been understood to reflect, among other things, poor information search strategies. Past research suggests that diagnostic reasoning may be more easily fostered when participants seek data to help in the selection of one of two competing courses of action as opposed to situations where they seek data to help infer which of two competing hypotheses is true. In the experiment reported here, we provide the first empirical evidence demonstrating that manipulating the relevance of the feature for which participants initially receive information determines whether they will make a nominally diagnostic or pseudodiagnostic selection. The discussion of these findings focuses on implications for the ability to engage in diagnostic hypothesis testing.     

Relevance-driven information search in “pseudodiagnostic” reasoning

When faced with two competing hypotheses, people sometimes prefer to look at multiple sources of information in support of one hypothesis rather than to establish the diagnostic value of a single piece of information for the two hypotheses. This is termed pseudodiagnostic reasoning and has often been understood to reflect, among other things, poor information search strategies. Past research suggests that diagnostic reasoning may be more easily fostered when participants seek data to help in the selection of one of two competing courses of action as opposed to situations where they seek data to help infer which of two competing hypotheses is true. In the experiment reported here, we provide the first empirical evidence demonstrating that manipulating the relevance of the feature for which participants initially receive information determines whether they will make a nominally diagnostic or pseudodiagnostic selection. The discussion of these findings focuses on implications for the ability to engage in diagnostic hypothesis testing.     

Error, falsification and scientific inference: An experimental investigation

This paper reports the results of four experiments designed to test the methodological falsificationist's assumption that replication is sufficient to prevent the possibility of error from being used to immunize hypotheses against disconfirmation. The first three experiments compare the performance of subjects on tasks that simulate scientific reasoning under two conditions: (1) where there is a 0-20% possibility of error in experimental results, but no actual error; and (2) a control condition.

All experiments used Wason's 2-4-6 task, in which subjects propose triples and are told whether each corresponds to a rule. In Experiment 1, subjects in the possible-error condition proposed significantly more triples than control subjects. Experiment 2 added colour and letter dimensions to the 2-4-6 task; possible-error subjects proposed significantly more triples and replicated the same triple more often than control subjects. Experiment 3 made replication more difficult by limiting the number of experiments subjects could perform and by altering the rule to make the results of the current trial dependent on previous ones. Control subjects solved this problem significantly more often than possible-error subjects.

Experiment 4 was run in a manner very similar to Experiment 1, except that an actual 20% error condition was added. Subjects in this condition solved the rule significantly less often than subjects in other conditions, and also took more time and replicated more often. Implications of these results for the methodological falsificationist's position are discussed.     

Reasoning about the referent of a picture versus reasoning about the picture as the referent: An effect of visual realism

Research on picture perception and picture-based problem solving has generally considered the information that enables one to “see” and think about a picture’s subject matter. However, people often reason about a picture or representation as the referent itself. The question addressed here is whether pictorial features themselves help determine when one reasons about the referent of an image, as with an engrossing movie, and when one reasons about the image in its own right, as with abstract art. Two experiments tested the hypothesis that pictures with relatively high fidelity to their referents lead people to think about those referents, whereas pictures with relatively low fidelity lead people to think about the picture as a referent. Subjects determined whether marks on the bottom and top boards of an open hinge would meet if the hinge were closed. Accuracy and latency results indicated that subjects who saw realistic displays simulated the physical behavior of the hinge through analog imagery. In contrast, subjects who saw schematic displays tended to reason about static features of the display such as line lengths and angles. The results demonstrate that researchers must be cautious when generalizing from reasoning about diagrammatic materials to reasoning about the referents themselves.     

Do images involuntarily trigger search? A test of Pillsbury's hypothesis

Pillsbury (1908) suggested that deciding to search for something in a scene consists of nothing more than forming a visual image of the target. If so, imaging should trigger search even when it would be more advantageous not to search. Subjects were cued to form an image of a specified object (e.g., tiger) and to press a key when they had done so. This initiated the presentation of a sequence of pictures, with a single target digit interspersed; the subject's task was to report this digit. The sequence contained a picture of the same type of object that the subject had just imaged (e.g., a tiger), either before or after the target digit. If this picture was detected involuntarily, an attentional blink should have impaired digit detection when the picture preceded the digit. This was confirmed in two experiments, even when instructions specifically encouraged subjects to discard the image and to avoid searching for it. The results support Pillsbury's hypothesis.     

Epistemic and Dialectical Models of Begging the Question

This paper addresses the problem posed by the current split between the two opposed hypotheses in the growing literature on the fallacy of begging the question the epistemic hypothesis, based on knowledge and belief, and the dialectical one, based on formal dialogue systems. In the first section, the nature of split is explained, and it is shown how each hypothesis has developed. To get the beginning reader up to speed in the literature, a number of key problematic examples are analyzed illustrating how both approaches can be applied. Useful tools are brought to bear on them, including the automated argument diagramming system Araucaria, and profiles of dialogue used to represent circular argumentation in a dialogue tableau format. These tools are used to both to model circular reasoning and to provide the contextual evidence needed to properly determine whether the circular reasoning in a given case is better judged fallacious or not. A number of technical problems that have impeded the development of both hypotheses are studied. One central problem is the distinction between argument and explanation. It is concluded that the best way to move forward and solve these problems is to reformulate the two hypotheses in such a way that they might be able to co-exist. On this basis, a unified methodology is proposed that allows each hypothesis to move forward as a legitimate avenue for research using the same tools.     

Scientific reasoning in early and middle childhood: The development of domain‐general evidence evaluation,experimentation, and hypothesis generation skills

According to Klahr's (2000, 2005; Klahr & Dunbar, 1988) Scientific Discovery as Dual Search model, inquiry processes require three cognitive components: hypothesis generation, experimentation, and evidence evaluation. The aim of the present study was to investigate (a) when the ability to evaluate perfect covariation, imperfect covariation, and non‐covariation evidence emerges, (b) when experimentation emerges, (c) when hypothesis generation skills emerge, and (d), whether these abilities develop synchronously during childhood. We administered three scientific reasoning tasks referring to the three components to 223 children of five age groups (from age 4.0 to 13.5 years). Our results show that the three cognitive components of domain‐general scientific reasoning emerge asynchronously. The development of domain‐general scientific reasoning begins with the ability to handle unambiguous data, progresses to the interpretation of ambiguous data, and leads to a flexible adaptation of hypotheses according to the sufficiency of evidence. When children understand the relation between the level of ambiguity of evidence and the level of confidence in hypotheses, the ability to differentiate conclusive from inconclusive experiments accompanies this development. Implications of these results for designing science education concepts for young children are briefly discussed.     

Evaluation of evidence for sufficiency, for necessity, and for necessity-and-sufficiency

We report two experiments with 120 undergraduate subjects. The tasks presented clearly articulated hypotheses concerning necessity, sufficiency, and necessity-and-sufficiency, together with possible combinations of treatment conditions, and required subjects to judge whether, according to the hypotheses, the relevant outcomes will occur, will not occur, or might occur. The patterns of responses reveal how subjects understand the extensions of the concepts, i.e. what each hypothesis permits, requires, and excludes. Only necessity-and-sufficiency hypotheses generally led to logically adequate responses, and the most typical error for both necessity hypotheses and sufficiency hypotheses was to treat them as extensionally equivalent to necessity-and-sufficiency. This error tendency was more evident for necessity than for sufficiency hypotheses. Further, although responses to both necessity hypotheses and sufficiency hypotheses were affected by the complexity of the hypothesized conditions, responses to necessity-and-sufficiency hypotheses were not. We argue that the results are not a function of a response bias but reflect differences in the way the hypotheses are understood. Implications for the causal attribution literature are discussed.     

Collaborative Discovery in a Scientific Domain

This study compares Pairs of subjects with Single subjects in a task of discovering scientific laws with the aid of experiments. Subjects solved a molecular genetics task in a computer micro-world (Dunbar, 1993). Pairs were more successful in discovery than Singles and participated more actively in explanatory activities (i.e., entertaining hypotheses and considering alternative ideas and justifications). Explanatory activities were effective for discovery only when the subjects also conducted crucial experiments. Explanatory activities were facilitated when paired subjects made requests of each other for explanation and focused on them. The study extends from individual to collaborative discovery activities the importance to the discovery process of setting goals to find hypotheses and evidence (Dunbar, 1993) and to construct explanations of phenomena and processes encountered in examples (Chi, Bassok, Lewis, & Glaser, 1989).     

Diagnostic hypothesis generation and human judgment

Diagnostic hypothesis-generation processes are ubiquitous in human reasoning. For example, clinicians generate disease hypotheses to explain symptoms and help guide treatment, auditors generate hypotheses for identifying sources of accounting errors, and laypeople generate hypotheses to explain patterns of information (i.e., data) in the environment. The authors introduce a general model of human judgment aimed at describing how people generate hypotheses from memory and how these hypotheses serve as the basis of probability judgment and hypothesis testing. In 3 simulation studies, the authors illustrate the properties of the model, as well as its applicability to explaining several common findings in judgment and decision making, including how errors and biases in hypothesis generation can cascade into errors and biases in judgment.     

The Science to Save Us from Philosophy of Science

Are knowledge and belief pivotal in science, as contemporary epistemology and philosophy of science nearly universally take them to be? I defend the view that scientists are not primarily concerned with knowing and that the methods of arriving at scientific hypotheses, models and scenarios do not commit us having stable beliefs about them. Instead, what drives scientific discovery is ignorance that scientists can cleverly exploit. Not an absence or negation of knowledge, ignorance concerns fundamental uncertainty, and is brought out by retroductive (abductive) inferences, which are roughly characterised as reasoning from effects to causes. I argue that recent discoveries in sciences that coped with under-structured problem spaces testify the prevalence of retroductive logic in scientific discovery and its progress. This puts paid to the need of finding epistemic justification or confirmation to retroductive methodologies. A scientist, never frightened of unknown unknowns, strives to advance the forefront of uncertainty, not that of belief or knowledge. Far from rendering science irrational, I conclude that catering well for the right conditions in which to cultivate ignorance is a key to how fertile retroductive inferences (true guesses) arise.