Cochrane Review as a “Warranting Device” for Reasoning About Health

Contemporary reasoning about health is infused with the work products of experts, and expert reasoning about health itself is an active site for invention and design. Building on Toulmin’s largely undeveloped ideas on field-dependence, we argue that expert fields can develop new inference rules that, together with the backing they require, become accepted ways of drawing and defending conclusions. The new inference rules themselves function as warrants, and we introduce the term “warranting device” to refer to an assembly of the rule plus whatever material, procedural, and institutional resources are required to assure its dependability. We present a case study on the Cochrane Review, a new method for synthesizing evidence across large numbers of scientific studies. After reviewing the evolution and current structure of the device, we discuss the distinctive kinds of critical questions that may be raised around Cochrane Reviews, both within the expert field and beyond. Although Toulmin’s theory of field-dependence is often criticized for its relativism, we find that, as a matter of practical fact, field-specific warrants do not enjoy immunity from external critique. On the contrary, they can be opened to evaluation and critique from any interested perspective.     

When Experts Argue: Explaining the Best and the Worst of Reasoning

Expert reasoning is responsible for some of the most stunning human achievements, but also for some of the most disastrous decisions ever made. The argumentative theory of reasoning has proven very effective at explaining the pattern of reasoning’s successes and failures. In the present article, it is expanded to account for expert reasoning. The argumentative theory predicts that reasoning should display a strong confirmation bias. If argument quality is not sufficiently high in a domain, the confirmation bias will make experts tap into their vast knowledge to defend whatever opinion they hold, with polarization and overconfidence as expected results. By contrast, experts should benefit even more from the power of group discussion to make the best of the confirmation bias—when they genuinely disagree that is, otherwise polarization is again likely to ensue. When experts interact with laymen other mechanisms can take the lead, in particular trust calibration and consistency checking. They can yield poor outcomes if experts do not have a sustained interaction with laymen, or if the laymen have strong opinions when they witness a debate between experts. Seeing reasoning as a mechanism of epistemic vigilance aimed at finding and evaluating arguments helps make better sense of expert reasoning performance, be it in individual ratiocination, in debates with other experts, or in interactions with laymen.     

知识图式对贝叶斯推理的影响

选取具有概率知识的大学生(专家)和没有概率知识的大学生(新手)为被试,进行贝叶斯推理中的概率估计,探讨知识图式对贝叶斯推理的影响。结果表明,具有概率知识背景的大学生比没有概率知识背景的大学生概率估计的准确性更高,反应时更长,说明知识图式影响概率信息的搜索和判断。     

When Expert Disagreement Supports the Consensus

It is often suggested that disagreement among scientific experts is a reason not to trust those experts, even about matters on which they are in agreement. In direct opposition to this view, I argue here that the very fact that there is disagreement among experts on a given issue provides a positive reason for non-experts to trust that the experts really are justified in their attitudes towards consensus theories. I show how this line of thought can be spelled out in three distinct frameworks for non-deductive reasoning: namely, Bayesian Confirmation Theory, Inference to the Best Explanation, and Inferential Robustness Analysis.     

Development of categorization and reasoning in the natural world: novices to experts,naive similarity to ecological knowledge

Two experiments investigate the role of similarity and causal-ecological knowledge in expert and novice categorization and reasoning. In Experiment 1, university undergraduates and commercial fishermen sorted marine creatures into groups; although there was substantial agreement, novices sorted largely on the basis of appearance, whereas experts often cited commercial, ecological, or behavioral factors, and systematically subdivided fish on the basis of ecological niche. In Experiment 2, experts and novices were asked to generalize a blank property or novel disease from a pair of marine creatures. Novices relied on similarity to guide generalizations. Experts used similarity to reason about blank properties but ecological relations to reason about diseases. Expertise appears to involve knowledge of multiple relations among entities and context-sensitive application of those relations.     

On the nature and development of conditional reasoning in early adolescence

Two studies investigated the conditional reasoning capabilities of eighth grade learners. In Study 1, we analysed conditional reasoning performances for differences in learners' abilities to solve problems correctly and propensities to be tricked into responding to problems in a biconditional manner. There seemed to be a developmental progression in conditional reasoning ability. Most young adolescents reason using memory of domain-specific memories. However, pragmatic inferential rules may serve as an intermediate level of abstration in reasoners as they progress from the ability to reason using only domain-specific experience to the ability to use content-free syntactic rules. In addition, correlation between scores on the Test of Logical Thinking and conditional reasoning performances suggested that ability to reason deductively using inferential rules at progressively higher levels of abstraction is related to cognitive development. In Study 2, we assessed the effectiveness of an instructional application of pragmatic inferential rule theory for development of conditional reasoning abilities. Pragmatic reasoning skill was developed throug engagement in Prolog programming activities involving hierarchical knowledge domains.     

Using Historical Knowledge to Reason About Contemporary Political Issues: An Expert–Novice Study

People often justify history's place in the curriculum by its relationship to citizenship, yet there is little research to help educators picture how people use historical knowledge for civic purposes. This expert–novice study used the think-aloud method to examine how eight political scientists and eight high school students employed historical knowledge to reason about a political issue. Findings indicated that detailed historical narrative played an important role for experts’ reasoning, and the experts used narrative to frame the issue, support their positions, and evaluate historical claims. Participating students used narrative as well, but their narratives were lacking in detail. They never used history to frame the problem, contextualize documents, or to support their positions, and they rarely used narrative to evaluate claims. These differences in how experts and novices use history to make sense of the present have implications for history instruction and research in history and civic education.     

An expert consensus on the most effective components of cognitive behavioural therapy for adults with depression: a modified Delphi study

ABSTRACT

Designing new approaches to delivering cognitive behavioural therapy (CBT) requires an understanding of the key components. This study aimed to establish an expert consensus on the effective components of CBT for depressed adults. An international panel of 120 CBT experts was invited to participate in a modified Delphi study. Thirty-two experts participated in round 1; 21 also provided data in round 2. In round 1, experts rated the effectiveness of 35 content and process components. A priori rules identified components carried forward to round 2, in which experts re-rated items and final consensus items were identified. Consensus was achieved for nine content components (ensuring understanding; developing and maintaining a good therapeutic alliance; explaining the rationale for CBT; eliciting feedback; identifying and challenging avoidant behaviour; activity monitoring; undertaking an initial assessment; relapse prevention methods; homework assignments); and three process components (ensuring therapist competence; scheduling sessions flexibly; scheduling sessions for 45–60 mins). Five of the twelve components identified were generic therapeutic competences rather than specific CBT items. There was less agreement about the effectiveness of cognitive components of CBT. This is an important first step in the development of novel approaches to delivering CBT that may increase access to treatment for patients.     

Supposition and representation in human reasoning

We report the results of three experiments designed to assess the role of suppositions in human reasoning. Theories of reasoning based on formal rules propose that the ability to make suppositions is central to deductive reasoning. Our first experiment compared two types of problem that could be solved by a suppositional strategy. Our results showed no difference in difficulty between problems requiring affirmative or negative suppositions and very low logical solution rates throughout. Further analysis of the error data showed a pattern of responses, which suggested that participants reason from a superficial representation of the premises in these arguments and this drives their choice of conclusion. Our second experiment employed a different set of suppositional problems but with extremely similar proofs in terms of the rules applied and number of inferential steps required. As predicted by our interpretation of reasoning strategies employed in Experiment 1, logical performance was very much higher on these problems. Our third experiment showed that problems that could be solved by constructing an initial representation of the premises were easier than problems in which this representation was not sufficient. This effect was independent of the suppositional structure of the problems. We discuss the implications of this research for theories of reasoning based on mental models and inference rules.     

专家医生的知识结构及诊断推理方式

医学专长研究中“中间者效应”的发现,引发了研究者对专家医生知识结构的探讨。在“知识打包”的基础上,医生的临床知识以“疾病脚本”的方式组织起来。随着临床经验的增加,专家医生积累了丰富的疾病脚本。在临床诊断中,他们无需对病人所有的体征和症状进行仔细地和系统地分析,而是通过非分析性的推理方式——“模式识别”或“样例识别”便可自动激活与之匹配的疾病脚本,据此对病人做出迅速而准确的诊断。医学专长的本质就在于专家医生以“疾病脚本”的方式组织起来的知识结构。“适应性专长”代表了未来医学专长研究的新方向     

Examining Reasoning Skill Differences Between Expert and Novice Counselors

There is increasing interest in identifying the complex cognitive skills that constitute counseling expertise. Research from cognitive psychology suggests significant differences between experts and novices in their reasoning. This study examines the reasoning of expert and novice counselors engaged in a diagnostic task. Cases varied on “problem structure”—the extent to which problem-relevant information was clear and apparent. While an expertise by problem structure interaction was hypothesized, most of the differences were found across case type. Results suggest that the structure of the diagnostic problem may be an important variable affecting reasoning. The importance of problem structure for future research and training is discussed.     

Expertise, Democracy, and Applied Ethics

Is expertise in applied ethics compatible with individual autonomy and democratic self-governance? This depends on whether a ‘tracking condition’ is satisfied for expert claims about issues in applied ethics. This condition requires that, when expert deliberations are properly conducted they ‘track’ the courses of reasoning that the experts’ clients would themselves have undertaken if they had (perhaps subject to certain conditions) considered the matters for themselves. Pluralism of the kind thematised by Isaiah Berlin and Stuart Hampshire suggests that the tracking condition typically will not be satisfied and, hence, that whatever experts are praticising in applied ethics they are doing it contrary to democratic principles of autonomy and self-government. The implications of this result are sketched and some standard objections briefly considered.     

Pertinence Generation in Radiological Diagnosis: Spreading Activation and the Nature of Expertise

An empirical study of human expert reasoning processes is presented. Its purpose is to test a model of how a human expert's cognitive system learns to detect, and does detect, pertinent data and hypotheses. This process is called pertinence generation. The model is based on the phenomenon of spreading activation within semantic networks. Twenty-two radiologists were asked to produce diagnoses from two very difficult X-ray films. As the model predicted, pertinence increased with experience and with semantic network integration. However, the experts whose daily work involved explicit reasoning were able, in addition, to go beyond and to generate more pertinence. The results suggest that two qualitatively different kinds of expertise, basic and super, should be distinguished. A reinterpretation of the results of Lesgold et al. (1988) is proposed, suggesting that apparent nonmonotonicities in performance are not representative of common radiological expertise acquisition but result from the inclusion of basic and super expertise on the same curve.     

Rips的“证明心理学理论”述评

该文对Rips提出的“证明心理学理论”做了综合述评。这一理论主要包含三方面的内容:对推理过程与人类记忆相互关系的解释;(2)根据逻辑学中的“自然推理规则”进行修正后用于解释人类推理过程的推理规则;(3)关于如何控制推理过程的论述。Rips认为他于1983年设计并实施的以“自然推理系统”所含各推理规则为实验材料的实验结果支持该理论的基本观点。     

Legal and psychological bases of expert testimony: Surveys of the law and of jurors

This article reviews the basic rules of evidence and procedure that provide the basis for the introduction of expert testimony into court. Second, it reviews some of the findings of social psychology regarding expert testimony, primarily the ways in which the structure of society affects what information will reach the court and what is the readiness of the factfinder to deal with that information. Finally, it reports original data on jurors' assessments of a variety of witnesses (viz. physicians, psychiatrists, psychologists, chemists, document examiners, polygraph examiners, police, eyewitnesses, firearms experts, and accountants and appraisers). Jurors' differential assessments of the different experts and differences among juror subgroups are reported and some explanations for a few of those differences are tested.     

Casuistic Reasoning in Expert Narratives on Healthy Eating

Individualisation of healthy eating is a broad tendency in Western culture and is becoming a popular ideal in nutrition science. Public perception of dietary knowledge is central to this individualisation and many experts now aim to present dietary knowledge in a way which relates to individual experience, as well as a casuistic style in which particular cases are discussed in relation to general, universal principles. Analysis of casuistic narrative styles in the public communication about healthy eating by experts makes explicit their flexible use of both particular cases and universal claims, in which clinical and personal cases are narratively employed by the experts to invite readers to personally consider dietary knowledge. Such casuistic narration by experts relates human agency and particular bodies to universal concepts in a way that has generative and critical functions with respect to dietary knowledge and understanding. The public articulation of clinical narratives by experts make salient the intersubjective emergence and accumulation of dietary knowledge between expert reasoning and lay understanding. Personal narratives have an essential role in presenting casuistic reasoning through experts’ personal lives and making their own bodies relevant to public communication about healthy eating. This emphasises the experts’ personal agency in relationship to common norms and general knowledge. Furthermore, both clinical and personal narratives are used by experts in their assessment of the shortcomings in public health messages. Experts’ casuistic narration thus does not merely construct credibility but has epistemic functions in constructing dietary knowledge tangible with respect to actual eating practices.     

Using Statistical Natural Language Processing for Understanding Complex Responses to Free-Response Tasks

Whereas great advances have been made in the statistical sophistication of assessments in terms of evidence accumulation and task selection, relatively little statistical work has explored the possibility of applying statistical techniques to data for the purposes of determining appropriate domain understanding and to generate task-level scoring rules. Now that complex tasks are becoming increasingly prevalent, the inattention to item-level scoring is becoming more problematic. This study utilizes exploratory techniques to examine the differences between experts and novices in command usage and troubleshooting strategies in the field of computer networking. Participants were students and instructors of the Cisco Networking Academy Program as well as experts from the field of networking. Each participant was asked to perform troubleshooting tasks and a log of their actions was recorded. Log files containing all commands that participants entered while completing the troubleshooting tasks were analyzed using techniques of Statistical Natural Language Processing (SNLP). Results indicated that experts and novices differed in the types of commands that were used as well as in the sequence of those commands. Moreover, some patterns of examinee response that were found were entirely unexpected, leading to a rethinking of the appropriate conceptualization of the domain and the tasks. Previous assumptions about expert novice differences were shown to be faulty along with previously constructed scoring rules based on those assumptions. Comprehensive research in the application of statistical techniques to the understanding of domains and the validation of scoring rules are recommended.     

Papineau’s Theoretical Rationality and the Anthropological Difference

A common view in philosophy is that the way human beings reason is not only gradually better, but that our way of reasoning is fundamentally distinctive. Findings in the psychology of reasoning challenge the traditional view according to which human beings reason in accordance with the laws of logic and probability theory, but rather suggest that human reasoning consists in the application of domain specific rules of thumb similar to those that we ascribe to some intelligent non-human animals as well. However, this view on human reasoning is unable to explain human accomplishments like technological innovations or scientific progress. David Papineau offers a theory of human theoretical rationality that is consistent with the psychological view on human reasoning but that can also explain how humans sometimes are able to transcend the limitations of their biologically quick and dirty modes of thought and thereby reach a high level of accuracy. Papineau claims that the abilities that constitute theoretical rationality are unique to the human species and thus, that human reasoning is fundamentally distinctive after all. In this paper I am going to discuss to what extent these abilities in fact are unique to our species and whether this theoretical rationality can be called an anthropological difference.     

DEVELOPMENT AND VALIDATION OF A COMPUTERIZED INTERPRETATION SYSTEM FOR PERSONNEL TESTS

A computerized system was developed for generating narrative interpretations of scores from a battery of personnel screening tests. The report structure and interpretive statement library were designed to capture the test expertise and interpretive strategies of a panel of testing experts. This was accomplished by enumerating the questions that the experts believed the battery could answer, developing answers to these questions, and devising rules for selecting the appropriate answers based on test-battery scores. The accuracy, thoroughness, readability, and coherence of the computer-generated reports were evaluated in comparison to reports generated by human experts for the same examinees. Results of the evaluation showed the computerized reports to be more accurate and thorough, as readable, and somewhat less coherent than interpretations generated by the typical human expert. The computerized system development and validation strategies described are useful for other applications in which numbers are interpreted in a narrative report format.     

Strategies in temporal reasoning

Researchers currently working on relational reasoning typically argue that mental model theory (MMT) is a better account than the inference rule approach (IRA). They predict and observe that determinate (or one-model) problems are easier than indeterminate (or two-model) problems, whereas according to them, IRA should lead to the opposite prediction. However, the predictions attributed to IRA are based on a mistaken argument. The IRA is generally presented in such a way that inference rules only deal with determinate relations and not with indeterminate ones. However, (a) there is no reason to presuppose that a rule-based procedure could not deal with indeterminate relations, and (b) applying a rule-based procedure to indeterminate relations should result in greater difficulty. Hence, none of the recent articles devoted to relational reasoning currently presents a conclusive case for discarding IRA by using the well-known determinate vs indeterminate problems comparison.