Meehl's contribution to clinical versus statistical prediction

Paul E. Meehl's work on the clinical versus statistical prediction controversy is reviewed. His contributions included the following: putting the controversy center stage in applied psychology; clarifying concepts underpinning the debate (especially his crucial distinction between ways of gathering data and ways of combining them) as well as establishing that the controversy was real and not concocted, analyzing clinical inference from both theoretical and probabilistic points of view, and reviewing studies that compared the accuracy of these 2 methods of data combination. Meehl's (1954/1996) conclusion that statistical prediction consistently outperforms clinical judgment has stood up extremely well for half a century. His conceptual analyses have not been significantly improved since he published them in the 1950s and 1960s. His work in this area contains several citation classics, which are part of the working knowledge of all competent applied psychologists today.     

Clinical and statistical approaches to the assessment of brain damage in children

In an attempt at innovation in the clinical versus statistical prediction controversy, the present research examined three methods of assessing organic brain damage in a sample of sixty-eight 9- to 14-year-old children. Methods compared were clinical judgment (CP); existing actuarial norms (EAN); and derived statistical prediction (DSP), based on three psychological tests. The results were as follows: (a) None of the methods classified subjects significantly better than chance (p<.05); (b) there was no significant difference (p<.05) among the methods in terms of their rates of accurate classification; (c) CP based on combined tests did not result in higher predictive accuracy than those based on an individual test; (d) CP made after receiving feedback on accuracy showed an increase in number of correct classifications over prior CP; and (e) CP based on knowledge of the statistical prediction in addition to test data showed a higher rate of correct classification than DSP alone.     

解释水平视角下的道德判断

道德判断对社会合作有重要意义。近20年来涌现出大量对道德判断的研究。经典的社会直觉模型和双加工模型已经不足以解释所有的研究发现。近年来,解释水平理论为道德判断的研究提供了一个新的视角。通过文献梳理发现,基于解释水平理论的道德判断研究可以分为两难的道德判断研究和一般的道德判断研究。两难的道德判断研究已经有了较为一致的结论,但是一般的道德判断研究还存在争议。解释水平影响道德判断的深层原因可能是通过关注点影响道德判断的标准。未来的研究可以着眼于跨文化研究和深层机制的剖析,为解释水平视角下的道德判断研究提供更多的实验证据,加深人们对道德判断的理解。     

Clinical versus mechanical prediction: a meta-analysis

The process of making judgments and decisions requires a method for combining data. To compare the accuracy of clinical and mechanical (formal, statistical) data-combination techniques, we performed a meta-analysis on studies of human health and behavior. On average, mechanical-prediction techniques were about 10% more accurate than clinical predictions. Depending on the specific analysis, mechanical prediction substantially outperformed clinical prediction in 33%-47% of studies examined. Although clinical predictions were often as accurate as mechanical predictions, in only a few studies (6%-16%) were they substantially more accurate. Superiority for mechanical-prediction techniques was consistent, regardless of the judgment task, type of judges, judges' amounts of experience, or the types of data being combined. Clinical predictions performed relatively less well when predictors included clinical interview data. These data indicate that mechanical predictions of human behaviors are equal or superior to clinical prediction methods for a wide range of circumstances.     

JUDGMENTAL VERSUS STATISTICAL PREDICTION:

Abstract— The relative predictive accuracy of humans and statistical models has long been the subject of controversy even though models have demonstrated superior performance in many studies. We propose that relative performance depends on the amount of contextual information available and whether it is distributed symmetrically to humans and models. Given their different strengths, human and statistical predictions can be profitably combined to improve prediction.     

Signal detection evidence for limited capacity in visual search

The nature of capacity limits (if any) in visual search has been a topic of controversy for decades. In 30 years of work, researchers have attempted to distinguish between two broad classes of visual search models. Attention-limited models have proposed two stages of perceptual processing: an unlimited-capacity preattentive stage, and a limited-capacity selective attention stage. Conversely, noise-limited models have proposed a single, unlimited-capacity perceptual processing stage, with decision processes influenced only by stochastic noise. Here, we use signal detection methods to test a strong prediction of attention-limited models. In standard attention-limited models, performance of some searches (feature searches) should only be limited by a preattentive stage. Other search tasks (e.g., spatial configuration search for a "2" among "5"s) should be additionally limited by an attentional bottleneck. We equated average accuracies for a feature and a spatial configuration search over set sizes of 1-8 for briefly presented stimuli. The strong prediction of attention-limited models is that, given overall equivalence in performance, accuracy should be better on the spatial configuration search than on the feature search for set size 1, and worse for set size 8. We confirm this crossover interaction and show that it is problematic for at least one class of one-stage decision models.     

Two Models of Moral Judgment

This paper compares two theories and their two corresponding computational models of human moral judgment. In order to better address psychological realism and generality of theories of moral judgment, more detailed and more psychologically nuanced models are needed. In particular, a motivationally based theory of moral judgment (and its corresponding computational model) is developed in this paper that provides a more accurate account of human moral judgment than an existing emotion‐reason conflict theory. Simulations based on the theory capture and explain a range of relevant human data. They account not only for the original data that were used to support the emotion – reason conflict theory, but also for a wider range of data and phenomena.     

Violence risk: Re-defining variables from the first-person perspective

Over the past 25 years, there have been notable advances in violence risk assessment of mentally ill individuals using actuarial methods to define high versus low risk groups. A focus on readily observable risk factors, however, has led to a relative neglect of how the offender's subjective states may be valuable to consider in research on the ongoing assessment and prevention of violence. We argue for the relevance of considering idiographic features of subjective experience in the development of structured assessment methods. We then identify three heuristic groups of existing constructs related to aggressive and illegal behavior that may capture modifiable, time-varying aspects of mental functioning leading up to involvement in an act of violence. These hypothesized domains are: (i) construal of intent and cause; (ii) normative reference points; and (iii) emotion recognition and regulation. We suggest that risk state for violence can be studied in a parsimonious and direct manner through systematic research on coded speech samples. The coding method for such an assessment procedure would be almost identical to existing structured clinical judgment instruments with the difference that variables be defined from a first-person point of view. Some implications of this approach for the tertiary prevention of violence in high-risk individuals are described.     

直觉还是推理,道德判断如何形成?

道德判断形成的机制一直存在争议,许多研究者通过实证研究来探索诸因素对道德判断的影响,这有助于理清不同因素的作用,但却难以构建道德判断形成的理论框架。文章系统性地回顾和总结了道德判断形成的五个经典理论模型及其相关理论,并结合实证研究的成果梳理了道德判断形成中理性、情绪、认知、直觉和推理之间的关系。基于此,构建了以直觉和推理为基本路径,基于规则的认知和情绪为调节因素的道德判断形成的理论框架。现有的理论模型和实证研究在该框架下均可以的得到较好的解释。同时,根据该框架,建议未来的实证研究中应当注重对影响因素的分类以及对实验情景的控制。     

Recognizing Racit Knowledge in Medical Epistemology

The evidence-based medicine movement advocates basing all medical decisions on certain types of quantitative research data and has stimulated protracted controversy and debate since its inception. Evidence-based medicine presupposes an inaccurate and deficient view of medical knowledge. Michael Polanyi’s theory of tacit knowledge both explains this deficiency and suggests remedies for it. Polanyi shows how all explicit human knowledge depends on a wealth of tacit knowledge which accrues from experience and is essential for problem solving. Edmund Pellegrino’s classic treatment of clinical judgment is examined, and a Polanyian critique of this position demonstrates that tacit knowledge is necessary for understanding how clinical judgment and medical decisions involve persons. An adequate medical epistemology requires much more qualitative research relevant to the clinical encounter and medical decision making than is currently being done. This research is necessary for preventing an uncritical application of evidence-based medicine by health care managers that erodes good clinical practice. Polanyi’s epistemology shows the need for this work and provides the structural core for building an adequate and robust medical epistemology that moves beyond evidence-based medicine.An erratum to this article can be found at     

Probabilistic Mental Models with Continuous Predictors

Gigerenzer and his colleagues have sought to develop psychologically plausible models of human judgment. Their models are classified as ones of bounded rationality based on a principle of one-reason decision making. The models associated with the theory of Probabilistic Mental Models (PMM) have been developed for tasks in which all predictors are binary. This article extends PMM to the case of continuous predictors. The current model employs the limitation on the number of categories people use in making absolute judgments along a single perceptual dimension (7 +/- 2; Miller, 1956). The algorithm transforms each continuous predictor to be consistent with this limitation, then implements a step-down one-reason decision procedure similar to previous PMM models. Like previous PMM models, the 7 +/- 2 model predicts binary judgments as well as a multiple-regression model. However, the model does not successfully predict the probability judgments of individual participants, which is also true of all other models in the literature. Copyright 2000 Academic Press.     

Effects of familiarity level and repetition on recognition accuracy

Theories of recognition memory based on signal detection theory posit that a recognition decision is similar to a psychophysical judgment. Like a judgment of stimulus brightness or loudness, a recognition judgment is based on the value of a unidimensional signal computed for the test item. This signal has been called the strength or familiarity value. One prediction of these models is that the ability to discriminate between a studied and nonstudied test item depends on the ability to detect the difference in their familiarities. This ability in turn is influenced by the items' familiarity levels; discrimination should become more difficult as the familiarity of both items increases. This prediction was supported in 3 experiments using a forced-choice procedure. Also, accuracy was higher when the list contained repeated items rather than a comparable number of distinct items.     

Prediction and Clinical Inference: Forty Years Later

The earlier controversy between clinical and statistical prediction has faded into the shadows. Few people today challenge the necessity for both approaches in seeking to develop and apply knowledge. In more contemporary terms, the issue of which prediction method should be preferred involves choosing between a quest for historical truth (i.e., correspondence demonstrated by staristical methods) and a quest for narrative truth (i.e., coherence achieved by clinical formulations).     

Prediction and clinical inference: forty years later

The earlier controversy between clinical and statistical prediction has faded into the shadows. Few people today challenge the necessity for both approaches in seeking to develop and apply knowledge. In more contemporary terms, the issue of which prediction method should be preferred involves choosing between a quest for historical truth (i.e., correspondence demonstrated by statistical methods) and a quest for narrative truth (i.e., coherence achieved by clinical formulations).     

Improving Predictive Accuracy with a Combination of Human Intuition and Mechanical Decision Aids

This study examines the intuitive combination of human judgment and mechanical prediction under varied information conditions. As expected, mechanical prediction outperformed human intuition when based on the same information, but a combined approach was best when judges had access to relevant information not captured by the model (information asymmetry). The model was useful for differentiating between the event outcomes (improved slope), while eliminating the bias caused by base-rate neglect. Human intuition was useful for incorporating relevant information outside the scope of the model, resulting in improved slope and reduced judgment scatter. The addition of irrelevant information was detrimental to judgment accuracy, causing an increase in bias and a reduction in slope. These results provide insight intohowandwhencombining mechanical prediction and human intuition is likely to result in improved accuracy.     

Human covariation judgment: Accuracy and strategy

Judgment strategy is proposed as a contributor to the variability of findings in past research on human covariation judgment. Mathematically sophisticated judgment strategies will accurately judge all event covariations. However, faulty judgment rules will also produce correct judgments of many event relationships. Several methods have been used in past research to identify subjects' strategies of covariation judgment. Each of these methods indicates that humans employ simplistic, error-prone rules to judge event relationships. Shifts in covariation strategy use are proposed as a source of past findings that judgment accuracy depends on the decision conditions.     

重复学习判断中的低自信效应

学习判断是元认知判断的一种重要形式,是对未来回忆成绩的预测。重复学习判断中的低自信效应是一个有趣而反常的现象,指在重复学习测验中,学习判断往往低估了回忆成绩的增加,出现低自信的一种现象。目前,大部分实验证据支持重复学习判断中的低自信效应是普遍存在的,但也有一些实验结果表明该效应是受一定条件限制的。对于这种低自信效应的作用机制,研究者提出了不同的理论模型（如,锚定调整假说、记忆偏向消除理论、过去测验记忆的启发式假说等）进行了深入探讨。最后,文章还指出了此领域现有研究的局限和有待研究的问题     

Forensic neuropsychology: A reply to the method skeptics

Various critics or method skeptics have contended that clinical neuropsychology is not sufficiently developed as a science to be offered as evidence in legal or trial proceedings. The present article attempts to balance the extreme position of the method skeptics with an overview of legal and research data that support forensic applications of neuropsychology. It is suggested that clinical evidence can usefully inform legal decision making and that the modern trend has been for courts to be increasingly open to such expert testimony. The relevance of studies of clinical judgment, experience, and actuarial prediction is discussed, and neuropsychological assessment validity is specifically addressed. It is concluded that the arguments of the method skeptics should guide future research and caution forensic neuropsychologists, but that a retreat from the courtroom is unwarranted.     

Policy Capturing with Ridge Regression

A policy capturing method combining human judgment with ridge regression is offered which results in superior judgment policy models. The new method (termed smart ridge regression) was tested against four others in seven judgment policy capturing applications. Performance criteria were two cross-validation indices: cross-validated multiple correlation and mean squared error of prediction of new judgments. Smart ridge regression was found to outperform ordinary least squares regression and conventional ridge regression, as well as subjective weighting and equal weighting of cues.     

The rhetoric of the reasoned social scientific fact

An analysis is provided for one possible practical link between rhetorical and social scientific inquiry. That link is found in the rhetoric of the reasoned social scientific fact. Understanding this point of intersection involves grounding a rhetorical theory of how to create and to evaluate arguments (a rhetorical theory of invention and judgment) in the practical problems that confront contemporary social scientists during their efforts to construct reasoned social facts. The applicability of this invention and judgment framework to analysis of the rhetoric of social science is illustrated with reference to a controversy over the legitimacy of rules theoretic explanations of human communication processes. Implications of the practical link between rhetorical and social scientific inquiry are then drawn out.