When good theories make bad predictions

Chaos-related obstructions to predictability have been used to challenge accounts of theory validation based on the agreement between theoretical predictions and experimental data (Rueger & Sharp, 1996. The British Journal for the Philosophy of Science, 47, 93–112; Koperski, 1998. Philosophy of Science, 40, 194–212). These challenges are incomplete in two respects: (a) they do not show that chaotic regimes are unpredictable in principle (i.e., with unbounded resources) and, as a result, that there is something conceptually wrong with idealized expectations of correct predictions from acceptable theories, and (b) they do not explore whether chaos-induced predictive failures of deterministic models can be remedied by stochastic modeling. In this paper we appeal to an asymptotic analysis of state space trajectories and their numerical approximations to show that chaotic regimes are deterministically unpredictable even with unbounded resources. Additionally, we explain why stochastic models of chaotic systems, while predictively successful in some cases, are in general predictively as limited as deterministic ones. We conclude by suggesting that the way in which scientists deal with such principled obstructions to predictability calls for a more comprehensive approach to theory validation, on which experimental testing is augmented by a multifaceted mathematical analysis of theoretical models, capable of identifying chaos-related predictive failures as due to principled limitations which the world itself imposes on any less-than-omniscient epistemic access to some natural systems. We give special thanks to two anonymous reviewers for their helpful comments that have substantially contributed to the final version of this paper     

Testing psychological theories about inference making

Two contrasting views about elaborative inferences in text comprehension are that they occur as the text is read and that they are deferred until they are made necessary by a question about the inferable information. This paper develops these views in greater detail and examines some of their consequences. The model of memory performance underlying the first of these theories has two quite different interpretations: the standard one and one in which inferable information, whether it is implicit or explicit in a text, is not encoded into memory for content. Two experiments are reported. The results are not consistent with the most straightforward interpretation of the claim that inferences are always deferred. However, that theory can be modified so that it makes the same predictions as its rivals. The various interpretations of the formal model underlying all three accounts of inference making can be distinguished only in terms of parsimony, explanatory adequacy, and their ability to handle results from other experimental paradigms, such as sentence verification.     

Expectation theories of decision making for duplex gambles

Complete tests of subjectively expected utility (SEU), subjectively expected value (SEV), expected utility (EU) and expected value (EV) theories were made for duplex gambles without measuring subjective probability or subjective utility. All gambles were hypothetical and offered on booklets. The duplex gambles consisted of winning gambles, which offered a chance to win a certain amount of money or to break even; and losing gambles, which offered a chance to lose a certain amount of money or break even.The results indicated that SEU, SEV and EU theories could not account for the strategies of 33%, 53% and 86% of the Ss respectively in the losing form of gambles, while EV theory accounted for 78% of the behavior of Ss.In the winning form of gambles, SEU, SEV and EU theory held for 77%, 65%, and 54% of the Ss respectively, while EV theory held for only 40% of the Ss. Suggestions for further research were made.     

Computing the uncomputable; or,The discrete charm of second-order simulacra

We examine a case in which non-computable behavior in a model is revealed by computer simulation. This is possible due to differing notions of computability for sets in a continuous space. The argument originally given for the validity of the simulation involves a simpler simulation of the simulation, still further simulations thereof, and a universality conjecture. There are difficulties with that argument, but there are other, heuristic arguments supporting the qualitative results. It is urged, using this example, that absolute validation, while highly desirable, is overvalued. Simulations also provide valuable insights that we cannot yet (if ever) prove.     

Falsifying mental models: Testing the predictions of theories of syllogistic reasoning

Four experiments are reported that tested the claim, drawn from mental models theory, that reasoners attempt to construct alternative representations of problems that might falsify preliminary conclusions they have drawn. In Experiment 1, participants were asked to indicate which alternative conclusion(s) they had considered in a syllogistic reasoning task. In Experiments 2-4, participants were asked to draw diagrams consistent with the premises, on the assumption that these diagrams would provide insights into the mental representation being used. In none of the experiments was there any evidence that people constructed more models for multiple-model than for single-model syllogisms, nor was there any correlation between number of models constructed and overall accuracy. The results are interpreted as showing that falsification of the kind proposed by mental models theory may not routinely occur in reasoning.     

Learning categories by making predictions: An investigation of indirect category learning

Categories are learned in many ways, but studies of category learning have generally focused on classification learning. This focus may limit the understanding of categorization processes. Two experiments were conducted in which participants learned categories of animals by predicting how much food each animal would eat. We refer to this as indirect category learning, because the task andthe feedback were not directly related to category membership, yet category learning was necessary for good performance in the task. In the first experiment, we compared the performance of participants who learned the categories indirectly with the performance of participants who first learned to classify the objects. In the second experiment, we replicated the basic findings and examined attention to different features during the learning task. In both experiments, participants who learned in the prediction-only condition displayed a broader distribution of attention than participants who learned in the classification-and-prediction condition did. Some participants in the prediction-only group learned the family resemblance structure of the categories, even when a perfect criterial attribute was present. In contrast, participants who first learned to classify the objects tended to learn the criterial attribute.     

The testability of psychiatric patients in an operant conditioning situation

The Psychological Record -     

Empirically and clinically useful decision making in psychotherapy: differential predictions with treatment response models

In the delivery of clinical services, outcomes monitoring (i.e., repeated assessments of a patient's response to treatment) can be used to support clinical decision making (i.e., recurrent revisions of outcome expectations on the basis of that response). Outcomes monitoring can be particularly useful in the context of established practice research networks. This article presents a strategy to disaggregate patients into homogeneous subgroups to generate optimal expected treatment response profiles, which can be used to predict and track the progress of patients in different treatment modalities. The study was based on data from 618 diagnostically diverse patients treated with either a cognitive-behavioral treatment protocol (n = 262) or an integrative cognitive-behavioral and interpersonal treatment protocol (n = 356). The validity of expected treatment response models to predict treatment in those 2 protocols for individual patients was evaluated. The ways such a procedure might be used in outpatient centers to learn more about patients, predict treatment response, and improve clinical practice are discussed.     

Elaboration and numerical anchoring: Implications of attitude theories for consumer judgment and decision making

Researchers across many domains have examined the impact of externally presented numerical anchors on perceiver judgments. In the traditional paradigm, “anchored” judgments are typically explained as a result of elaborate thinking (i.e., confirmatory hypothesis testing that selectively activates anchor-consistent information in memory). Consistent with a long tradition in attitude change, we suggest that the same judgments can result from relatively thoughtful or non-thoughtful processes, with more thoughtful processes resulting in judgments that have more lasting impact. We review recent anchoring research consistent with this elaboration-based perspective and discuss implications for past anchoring results and theory in judgment and decision making.     

The dimensions of stage theories

Stage theories are commonly used to represent the structure of cognitive processes in the domains of cognitive processing and cognitive development. Hierarchical unidimensional theories consisting of a single sequence of stages through which development proceeds in one direction only have predominated. However, a review of stage theories revealed that a two-dimensional, interactive (multidimensional) structure is warranted. Particularly in the area of cognitive development, results of research have supported subdivision of the stages of prominent unidimensional theories. Following this trend a small number of theorists have developed multidimensional theories, most of which consist of a sequence of stages in which each stage contains a number of subsidiary stages or levels. The emergence of a similar trend in other domains is examined, and the prospects of using an existing multidimensional framework to enable future comparisons and integration of different stage theories is explored.     

The dimensions of stage theories

Stage theories are commonly used to represent the structure of cognitive processes in the domains of cognitive processing and cognitive development. Hierarchical unidimensional theories consisting of a single sequence of stages through which development proceeds in one direction only have predominated. However, a review of stage theories revealed that a two-dimensional, interactive (multidimensional) structure is warranted. Particularly in the area of cognitive development, results of research have supported subdivision of the stages of prominent unidimensional theories. Following this trend a small number of theorists have developed multidimensional theories, most of which consist of a sequence of stages in which each stage contains a number of subsidiary stages or levels. The emergence of a similar trend in other domains is examined, and the prospects of using an existing multidimensional framework to enable future comparisons and integration of different stage theories is explored.