Solving probabilistic and statistical problems: a matter of information structure and question form

Is the human mind inherently unable to reason probabilistically, or is it able to do so only when problems tap into a module for reasoning about natural frequencies? We suggest an alternative possibility: naive individuals are able to reason probabilistically when they can rely on a representation of subsets of chances or frequencies. We predicted that naive individuals solve conditional probability problems if they can infer conditional probabilities from the subset relations in their representation of the problems, and if the question put to them makes it easy to consider the appropriate subsets. The results of seven studies corroborated these predictions: when the form of the question and the structure of the problem were framed so as to activate intuitive principles based on subset relations, naive individuals solved problems, whether they were stated in terms of probabilities or frequencies. Otherwise, they failed with both sorts of information. The results contravene the frequentist hypothesis and the evolutionary account of probabilistic reasoning.     

Representation facilitates reasoning: what natural frequencies are and what they are not

A good representation can be crucial for finding the solution to a problem. Gigerenzer and Hoffrage (Psychol. Rev. 102 (1995) 684; Psychol. Rev. 106 (1999) 425) have shown that representations in terms of natural frequencies, rather than conditional probabilities, facilitate the computation of a cause's probability (or frequency) given an effect--a problem that is usually referred to as Bayesian reasoning. They also have shown that normalized frequencies--which are not natural frequencies--do not lead to computational facilitation, and consequently, do not enhance people's performance. Here, we correct two misconceptions propagated in recent work (Cognition 77 (2000) 197; Cognition 78 (2001) 247; Psychol. Rev. 106 (1999) 62; Organ. Behav. Hum. Decision Process. 82 (2000) 217): normalized frequencies have been mistaken for natural frequencies and, as a consequence, "nested sets" and the "subset principle" have been proposed as new explanations. These new terms, however, are nothing more than vague labels for the basic properties of natural frequencies.     

Frequency interpretation of ambiguous statistical information facilitates Bayesian reasoning

The idea that naturally sampled frequencies facilitate performance in statistical reasoning tasks because they are a cognitively privileged representational format has been challenged by findings that similarly structured numbers presented as chances similarly facilitate performance, on the basis of the claim that these are technically single-event probabilities. A crucial opinion, however, is that of the research participants, who possibly interpret chances as de facto frequencies. A series of experiments here indicate that not only is performance improved by clearly presented natural frequencies, rather than chances phrasing, but also that participants who interpreted chances as frequencies, rather than as probabilities, were consistently better at statistical reasoning. This result was found across different variations of information presentation and across different populations.     

Naive probability: a mental model theory of extensional reasoning

This article outlines a theory of naive probability. According to the theory, individuals who are unfamiliar with the probability calculus can infer the probabilities of events in an extensional way: They construct mental models of what is true in the various possibilities. Each model represents an equiprobable alternative unless individuals have beliefs to the contrary, in which case some models will have higher probabilities than others. The probability of an event depends on the proportion of models in which it occurs. The theory predicts several phenomena of reasoning about absolute probabilities, including typical biases. It correctly predicts certain cognitive illusions in inferences about relative probabilities. It accommodates reasoning based on numerical premises, and it explains how naive reasoners can infer posterior probabilities without relying on Bayes's theorem. Finally, it dispels some common misconceptions of probabilistic reasoning.     

Children can solve Bayesian problems: the role of representation in mental computation

Can children reason the Bayesian way? We argue that the answer to this question depends on how numbers are represented, because a representation can do part of the computation. We test, for the first time, whether Bayesian reasoning can be elicited in children by means of natural frequencies. We show that when information was presented to fourth, fifth, and sixth graders in terms of probabilities, their ability to estimate the Bayesian posterior probability was zero. Yet when the same information was presented in natural frequencies, Bayesian reasoning showed a steady increase from fourth to sixth grade, reaching an average level of 19, 39, and 53%, respectively, in two studies. Sixth graders' performance with natural frequencies matched the performance of adults with probabilities. But this general increase was accompanied by striking individual differences. More than half of the sixth graders solved most or all problems, whereas one third could not solve a single one. An analysis of the children's responses provides evidence for the use of three non-Bayesian strategies. These follow an overlapping wave model of development and continue to be observed in the minds of adults. More so than adults' probabilistic reasoning, children's reasoning depends on a proper representation of information.     

Motivated reasoning and verbal vs. numerical probability assessment: Evidence from an accounting context

A perplexing yet persistent empirical finding is that individuals assess probabilities in words and in numbers nearly equivalently, and theorists have called for future research to search for factors that cause differences. This study uses an accounting context in which individuals are commonly motivated to reach preferred (rather than accurate) conclusions. Within this context, I predict new differences between verbal and numerical probability assessments, as follows: first, individuals will justify an optimistic verbal assessment (e.g., somewhat possible) by retaining the option of re-defining it, in case of negative outcomes, as though the phrase really means something different, and, for that matter, means more things. This re-definition will maintain some connection to the original meaning of the phrase, but de-emphasized relative to the new meaning. Second, based on this behavior, I also predict individuals’ verbal probability assessments to be (1) more biased and yet (2) perceived as more justifiable than their numerical assessments. I find supportive evidence in an experiment designed to test the hypotheses. This study contributes to motivated reasoning and probability assessment theories (1) with new evidence of how individuals can word-smith in multiple attributes of a phrase to justify reaching a preferred conclusion, and (2) with new, reliable differences between verbal and numerical probability assessments. This study has important theoretical and practical implications relevant to organizational contexts in which people assess the likelihoods of uncertainties in words or numbers, and with motivations to reach a preferred conclusion.     

Mechanistic probability

I describe a realist, ontologically objective interpretation of probability, ??far-flung frequency (FFF) mechanistic probability??. FFF mechanistic probability is defined in terms of facts about the causal structure of devices and certain sets of frequencies in the actual world. Though defined partly in terms of frequencies, FFF mechanistic probability avoids many drawbacks of well-known frequency theories and helps causally explain stable frequencies, which will usually be close to the values of mechanistic probabilities. I also argue that it??s a virtue rather than a failing of FFF mechanistic probability that it does not define single-case chances, and compare some aspects of my interpretation to a recent interpretation proposed by Strevens.     

The Directionality of Verbal Probability Expressions: Effects on Decisions,Predictions, and Probabilistic Reasoning

Verbal phrases denoting uncertainty are of two kinds: positive, suggesting the occurrence of a target outcome, and negative, drawing attention to its nonoccurrence (Teigen & Brun, 1995). This directionality is correlated with, but not identical to, high and low p values. Choice of phrase will in turn influence predictions and decisions. A treatment described as having “some possibility” of success will be recommended, as opposed to when it is described as “quite uncertain,” even if the probability of cure referred to by these two expressions is judged to be the same (Experiment 1). Individuals who formulate their chances of achieving a successful outcome in positive terms are supposed to make different decisions than individuals who use equivalent, but negatively formulated, phrases (Experiments 2 and 3). Finally, negative phrases lead to fewer conjunction errors in probabilistic reasoning than do positive phrases (Experiment 4). For instance, a combination of 2 “uncertain” outcomes is readily seen to be “very uncertain.” But positive phrases lead to fewer disjunction errors than do negative phrases. Thus verbal probabilistic phrases differ from numerical probabilities not primarily by being more “vague,” but by suggesting more clearly the kind of inferences that should be drawn.     

Within-subject consistency and between-subject variability in Bayesian reasoning strategies

It is well known that people tend to perform poorly when asked to determine a posterior probability on the basis of a base rate, true positive rate, and false positive rate. The present experiments assessed the extent to which individual participants nevertheless adopt consistent strategies in these Bayesian reasoning problems, and investigated the nature of these strategies. In two experiments, one laboratory-based and one internet-based, each participant completed 36 problems with factorially manipulated probabilities. Many participants applied consistent strategies involving use of only one of the three probabilities provided in the problem, or additive combination of two of the probabilities. There was, however, substantial variability across participants in which probabilities were taken into account. In the laboratory experiment, participants’ eye movements were tracked as they read the problems. There was evidence of a relationship between information use and attention to a source of information. Participants’ self-assessments of their performance, however, revealed little confidence that the strategies they applied were actually correct. These results suggest that the hypothesis of base rate neglect actually underestimates people’s difficulty with Bayesian reasoning, but also suggest that participants are aware of their ignorance.     

Protecting Against Low‐Probability Disasters: The Role of Worry

We carry out a large monetary stakes insurance experiment with very small probabilities of losses and ambiguous as well as exact probabilities. Many individuals do not want to pay anything for insurance whether the probabilities are given exactly or are ambiguous. Many others, however, are willing to pay surprisingly large amounts. With ambiguity, the percentage of those paying nothing is smaller and the willingness to pay (WTP) of the other individuals larger than with exact probabilities. Comparing elasticities with ambiguity, we find that worry is much more important than subjective probability in determining WTP for insurance. Furthermore, when the ambiguous loss probability is increased by a factor of 1000, it has almost no effect on WTP. Copyright © 2011 John Wiley & Sons, Ltd.     

Retrospective Frequency Formats Promote Consistent Experience‐Based Bayesian Judgments

On the basis of their experiences with pregnant patients in their practice, obstetrician/gynecologists estimated the posterior probability of Down syndrome given a positive screening result. They also estimated the base rate of Down syndrome in their practice, along with the hit and false alarm rates for the screening test; for each subject, these numbers were combined to calculate a posterior probability to which the initial estimated posterior probability could be compared. Physicians gave highly consistent estimates when asked to think about their past experiences in terms of event frequencies. However, those told to respond using single event probabilities or to use past experiences to predict prospective frequencies gave inconsistent Bayesian estimates. Thus, when making Bayesian judgments based on real life experience, natural frequency formats only lead to better judgments, compared with single event probability formats, if people think retrospectively, not when using past experiences to make prospective predictions. Copyright © 2011 John Wiley & Sons, Ltd.     

概率大小、损益结果和认知闭合需要对模糊规避的影响

模糊规避是指在相同奖赏的情况下,决策者会力图规避从主观上判断具有模糊概率的事件而偏好具有相同精确概率的事件。本研究探讨了概率大小、损益结果和认知闭合需要对模糊规避的影响。研究发现,在小概率受益的情况下,个体倾向于模糊寻求;在中概率受益的情况下,个体倾向于模糊规避;在高概率受益的情况下,个体倾向于模糊规避;在小概率损失的情况下,个体倾向于模糊规避;在中概率损失的情况下,个体倾向于模糊规避;在高概率损失的情况下,个体倾向于模糊寻求。但是,研究并未发现认知闭合需要对模糊规避有预测作用。     

When Equal Chances = Good Chances: Verbal Probabilities and the Equiprobability Effect

When six equally qualified candidates compete for the same position, p = 1/6 for each. People seem to accept this principle more readily for numerical than for verbal probabilities. Equal chances with three to six alternatives are often verbally described in a positive vein as "entirely possible" or "a good chance" and rarely negatively as "doubtful" or "improbable." This equiprobability effect of verbal probabilities is demonstrated in five studies describing job applicants, lottery players, competing athletes, and examination candidates. The equiprobability effect is consistent with a causal (propensity) view of probabilities, where chances are believed to reflect the relative strength of facilitating and preventive causes. If important conditions in support of the target outcome are present (the candidate is qualified for the position), and there is little to prevent it from occurring (no stronger candidates), chances appear to be good. In the presence of obstacles (one stronger candidate), or in the absence of facilitating conditions (the candidate is poorly qualified), chances appear to be poor, even when numerical p values remain constant. The findings indicate that verbal and numerical probability estimates can reflect different intuitions. Copyright 2001 Academic Press.     

Frequency versus probability formats in statistical word problems

Three experiments examined people's ability to incorporate base rate information when judging posterior probabilities. Specifically, we tested the (Cosmides, L., & Tooby, J. (1996). Are humans good intuitive statisticians after all? Rethinking some conclusions from the literature on judgement under uncertainty. Cognition, 58, 1-73) conclusion that people's reasoning appears to follow Bayesian principles when they are presented with information in a frequency format, but not when information is presented as one case probabilities. First, we found that frequency formats were not generally associated with better performance than probability formats unless they were presented in a manner which facilitated construction of a set inclusion mental model. Second, we demonstrated that the use of frequency information may promote biases in the weighting of information. When participants are asked to express their judgements in frequency rather than probability format, they were more likely to produce the base rate as their answer, ignoring diagnostic evidence.     

Teaching Bayesian reasoning in less than two hours

The authors present and test a new method of teaching Bayesian reasoning, something about which previous teaching studies reported little success. Based on G. Gigerenzer and U. Hoffrage's (1995) ecological framework, the authors wrote a computerized tutorial program to train people to construct frequency representations (representation training) rather than to insert probabilities into Bayes's rule (rule training). Bayesian computations are simpler to perform with natural frequencies than with probabilities, and there are evolutionary reasons for assuming that cognitive algorithms have been developed to deal with natural frequencies. In 2 studies, the authors compared representation training with rule training; the criteria were an immediate learning effect, transfer to new problems, and long-term temporal stability. Rule training was as good in transfer as representation training, but representation training had a higher immediate learning effect and greater temporal stability.     

Culture and Conflict

Members of different cultures sample with diverse probabilities different kinds of information from their environment. Some sample the content of communications more than the context (e.g. tone of voice, gestures), whereas others do the reverse. Some sample processes internal to individuals (e.g. attitudes, beliefs) whereas others sample processes external to individuals (e.g. social influences, roles) with higher probabilities. Some give greater weights to ascribed attributes of persons, such as ethnicity, race, religion, and others to achieved attributes, such as beliefs, attitudes, or a record of achievements. These differences have profound implications for the probability of conflict and the type of conflict that will develop between individuals and groups.     

Probabilities on Sentences in an Expressive Logic

Automated reasoning about uncertain knowledge has many applications. One difficulty when developing such systems is the lack of a completely satisfactory integration of logic and probability. We address this problem directly. Expressive languages like higher-order logic are ideally suited for representing and reasoning about structured knowledge. Uncertain knowledge can be modeled by using graded probabilities rather than binary truth values. The main technical problem studied in this paper is the following: Given a set of sentences, each having some probability of being true, what probability should be ascribed to other (query) sentences? A natural wish-list, among others, is that the probability distribution (i) is consistent with the knowledge base, (ii) allows for a consistent inference procedure and in particular (iii) reduces to deductive logic in the limit of probabilities being 0 and 1, (iv) allows (Bayesian) inductive reasoning and (v) learning in the limit and in particular (vi) allows confirmation of universally quantified hypotheses/sentences. We translate this wish-list into technical requirements for a prior probability and show that probabilities satisfying all our criteria exist. We also give explicit constructions and several general characterizations of probabilities that satisfy some or all of the criteria and various (counter)examples. We also derive necessary and sufficient conditions for extending beliefs about finitely many sentences to suitable probabilities over all sentences, and in particular least dogmatic or least biased ones. We conclude with a brief outlook on how the developed theory might be used and approximated in autonomous reasoning agents. Our theory is a step towards a globally consistent and empirically satisfactory unification of probability and logic.     

Probabilities of counterfactuals and counterfactual probabilities

Probabilities figure centrally in much of the literature on the semantics of conditionals. I find this surprising: it accords a special status to conditionals that other parts of language apparently do not share. I critically discuss two notable ‘probabilities first’ accounts of counterfactuals, due to Edgington and Leitgeb. According to Edgington, counterfactuals lack truth values but have probabilities. I argue that this combination gives rise to a number of problems. According to Leitgeb, counterfactuals have truth conditions-roughly, a counterfactual is true when the corresponding conditional chance is sufficiently high. I argue that problems arise from the disparity between truth and high chance, between approximate truth and high chance, and from counterfactuals for which the corresponding conditional chances are undefined. However, Edgington, Leitgeb and I can unite in opposition to Stalnaker and Lewis-style ‘similarity’ accounts of counterfactuals.     

Replication is not coincidence: Reply to Iverson, Lee, and Wagenmakers (2009)

Iverson, Lee, and Wagenmakers (2009) claimed that Killeen’s (2005) statistic prep overestimates the “true probability of replication.” We show that Iverson et al. confused the probability of replication of an observed direction of effect with a probability of coincidence—the probability that two future experiments will return the same sign. The theoretical analysis is punctuated with a simulation of the predictions of prep for a realistic random effects world of representative parameters, when those are unknown a priori. We emphasize throughout that prep is intended to evaluate the probability of a replication outcome after observations, not to estimate a parameter. Hence, the usual conventional criteria (unbiasedness, minimum variance estimator) for judging estimators are not appropriate for probabilities such as p and p_rep.     

Subjective randomness and natural scene statistics

Accounts of subjective randomness suggest that people consider a stimulus random when they cannot detect any regularities characterizing the structure of that stimulus. We explored the possibility that the regularities people detect are shaped by the statistics of their natural environment. We did this by testing the hypothesis that people’s perception of randomness in two-dimensional binary arrays (images with two levels of intensity) is inversely related to the probability with which the array’s pattern would be encountered in nature. We estimated natural scene probabilities for small binary arrays by tabulating the frequencies with which each pattern of cell values appears. We then conducted an experiment in which we collected human randomness judgments. The results show an inverse relationship between people’s perceived randomness of an array pattern and the probability of the pattern appearing in nature.