Predicting binary choices from probability phrase meanings

The issues of how individuals decide which of two events is more likely and of how they understand probability phrases both involve judging relative likelihoods. In this study, we investigated whether derived scales representing probability phrase meanings could be used within a choice model to predict independently observed binary choices. If they can, this simultaneously provides support for our model and suggests that the phrase meanings are measured meaningfully. The model assumes that, when deciding which of two events is more likely, judges take a single sample from memory regarding each event and respond accordingly. The model predicts choice probabilities by using the scaled meanings of individually selected probability phrases as proxies for confidence distributions associated with sampling from memory. Predictions are sustained for 34 of 41 participants but, nevertheless, are biased slightly low. Sequential sampling models improve the fit. The results have both theoretical and applied implications.     

Verbal probabilities: a question of frame?

Verbal expressions of probability and uncertainty are of two kinds: positive (‘probable’, ‘possible’) and negative (‘not certain’, ‘doubtful’). Choice of term has implications for predictions and decisions. The present studies show that positive phrases are rated to be more optimistic (when the target outcome is positive), and more correct, when the target outcome actually occurs, even in cases where positive and negative phrases are perceived to convey the same probabilities (Experiments 1 and 2). Selection of phrase can be determined by linguistic frame. Positive quantifiers (‘some’, ‘several’) support positive probability phrases, whereas negative quantifiers (‘not all’) suggest negative phrases (Experiment 3). Positive frames induced by numeric frequencies (e.g. the number of students to be admitted) imply positive probability phrases, whereas negative frames (e.g. the number of students to be rejected) call for negative probability phrases (Experiment 4). It is concluded that choice of verbal phrase is based not only on level of probability, but also on situational and linguistic cues. Copyright © 2002 John Wiley & Sons, Ltd.     

Directional verbal probabilities: inconsistencies between preferential judgments and numerical meanings

Verbal probability expressions (e.g., it is possible or doubtful) convey not only vague numerical meanings (i.e., probability) but also semantic functions, called directionality. We performed two experiments to examine whether preferential judgments are consistent with numerical meanings of verbal probabilities regardless of directionality. The results showed that because of the effects of directionality, perceived degrees of certainty for verbal probabilities differed between a binary choice and a numerical translation (Experiment 1), and decisions based on a verbal probability do not correspond to those based on a numerical translation for verbal probabilities (Experiment 2). These findings suggest that directionality of verbal probabilities is an independent feature from numerical meanings; hence numerical meanings of verbal probability alone remain insufficient to explain the effects of directionality on preferential judgments.     

Dyadic decisions with numerical and verbal probabilities

The goal of the research was to compare decisions under risk in a situation in which forecasters (F) communicate to decision makers (DM) either numerically (e.g., .70) or verbally (e.g., likely) about the chances that a binary event will occur. Following each forecast, the DM bid for a winning or losing lottery based on the event. In Experiment 1 Fs and DMs also provided numerical translations of each verbal forecast after the DMs' bid. In Experiment 2 the DMs provided membership functions over the [0, 1] interval for each phrase used by the Fs. The primary results were: (a) extreme similarity in the DM's bids and rates of bidding under the two modes of communication; (b) greater variability in bids to specific verbal than numerical forecasts; (c) a pattern of bids, in which DMs demonstrated risk seeking for gains and risk neutrality for losses; (d) DMs' numerical translations in Experiment 1 were closer to .50 than were those of Fs; and (e) phrases selected by Fs had high membership values to DMs for the probabilities the Fs were attempting to describe. Points (a), (b), (d), and (e) are consistent with the ν-μ model which assumes that the vague meaning of a probability phrase can be represented by a membership function over the [0, 1] interval, and that in reaching a decision the DM focuses on a range of probabilities with sufficiently high membership. Point (c) is speculatively attributed to social aspects of the dyadic situation, and requires further investigation.     

Spicy Adjectives and Nominal Donkeys: Capturing Semantic Deviance Using Compositionality in Distributional Spaces

Sophisticated senator and legislative onion. Whether or not you have ever heard of these things, we all have some intuition that one of them makes much less sense than the other. In this paper, we introduce a large dataset of human judgments about novel adjective‐noun phrases. We use these data to test an approach to semantic deviance based on phrase representations derived with compositional distributional semantic methods, that is, methods that derive word meanings from contextual information, and approximate phrase meanings by combining word meanings. We present several simple measures extracted from distributional representations of words and phrases, and we show that they have a significant impact on predicting the acceptability of novel adjective‐noun phrases even when a number of alternative measures classically employed in studies of compound processing and bigram plausibility are taken into account. Our results show that the extent to which an attributive adjective alters the distributional representation of the noun is the most significant factor in modeling the distinction between acceptable and deviant phrases. Our study extends current applications of compositional distributional semantic methods to linguistically and cognitively interesting problems, and it offers a new, quantitatively precise approach to the challenge of predicting when humans will find novel linguistic expressions acceptable and when they will not.     

Selection of verbal probabilities: A solution for some problems of verbal probability expression

A method for verbal expression of degree of uncertainty is described. It requires the subject to select a phrase from a list that spans the full range of probabilities. In a second, optional, step, the subject indicates the numerical meaning of each phrase. The method avoids two problems of verbal probabilities—the indefinitely large lexicon and the individual differences in the interpretation of words. To test whether context and ordinal position might bias subjects' selection or interpretation of the verbal expressions in the list, the list order was varied. When the verbal expressions were arranged in random order, ordinal position had a significant effect on the selection of expressions. However, these effects did not occur when the phrases were listed in ascending or descending order. Considerations of accuracy and interpersonal agreement also support the use of ordered phrase lists.     

Is guilt ‘likely’ or ‘not certain’?: Contrast with previous probabilities determines choice of verbal terms

This research focuses on what determines speakers' choice of positive and negative probability phrases (e.g., “a chance” vs. “not certain”) in a legal context. We argue that choice of phrase to describe an event's probability of occurrence can be determined by the contrast between its current p value and an earlier p value, and not by that current value alone. Three experiments were conducted describing scenarios where profilers communicated a suspect's probability of guilt to the police. In the first study, a probability estimate is revised upwards or downwards. In the second one, the probability estimate of a speaker is higher or lower than that given by a previous speaker. In both cases, participants expected upward trends to lead to positive phrases, whereas downward trends were associated with negative phrases. In a third study, participants had to select probability phrases to characterize two different suspects. No contrast effects were found. We conclude that verbal probability directionality has primarily an argumentative function, where positive phrases are selected when probabilities are contrasted with smaller p values, and negative when contrasted with higher p values.     

Eye movements reveal the on-line computation of lexical probabilities during reading

Skilled readers are able to derive meaning from a stream of visual input with remarkable efficiency. In this article, we present the first evidence that statistical information latent in the linguistic environment can contribute to an account of reading behavior. In two eye-tracking studies, we demonstrate that the transitional probabilities between words have a measurable influence on fixation durations, and using a simple Bayesian statistical model, we show that lexical probabilities derived by combining transitional probability with the prior probability of a word's occurrence provide the most parsimonious account of the eye movement data. We suggest that the brain is able to draw upon statistical information in order to rapidly estimate the lexical probabilities of upcoming words: a computationally inexpensive mechanism that may underlie proficient reading.     

The effect of constant outcome value on judgments and decision making given linguistic probabilities

Two experiments were performed to determine whether judgments of the relative chances of two independent events occurring are biased by constant outcome values contingent on the events when the uncertainties are specified by linguistic expressions (e.g. doubtful). In Experiment 1, subjects directly judged the relative chances of the two events, of which one was represented by a spinner and the other by a linguistic probability expression. In Experiment 2, only linguistic probability expressions were used to describe the two events and a betting procedure was used. A bias was evident in both studies, such that the relative judgments tended to favour the event with the positive rather than the negative contingent outcome. The bias was smaller for the low- than for the high-probability phrases. Individual differences were great, with the bias appearing strongly in only about one-third of the population. Theoretical implications of the present and related results are discussed.     

Memory for sentences and noun phrases of extreme depth

Memory for orally presented noun phrases and sentences was investigated in two experiments. Syntactic depth (D) of the phrases and sentences was varied with D approaching the values of 7 ± 2. Consistent with recent investigations, these studies found D not to be generally related to recall of sentences, phrases, or individual words within sentences. However, recall of individual words from an adverbial noun phrase apparently was related to D. The apparent relationship is accounted for not by depth of individual words, but in terms primarily of superior noun retention and its interaction with serial position effects. In addition, conditional recall probabilities for another type of phrase, a noun phrase with prenominal adjectives, indicate stronger relationships between adjectives and the noun than between successive adjectives. It is concluded that the Yngve grammar is deficient as a psycholinguistic model.     

Interpreting elliptical verb phrases

Elliptical verb phrases are anaphoric expressions whose correct interpretation depends on the exact form of the preceding text as well as its meaning. However, people are not very good at remembering surface details of what they read or hear, so how do they understand such expressions? One alternative to a linguistically based interpretation of elliptical verb phrases is to assign them meanings that are plausible, given general knowledge about the situation being described. In an experiment, subjects read passages in which context provided a plausible interpretation for an elliptical verb phrase that either was or was not at odds with the linguistically correct interpretation. There was a tendency for subjects to assign plausible, but incorrect, meanings to elliptical verb phrases. This tendency increased with the distance between the elliptical verb phrase and its antecedent. Incorrect interpretations were assigned slowly, and the speed of assignment increased with distance from the antecedent. These results suggest that people try to interpret elliptical verb phrases correctly, but that they are not always able to do so. Furthermore, they indicate that the intuition that elliptical verb phrases are easy to interpret is misleading. If text is to be comprehensible, speakers and writers must use such expressions with care.     

Novelty and processing demands in conceptual combination

In this article, we sought to isolate the processing demands of combining the concepts of modifier‐noun phrases from those of other language comprehension processes. Probe reaction time (RT) was used as an indication of the processing resources required for combining concepts. Phrase frequency (as measured by Google hit rates) was used as a metric of the degree of conceptual combination required for each phrase. Participants were asked to interpret modifier‐noun phrases using a sense‐nonsense decision (Experiment 1) and a phrase meaning access task (Experiment 2). Experiment 2 also used a lexical decision task to activate the word's individual meanings. Regression analyses for both experiments indicated that phrase frequency (indicating novelty) predicts a significant portion of the probe RT variance, such that low‐frequency phrases required more processing resources than high‐frequency phrases, when controlling for associative strength, word frequency, letter length, and lexical‐semantic activation. Overall, this study indicates that conceptual combination requires processing resources beyond those of other language processes.     

Verbal probabilities: An alternative approach

Previous studies of verbal probabilities have tried to place expressions like a chance, possible, and certain on 0–1 numerical probability scales. We ask instead, out of a range of outcomes, which outcome a verbal probability suggests. When, for instance, a sample of laptop batteries lasts from 1.5 to 3.5 hours, what is a certain and what is a possible duration? Experiment 1 showed that speakers associate certain with low values and possible with (unlikely) high or maximal values. In Experiment 2, this methodology was applied to several positive and negative verbal probability phrases, showing a preference for high rather than low or middle values in a distribution. Experiment 3 showed that such maxima are not universally described by large numbers. For instance, maximum speed is often described in terms of a small number of time units. What can (possibly) happen is accordingly sometimes described with very low and sometimes with very high values, depending upon focus of interest. Finally, participants in Experiment 4 were given the role of hearers rather than speakers and were asked to infer outcome ranges from verbal probabilities. Hearers appeared to be partly aware of speakers' tendencies to describe outcomes at the top of the range.     

The Directionality of Verbal Probability Expressions: Effects on Decisions,Predictions, and Probabilistic Reasoning

Verbal phrases denoting uncertainty are of two kinds: positive, suggesting the occurrence of a target outcome, and negative, drawing attention to its nonoccurrence (Teigen & Brun, 1995). This directionality is correlated with, but not identical to, high and low p values. Choice of phrase will in turn influence predictions and decisions. A treatment described as having “some possibility” of success will be recommended, as opposed to when it is described as “quite uncertain,” even if the probability of cure referred to by these two expressions is judged to be the same (Experiment 1). Individuals who formulate their chances of achieving a successful outcome in positive terms are supposed to make different decisions than individuals who use equivalent, but negatively formulated, phrases (Experiments 2 and 3). Finally, negative phrases lead to fewer conjunction errors in probabilistic reasoning than do positive phrases (Experiment 4). For instance, a combination of 2 “uncertain” outcomes is readily seen to be “very uncertain.” But positive phrases lead to fewer disjunction errors than do negative phrases. Thus verbal probabilistic phrases differ from numerical probabilities not primarily by being more “vague,” but by suggesting more clearly the kind of inferences that should be drawn.     

Posterior probabilities for a consensus ordering

In the situation where subjects independently rank order a fixed set of items, the idea of a consensus ordering of the items is defined and employed as a parameter in a class of probability models for rankings. In the context of such models, which generalize those of Mallows, posterior probabilities may be easily formed about the population consensus ordering. An example of rankings obtained by the Graduate Record Examination Board is presented to demonstrate the adequacy of these generalized Mallows' models for describing actual data sets of rankings and to illustrate convenient summaries of the posterior probabilities for the consensus ordering.The authors thank Leann Birch for permission to refer to her experiment on cracker preferences, and the Graduate Record Examination Board for permission to use primary data from the Pike Report in our example. We also thank the referees for helpful comments.     

Physical probabilities

A conception of probability as an irreducible feature of the physical world is outlined. Propensity analyses of probability are examined and rejected as both formally and conceptually inadequate. It is argued that probability is a non-dispositional property of trial-types; probabilities are attributed to outcomes as event-types. Brier's Rule in an objectivist guise is used to forge a connection between physical and subjective probabilities. In the light of this connection there are grounds for supposing physical probability to obey some standard set of axioms. However, there is no a priori reason why this should be the case.     

Understanding conditional probabilities

In two experiments, subjects were asked to judge whether the probability of A given B was greater than, equal to, or less than the probability of B given A for various events A and B. In addition, in Experiment 2, subjects were asked to estimate the conditional probabilities and also to calculate conditional probabilities from contingency data. For problems in which one conditional probability was objectively larger than the other, performance ranged from about 25–80% correct, depending on the nature of A and B. Changes in the wording of problems also affected performance, although less dramatically. Patterns of responses consistent with the existence of a causal bias in judging probabilities were observed with one of the wordings used but not with the other. Several features of the data suggest that a major source of error was the confusion between conditional and joint probabilities.     

Unknown probabilities

From a point of view like de Finetti's, what is the judgmental reality underlying the objectivistic claim that a physical magnitude X determines the objective probability that a hypothesis H is true? When you have definite conditional judgmental probabilities for H given the various unknown values of X, a plausible answer is sufficiency, i.e., invariance of those conditional probabilities as your probability distribution over the values of X varies. A different answer, in terms of conditional exchangeability, is offered for use when such definite conditional probabilities are absent.     

Predicting the directionality of probability words from their membership functions

Teigen and Brun have suggested that distinct from their numerical implications, most probability phrases are either positive or negative, in that they encourage one to think of reasons why the target event will or will not occur. We report two experiments testing our hypotheses that (a) the direction of a phrase can be predicted from properties of its membership function, and (b) this relation is invariant across contexts, and (c) —originally formulated by Teigen and Brun (1999)—that strong modifiers intensify phrase directionality. For each phrase, participants encoded membership functions by judging the degree to which it described the numerical probabilities 0.0, 0.1, …, 1.0, and also completed sentences including the target phrase. The types of reasons given in the sentence completion task were used to determine the phrase's directionality. The results support our hypotheses (a) and (b) regarding the relation between directionality and the membership functions, but we found only limited support for hypothesis (c) regarding the effects of modifiers on directionality. A secondary goal, to validate an efficient method of encoding membership functions, was also achieved. Copyright © 2003 John Wiley & Sons, Ltd.     

Verbal probabilities: Ambiguous, context-dependent, or both?

To investigate the communication value of verbal probabilistic phrases, like “possibly,” “probably,” and “perhaps,” three experiments were conducted. Subjects were asked to judge the degree of probability expressed by such phrases in different contexts: in sentences reflecting opinions on current events, in a medical discussion of treatment effectiveness, and in videotaped news reports. Judgments of degree of probability were performed in the first study on a 0–100% probability scale and in the other two on 7-point rating scales. Results indicated that different contexts influence the interpretation of probability terms and in many cases, but not always, lead to higher between-subject variability than when the terms are judged in isolation, presumably because the interpretation of probability terms tends to be correlated with the judges' personal opinions on the topics. Special communication problems arise from the fact that most people are not fully aware of the ambiguity of these phrases and underestimate the variability of such ratings in the general population. Miscommunication between experts and the general public was illustrated by answers to a questionnaire given to general medical practitioners and to parents of small children. The latter preferred numerical probabilities to words, but thinking from an individual-oriented perspective, they often misunderstood the intended statistical meanings.