A re‐examination of probability matching and rational choice

In a typical probability learning task participants are presented with a repeated choice between two response alternatives, one of which has a higher payoff probability than the other. Rational choice theory requires that participants should eventually allocate all their responses to the high‐payoff alternative, but previous research has found that people fail to maximize their payoffs. Instead, it is commonly observed that people match their response probabilities to the payoff probabilities. We report three experiments on this choice anomaly using a simple probability learning task in which participants were provided with (i) large financial incentives, (ii) meaningful and regular feedback, and (iii) extensive training. In each experiment large proportions of participants adopted the optimal response strategy and all three of the factors mentioned above contributed to this. The results are supportive of rational choice theory. Copyright © 2002 John Wiley & Sons, Ltd.     

High Level of Construal and Psychological Distance Reduce Melioration

We examined how level of construal and psychological distance affected performance in a task in which value of alternatives depended on frequency of choice. Melioration, a sub‐optimal choice strategy that fails to take into account such value changes, has been found to be prevalent and difficult to change. Participants repeatedly chose between two buttons to earn as many coins as possible. Button A earned N coins, button B earned N + 3 coins, and N was the number of A presses in the last five trials. In this situation, B has a higher immediate payoff, but because choosing A increases the return from both buttons for the next five trials, maximizing A choices is the optimal strategy. We predicted that because a global perspective is needed to appreciate the dependency of value on frequency of choice, frequency of choosing the globally maximizing alternative (A) would increase when participants apply a higher level of construal and assume a more psychologically distal perspective. Study 1 manipulated construal level through segmenting the sequence of trials into segments of different size, with bigger segments representing higher level of construal. In Study 2, participants played a practice game, and we manipulated the probability of playing the actual game. Both a higher level of construal and a lower probability of playing the actual game (higher distance) increased the number of A choices. These results suggest that a broad perspective that is induced by high level of construal or by psychological distancing may reduce sub‐optimal, meliorating choice. Copyright © 2014 John Wiley & Sons, Ltd.     

Eye Movements in Strategic Choice

In risky and other multiattribute choices, the process of choosing is well described by random walk or drift diffusion models in which evidence is accumulated over time to threshold. In strategic choices, level‐k and cognitive hierarchy models have been offered as accounts of the choice process, in which people simulate the choice processes of their opponents or partners. We recorded the eye movements in 2 × 2 symmetric games including dominance‐solvable games like prisoner's dilemma and asymmetric coordination games like stag hunt and hawk–dove. The evidence was most consistent with the accumulation of payoff differences over time: we found longer duration choices with more fixations when payoffs differences were more finely balanced, an emerging bias to gaze more at the payoffs for the action ultimately chosen, and that a simple count of transitions between payoffs—whether or not the comparison is strategically informative—was strongly associated with the final choice. The accumulator models do account for these strategic choice process measures, but the level‐k and cognitive hierarchy models do not. © 2015 The Authors. Journal of Behavioral Decision Making published by John Wiley & Sons Ltd.     

Leader‐driven distortion of probability and payoff information affects choices between risky prospects

Research indicating that decision makers often distort new information to favor nascent preferences has focused primarily on riskless choice rather than risky choice. In addition, the critical assumption that information distortion mediates the link between the initially preferred alternative and the final choice has not been tested in a compelling manner. In an experiment designed to fill these gaps, participants made six choices involving pairs of hypothetical three‐outcome monetary gambles. We manipulated initial preferences by varying the order in which gamble features were presented. Multilevel regressions indicated that participants distorted their evaluations of precise probabilities and payoffs in the direction of their emerging preferences and that they used their biased evaluations to update those preferences. As expected, information distortion mediated the effect of initial preferences on final choices and final preference strength. In a follow‐up experiment, we compared a standard measure of information distortion (based on comparisons to mean ratings in a no‐choice control task) to a more personalized measure (based on participants' own ratings in the control task) and found the mean‐based measure to be superior for the probability and payoff information in question. Other findings in Experiment 2 corroborated the results of Experiment 1. In both studies, the distortion of quantitative inputs in a simple task highlights the non‐normative circularity of the choice process. Copyright © 2010 John Wiley & Sons, Ltd.     

Rapid decisions from experience

In many everyday decisions, people quickly integrate noisy samples of information to form a preference among alternatives that offer uncertain rewards. Here, we investigated this decision process using the Flash Gambling Task (FGT), in which participants made a series of choices between a certain payoff and an uncertain alternative that produced a normal distribution of payoffs. For each choice, participants experienced the distribution of payoffs via rapid samples updated every 50 ms. We show that people can make these rapid decisions from experience and that the decision process is consistent with a sequential sampling process. Results also reveal a dissociation between these preferential decisions and equivalent perceptual decisions where participants had to determine which alternatives contained more dots on average. To account for this dissociation, we developed a sequential sampling rank-dependent utility model, which showed that participants in the FGT attended more to larger potential payoffs than participants in the perceptual task despite being given equivalent information. We discuss the implications of these findings in terms of computational models of preferential choice and a more complete understanding of experience-based decision making.     

Navigating through abstract decision spaces: Evaluating the role of state generalization in a dynamic decision-making task

Research on dynamic decision-making tasks, in which the payoffs associated with each choice vary with participants’ recent choice history, shows that humans have difficulty making long-term optimal choices in the presence of attractive immediate rewards. However, a number of recent studies have shown that simple cues providing information about the underlying state of the task environment may facilitate optimal responding. In this study, we examined the mechanism by which this state knowledge influences choice behavior. We examined the possibility that participants use state information in conjunction with changing payoffs to extrapolate payoffs in future states. We found support for this hypothesis in an experiment in which generalizations based on this state information worked to the benefit or detriment of task performance, depending on the task’s payoff structure.     

Indeterminism, Explanation, and Luck

I first adumbrate pertinent aspectsof Robert Kane's libertarian theory of free choice oraction and an objection of luck that has been levelledagainst the theory. I then consider Kane's recentresponses to this objection. To meet these responses,I argue that the view that undetermined choices (ofthe sort implied by Kane's theory) are a matter ofluck is associated with a view about actionexplanation, to wit: when Jones does A and hisdoing of A is undetermined, and when hiscounterpart, Jones*, in the nearest possibleworld in which the past and the laws are held constantuntil the moment of choice does B instead, thereis no explanation (deterministic or indeterministic)of the difference in outcome – Jones's A-ing butJones*'s B-ing – in terms of prior reasonsor motives of either agent. Absence of such anexplanation is one crucial factor that underliesthe charge that Jones's A-ing and Jones*'sB-ing are matters of luck. I argue that thissort of luck is incompatible with responsibility.     

Temporal Discounting Mechanisms of Future‐oriented Coping: Evidence from Delay Discounting and Task Prioritization Paradigms

Four studies were conducted to examine the relationship between future‐oriented coping and temporal discounting under different situational conditions. In Study 1, 138 participants were primed with either stressful or neutral stimuli, followed by a delay‐discounting task. In Study 2, 118 participants were primed with either stressful or neutral stimuli, followed by a task‐prioritization activity. The results of both studies indicated that future‐oriented coping had a significant negative association with temporal discounting or the number of rational choices in the neutral‐priming condition, but the relationship was not significant in the stress‐priming condition. In Study 3, qualitative data revealed that the major reason for shifting choices from larger but later payoffs to smaller but sooner rewards in a stressful condition was to reduce the stressful mood, create a positive mood, and promote a sense of accomplishment. This explanation was corroborated by Study 4, in which one group was allowed to choose an immediate payoff and the other group was blocked from choosing that immediate payoff. We confirmed that post‐test anxiety was significantly lower in the immediate payoff group compared with the delayed‐payoff group in proactive and preventive coping, using pre‐test anxiety as a covariate. Preventive coping helped to reduce anxiety levels in a stressful condition only when there was a choice to obtain an immediate payoff. These findings underscore the relationship between future‐oriented coping and temporal discounting, as well as the flexibility of discounting in the face of stress. Copyright © 2015 John Wiley & Sons, Ltd.     

PIGEONS (COLUMBA LIVIA) APPROACH NASH EQUILIBRIUM IN EXPERIMENTAL MATCHING PENNIES COMPETITIONS

The game of Matching Pennies (MP), a simplified version of the more popular Rock, Papers, Scissors, schematically represents competitions between organisms with incentives to predict each other's behavior. Optimal performance in iterated MP competitions involves the production of random choice patterns and the detection of nonrandomness in the opponent's choices. The purpose of this study was to replicate systematic deviations from optimal choice observed in humans when playing MP, and to establish whether suboptimal performance was better described by a modified linear learning model or by a more cognitively sophisticated reinforcement‐tracking model. Two pairs of pigeons played iterated MP competitions; payoffs for successful choices (e.g., “Rock” vs. “Scissors”) varied within experimental sessions and across experimental conditions, and were signaled by visual stimuli. Pigeons' behavior adjusted to payoff matrices; divergences from optimal play were analogous to those usually demonstrated by humans, except for the tendency of pigeons to persist on prior choices. Suboptimal play was well characterized by a linear learning model of the kind widely used to describe human performance. This linear learning model may thus serve as default account of competitive performance against which the imputation of cognitively sophisticated processes can be evaluated.     

Evaluating the reliance on past choices in adaptive learning models

Adaptive learning models are used to predict behavior in repeated choice tasks. Predictions can be based on previous payoffs or previous choices of the player. The current paper proposes a new method for evaluating the degree of reliance on past choices, called equal payoff series extraction (EPSE). Under this method a simulated player has the same exact choices as the player but receives equal constant payoffs from all of the alternatives. Success in predicting the next choice ahead for this simulated player therefore relies strictly on mimicry of previous choices of the actual player. This allows determining the marginal fit of predictions that are not based on the actual task payoffs. To evaluate the reliance on past choices under different models, an experiment was conducted in which 48 participants completed a three-alternative choice task in four task conditions. Two different learning rules were evaluated: an interference rule and a decay rule. The results showed that while the predictions of the decay rule relied more on past choices, only the reliance on past payoffs was associated with improved parameter generality. Moreover, we show that the Equal Payoff Series can be used as a criterion for optimizing parameters resulting in better parameter generalizability.     

Action Dynamics in Intertemporal Choice Reveal Different Facets of Decision Process

Two experiments were conducted to examine decision process in intertemporal choice with spatial and temporal action dynamic measures generated from cursor movement. The effects of the context valence (gain and loss) and magnitude of the payoffs (small and large) were investigated. Other factors examined were differences between decisions made at versus away from the indifference point and response variability dependent on what was selected (delayed versus non‐delayed/less‐delayed payoff). Using principal components analysis, decision process is described with three orthogonal components. General decision difficulty is encompassed by two components: (1) Conflict, depicted by Idle time and deviations from a direct choice path, and (2) decision uncertainty or Wavering, described by left‐right directional flips. A general Locomotion factor was also present. In Experiment 1, Conflict was best at capturing the gain/loss and magnitude context effects. Greater Conflict was observed in the loss context and the small magnitude condition. When choices were closer to the indifference point, Wavering was most salient. In Experiment 2, when selecting larger, delayed payoffs, both Conflict and Wavering increased suggesting that controlling the temptation of selecting the sooner, smaller reward entails effort. By analyzing trajectories, the study advances knowledge of the construct validity of different action dynamic measures and supports the distinction of decision uncertainty and decision conflict. Copyright © 2015 John Wiley & Sons, Ltd.     

Is everyday causation deterministic or probabilistic?

One view of causation is deterministic: A causes B means that whenever A occurs, B occurs. An alternative view is that causation is probabilistic: the assertion means that given A, the probability of B is greater than some criterion, such as the probability of B given not-A. Evidence about the induction of causal relations cannot readily decide between these alternative accounts, and so we examined how people refute causal assertions. In four experiments most participants judged that a single counterexample of A and not-B refuted assertions of the form, A causes B. And, as a deterministic theory based on mental models predicted, participants were more likely to request multiple refutations for assertions of the form, A enables B. Similarly, refutations of the form not-A and B were more frequent for enabling than causal assertions. Causation in daily life seems to be a deterministic concept.     

Allocation of effort as a function of payoffs for individual tasks in a multitasking environment

SYNWORK1 software allows the examination of how payoffs affect the allocation of effort by people when they perform four different tasks (memory search, arithmetic, visual monitoring, and auditory monitoring). In a previous study (Wang, Proctor, & Pick, 2007), we showed that participants adopted multitasking strategies allocating relative effort appropriate to payoff differences between the arithmetic and memory tasks, but that they exhibited residual effects of prior payoffs when the payoffs were switched. In the present study, we varied the payoff in two different experiments for only one of these tasks, the memory task in Experiment 1 and the arithmetic task in Experiment 2. Doing this allowed consideration of performance for both the task for which payoff changed explicitly and the other cognitive task, for which the payoff difference was implicit (i.e., relative to the explicit payoff that was manipulated). Although participants adjusted performance on the task for which the payoff explicitly varied, the payoff manipulation had less effect than did the explicit payoff manipulations for both tasks used previously. Also, the change in effort on a task resulting from explicitly increasing its payoff was less than that from decreasing the payoff. SYNWORK1 is a good environment for studying multitasking, but has several limitations that need to be addressed to provide a synthetic work environment that allows investigation of a wider range of theoretically relevant issues.     

On adaptation, maximization, and reinforcement learning among cognitive strategies

Analysis of binary choice behavior in iterated tasks with immediate feedback reveals robust deviations from maximization that can be described as indications of 3 effects: (a) a payoff variability effect, in which high payoff variability seems to move choice behavior toward random choice; (b) underweighting of rare events, in which alternatives that yield the best payoffs most of the time are attractive even when they are associated with a lower expected return; and (c) loss aversion, in which alternatives that minimize the probability of losses can be more attractive than those that maximize expected payoffs. The results are closer to probability matching than to maximization. Best approximation is provided with a model of reinforcement learning among cognitive strategies (RELACS). This model captures the 3 deviations, the learning curves, and the effect of information on uncertainty avoidance. It outperforms other models in fitting the data and in predicting behavior in other experiments.     

Effective choice in all the symmetric 2 × 2 games

Solution concepts for game play implicitly assume what information is relevant for choosing moves. Maximin and Eliminating Dominated Strategies use payoff order; mixed strategies and Harsanyi??s risk dominance use payoff values. ??Cooperative?? strategies use previous choices, ignoring payoffs altogether. We first define the 12 symmetric 2 × 2 games as a continuous payoff space then use this space to evaluate strategies based on different types of information. Strategic success is shown to be sensitive to actual payoffs and seldom consistent for all games of an ordinally defined category such as Chicken. A new hybrid strategy using both payoff and choice information is introduced and shown to produce efficient outcomes in all symmetric 2 × 2 games when employed by both players. It does well in evolutionary tournaments against strategies of other types, especially in the difficult distribution games including Battle-of-the-Sexes.     

Too good to be true? Psychological responses to uncommon options in risk–reward environments

Risks and rewards, or payoffs and probabilities, are inversely related in many choice environments. We investigated people's psychological responses to uncommon combinations of risk and reward that deviate from learned regularities (e.g., options that offer a high payoff with an unusually high probability) as they evaluated risky options. In two experiments (N = 183), participants first priced monetary gambles drawn from environments in which risks and rewards were negatively correlated, positively correlated, or uncorrelated. In later trials, they evaluated gambles with uncommon combinations of risk and reward—that is, options that deviated from the respective environment's risk–reward structure. Pricing, response times, and (in Experiment 2) pupil dilation were recorded. In both experiments, participants took more time when responding to uncommon compared to foreseeable options or when the same options were presented in an uncorrelated risk–reward environment. This result was most pronounced when the uncommon gambles offered higher expected values compared to the other gambles in the set. Moreover, these uncommon, high‐value options were associated with an increase in pupil size. These results suggest that people's evaluations of risky options are based not only on the options' payoffs and probabilities but also on the extent to which they fit the risk–reward structure of the environment.     

Delay discounting by pathological gamblers

Discounting of delayed rewards by pathological gamblers was compared to discounting of delayed rewards by matched control nongambling participants. All participants completed a hypothetical choice task in which they made repeated choices between dollars 1,000 available after a delay and an equal or lesser amount of money available immediately. The delay to the large amount of money was varied from 1 week to 10 years across conditions. Indifference points between immediate money and delayed money were identified at each delay condition by varying the amount of immediate money across choice trials. For the majority of participants, indifference points decreased monotonically across delays. Overall, gamblers discounted the delayed rewards more steeply than did control participants.     

RISKY CHOICE IN PIGEONS AND HUMANS: A CROSS‐SPECIES COMPARISON

Pigeon and human subjects were given repeated choices between variable and adjusting delays to token reinforcement that titrated in relation to a subject's recent choice patterns. Indifference curves were generated under two different procedures: immediate exchange, in which a token earned during each trial was exchanged immediately for access to the terminal reinforcer (food for pigeons, video clips for humans), and delayed exchange, in which tokens accumulated and were exchanged after 11 trials. The former was designed as an analogue of procedures typically used with nonhuman subjects, the latter as an analogue to procedures typically used with human participants. Under both procedure types, different variable‐delay schedules were manipulated systematically across conditions in ways that altered the reinforcer immediacy of the risky option. Under immediate‐exchange conditions, both humans and pigeons consistently preferred the variable delay, and indifference points were generally ordered in relation to relative reinforcer immediacies. Such risk sensitivity was greatly reduced under delayed‐exchange conditions. Choice and trial‐initiation response latencies varied directly with indifference points, suggesting that local analyses may provide useful ancillary measures of reinforcer value. On the whole, the results indicate that modifying procedural features brings choices of pigeons and humans into better accord, and that human—nonhuman differences on risky choice procedures reported in the literature may be at least partly a product of procedural differences.