Evaluating the reliance on past choices in adaptive learning models

Adaptive learning models are used to predict behavior in repeated choice tasks. Predictions can be based on previous payoffs or previous choices of the player. The current paper proposes a new method for evaluating the degree of reliance on past choices, called equal payoff series extraction (EPSE). Under this method a simulated player has the same exact choices as the player but receives equal constant payoffs from all of the alternatives. Success in predicting the next choice ahead for this simulated player therefore relies strictly on mimicry of previous choices of the actual player. This allows determining the marginal fit of predictions that are not based on the actual task payoffs. To evaluate the reliance on past choices under different models, an experiment was conducted in which 48 participants completed a three-alternative choice task in four task conditions. Two different learning rules were evaluated: an interference rule and a decay rule. The results showed that while the predictions of the decay rule relied more on past choices, only the reliance on past payoffs was associated with improved parameter generality. Moreover, we show that the Equal Payoff Series can be used as a criterion for optimizing parameters resulting in better parameter generalizability.     

Doomed to repeat the successes of the past: History is best forgotten for repeated choices with nonstationary payoffs

Many everyday tasks involve repeated choices in which past outcomes are used to estimate payoffs but in which present payoffs may differ from past ones. Two experiments with 10 decision problems employing the decisions-from-feedback paradigm examined the choice between two risky options, wherein the payoff probabilities for one option could change over a sequence of trials. Participants either saw the outcomes associated with each option, or additionally were given a “history” summarizing the outcomes of previous trials. Participants adapted quickly to new problems, but adapted slowly to payoff changes. Providing a history improved initial choices, but had a null or negative impact on later ones—although, appropriately, the summary received less weight in later trials. An associative choice model captured changes in preference, but not initial patterns of choice. The findings emphasize the adaptive value of forgetting in unstable decision environments, but illustrate how providing additional relevant information may hinder this.     

Maximizing Scales Do Not Reliably Predict Maximizing Behavior in Decisions from Experience

In this paper, we explore the relationships between psychometric and behavioral measures of maximization in decisions from experience (DfE). In two experiments, we measured choice behavior in two experimental paradigms of DfE and self‐reported maximizing tendencies using three prominent scales of maximization. In the repeated consequentialist choice paradigm, participants made repeated choices between two unlabeled options and received consequential feedback on each trial. In the sampling paradigm, participants freely sampled from two options and received feedback on their sampling before making a single consequential choice. Individuals exhibited different degrees of maximizing behavior in both paradigms and across different payoff distributions, but none of the maximizing scales predicted this behavior. These results indicate that maximization scales address constructs that are different from the maximization behavior observed in DfE, and that these measures will need to be improved to reflect behavioral aspects of choice and search from experience. Copyright © 2017 John Wiley & Sons, Ltd.     

Navigating through abstract decision spaces: Evaluating the role of state generalization in a dynamic decision-making task

Research on dynamic decision-making tasks, in which the payoffs associated with each choice vary with participants’ recent choice history, shows that humans have difficulty making long-term optimal choices in the presence of attractive immediate rewards. However, a number of recent studies have shown that simple cues providing information about the underlying state of the task environment may facilitate optimal responding. In this study, we examined the mechanism by which this state knowledge influences choice behavior. We examined the possibility that participants use state information in conjunction with changing payoffs to extrapolate payoffs in future states. We found support for this hypothesis in an experiment in which generalizations based on this state information worked to the benefit or detriment of task performance, depending on the task’s payoff structure.     

Do consumers ever learn? Analysis of segment behavior in experimental markets

Economists argue that, despite cognitive limitations, economic agents arrive at optimal choice rules by learning. The assumption is that consumers, for example, are adaptively rational. Adaptive rationality raises a host of issues. We address three of these in the context of experimental markets: do consumers differ on the basis of learning; how do these differences, when aggregated, affect market efficiency; and how do consumers learn? Analysis of our experimental data reveals the following. First, multiple segments of consumers exist on the basis of learning. Second, the largest segment consists of subjects who do not learn despite timely feedback and motivation. Third, although some consumers do learn to make optimal choices, the effect of this segment on market efficiency is cancelled by an equal number of subjects who ‘learn' false relations. Finally, although subjects do not learn strict rationality even with experience, they are in the aggregate not so irrational as to allow highly suboptimal brands to survive. Further analysis of how consumers learn, specifically on the cues (signals) and the rules consumers employ in making choices over time leads to the following two conclusions. First, some signals make learning more easy than others: for example, providing market share information improves learning but not as much as providing quality information does. Second, people employ different rules depending upon the type of information they have. For example, consumers making decisions based only on price information are more likely to use a heuristic like ‘buy a medium‐priced product provided it has not failed in the past'. Consumers making decisions based on price and quality information may employ a heuristic such as ‘buy top quality products regardless of price'. We discuss the implications of these findings for theory and practice. Copyright © 2000 John Wiley & Sons, Ltd.     

Decision making under uncertainty: a comparison of simple scalability, fixed-sample, and sequential-sampling models

The purpose of this article is to investigate the learning and memory processes involved in decision making under uncertainty. In two different experiments, subjects were given a choice between a certain alternative that produced a single known payoff and an uncertain alternative that produced a normal distribution of payoffs. Initially this distribution was unknown, and in the first experiment it was learned through feedback from past decisions, whereas in the second experiment it was learned by observing sample outcomes. In the first experiment, a response deadline was used to limit the amount of time available for making a decision. In the second experiment, an observation cost was used to limit the number of samples that could be purchased. The mean and variance of the uncertain alternative and the value of the certain alternative were factorially manipulated to study their joint effects on choice probability, choice response time (Experiment 1), and number of observations purchased (Experiment 2). Algebraic-deterministic theories developed for decision making with simple gambles fail to explain the present results. Two new models are developed and tested--fixed- and sequential-sampling models--that attempt to describe the learning and memory processes involved in decision making under uncertainty.     

Effects of problem frame and gender on principals’ decision making

Research indicates people’s decisions can sometimes be influenced by seemingly trivial differences in the framing (i.e., wording) of alternative options. The tendency to prefer risk averse options when framed positively and risky options when framed negatively is known as the framing effect. The current study examined the susceptibility of school principals to the framing effect. Additionally, analytical and intuitive decision styles, the degree to which one’s typical goal is to maximize (rather than satisfice), gender, and years of experience as a principal were measured to assess whether they are predictive of principals’ choices, and to test whether they moderate the effects of framing on choice. Seventy-one principals completed six decision problems (framed either positively or negatively) and instruments assessing decision style, typical decision goal, gender, and experience. Analyses demonstrated that principals are influenced by framing. Although the positively and negatively framed versions of the decision problems were objectively identical, negative framing resulted in more risky choices. Additionally, regardless of frame, men made more risky choices than women. There was no evidence that experience, decision style, or the degree to which one’s typical decision goal was to maximize, decreased framing effects. Several potential debiasing strategies are described, and limitations are noted.     

The effect of foregone outcomes on choices from experience

We examined the relative weight given to obtained and foregone outcomes (i.e., outcomes from the non-chosen options) in repeated choices using cognitive modeling. Previous modeling studies have yielded mixed results. When participants' choices are analyzed by models that predict the next choice ahead in a sequence of decisions, the results imply that people give less weight to foregone than to obtained outcomes. In contrast, in simulation models of n trials ahead, the results imply that, on average, people give equal weight to foregone and obtained outcomes. Using datasets of experience-based binary choices with fixed (stationary) payoff distributions (Erev & Haruvy, in press) and dynamic (nonstationary) payoff distributions (Rakow & Miler, 2009), we employed generalization tests at the individual level to examine whether the findings derived from the one-step-ahead method are due to overfitting. The results of trial-ahead model fitting implied that for the nonstationary tasks only, foregone outcomes received lower weight. However, when this dataset was assessed via generalization criteria at the individual level, equal weighting of foregone and obtained outcomes was the best assumption. This implies that overfitting is implicated in the superior fit of models that assume discounting of foregone outcomes.     

The gap between medical and monetary choices under risk persists in decisions for others

Decisions under risk in the medical domain have been found to systematically diverge from decisions in the monetary domain. When making choices between monetary options, people commonly rely on a decision strategy that trades off outcomes with their probabilities; when making choices between medical options, people tend to neglect probability information. In two experimental studies, we tested to what extent differences between medical and monetary decisions also emerge when the decision outcomes affect another person. Using a risky choice paradigm for medical and monetary decisions, we compared hypothetical decisions that participants made for themselves to decisions for a socially distant other (Study 1) and to recommendations as financial advisor or doctor (Study 2). In addition, we examined people's information search in a condition in which information about payoff distributions had to be learned from experiential sampling. Formal modeling and analyses of search behavior revealed a similarly pronounced gap between medical and monetary decisions in decisions for others as in decisions for oneself. Our results suggest that when making medical decisions, people try to avoid the worst outcome while neglecting its probability—even when the outcomes affect others rather than themselves.     

Rapid decisions from experience

In many everyday decisions, people quickly integrate noisy samples of information to form a preference among alternatives that offer uncertain rewards. Here, we investigated this decision process using the Flash Gambling Task (FGT), in which participants made a series of choices between a certain payoff and an uncertain alternative that produced a normal distribution of payoffs. For each choice, participants experienced the distribution of payoffs via rapid samples updated every 50 ms. We show that people can make these rapid decisions from experience and that the decision process is consistent with a sequential sampling process. Results also reveal a dissociation between these preferential decisions and equivalent perceptual decisions where participants had to determine which alternatives contained more dots on average. To account for this dissociation, we developed a sequential sampling rank-dependent utility model, which showed that participants in the FGT attended more to larger potential payoffs than participants in the perceptual task despite being given equivalent information. We discuss the implications of these findings in terms of computational models of preferential choice and a more complete understanding of experience-based decision making.     

Decisions from experience: Why small samples?

In many decisions we cannot consult explicit statistics telling us about the risks involved in our actions. In lieu of such data, we can arrive at an understanding of our dicey options by sampling from them. The size of the samples that we take determines, ceteris paribus, how good our choices will be. Studies of decisions from experience have observed that people tend to rely on relatively small samples from payoff distributions, and small samples are at times rendered even smaller because of recency. We suggest one contributing and previously unnoticed reason for reliance on frugal search: Small samples amplify the difference between the expected earnings associated with the payoff distributions, thus making the options more distinct and choice easier. We describe the magnitude of this amplification effect, and the potential costs that it exacts, and we empirically test four of its implications.     

Loss aversion,diminishing sensitivity,and the effect of experience on repeated decisions

Three experiments are presented that explore the assertion that loss aversion and diminishing sensitivity drive the effect of experience on choice behavior. The experiments are focused on repeated choice tasks where decision makers choose repeatedly between alternatives and get feedback after each choice. Experiments 1a and 1b show that behavioral tendencies that were previously interpreted as indications of loss aversion in decisions from experience are better described as products of diminishing sensitivity to absolute payoffs. Experiment 2 highlights a nominal magnitude effect: A decrease in the magnitude of the nominal payoffs eliminates the evidence for diminishing sensitivity. These and related previous results can be captured with a model that assumes reliance on small samples of subjective experiences, and an increase in diminishing sensitivity with payoff variability. Copyright © 2008 John Wiley & Sons, Ltd.     

Risk taking in achievement-oriented situations: Do people really maximize affect or competence information?

Several studies of choice behavior (risk taking) in achievement-oriented situations are reanalyzed. The usual ways of pooling all choices over trials and subjects conceal the series of subjects' decisions and the dynamics inherent in these decisions. A basic strategy of subjects in an achievement-oriented choice situation seems to be to start with an easy task, choose a more difficult one whenever you succeed, and stay mostly at the same difficulty level whenever you fail. A computer model, in which such simple assumptions are made, generates preference functions over the order of difficulty levels that are indistinguishable from those found in empirical studies. It is concluded that the study of choice behavior in achievement-oriented situations should be based on the analysis of the series of single decisions by one subject. For this we need models that allow the predictions of such decisions and the prediction of action-controlling cognitions and emotions.     

Copying without rewards: socially influenced foraging decisions among brown capuchin monkeys

An individual’s foraging activity can be influenced by the choices made by nearby conspecifics. The interest shown in the location and characteristics of a feeding patch may depend on the feeding success of a conspecific there, a process that needs to be distinguished from choices guided by rewards to the observer itself. We investigated how rewards for both self and others influence the foraging choices of captive capuchin monkeys (Cebus apella). Thirteen adult capuchins observed familiar female conspecific models explore one of three opaque boxes under three conditions. In the first, there were no rewards available to either monkey; in the second, rewards were available to the model only; and in the third, both monkeys could retrieve a reward. Under all conditions, subjects more often explored the same box as the model than was expected by chance. Thus, without ever receiving a reward themselves or without seeing another receive rewards, subjects’ searches were directed at the box explored by another monkey. The tendency to match the model’s choice increased if the subject was rewarded. We compared these results to control conditions in which the model was either absent, or present but not allowed to demonstrate. Subjects’ located the reward less often in control conditions, than in the experimental conditions. We conclude that extrinsic rewards, while helpful, are not required for partners to influence the foraging choices of capuchins, and that the unrewarded copying of foraging choices demonstrated here may provide the basis for additional social influences on learning. This contribution is part of the Special Issue “A Socioecological Perspective on Primate Cognition” (Cunningham and Janson 2007)     

A re‐examination of melioration and rational choice

We examined how people allocate choices between two alternatives when the payoff from each alternative varied as a function of the allocation of recent choices. On any one trial alternative A had a higher immediate payoff than alternative B, but across all trials B had a higher overall payoff than A. Rational choice theory requires that participants allocate all their responses to the alternative with the greatest overall payoff irrespective of which has the higher immediate payoff. Melioration, in contrast, proposes that participants are motivated to choose the alternative with the higher immediate payoff, irrespective of the consequences for future returns. We report four experiments in which we varied the nature of the payoffs. Participants exhibited self‐control consistent with rational choice theory when payoffs varied in magnitude, but exhibited impulsiveness consistent with melioration when the payoffs varied in probability. Finally, we show that impulsivity when payoffs varied in probability can be overcome following un‐reinforced practice. Copyright © 2002 John Wiley & Sons, Ltd.