Gambling for Gatorade: risk-sensitive decision making for fluid rewards in humans

Determining how both humans and animals make decisions in risky situations is a central problem in economics, experimental psychology, behavioral economics, and neurobiology. Typically, humans are risk seeking for gains and risk averse for losses, while animals may display a variety of preferences under risk depending on, amongst other factors, internal state. Such differences in behavior may reflect major cognitive and cultural differences or they may reflect differences in the way risk sensitivity is probed in humans and animals. Notably, in most studies humans make one or a few choices amongst hypothetical or real monetary options, while animals make dozens of repeated choices amongst options offering primary rewards like food or drink. To address this issue, we probed risk-sensitive decision making in human participants using a paradigm modeled on animal studies, in which rewards were either small squirts of Gatorade or small amounts of real money. Possible outcomes and their probabilities were not made explicit in either case. We found that individual patterns of decision making were strikingly similar for both juice and for money, both in overall risk preferences and in trial-to-trial effects of reward outcome on choice. Comparison with decisions made by monkeys for juice in a similar task revealed highly similar gambling styles. These results unite known patterns of risk-sensitive decision making in human and nonhuman primates and suggest that factors such as the way a decision is framed or internal state may underlie observed variation in risk preferences between and within species.     

A discounting framework for choice with delayed and probabilistic rewards

When choosing between delayed or uncertain outcomes, individuals discount the value of such outcomes on the basis of the expected time to or the likelihood of their occurrence. In an integrative review of the expanding experimental literature on discounting, the authors show that although the same form of hyperbola-like function describes discounting of both delayed and probabilistic outcomes, a variety of recent findings are inconsistent with a single-process account. The authors also review studies that compare discounting in different populations and discuss the theoretical and practical implications of the findings. The present effort illustrates the value of studying choice involving both delayed and probabilistic outcomes within a general discounting framework that uses similar experimental procedures and a common analytical approach.     

A learning rule that explains how rewards teach attention

Many theories propose that top-down attentional signals control processing in sensory cortices by modulating neural activity. But who controls the controller? Here we investigate how a biologically plausible neural reinforcement learning scheme can create higher order representations and top-down attentional signals. The learning scheme trains neural networks using two factors that gate Hebbian plasticity: (1) an attentional feedback signal from the response-selection stage to earlier processing levels; and (2) a globally available neuromodulator that encodes the reward prediction error. We demonstrate how the neural network learns to direct attention to one of two coloured stimuli that are arranged in a rank-order. Like monkeys trained on this task, the network develops units that are tuned to the rank-order of the colours and it generalizes this newly learned rule to previously unseen colour combinations. These results provide new insight into how individuals can learn to control attention as a function of reward contingency.     

Driver performance, rewards and motivation: A simulator study

It has been suggested that effort invested in a task may be moderated by the perception of the nature of reward offered, e.g., whether social or financial in nature. If levels of payment for empirical work meaningfully influence the effort invested in tasks by participants, the implications may be serious and wide-ranging for the reliability and validity of published data. The study reported in this paper, examines this thesis in a driving context. Thirty-six participants were allocated to either a No payment, Low payment or Medium payment condition in a driving simulator. They undertook both easy and difficult driving scenarios in which performance was measured. Subjective workload and performance results indicated that task difficulty was successfully manipulated with the study. However, in contrast to the previous research in non-driving contexts, participants not receiving rewards were found to perform more poorly and experience increased time pressure.     

Preference for unpredictable food rewards occurs with high proportion of reinforced trials or alcohol if rewards are not delayed

Organisms typically prefer situations where reward and nonreward are predictable rather than unpredictable. Although many theories can account for this result (e.g., information theory and delay-reduction theory), a recently developed mathematical model (DMOD) also predicts that subjects prefer the unpredictable reward situation under conditions that substantially decrease aversiveness of unpredictable nonreward (Daly & Daly, 1982). Because a high proportion of reinforced trials (lenient schedule) and alcohol injections decrease aversive conditioning, these variables were tested with rats in five E-maze experiments. A choice to one side of the maze resulted in a stimulus uncorrelated with reward outcome (unpredictable situation). A choice to the other side resulted in stimuli correlated with reward and nonreward (predictable situation). The stimuli were not visible until after the choice was made. A lenient reinforcement schedule resulted in preference for the unpredictable reward situation if rewards were not delayed. Alcohol resulted in preference for the unpredictable reward situation if a medium five-pellet reward was given. A lenient reinforcement schedule combined with an alcohol injection resulted in faster acquisition of the preference for the unpredictable reward situation than did a lenient schedule combined with a saline control injection. These results pose a major challenge to most theories, yet were predicted by DMOD.     

A portable biofeedback device for autonomic responses

The design of a versatile and programmable transducer amplifier device with analogue display for self-monitoring of autonomic responses is described. The design features low cost, portability, and flexibility across direct-current transducer options (e.g., photoplethysmograph or thermistor). The device can be used for the visual or auditory display of continuous blood volume pulse or temperature measures where the relative amplitude or pulse rate is of concern. Auditory or visual biofeedback may be provided via the choice of a stacked bar-graph display or piezoelectric buzzer. A common circuit design to allow programming options for the estimation of heart rate, inter-beat interval, or pulse duration is provided.     

A smoke-mixing device for measuring nicotine preference

A smoke-mixing device is described that allows individuals to select their preferred nicotine deliveries from cigarette smoke on a puff-by-puff basis. This methodology offers advantages over previous strategies to investigate the role of nicotine as a reinforcing constituent of tobacco.