Self-control and choice in humans: Effects of video game playing as a positive reinforcer

Participants chose between reinforcement schedules differing in delay of reinforcement (interval between a choice response and onset of a video game) and/or amount of reinforcement (duration of access to the game). Experiment 1 showed that immediate reinforcement was preferred to delayed reinforcement with amount of reinforcement held constant, and a large reinforcer was preferred to a small reinforcer when both were obtainable immediately. Imposing a delay before the large reinforcer produced a preference for the immediate, small reinforcer in 40% of participants. This suggested a limited degree of “impulsivity.” In Experiment 2, unequal delays were extended by equal intervals, the amounts being kept equal. Preference for the shorter delay decreased, an effect that presumably makes possible the “preference reversal” phenomenon in studies of self-control. Overall, the results demonstrate that video game playing can produce useful, systematic data when used as a positive reinforcer for choice behavior in humans.     

The effect of number of options on choices involving delayed causation

A video game was adapted to investigate the effect of the number of alternatives on causal judgment in a complex environment involving targets with delayed outcomes. Participants were presented with groups of potential targets. Each target (the candidate cause) fired at random relative to the others, with one target in each group causing a delayed explosion (the effect).The participants were tasked with discriminating which of the potential targets was producing the effect. Experiment 1 revealed a main effect of delay on discriminability but no effect of the number of alternatives. However, latencies did increase as the number of alternatives increased, suggesting that discriminability was maintained by compensating with longer observation times. But, a similar increase in latency for longer delays did not offset the detrimental impact of an increase in cause-effect delay. Experiment 2 replicated the outcome under conditions in which participants were always required to observe for 16 s before making a decision; under the most difficult conditions, however, the limited viewing time started to reveal an effect of the number of alternatives on the participants' ability to discriminate a true cause from foils. Evidence suggests that participants' decisions were determined primarily by differences in the number of cause-effect contiguities experienced for the true cause versus the foils.     

Developmental continuity in the processes that underlie spatial recall

This study investigated whether children's spatial recall performance shows three separable characteristics: (1) biases away from symmetry axes (geometric effects); (2) systematic drift over delays; and (3) biases toward the exemplar distribution experienced in the task (experience-dependent effects). In Experiment 1, the location of one target within each geometric category was varied. Children's responses showed biases away from a midline axis that increased over delays. In Experiment 2, multiple targets were placed within each category at the same locations used in Experiment 1. After removing geometric effects, 6-year-olds'--but not 11-year-olds'--responses were biased toward the average remembered location over learning. In Experiment 3, children responded to one target more frequently than the others. Both 6- and 11-year-olds showed biases toward the most frequent target over learning. These results provide a bridge between the performance of younger children and adults, demonstrating continuity in the processes that underlie spatial memory abilities across development.     

Alan Baron: A pioneer in translational science

Primates take longer to choose between alternatives with smaller differences in value. This effect—a particular instance of the distance effect in symbolic comparisons—has not been replicated in birds. Instead, birds appear to respond independently to each alternative, such that the latency to choose depends primarily on the alternative of highest value. Three experiments tested for the distance effect in pigeons under conditions not previously considered. Experiment 1 presented pigeons with forced‐ and binary free‐choice trials, where each alternative was one of three possible delays to reinforcement (4, 8, and 16 s). Pigeons were exposed to the choice stimuli for different amounts of time and with different sample response requirements prior to the choice response. Experiment 2 added a fourth (0‐s delay) alternative. Experiment 3 substituted the 16‐s delay with a second 4‐s delay. In all experiments, pigeons systematically chose the shortest delay to reinforcement. Latency to choose the 4‐s delay did not vary when choosing against the 8‐s or 16‐s delay, regardless of whether choice stimuli were exposed for the duration of nine pecks (Experiment 1), or whether a 0‐s delay alternative was sometimes present (Experiment 2). Latency to choose the preferred of two identical alternatives (4‐s vs. 4‐s) was shorter than the latency to choose between different alternatives (4‐s vs. 8‐s; Experiment 3); this is the opposite of a distance effect. These results show no evidence of a distance effect in pigeon choice, consistent with the hypothesis that pigeons respond independently to each choice alternative.     

Effects of signaled versus unsignaled delay of reinforcement on choice

Pigeons chose between 5-s and 15-s delay-of-reinforcement alternatives. The first key peck to satisfy the choice schedule began a delay timer, and food was delivered at the end of the interval. Key pecks during the delay interval were measured, but had no scheduled effect. In Experiment 1, signal conditions and choice schedules were varied across conditions. During unsignaled conditions, no stimulus change signaled the beginning of a delay interval. During differential and nondifferential signal conditions, offset of the choice stimuli and onset of a delay stimulus signaled the beginning of a delay interval. During differential signal conditions, different stimuli were correlated with the 5-s and 15-s delays, whereas the same stimulus appeared during both delay durations during nondifferential signal conditions. Pigeons showed similar, extreme levels of preference for the 5-s delay alternative during unsignaled and differentially signaled conditions. Preference levels were reliably lower with nondifferential signals. Experiment 2 assessed preference with two pairs of unsignaled delays in which the ratio of delays was held constant but the absolute duration was increased fourfold. No effect of absolute duration was found. The results highlight the importance of delayed primary reinforcement effects and challenge models of choice that focus solely on conditioned reinforcement.     

Associations between a one-shot delay discounting measure and age, income, education and real-world impulsive behavior

There has been discussion over the extent to which delay discounting – as prototypically shown by a preference for a smaller-sooner sum of money over a larger-later sum – measures the same kind of impulsive preferences that drive non-financial behavior. To address this issue, a dataset was analyzed containing 42,863 participants’ responses to a single delay-discounting choice, along with self-report behaviors that can be considered as impulsive. Choice of a smaller-sooner sum was associated with several demographics: younger age, lower income, and lower education; and impulsive behaviors: earlier age of first sexual activity and recent relationship infidelity, smoking, and higher body mass index. These findings suggest that at least an aspect of delay discounting preference is associated with a general trait influencing other forms of impulsivity, and therefore that high delay discounting is another form of impulsive behavior.     

Differences in delay, not ratios, control choice in concurrent chains

In two experiments, pigeons were trained with concurrent-chains schedules, wherein responding to equal initial links measured preference between variable-interval terminal links. Absolute terminal-link duration was varied by keeping constant the difference between the terminal-link delays and forcing their ratio to change. Delay-reduction theory scales value relative to a common temporal context and requires that delay differences control choice. Thus, preference should remain invariant. Most competing accounts, including the matching law and a strong form of Weber's law, require that preference vary with the delay ratio. Experiment 1 employed standard concurrent chains, in which terminal-link position and color were confounded. Although average preference remained constant, individual preferences were highly variable and inconsistent, possibly due to carryover of position biases across conditions. In an attempt to reduce variability, Experiment 2 used a modified concurrent-chains procedure. Preference at different terminal-link durations was assessed simultaneously to prevent order effects, and terminal-link position was alternated randomly across trials to minimize the impact of position biases. In Experiment 2, both individual and mean preferences showed the constant-difference invariance. Overall, choice was controlled by terminal-link differences, not ratios.     

Effects of previous delay of reward,generalized effort,and deprivation on impulsiveness

Two experiments investigated the effects of learning and motivational variables upon rats' degree of impulsiveness (choice of a small, immediate reward versus a large, delayed reward). In Experiment 1, food was presented in one of four ways for 24 sessions: fixed-ratio lever pressing requirement, continuous-reinforcement lever-pressing requirement, long intervals between free food presentations, or short intervals between free food presentations. Exposure to the long intervals produced less subsequent impulsiveness in 48 choice trials than did the short intervals, there being no effect of the instrumental requirement. In Experiment 2, there were 48 choice trials in which two hunger levels were combined factorially with two delays of reward for the self-control alternative. Hunger level did not affect implusiveness when there was a short delay, but high hunger increased the impulsiveness when there was a long delay. These results indicate that impulsiveness is affected by prior learning and current motivation. Implications for Baum, Rachlin, and Green's matching formula and for frustration theory are discussed.     

Token reinforcement, choice, and self-control in pigeons.

Pigeons were exposed to self-control procedures that involved illumination of light-emitting diodes (LEDs) as a form of token reinforcement. In a discrete-trials arrangement, subjects chose between one and three LEDs; each LED was exchangeable for 2-s access to food during distinct posttrial exchange periods. In Experiment 1, subjects generally preferred the immediate presentation of a single LED over the delayed presentation of three LEDs, but differences in the delay to the exchange period between the two options prevented a clear assessment of the relative influence of LED delay and exchange-period delay as determinants of choice. In Experiment 2, in which delays to the exchange period from either alternative were equal in most conditions, all subjects preferred the delayed three LEDs more often than in Experiment-1. In Experiment 3, subjects preferred the option that resulted in a greater amount of food more often if the choices also produced LEDs than if they did not. In Experiment 4, preference for the delayed three LEDs was obtained when delays to the exchange period were equal, but reversed in favor of an immediate single LED when the latter choice also resulted in quicker access to exchange periods. The overall pattern of results suggests that (a) delay to the exchange period is a more critical determinant of choice than is delay to token presentation; (b) tokens may function as conditioned reinforcers, although their discriminative properties may be responsible for the self-control that occurs under token reinforcer arrangements; and (c) previously reported differences in the self-control choices of humans and pigeons may have resulted at least in part from the procedural conventions of using token reinforcers with human subjects and food reinforcers with pigeon subjects.     

Stimulus Equivalence: Effects Of A Default-response Option On Emergence Of Untrained Stimulus Relations

Human subjects were exposed to a concurrent-chains schedule in which reinforcer amounts, delays, or both were varied in the terminal links, and consummatory responses were required to receive points that were later exchangeable for money. Two independent variable-interval 30-s schedules were in effect during the initial links, and delay periods were defined by fixed-time schedules. In Experiment 1, subjects were exposed to three different pairs of reinforcer amounts and delays, and sensitivity to reinforcer amount and delay was determined based on the generalized matching law. The relative responding (choice) of most subjects was more sensitive to reinforcer amount than to reinforcer delay. In Experiment 2, subjects chose between immediate smaller reinforcers and delayed larger reinforcers in five conditions with and without timeout periods that followed a shorter delay, in which reinforcer amounts and delays were combined to make different predictions based on local reinforcement density (i.e., points per delay) or overall reinforcement density (i.e., points per total time). In most conditions, subjects' choices were qualitatively in accord with the predictions from the overall reinforcement density calculated by the ratio of reinforcer amount and total time. Therefore, the overall reinforcement density appears to influence the preference of humans in the present self-control choice situation.     

The effects of reward and punishment in violent video games on aggressive affect, cognition, and behavior

Three experiments examined the effects of rewarding and punishing violent actions in video games on later aggression-related variables. Participants played one of three versions of the same race-car video game: (a) a version in which all violence was rewarded, (b) a version in which all violence was punished, and (c) a nonviolent version. Participants were then measured for aggressive affect (Experiment 1), aggressive cognition (Experiment 2), and aggressive behavior (Experiment 3). Rewarding violent game actions increased hostile emotion, aggressive thinking, and aggressive behavior. Punishing violent actions increased hostile emotion, but did not increase aggressive thinking or aggressive behavior. Results suggest that games that reward violent actions can increase aggressive behavior by increasing aggressive thinking.     

Zebrafish choice behavior is sensitive to reinforcer rate,immediacy, and magnitude ratios

Behavioral flexibility has, in part, been defined by choice behavior changing as a function of changes in reinforcer payoffs. We examined whether the generalized matching law quantitatively described changes in choice behavior in zebrafish when relative reinforcer rates, delays/immediacy, and magnitudes changed between two alternatives across conditions. Choice was sensitive to each of the three reinforcer properties. Sensitivity estimates to changes in relative reinforcer rates were greater when 2 variable-interval schedules were arranged independently between alternatives (Experiment 1a) than when a single schedule pseudorandomly arranged reinforcers between alternatives (Experiment 1b). Sensitivity estimates for changes in relative reinforcer immediacy (Experiment 2) and magnitude (Experiment 3) were similar but lower than estimates for reinforcer rates. These differences in sensitivity estimates are consistent with studies examining other species, suggesting flexibility in zebrafish choice behavior in the face of changes in payoff as described by the generalized matching law.     

Sensitivity to reinforcer duration in a self-control procedure

In a concurrent-chains procedure, pigeons' responses on left and right keys were followed by reinforcers of different durations at different delays following the choice responses. Three pairs of reinforcer delays were arranged in each session, and reinforcer durations were varied over conditions. In Experiment 1 reinforcer delays were unequal, and in Experiment 2 reinforcer delays were equal. In Experiment 1 preference reversal was demonstrated in that an immediate short reinforcer was chosen more frequently than a longer reinforcer delayed 6 s from the choice, whereas the longer reinforcer was chosen more frequently when delays to both reinforcers were lengthened. In both experiments, choice responding was more sensitive to variations in reinforcer duration at overall longer reinforcer delays than at overall shorter reinforcer delays, independently of whether fixed-interval or variable-interval schedules were arranged in the choice phase. We concluded that preference reversal results from a change in sensitivity of choice responding to ratios of reinforcer duration as the delays to both reinforcers are lengthened.     

Local effects of delayed food

Five pigeons were trained on a procedure in which seven concurrent variable-interval schedules arranged seven different food-rate ratios in random sequence in each session. Each of these components lasted for 10 response-produced food deliveries, and components were separated by 10-s blackouts. We varied delays to food (signaled by blackout) between the two response alternatives in an experiment with three phases: In Phase 1, the delay on one alternative was 0 s, and the other was varied between 0 and 8 s; in Phase 2, both delays were equal and were varied from 0 to 4 s; in Phase 3, the two delays summed to 8 s, and each was varied from 1 to 7 s. The results showed that increasing delay affected local choice, measured by a pulse in preference, in the same way as decreasing magnitude, but we found also that increasing the delay at the other alternative increased local preference. This result casts doubt on the traditional view that a reinforcer strengthens a response depending only on the reinforcer's value discounted by any response-reinforcer delay. The results suggest that food guides, rather than strengthens, behavior.     

Delayed signal detection, differential reinforcement, and short-term memory in the pigeon.

In two discrete-trial delayed-detection experiments, six pigeons were trained on dependent concurrent variable-interval schedules. Pecking a red side key was reinforced when the brighter of two white lights (S1) had been presented on the center key, and pecking a green side key was reinforced when the duller of two white lights (S2) had been presented on the center key. Incorrect responses were red side-key pecks following S2 presentations and green side-key pecks following S1 presentations; these resulted in three-second blackouts. In Experiment 1, the time between presentation of S1 or S2 on the center key and the onset of the red and green side keys was varied nonsystematically from 0.06 seconds to 19.69 seconds across experimental conditions. Stimulus discriminability decreased as the stimulus-choice delay increased. A rectangular-hyperbolic function better described this decrease in discriminability over time than did a negative-exponential function. In Experiment 2, at each of three stimulus-choice delays (0.06, 3.85, and 10.36 seconds), relative reinforcer frequency for correct responses to the red and green side keys was varied by changing the values of the dependent concurrent variable-interval schedules. The sensitivity of choice to relative reinforcer frequency was independent of the decrease in stimulus discriminability with increasing stimulus-choice delay.     

Choice and delay of reinforcement: Effects of terminal-link stimulus and response conditions

In two experiments, pigeons were exposed to concurrent-chains schedules in which a single initial-link variable-interval schedule led to access to terminal links composed of fixed-interval or fixed-delay schedules. In Experiment 1, an 8-s (or 16-s) delay to reinforcement was associated with the standard key, while reinforcer delay values associated with the experimental key were varied from 4 to 32 s. The results of Experiment 1 showed undermatching of response ratios to delay ratios with terminal-link fixed-delay schedules, whereas in some pigeons matching or overmatching was evident with the fixed-interval schedules. In Experiment 2, one pair of reinforcer delay values, either 8 versus 16 s or 16 versus 32 s, was used. In the first condition of Experiment 2, different delays were associated with different keylight stimuli (cued condition). In the second condition, different terminal-link delays were associated with the same stimulus, either a blackout (uncued-blackout condition) or a white key (uncued-white condition). To examine the role of responses emitted during delays, the keys were retracted during a delay (key-absent condition) in the third condition and responses were required by a fixed-interval schedule in the fourth condition. Experiment 2 demonstrated that the choice proportions for the shorter delay were more extreme in the cued condition than in the uncued-blackout condition, and that the response requirement imposed by the fixed-interval schedules did not affect choice of the shorter delay, nor did the key-absent and key-present conditions. These results indicate that the keylight-stimulus conditions affected preference for the shorter of two delays and that the findings obtained in Experiment 1 depended mainly on the keylight-stimulus conditions of the terminal links (i.e., the conditioned reinforcing value of the terminal-link stimuli).     

Effects of mindful eating training on delay and probability discounting for food and money in obese and healthy-weight individuals

Obese individuals tend to behave more impulsively than healthy weight individuals across a variety of measures, but it is unclear whether this pattern can be altered. The present study examined the effects of a mindful eating behavioral strategy on impulsive and risky choice patterns for hypothetical food and money. In Experiment 1, 304 participants completed computerized delay and probability discounting tasks for food-related and monetary outcomes. High percent body fat (PBF) predicted more impulsive choice for food, but not small-value money, replicating previous work. In Experiment 2, 102 randomly selected participants from Experiment 1 were assigned to participate in a 50-min workshop on mindful eating or to watch an educational video. They then completed the discounting tasks again. Participants who completed the mindful eating session showed more self-controlled and less risk-averse discounting patterns for food compared to baseline; those in the control condition discounted similarly to baseline rates. There were no changes in discounting for money for either group, suggesting stimulus specificity for food for the mindful eating condition.     

Children’s incidental learning of the colors of objects and clothing

Few researchers have examined adults’ and children’s incidental recall of color information. In Experiment 1, we examined how well children recalled the colors of items seen in a first aid demonstration. Color recall of objects was above chance levels after three different time delays. However, recall of clothing color was poor. In Experiment 2, we examined whether colors of items from the same category (e.g., items of clothing) were more difficult to remember than distinct items (i.e., belonging to a range of different categories). Children and adults saw 18 items from three different categories. Color recall was above chance levels. The colors of distinct items were recalled more often than the color of category items. The results suggest that color is encoded automatically to some extent. However, participants had difficulty in recalling the color of clothing worn by the experimenter.     

Interaction of procedural factors in human performance on yoked schedules

The differential effects of reinforcement contingencies and contextual variables on human performance were investigated in two experiments. In Experiment 1, adult human subjects operated a joystick in a video game in which the destruction of targets was arranged according to a yoked variable-ratio variable-interval schedule of reinforcement. Three variables were examined across 12 conditions: verbal instructions, shaping, and the use of a consummatory response following reinforcement (i.e., depositing a coin into a bank). Behavior was most responsive to the reinforcement contingencies when the consummatory response was available, responding was established by shaping, and subjects received minimal verbal instructions about their task. The responsiveness of variable-interval subjects' behavior varied more than that of variable-ratio subjects when these contextual factors were altered. Experiment 2 examined resistance to instructional control under the same yoked-schedules design. Conditions varied in terms of the validity of instructions. Performance on variable-ratio schedules was more resistant to instructional control than that on variable-interval schedules.