Brain-stimulation reward: a review

During most of the first half of this century psychologists knew what they wanted to do but had no idea how to do it, and during the second half they have, for the most part, been so preoccupied with how to do it that they have forgotten what they wanted to do. When J. Olds and Milner (1954) announced that rats would stimulate themselves in the septal area, it appeared to open the door to understanding motivation and reinforcement in terms of the underlying physiology. In the ensuing 36 years some progress has been made in that direction, though far outstripped by the progress in methodology. In this review I trace the efforts that have been made to locate the structures involved in self-stimulation by lesions, drugs, determinations of their neurophysiological characteristics, and other more sophisticated methods. I then review experiments, none very recent, comparing brain-stimulation reward to natural rewards and finally indicate how the information so far collected may be incorporated into theories of learning and motivation.     

The reward probability index: design and validation of a scale measuring access to environmental reward

Behavioral models of depression implicate decreased response-contingent positive reinforcement (RCPR) as critical toward the development and maintenance of depression (Lewinsohn, 1974). Given the absence of a psychometrically sound self-report measure of RCPR, the Reward Probability Index (RPI) was developed to measure access to environmental reward and to approximate actual RCPR. In Study 1 (n = 269), exploratory factor analysis supported a 20-item two-factor model (Reward Probability, Environmental Suppressors) with strong internal consistency (α = .90). In Study 2 (n = 281), confirmatory factor analysis supported this two-factor structure and convergent validity was established through strong correlations between the RPI and measures of activity, avoidance, reinforcement, and depression (r = .65 to .81). Discriminant validity was supported via smaller correlations between the RPI and measures of social support and somatic anxiety (r = – .29 to – .40). Two-week test–retest reliability was strong (r = .69). In Study 3 (n = 33), controlling for depression symptoms, hierarchical regression supported the incremental validity of the RPI in predicting daily diary reports of environmental reward. The RPI represents a parsimonious, reliable, and valid measure that may facilitate understanding of the etiology of depression and its relationship to overt behaviors.     

Effects of reward and familiarity of reward agent on spontaneous play in preschoolers: a field study

A field experiment was conducted with preschool children to test the effect of rewards on a familiar, spontaneous play activity, in conditions as close as possible to the children's natural school context, and to examine the role of familiarity of the person who administered rewards. In three experimental conditions, children were rewarded either by their own teacher or by an unknown adult for playing with toys at the school playground and stayed with either the teacher or the unknown adult in the remaining part of reward sessions. Spontaneous play was significantly reduced by the reward relative to baseline levels and recovered after a 3-wk. interval. However, no difference due to familiarity/unfamiliarity of reward agent could be found. Results, discussed in terms of an incentive contrast hypothesis, attest to the generality and external validity of the undermining effects of rewards.     

Transfer between downshift in reward magnitude and continuous delay of reward

Two separate experiments were conducted to investigate transfer of persistence between delay and a downshift in reward magnitude. In the first experiment, experimental rats were initially downshifted in reward magnitude and later tested for persistence to continuous delay of large reward. It was found that these rats were more persistent to the effects of delay than control rats which did not receive prior experience with a downshift in reward magnitude. In the second experiment, experimental rats were first trained to receive large reward under delayed conditions and then tested for persistence to a downshift in reward magnitude. Compared to control rats which received no prior experience with delay, the experimental rats showed a significantly smaller negative contrast effect. The results were interpreted as supporting Amsel's theory of persistence as well as Capaldi's recent interpretation of contrast effects.     

Delay and amount of reward in a concurrent chain

Eight pigeons responded under a concurrent-chain schedule for rewards differing in both delay and amount, the larger reward being associated with the longer delay. Preference was examined as the absolute durations of the terminal-link delays were increased at four different delay ratios. Difficulties with other experiments of this type were controlled for by the use of (a) a single-tape initial link to equalize terminal-link entries, (b) a blackout following the more immediate reward to equalize terminal-link length, and (c) a photocell to measure reinforcer duration more accurately. Preference for the larger reward changed systematically as delays increased in all conditions, decreasing for the 6:1, 3:1, and 3:2 ratios, and increasing for the 1:1 ratio. These results were similar to, but significantly different from, those of previous investigations. The implications of these results for various models of concurrent-chain behavior are discussed.     

Visual discrimination learning with a 1-min delay of reward

Eight rats were successfully trained in a black-white discrimination with a 1-min delay of reward. The procedure was unusual in that the rat spent the delay outside the apparatus in its home cage. Immediately after the rat responded, whether correctly or incorrectly, it was removed from the choice compartment and placed in its home cage. When the delay ended, it was returned to the startbox. If the preceding response had been correct, the rat received a reward of sugar water; otherwise, it was allowed to make another choice response. Mediation by external cues was excluded because there was no difference in the way the rats were treated after a correct or an incorrect response until the delay interval ended. Mediation by proprioceptive stimuli was excluded because position was an irrelevant cue.     

Self-stimulating rats combine subjective reward magnitude and subjective reward rate multiplicatively

For rats that bar pressed for intracranial electrical stimulation in a 2-lever matching paradigm with concurrent variable interval schedules of reward, the authors found that the time allocation ratio is based on a multiplicative combination of the ratio of subjective reward magnitudes and the ratio of the rates of reward. Multiplicative combining was observed in a range covering approximately 2 orders of magnitude in the ratio of the rates of reward from about 1:10 to 10:1) and an order of magnitude change in the size of rewards. After determining the relation between the pulse frequency of stimulation and subjective reward magnitude, the authors were able to predict from knowledge of the subjective magnitudes of the rewards and the obtained relative rates of reward the subject's time allocation ratio over a range in which it varied by more than 3 orders of magnitude.     

Performance on a fixed-ratio schedule with correlated amount of reward

Four rats were trained to bar press on FR 9 TO 30 sec. They were reinforced with a large or small amount of water according to whether their final IRT was long or short respectively. Four control rats always received the small amount of reinforcement. The control animals produced the high rates of responding typical of fixed-ratio performance. The experimental animals, with one exception, developed superstitious behavior and maintained slow responding throughout the ratio. However, some features of the results pointed to a persistent influence of the factors which favor short IRTs.     

Learning when reward is delayed: a marking hypothesis

Rats were trained on spatial discriminations in which reward was delayed for 1 min. Experiment 1 tested Lett's hypothesis that responses made in the home cage during the delay interval are less likely to interfere with learning than responses made in the maze. Experimental subjects were transferred to their home cages during the delay interval, and control subjects were picked up but then immediately replaced in the maze. Contrary to Lett's hypothesis, both groups learned. Further experiments suggested that handling following a choice response was the crucial variable in producing learning: No learning occurred when handling was delayed (Experiment 2) or omitted (Experiment 3). One possible explanation for the fact that handling facilitated learning is that it served to mark the preceding choice response in memory so that subjects were then more likely to recall it when subsequently reinforced. In accordance with this interpretation, learning was found to be just as strong when the choice response was followed by an intense light or noise as by handling (Experiment 4). The implication of marking for other phenomena such as avoidance, quasi-reinforcement, and the paradoxical effects of punishment is also discussed.