Sodium amylobarbitone and responses to nonreward

Three experiments are reported testing two alternative hypotheses concerning the behavioural effects of sodium amylobarbitone (SA): (1) that it blocks the after-effect of nonreward; (2) that it blocks conditioned frustration, elicited by stimuli associated with nonreward. In support of (2) Experiment I showed that SA given in acquisition abolished the partial reinforcement extinction effect (PREE) when rats were run at one trial a day in an alley for food reward on a continuous (CRF) or partial (PRF) reinforcement schedule. Experiment II showed that, in the goal section, the effect of the drug on the PREE was due to its presence during acquisition and was not due to state dependency; but the effect of the drug in the start section was consistent with state dependency of the PREE. In Experiment III, in opposition to (1) and again in support of (2), SA given to rats trained to show patterned running for water reward on a single alternation schedule blocked patterning by increasing running speeds on nonreward trials, not by decreasing running speeds on rewarded trials.     

Transfer from discrimination under hunger to discrimination under thirst: The role of expectancy and habit

The experimental group (Group HA-TA) received food (F) and water (W) rewarded trials in an alternating sequence under hunger in Phase 1 and under thirst in Phase 2. Group HA-TA ran faster on F than on W trials in Phase 1, and faster on W than on F trials in Phase 2. Early in Phase 2 the difference between speeds on W and F trials was larger for Group HA-TA than for a group which received no runway training in Phase 1 (Group HO-TA), but later in Phase 2 this difference was larger for Group HO-TA than for Group HA-TA. Also in Phase 2 the difference between speeds on W and F trials was larger for Group HA-TA than for a group which received a random sequence of F and W trials under hunger in Phase 1, and smaller for Group HA-TA than for a group which received alternating F and W trials under thirst in both phases. To interpret these results it was assumed that for Group HA-TA the expectancies of reward formed in Phase 1 facilitated development of alternation performance in Phase 2, but that the S-R associative connections formed in Phase 1 inhibited ultimate development of alternation performance in Phase 2.     

Effects of sucrose concentrations on single alternation runway responding in rats

Rats received runway training under alternating reward and nonreward in which rewarded (R) trials provided 32 or 4% sucrose concentration from a drinking tube and nonrewarded (N) trials provided a dry tube or, for half of the rats in the 32% condition, plain water. Both 32 and 4% concentrations yielded faster running on R trials than on N trials; but this effect was reliable only for the 32% condition. Compared to 4% sucrose, 32% sucrose yielded reliably slower running on N trials and unreliably faster running on R trials.     

Schedules of reward shift in the double runway

Eighty food deprived rats received 62 trials in a double runway. On Trials 1-30, reward in the first goal box (GB1) was either always two food pellets or always zero pellets. All subjects received two pellets in the second goal box (GB2). On Trials 31-62 subjects in each preshift group (GB1 reward or GB1 nonreward) were shifted to the opposite GB1 reward level on 0, 25, 50, 75 or 100% of occasions. GB2 reward remained unaltered in all cases. For subjects experiencing reward decrease, second runway (A2) run and goal speeds after nonreward were generally enhanced, both within-group and in comparison with never rewarded controls. No such effect was evident on A2 start speed, nor was there any evidence to suggest that A2 performance after decreased reward was a function of the schedule of decrease. Increased GB1 reward resulted in general within-group impairment of A2 start and run speeds, with no effect on A2 goal performance. However, comparisons of speeds after increased reward with those of always rewarded controls revealed no difference on A2 start or run but indicated impairment of A2 goal performance. With the 50% schedule of reward increase, A2 run speeds after nonreward (the training level) exceeded those of never rewarded controls. Results are discussed with reference to McHose's contrast account of double runway phenomena and Amsel's frustration theory.     

Preference for unpredictable food rewards occurs with high proportion of reinforced trials or alcohol if rewards are not delayed

Organisms typically prefer situations where reward and nonreward are predictable rather than unpredictable. Although many theories can account for this result (e.g., information theory and delay-reduction theory), a recently developed mathematical model (DMOD) also predicts that subjects prefer the unpredictable reward situation under conditions that substantially decrease aversiveness of unpredictable nonreward (Daly & Daly, 1982). Because a high proportion of reinforced trials (lenient schedule) and alcohol injections decrease aversive conditioning, these variables were tested with rats in five E-maze experiments. A choice to one side of the maze resulted in a stimulus uncorrelated with reward outcome (unpredictable situation). A choice to the other side resulted in stimuli correlated with reward and nonreward (predictable situation). The stimuli were not visible until after the choice was made. A lenient reinforcement schedule resulted in preference for the unpredictable reward situation if rewards were not delayed. Alcohol resulted in preference for the unpredictable reward situation if a medium five-pellet reward was given. A lenient reinforcement schedule combined with an alcohol injection resulted in faster acquisition of the preference for the unpredictable reward situation than did a lenient schedule combined with a saline control injection. These results pose a major challenge to most theories, yet were predicted by DMOD.     

Control of instrumental behavior by deprivation stimuli

In Experiment 1, rats were given one trial per day in a straight alley under food deprivation on half of the trials and under water deprivation on the other half. Wet mash was available in the goal box under food deprivation for Group H and under water deprivation for Group T, the other deprivation being nonrewarded for each group. After 15--18 trials both groups ran significantly faster on their rewarded than on their nonrewarded deprivation days. A third group showed that random variation of alley color retarded formation of the discrimination. A fourth group was run in a conditional discrimination in which under food deprivation wet mash was available in a black alley, nonreward in a white alley, and vice versa under water deprivation. This group took 114 trials to begin running significantly faster in their rewarded than in their nonrewarded alley under each deprivation. In Experiment 2, it was shown that prior learning about deprivation cues "blocks" learning about alley color when alley color is subsequently presented in compound with the deprivation cue but that when both alley color and deprivation cues are relevant from the start of training, the rat learns about both cues. It is suggested that previous studies have underestimated the importance of deprivation cues by using conditional discrimination designs, choice measures rather than speeds, and parameters that are not optimal for discrimination learning.     

Stimulus control of responding in the early trials of differential conditioning

In Experiment 1 two groups of rats were given 12 differential conditioning trials, seven to the rewarded alley (S+) and five to the nonrewarded alley (S?), prior to being extinguished in both alleys. Group S?S+ received S+ trials, following S? trials in acquisition, while Group S+S? did not receive S+ trials following S? trials in acquisition. In extinction S+ and S? trials were presented according to a quasi-random sequence for both groups. Running on the last 3 trials of acquisition was found to be faster following S+ than following S? trials. Group S?S+ showed greater resistance to extinction and less discriminative responding in extinction than Group S+S?. These results suggest that responding in differential conditioning is controlled not merely by S+ and S? but by the memories of reward (S^R) and of nonreward (S^N) as well. When the joint effects of both classes of cues were considered, e.g., S^R+S+, responding in the early trials of differential conditioning was shown to be highly orderly. Experiment 2 was highly similar to Experiment 1 except that Groups S?S+ and S+S? were equated along dimensions not equated in Experiment 1. The results obtained in Experiment 2 were highly similar to those obtained in Experiment 1.     

Reward schedule effects in extinction: Intertrial interval, memory, and memory retrieval

Rats were trained in a runway such that partial reward occurred on Trial 1 of the day and consistent reward on subsequent massed trials (Group PRT1), or consistent reward occurred on Trial 1 of the day and partial reward on subsequent massed trials (Group PRTM). Under spaced (24-hr) extinction, Group PRT1 was more resistant to extinction than Group PRTM and under massed (1-min) extinction, Group PRTM was more resistant to extinction than Group PRT1. These findings suggest that (a) distinctive stimuli are associated with Trial 1 of the day and with subsequent massed trials, (b) these distinctive stimuli function as retrieval cues for memories, memory retrieval being independent of intertrial interval, and (c) behavior in extinction is controlled by a stimulus compound consisting of the memory of nonreward plus stimuli which accompany the memory of nonreward on rewarded acquisition trials.     

Visual Reinforcement Signals Interfere with the Effects of Reinforcer Magnitude Manipulations

Three experiments examined the effect of reinforcement magnitude on free-operant response rates. In Experiment 1, rats that received four food pellets responded faster than rats that received one pellet on a variable ratio 30 schedule. However, when the food hopper was illuminated during reinforcer delivery, there was no difference between the rates of response produced by the two magnitudes of reward. In Experiment 2, there was no difference in response rates emitted by rats receiving either one or four pellets of food as reward on a random interval (RI) 60-s schedule. In Experiment 3, rats responding on an RI 30-s schedule did so at a lower rate with four pellets as reinforcement than with one pellet. This effect was abolished by the illumination of the food hopper during reinforcement delivery. These results indicate that the influence of magnitude is obscured by manipulations which signal the delivery of reinforcement.     

Immediate reinforcement in delayed reward learning in pigeons

Pigeons were trained on simultaneous red-green discrimination procedures with delayed reward and sequences of stimuli during the delay. In Experiment 1, three stimuli appeared during the 60-second intervals between the correct responses and reward, and the incorrect responses and nonreward. The stimulus that immediately followed a correct response also preceded nonreward, and the stimulus that followed an incorrect response preceded reward. These stimuli were 10 or .33 second in duration for different groups. Stimuli during the remainder of the delay interval differed following correct and incorrect responses. Group 10 initially persisted in the nonrewarded choice, but shifted to a preponderance of rewarded responses after further training. Group .33 rapidly acquired the correct response. Similar results were obtained in Experiment 2 where delay intervals consisted of opposite sequences of two stimuli of equal duration and total delays were 6, 20, or 60 seconds. Early in training, generalization of differential conditioned-reinforcing properties from the conditions preceding reward and nonreward to postchoice conditions had a greater effect relative to backchaining than it did later. It was concluded that delayed-reward learning is best analyzed in terms of the conditioned-reinforcing value of the patterns of cues that follow immediately after rewarded and nonrewarded responses.     

Positive contrast following gradual and abrupt increases in reward magnitude using delay of reinforcement

In Experiment 1, rats were given a 1-pellet reward for 48 preshift trials. During a subsequent 20-trial postshift phase, one group was shifted to a 12-pellet reward on Trial 1, a second was shifted on Trial 11, and a third was given 1 more pellet each trial and then 12 pellets for the last 10 trials. The speeds of all three groups increased to a level above that of a control group given a 12-pellet food reward throughout training (positive contrast). In experiment 2, rats were shifted from 1 to 12 pellets either gradually or abruptly following either abbreviated training (9 trials) or extended training (20 trials). One group of control subjects received 12 pellets throughout training. The results revealed a positive contrast effect for gradually shifted subjects following extended training but not following abbreviated training. The abrupt shift procedure produced positive contrast following abbreviated training but only a marginal effect following extended training. These results indicate that, contingent upon the amount of preshift training, either gradual or abrupt reward increases may produce positive contrast.     

Organized responding in instrumental learning: Chunks and superchunks

Two investigations attempted to determine if rats could learn that the second series of runway trials, the test series, was the same as the first series, the study series. Series were constructed from runway responses which terminated either in food reward (R) or nonreward (N). In the series RNR, for example, three successive responses terminated in R, N, and R, respectively. Rats manifested mastery of a series by running fast to R and slow to N. In Experiment 1 the test series (either RNR or RNN) occurred in a black runway about 15 s after the study series presented in a white runway. In Experiment 2 the test series (either RN or RRN) occurred in a gray runway about 15 min after the study series, also in a gray runway. In both experiments rats learned that the study series was the same as the test series. A hierarchical interpretation was suggested in which runway trials are organized into series, series being organized into lists.     

Double drive learning in rats without previous selective reinforcement

Can rats which have never been selectively satiated by food or water respond appropriately by choosing the relevant reward when they are made hungry or thirsty ? A group of 21 rats was put on a diet of wet mash and trained both hungry and thirsty in a T-maze with water on one side and food on the other. 8 out of 10 of the animals which were made thirsty on the test-run chose correctly in spite of an established preference for the other reward. Only 1 out of the 11 animals made hungry chose correctly. Possible explanations of this result are examined.     

Muscimol Induces Retrograde Amnesia for Changes in Reward Magnitude

These experiments examined the effect of the GABA_A, agonist, muscimol (MUS), on memory for changes in reward magnitude. In Experiment 1 rats were trained to run a straight alley for either a large or small food reward. After reaching asymptotic performance rats in the high reward group were shifted to the small food reward. Half the animals received 1.0 or 3.0 mg/kg (ip) of MUS or the equivalent volume of saline immediately after training. Shifted training continued for 3 more days and no further injections were given. Shifted saline animals displayed an increase in response latencies compared to unshifted controls with a sharp peak on the day after the shift. Shifted MUS receiving 1.0 mg/kg performed comparably to shifted saline animals. In contrast, shifted MUS animals receiving 3.0 mg/kg displayed performance comparable to shifted saline animals on the day of the shift but displayed a sharp increase in response latencies on the second day after the shift. These findings indicate that post-training systemic MUS injections delay the peak increase in response latencies and suggest that MUS induces retrograde amnesia for reward reduction. Experiment 2 examined the effect of MUS on the memory of a reward increase. Rats were first trained as in Experiment 1 and rats under the high reward condition were then shifted to the small reward. On the next training session, the large food reward was reinstated. Immediately after the session all animals were injected with saline or 3.0 mg/kg of MUS. The large food reward was continued for the remainder of training and no further injections were given. On the following session, the performance of the shifted saline animals was comparable to that of the unshifted controls while shifted MUS animals displayed significantly higher response latencies. The findings that MUS prevented the reduction in response latencies seen in saline-injected animals suggest that MUS also induces retrograde amnesia for reward increases.     

Are schedule-induced drinking and displacement activities causally related?

The acquisition of schedule-induced drinking was studied in hungry rats given food according to an intermittent (fixed-time 45-s) schedule, in three separate experiments. Rate of acquisition of induced drinking was not affected by making the rats thirsty as well as hungry (Experiment I), nor by offering them a palatable saline solution instead of water (Experiment II). Thirst and palatability did affect the asymptotic level of induced drinking. Increasing the level of food deprivation (Experiment III) increased both the rate of acquisition of induced drinking and its asymptotic level. Thus, rate of acquisition of induced drinking was found to depend more on causal factors relevant to feeding than on causal factors relevant to drinking, while asymptotic level of drinking was affected by both types of causal factor. This suggests that the occurrence of induced drinking does not depend primarily on a disinhibition mechanism of the type thought to underlie displacement activities, and hence does not support the idea that induced drinking is a type of displacement activity.     

Resistance to satiation: Reinforcing effects of food and eating under satiation

It has been shown previously that rats which have learned a response when hungry will continue to make that response when tested satiated, a phenomenon labeled resistance to satiation. Here we showed that rats which were previously trained hungry will learn a new response for the opportunity to consume pellets in a new situation when tested satiated. In four experiments various groups received each of the components of the training given when rats learn an instrumental response when hungry. Rats were placed in the goalbox of a straight alley and given food pellets when hungry or were hungry only in their home cages prior to running a straight alley in the satiated test in Experiment 1. In Experiments 2, 3, and 4 learning of a differential conditioning problem for pellets in S+ (nonreward in S?) was measured in the satiated test. Groups given pellets in their home cages when hungry with or without alley exposure learned to run more rapidly in S+ than in S? in the satiated test phase. The tendency to eat pellets in the apparatus and the reinforcing effect of eating the pellets was larger for rats which ate the pellets when hungry in their home cage than for rats which ate the pellets when satiated in their home cage. Being hungry in the home cage with no pellets was not sufficient to produce eating or running for pellets in the satiated test, indicating that any inherent reinforcing effect of the pellets is not sufficient to produce eating or running, and that incomplete satiation cannot account for the learning. These data indicate that a reinforcing effect of eating pellets under satiation is an important determiner of resistance to satiation.     

Resistance to extinction and punishment following training with shock and non-reinforcement: Failure to obtain cross-tolerance

Two experiments investigated the phenomenon of cross-tolerance between the partial reinforcement extinction effect (PREE) and the partial punishment effect (PPE). Three groups of rats were trained in acquisition to run in a straight alley. The continuously reinforced (CRF) group received a reward on every trial. The partially reinforced (PRF) group was rewarded on a quasi-random 50% schedule. The partially punished (PP) group received food reward on every trial but, in addition, received foot shocks of gradually increasing intensity in the goal box on a random 50% of the trials. In the test stage, half of the animals in each training condition were tested in extinction, where no reward was given on any of the trials, and the other half were tested in punishment, with both food and shock presented on each trial. Experiment 1 used a 1-trial/day procedure; Experiment 2 used a multi-trial procedure. In both procedures, clear PREE and PPE were obtained. In the 1-trial/day procedure, no cross-tolerance was evident: animals trained on a PRF or PP schedule did not show increased resistance to punishment and extinction, respectively. In the multi-trial procedure, only weak cross-tolerance was obtained in animals trained on partial reinforcement and tested in punishment.     

Multiple determinants of the effects of reinforcement magnitude on free-operant response rates

Four experiments examined the effects of increasing the number of food pellets given to hungry rats for a lever-press response. On a simple variable-interval 60-s schedule, increased number of pellets depressed response rates (Experiment 1). In Experiment 2, the decrease in response rate as a function of increased reinforcement magnitude was demonstrated on a variable-interval 30-s schedule, but enhanced rates of response were obtained with the same increase in reinforcement magnitude on a variable-ratio 30 schedule. In Experiment 3, higher rates of responding were maintained by the component of a concurrent variable-interval 60-s variable-interval 60-s schedule associated with a higher reinforcement magnitude. In Experiment 4, higher rates of response were produced in the component of a multiple variable-interval 60-s variable-interval 60-s schedule associated with the higher reinforcement magnitude. It is suggested that on simple schedules greater reinforcer magnitudes shape the reinforced pattern of responding more effectively than do smaller reinforcement magnitudes. This effect is, however, overridden by another process, such a contrast, when two magnitudes are presented within a single session on two-component schedules.     

Reward, novelty and spontaneous alternation

It has been suggested that rats' propensity for “win-shift” behaviour in spatial memory and spontaneous alternation tests reflects a species-specific foraging strategy which leads them to avoid places where they have recently found food. An alternative explanation is that they avoid places which are familiar. In three experiments using a T-maze, we evaluated these accounts by comparing the probability of avoiding or re-entering a recently visited arm, as a function of whether food had or had not been found on the previous visit. Each rat received a series of 16 exposure-test trial pairs over 8 days. Neither alternation nor repetition of the previous choice was differentially reinforced. Experiments I and II forced rats to enter a specific arm before a subsequent choice, and differed in the overall probability of reward; in Experiment III all choices were free. In all three experiments the probability of alternating was greater after nonreward than after reward. This effect occurred more reliably on later tests within a day, little difference appearing on earlier tests. It was concluded that there was no evidence for a spontaneous “win-shift” tendency as such, and that these and other results can be adequately accounted for by a combination of exploratory tendencies (spontaneous alternation) and the conventional effects of reward.     

Effects of reinforcer magnitude on responding under differential-reinforcement-of-low-rate schedules of rats and pigeons

Experiment I investigated the effects of reinforcer magnitude on differential-reinforcement-of-low-rate (DRL) schedule performance in three phases. In Phase 1, two groups of rats (n = 6 and 5) responded under a DRI. 72-s schedule with reinforcer magnitudes of either 30 or 300 microl of water. After acquisition, the water amounts were reversed for each rat. In Phase 2, the effects of the same reinforcer magnitudes on DRL 18-s schedule performance were examined across conditions. In Phase 3, each rat responded unider a DR1. 18-s schedule in which the water amotnts alternated between 30 and 300 microl daily. Throughout each phase of Experiment 1, the larger reinforcer magnitude resulted in higher response rates and lower reinforcement rates. The peak of the interresponse-time distributions was at a lower value tinder the larger reinforcer magnitude. In Experiment 2, 3 pigeons responded under a DRL 20-s schedule in which reinforcer magnitude (1-s or 6-s access to grain) varied iron session to session. Higher response rates and lower reinforcement rates occurred tinder the longer hopper duration. These results demonstrate that larger reinforcer magnitudes engender less efficient DRL schedule performance in both rats and pigeons, and when reinforcer magnitude was held constant between sessions or was varied daily. The present results are consistent with previous research demonstrating a decrease in efficiency as a function of increased reinforcer magnituide tinder procedures that require a period of time without a specified response. These findings also support the claim that DRI. schedule performance is not governed solely by a timing process.