Acquisition of lever-press responding in rats with delayed reinforcement: A comparison of three procedures

The present study examined the acquisition of lever pressing in rats under three procedures in which food delivery was delayed by 4, 8, and 16 seconds relative to the response. Under the nonresetting delay procedure, food followed the response selected for reinforcement after a specified interval elapsed; responses during this interval had no programmed effect. Under the resetting procedure, the response selected for reinforcement initiated an interval to food delivery that was reset by each subsequent response. Under the stacked delay procedure, every response programmed delivery of food t seconds after its occurrence. Two control groups were studied, one that received food immediately after each lever press and another that never received food. With the exception of the group that did not receive food, responding was established with every procedure at every delay value without autoshaping or shaping. Although responding was established under the resetting delay procedure, response rates were generally not as high as under the other two procedures. These findings support the results of other recent investigations in demonstrating that a response not previously reinforced can be brought to strength by delayed reinforcement in the absence of explicit training.     

Free-operant acquisition with delayed reinforcement

The acquisition of free-operant lever pressing by hungry rats was investigated under a schedule in which the first lever press in each second programmed a reinforcer delivery after a fixed delay. In two studies acquisition was observed under programmed delays ranging between 0 and 32 sec, although animals trained with a delay of 64 sec pressed no more than yoked, non-contingent controls. In the final study an attempt was made to enhance sensitivity to the instrumental contingency under a 64-sec delay by exposing the animals to the operant chamber in the absence of the lever prior to each training session. Only animals receiving such exposure pressed significantly more than their non-contingent controls.     

The effects of lateral, medial, or complete septal lesions on positive conditioned suppression in rats

This study investigated the effects of lateral, medial, or complete septal lesions in rats on lever pressing and US-approach behaviors during the presentation of a light followed by two free food pellets. Ten days after surgery, groups of rats received 20 sessions of a random interval (RI) 60-s schedule of food reinforcement. The positive conditioned suppression paradigm consisted of 20 sessions of a 10-s light paired with free food while the rats were responding on the RI 60-s schedule. All groups of rats significantly suppressed lever pressing during the 10-s light presentation. During the last four sessions of the 10-s light condition, rats with lateral septal lesions had a significant increase in food-tray entries during the light presentation, while the other groups decreased lever pressing without a change in food-tray entries during the 10-s light presentation. Although rats with septal damage generally have difficulty inhibiting responses in a variety of operant situations, rats with lateral, medial, or complete septal lesions showed no impairment in positive conditioned suppression. This study also suggests that when the spatial arrangement of the CS and the type of US were manipulated to minimize sign-tracking and goal-tracking behaviors, emotional reactions during the presentation of the CS must be considered a factor influencing positive conditioned suppression.     

Second-order schedules of token reinforcement: effects of varying the schedule of food presentation

In the initial link of a complex schedule, one discriminative stimulus was presented and lever pressing produced tokens on fixed-ratio schedules. In the terminal link, signalled by a second discriminative stimulus, deposits of the tokens produced food. With two rats, the terminal link was presented after each sixth component schedule of token reinforcement was completed. With the other two rats, the terminal link was presented following the first component schedule completed after a fixed interval. During the terminal link, each token deposit initially produced food. The schedule of food presentation was subsequently increased such that an increasing number of token deposits in the terminal link was required for each food presentation. Rates of lever pressing in the initial link were inversely related to the schedule of food presentation in the terminal link. These results are similar to those of experiments that have varied schedules of food presentation in chained schedules. Rates and patterns of responding controlled throughout the initial link were more similar to those ordinarily controlled by second-order brief-stimulus schedules than to those controlled by comparable extended chained schedules.     

Conditional acceleration and external disinhibition of operant lever pressing by prereward, neutral, and reinforcing stimuli

Rats responding under a differential-reinforcement-of-low-rate schedule increased their rates of lever pressing during a 20-second click/flash stimulus that preceded the delivery of a response-independent food pellet. The increase could not be attributed to suppression of collateral behavior that has been said to mediate temporally-spaced responding. We propose that the prereward stimulus functioned as an external disinhibitor of lever pressing that had been inhibited by the constraints of the operant schedule. Support is derived from the observed disinhibitory effects of a 10-second unpaired click/flash stimulus and of unsignaled, response-independent pellets that were presented while the animals were responding under the same schedule.     

Effects of signaled and unsignaled shock on schedule-controlled lever pressing and schedule-induced licking: Shock intensity and body weight

Schedule-controlled lever pressing and schedule-induced licking were studied in rats under a multiple fixed-interval fixed-interval schedule of food reinforcement. Following acquisition of stable rates of pressing and licking, a multiple variable-time variable-time schedule of electric-shock delivery was superimposed upon the baseline schedule. In only one component of the multiple schedule, a 5-sec stimulus preceded each shock (signaled shock). In the other component shock was unsignaled. Several shock intensities (Experiment 1) and body weights (Experiment 2) were studied. Lever pressing and licking were affected similarly by experimental manipulations, although with parametric differences. Depending upon shock intensity and body weight, rates of lever pressing and licking were hardly suppressed, suppressed primarily in the unsignaled shock component (differential suppression), or markedly suppressed in both components. Differential suppression during components with signaled and unsignaled shock and conditioned suppression of responding during the preshock stimulus appeared not to be functionally related. Differential suppression depended more on the discriminability of shock-free time, and on shock intensity, body weight, and the type of response than on the “preparatory” behavior preceding shock.     

Body weight and response acquisition with delayed reinforcement.

The relation between body weight and responding established with unsignaled delayed reinforcement was investigated. In three experiments, naive rats were deprived to either 70%, 80%, or 90% of ad libitum weight and were then exposed to tandem variable-interval 15-s differential-reinforcement-of-other-behavior 30-s schedules. The tandem schedule defined a resetting unsignaled delay-of-reinforcement procedure. In the first experiment, speed of magazine training, acquisition of lever pressing, and final rate of lever pressing were related to body weight. In the next experiment, lever pressing was established and maintained in rats that were magazine trained at 70% of ad libitum weight but that were then exposed to the delay procedure at 90% of ad libitum weight. Responding did not change consistently either across or within subjects in subsequent conditions in which body weight was manipulated. In the final experiment, lever pressing was established and maintained with delayed reinforcement in the absence of magazine training for each of 2 rats at 70% and for 1 of 2 rats at 90% of ad libitum weight. The results further illuminate the conditions under which responding can be established in the absence of training and when such responses are reinforced only following an unsignaled delay period.     

Second-order schedules of token reinforcement: comparisons of performance under fixed-ratio and variable-ratio exchange schedules

Rats' lever pressing produced tokens according to a 20-response fixed-ratio schedule. Sequences of token schedules were reinforced under a second-order schedule by presentation of periods when tokens could be exchanged for food pellets. When the exchange period schedule was a six-response fixed ratio, patterns of completing the component token schedules were bivalued, with relatively long and frequent pauses marking the initiation of each new sequence. Altering the exchange period schedule to a six-response variable ratio resulted in sharp reductions in the frequency and duration of these initial pauses, and increases in overall rates of lever pressing. These results are comparable to those ordinarily obtained under simple fixed-ratio and variable-ratio schedules.     

Punishment of schedule-induced drinking in rats by signaled and unsignaled delays in food presentation

Food-deprived rats were exposed to a fixed-time 60-s schedule of food-pellet presentation and developed schedule-induced drinking. Using an ABA reversal design, three experiments investigated the effects of events then made dependent on licks. In Experiment 1, lick-dependent signaled delays (10 s) in food presentation in general led to decreased drinking, which recovered when the signaled delays were discontinued. The drinking of yoked-control rats, which received food at the same times as those exposed to the signaled-delay contingency, showed much smaller changes. Experiment 2 showed that 10-s lick-dependent signals alone did not reduce drinking. In Experiment 3, when licks produced unsignaled 10-s delays in food there were less marked and more gradual changes in drinking than in Experiment 1, although these effects again were greater than with yoked-control animals. We concluded that both signaled and unsignaled delays functioned as punishers of drinking. These findings support the view that schedule-induced drinking, like operant behavior, is subject to control by its consequences.     

Preference for less frequent shock under fixed-interval schedules of electric-shock presentation.

Lever pressing by 2 squirrel monkeys was maintained under fixed-interval 6-min and fixed-interval 2-min schedules of electric-shock presentation. Preference for these schedules was assessed during three experimental phases. In all phases, responses on one lever produced shock according to one or the other fixed-interval schedule, and responses on a second, changeover, lever switched between schedules. The opportunity to change over was presented during separate choice periods (during which the fixed-interval schedules did not operate) that followed the first through fourth shocks in each schedule. If no changeover occurred during those choice periods, a changeover automatically occurred following the fifth shock. In Phase I, durations of the choice periods were fixed. In Phase II, the choice periods equaled a proportion of their respective fixed interval. During Phase III (completed with 1 monkey) a response on the changeover lever during a given choice period reinstated the most recent fixed interval, and a failure to respond resulted in a changeover. During each of these phases, distinct preferences developed for the 6-min schedule. These results suggest that the maintenance of lever pressing by fixed-interval presentation of electric shock may not be an example of positive reinforcement, and that the response-maintaining characteristics of shock presentation may derive from other properties of the schedule.     

Differential reinforcement of other behavior (DRO): a yoked-control comparison

After training to press a lever on a variable-interval 30-sec schedule, one group of rats was shifted to a differential-reinforcement-of-other-behavior 10-sec schedule, while a second group was shifted to a noncontingent yoked-control schedule that provided the same frequency and distribution of reinforcement. Then, both groups were extensively retrained on the variable-interval schedule, after which the first group was shifted to a series of differential-reinforcement-of-other-behavior 30-sec sessions alternating daily with variable-interval 30-sec sessions, while the second group was treated like the first on variable-interval days and yoked with the first as before on differential-reinforcement-of-other-behavior days. In both phases, response-decrement was more rapid and more marked in the differential-reinforcement-of-other-behavior animals than in the controls. The difference was due, at least in large measure, to sustainment of response in the control animals by adventitious reinforcement. All the differential-reinforcement-of-other-behavior animals developed “other” behavior—the same distinctive pattern of waiting at the foodcup—but there was no direct evidence that it contributed in any way to the decrement in lever pressing.     

Effects of D-amphetamine on response acquisition with immediate and delayed reinforcement.

The present study examined in 8-hour sessions the effects of d-amphetamine (1.0, 5.6, and 10 mg/kg) on the acquisition of lever-press responding in rats that were exposed to procedures in which water delivery was delayed by 0, 8, or 16 seconds relative to the response that produced it. Both nonresetting- and resetting-delay conditions were studied. Although neither shaping nor autoshaping occurred, substantial levels of operative-lever responding developed under all conditions in which responses produced water. The lowest dose (1.0 mg/kg) of d-amphetamine either had no effect on or increased operative-lever pressing, whereas higher doses typically produced an initial reduction in lever pressing. Nonetheless, overall rates of operative-lever pressing at these doses were as high as, or higher than, those observed with vehicle. Thus, response acquisition was observed under all reinforcement procedures at all drug doses. In the absence of the drug, most responding occurred on the operative lever when reinforcement was immediate. Such differential responding also developed under both nonresetting- and resetting-delay procedures when the delay was 8 seconds, but not when it was 16 seconds. d-Amphetamine did not affect the development of differential responding under any procedure. Thus, consistent with d-amphetamine's effects under repeated acquisition procedures, the drug had no detrimental effect on learning until doses that produced general behavioral disruption were administered.     

Temporal control in a complex environment: An analysis of schedule-related behavior

Three experiments conducted in an automated ten-compartment chamber recorded collateral activities of rats reinforced for lever pressing on differential-reinforcement-of-low-rate schedules. In Experiment 1, the rate of lever pressing increased when stimulus support for collateral activities was removed, thus confirming earlier findings. However, there were no temporal or sequential patterns of collateral activities that predicted operant responding. In Experiment 2, the rate of lever pressing increased only if (a) access to all stimulus support for collateral activities was simultaneously prevented, and (b) the rat was forced to remain in the presence of the lever and food tray. The availability of any of the stimuli related to collateral activity was sufficient to keep lever-pressing rates from increasing. Experiment 3 examined collateral activities under a signaled differential-reinforcement-of-low-rate schedule. Preventing access to stimuli supporting collateral activities had little effect on stable lever pressing when the signal was maintained. When the signal was removed, collateral activities continued, but lever-pressing rates increased in three of the four rats and rates of food presentation declined in all rats. Hypotheses that collateral activities have (a) a timekeeping or discriminative function, or (b) directly inhibit operant responding were not supported. The results suggest that collateral activities may facilitate operant responding by simply removing the subject from the presence of reinforcement-related stimuli.     

Observing behavior in squirrel monkeys under a multiple schedule of reinforcement availability

Observing behavior of two squirrel monkeys was examined under a multiple schedule of four components. Lever (observing) responses produced either a stimulus indicating the availability of food or another stimulus indicating food was not available. Key responses in the presence of the food-available stimulus produced food on a continuous reinforcement schedule. In the absence of food-available stimuli, responding on the key had no scheduled consequences. Observing responses produced food-available stimuli according to three different random-interval schedules with mean interstimulus availability times of 1, 2, and 4 min. In the fourth component of the multiple schedule (observing extinction) food-available stimuli never occurred. Each component of the schedule was correlated with a distinctive auditory stimulus. Observing rates decreased with decreasing frequency of the food-available stimulus. Observing rates during extinction continued decreasing when the brief stimulus indicating food unavailability was no longer produced by lever pressing. When the brief stimulus was reinstated response rates increased abruptly.     

Summation of punishment suppression

In two experiments, eight rats were trained to lever press with food on a variable-interval schedule. Bar pressing produced shock on a variable-interval schedule in the presence of two independently presented stimuli, a light and a tone. Two rats in each experiment received alternative presentations of the light and the tone and were consequently always in the presence of a stimulus that signalled variable-interval punishment. The other two rats in each experiment were treated similarly except that they received periods in which neither light nor tone was present. During these periods, bar pressing was not punished. The two stimuli that signalled punishment were then presented simultaneously to evaluate the effect of stimulus compounding on response suppression. The subjects trained without punishment-free periods did not show summation to the compound stimulus; the subjects trained with punishment-free periods showed summation of suppression. The major difference between the two experiments was the longer mean interval of variable-interval punishment used in the second experiment. This manipulation made the summation effect more resistant to extinction and thus increased its magnitude.     

Choice of timeout during response-independent food schedules

Rats' lever pressing terminated visual or auditory stimuli associated with fixed-time or variable-time schedules of food delivery and produced a timeout period during which food delivery could not occur. Lever pressing during a timeout period reinstated the food-associated stimuli and again permitted food delivery according to the fixed-time or variable-time schedules. The mean interfood interval ranged from 1 minute to 16 minutes (variable-time schedules) or 32 minutes (fixed-time schedules); the timer controlling schedule intervals did not stop during timeout periods. The percentage of session time spent in timeout increased when the mean interfood intervals were lengthened and decreased when the mean interfood intervals were shortened. Timeouts were initiated most frequently about half way between successive food deliveries (fixed-time schedules) or after 15 seconds or more had lapsed since the last food delivery (variable-time schedules). Elimination of food delivery increased the percentage of session time spent in timeout, and elimination of the timeout contingency decreased lever press rates. When timeout was produced only when the lever was held in the depressed position, little time was spent in timeout. The main determinants of timeout initiation and termination appeared to be the rate of food delivery, freedom of movement during timeout, and the stimulus change associated with initiation and termination of timeout.     

Techniques for establishing schedules with wheel running as reinforcement in rats.

In three experiments, access to wheel running was contingent on lever pressing. In each experiment, the duration of access to running was reduced gradually to 4, 5, or 6 s, and the schedule parameters were expanded gradually. The sessions lasted 2 hr. In Experiment 1, a fixed-ratio 20 schedule controlled a typical break-and-run pattern of lever pressing that was maintained throughout the session for 3 rats. In Experiment 2, a fixed-interval schedule of 6 min maintained lever pressing throughout the session for 3 rats, and for 1 rat, the rate of lever pressing was positively accelerated between reinforcements. In Experiment 3, a variable-ratio schedule of 20 or 35 was in effect and maintained lever pressing at a very stable pace throughout the session for 2 of 3 rats; for 1 rat, lever pressing was maintained at an irregular rate. When the session duration was extended to successive 24-hr periods, with food and water accessible in Experiment 3, lever pressing settled into a periodic pattern occurring at a high rate at approximately the same time each day. In each experiment, the rats that developed the highest local rates of running during wheel access also maintained the most stable and highest rates of lever pressing.     

Schedules of food postponement: II. Maintenance of behavior by food postponement and effects of the schedule parameter

In Experiment I, food-deprived, feeder-trained squirrel monkeys pressed a lever to postpone brief electric shocks (Response-Shock=Shock-Shock interval=30 seconds). Forty-one three-hour sessions of shock postponement were followed by 120 sessions of concurrent shock and food postponement. The shock schedule was unchanged and the food schedule was Response-food interval–20 seconds, Food-food interval 10 seconds. After concurrent shock and food postponement, the shock schedule was discontinued and 40 sessions of food postponement ensued, followed by 53 sessions of extinction. After extinction, food postponement was resumed for 11 sessions. Stable responding with low food rates was maintained under food-postponement after the concurrent schedule. Responding decreased to low levels under extinction and recovered immediately to previous levels when the food-postponement schedule was re-instated. In Experiment II, a parameter of the food-postponement schedule was studied sequentially. Using the same subjects, the Response-food–Food-food interval was manipulated from four seconds to 80 seconds with several orders of presentation. Relations of response rates and food rates to the parameter were similar to those seen under shock postponement. Exposure to very short postponement times (four seconds), resulting in very high food rates, decreased but did not abolish subsequent responding at longer postponement times. Results are discussed from the point of view that reinforcing functions of stimuli consequent on responding depend on a prior history of scheduled contact with those stimuli.     

Compounding of discriminative stimuli correlated with chained and multiple schedules

In studies of stimulus compounding (1) the stimuli are presented randomly, (2) primary reinforcement is correlated with each stimulus, (3) a specific response is emitted during each stimulus, and (4) the response is necessary to produce the reinforcer. The present experiments assessed the importance of these procedures by (1) presenting light and tone stimuli in fixed order, (2) removing reinforcement (food) during one stimulus, (3) preventing the response (lever pressing) from being emitted, and (4) eliminating the contingency between lever pressing and food. These variables were presented in various combinations within the context of chained and multiple schedules. When the stimuli were combined in the schedule component correlated with each stimulus, the frequency of lever pressing increased in most instances (additive summation). This suggests that the effect of combining stimuli was not closely tied to the specific procedures used in previous experiments. However, presenting the stimuli in a fixed order did have an effect: the level of responding to the compound was generally greatest when the stimuli were combined in the component correlated with the higher frequency of lever pressing to the single stimulus. Additive summation failed to occur consistently when response-independent food was correlated with each stimulus, and when both lever pressing and food were eliminated during one stimulus.     

Effects of reinforcer probability, delay, and response requirements on the choices of rats and pigeons: possible species differences

In Experiment 1 with rats, a left lever press led to a 5-s delay and then a possible reinforcer. A right lever press led to an adjusting delay and then a certain reinforcer. This delay was adjusted over trials to estimate an indifference point, or a delay at which the two alternatives were chosen about equally often. Indifference points increased as the probability of reinforcement for the left lever decreased. In some conditions with a 20% chance of food, a light above the left lever was lit during the 5-s delay on all trials, but in other conditions, the light was only lit on those trials that ended with food. Unlike previous results with pigeons, the presence or absence of the delay light on no-food trials had no effect on the rats' indifference points. In other conditions, the rats showed less preference for the 20% alternative when the time between trials was longer. In Experiment 2 with rats, fixed-interval schedules were used instead of simple delays, and the presence or absence of the fixed-interval requirement on no-food trials had no effect on the indifference points. In Experiment 3 with rats and Experiment 4 with pigeons, the animals chose between a fixed-ratio 8 schedule that led to food on 33% of the trials and an adjusting-ratio schedule with food on 100% of the trials. Surprisingly, the rats showed less preference for the 33% alternative in conditions in which the ratio requirement was omitted on no-food trials. For the pigeons, the presence or absence of the ratio requirement on no-food trials had little effect. The results suggest that there may be differences between rats and pigeons in how they respond in choice situations involving delayed and probabilistic reinforcers.