The relative aversiveness of signalled vs unsignalled avoidance

Subjects avoided shock by pressing on one lever under an unsignalled condition, but by pressing a separate lever they changed the condition to signalled avoidance for 1-min periods. Signalled avoidance periods were identified by a correlated stimulus. All eight subjects responded to change the unsignalled schedule to a signalled one. Once contact with signalled avoidance was made, subjects continued responding to remain in that condition. Other tests showed that changeover responding was greater when the correlated stimulus was presented without the signal than when the signal was presented without the correlated stimulus. An analysis based upon shock and shock-free periods is presented.     

Choosing between predictable shock schedules: Long- versus short-duration signals

Two experiments assessed the relative aversiveness of different duration preshock signals (5 and 20 seconds) and different duration stimuli identifying shock-free periods. In the first experiment, the responding of 15 of 18 rats was maintained when it produced changes from a predictable-shock condition with a 5-second preshock signal to an identical schedule with a 20-second preshock signal; responding was not maintained when it produced the opposite changes. These results occurred with intershock intervals of both 120 seconds and 240 seconds. The second experiment assessed whether changing to the 20-second schedule was maintained by properties of the preshock signals identifying the shock periods or by properties of the stimuli identifying the shock-free periods. Four subjects were given training with the two signaled schedules in an operant chamber and then later given off-baseline preference tests in a shuttlebox. When given a choice between preshock signals, subjects chose the 5-second signals over the 20-second signals. However, when given a choice between stimuli identifying shock-free periods, subjects chose the stimulus identifying the shorter shock-free periods (i.e., the one previously correlated with the 20-second signals). These findings are discussed within the Rescorla and Wagner model of stimulus compounds and within the context of safety as a contrast phenomenon.     

Techniques for establishing schedules with wheel running as reinforcement in rats.

In three experiments, access to wheel running was contingent on lever pressing. In each experiment, the duration of access to running was reduced gradually to 4, 5, or 6 s, and the schedule parameters were expanded gradually. The sessions lasted 2 hr. In Experiment 1, a fixed-ratio 20 schedule controlled a typical break-and-run pattern of lever pressing that was maintained throughout the session for 3 rats. In Experiment 2, a fixed-interval schedule of 6 min maintained lever pressing throughout the session for 3 rats, and for 1 rat, the rate of lever pressing was positively accelerated between reinforcements. In Experiment 3, a variable-ratio schedule of 20 or 35 was in effect and maintained lever pressing at a very stable pace throughout the session for 2 of 3 rats; for 1 rat, lever pressing was maintained at an irregular rate. When the session duration was extended to successive 24-hr periods, with food and water accessible in Experiment 3, lever pressing settled into a periodic pattern occurring at a high rate at approximately the same time each day. In each experiment, the rats that developed the highest local rates of running during wheel access also maintained the most stable and highest rates of lever pressing.     

Intercurrent and reinforced behavior under multiple spaced-responding schedules

Lever pressing in rats was reinforced with food under a multiple spaced-responding schedule. A lever, food cup, and drinking tube were mounted in a running wheel so that lever pressing, running, and licking could be recorded. Running and licking had no scheduled consequences. Lever pressing was reinforced under a multiple schedule with three spaced-responding components and an extinction component. Each component was associated with a different auditory stimulus. Spaced-responding components reinforced only lever presses terminating interresponse times equal to or greater than 10, 20, or 60 sec, respectively. Rates of lever pressing, reinforcement, and licking all decreased as schedule parameter increased. Efficiency of spaced responding, as measured by reinforcements per response, also decreased. Rate of wheel running either increased or increased and then decreased with increasing schedule parameter. Individual running rates differed substantially. Neither licking nor running rate correlated with individual differences in efficiency. Analysis of conditional probabilities among the several response classes showed that, as the schedule requirement increased, the probability of running after a lever press increased and the probability of licking after a lever press decreased. After reinforcement, one subject always pressed the lever next. In the other subjects, the conditional probability of lever pressing, given reinforcement, increased while the probability of licking, given reinforcement, decreased with increasing schedule requirement. Results are discussed in relation to the concepts of schedule-induced and mediating behavior.     

Signal functions in discriminated avoidance behavior

Four rats were trained to lever press under a discriminated avoidance/escape schedule in which separately signalled safe and warning periods were 100 sec and 32 sec respectively. The auditory and not the visual component of the compound warning signal became associated with the discriminative control of lever pressing. Avoidance behavior also came under temporal control, in that the probability of lever pressing increased as the warning period progressed. Timing began with the onset of the warning signal rather than the offset of the safe signal. However, after the warning signal had been progressively eliminated, timing began with the offset of the safe signal. When neither signal was normally available, the temporal distribution of avoidance behavior changed markedly. Drifts in the temporal distribution of lever pressing occurred throughout the study; these were manipulated for two animals.     

Preference for less frequent shock under fixed-interval schedules of electric-shock presentation.

Lever pressing by 2 squirrel monkeys was maintained under fixed-interval 6-min and fixed-interval 2-min schedules of electric-shock presentation. Preference for these schedules was assessed during three experimental phases. In all phases, responses on one lever produced shock according to one or the other fixed-interval schedule, and responses on a second, changeover, lever switched between schedules. The opportunity to change over was presented during separate choice periods (during which the fixed-interval schedules did not operate) that followed the first through fourth shocks in each schedule. If no changeover occurred during those choice periods, a changeover automatically occurred following the fifth shock. In Phase I, durations of the choice periods were fixed. In Phase II, the choice periods equaled a proportion of their respective fixed interval. During Phase III (completed with 1 monkey) a response on the changeover lever during a given choice period reinstated the most recent fixed interval, and a failure to respond resulted in a changeover. During each of these phases, distinct preferences developed for the 6-min schedule. These results suggest that the maintenance of lever pressing by fixed-interval presentation of electric shock may not be an example of positive reinforcement, and that the response-maintaining characteristics of shock presentation may derive from other properties of the schedule.     

The relative aversiveness of signalled vs unsignalled escapable and inescapable shock

In the first study, subjects escaped shock by pressing on a lever under an unsignalled condition. By pressing a different lever they changed the condition to signalled escape for three minute periods. The second study used the changeover procedure to study inescapable-unavoidable shock. Seven rats were used in each study. All subjects in both studies changed over from unsignalled to signalled conditions. Once contact with the signal condition was made, subjects responded to remain in that condition. The three different extinction conditions showed that the correlated stimulus without the signal had greater control over responding than the signal without the correlated stimulus. An analysis based upon shock and shock-free periods was presented.     

Summation of punishment suppression

In two experiments, eight rats were trained to lever press with food on a variable-interval schedule. Bar pressing produced shock on a variable-interval schedule in the presence of two independently presented stimuli, a light and a tone. Two rats in each experiment received alternative presentations of the light and the tone and were consequently always in the presence of a stimulus that signalled variable-interval punishment. The other two rats in each experiment were treated similarly except that they received periods in which neither light nor tone was present. During these periods, bar pressing was not punished. The two stimuli that signalled punishment were then presented simultaneously to evaluate the effect of stimulus compounding on response suppression. The subjects trained without punishment-free periods did not show summation to the compound stimulus; the subjects trained with punishment-free periods showed summation of suppression. The major difference between the two experiments was the longer mean interval of variable-interval punishment used in the second experiment. This manipulation made the summation effect more resistant to extinction and thus increased its magnitude.     

Effects of fixed-time shocks and brief stimuli on food-maintained behavior of rats

When a fixed-time schedule of shocks was presented to rats lever pressing for food on a random-interval schedule, a pattern of behavior developed with a high rate of pressing after shock declining to near zero before the next shock was delivered. Once this pattern had stabilized, one-quarter of the shocks were replaced with brief auditory stimuli (tones) in a random sequence. Tone maintained behavior similar to shock, although tone was never paired with shock. Both tone and shocks elicited responding when presented at various times as probe stimuli, and responding was usually totally suppressed if neither stimulus occurred at the beginning of the fixed-time interval. When other stimuli were paired with tone and shock, only those paired with tone gained discriminative control and elicited responding. These findings suggest that stimuli that signal a shock-free, or safe, period will maintain the pattern of behavior generated by shock on a fixed-time schedule. There is a parallel between this phenomenon and the control of behavior on second-order schedules of positive reinforcement with nonpaired brief stimuli.     

Motivational control of impulsive behavior interacts with choice opportunities

Impulsive behavior has been investigated through choice between a smaller/immediate reinforcer and a larger/delayed reinforcer, or through performance on a differential reinforcement of low rate (DRL) schedule. In the present study, we investigated a methodological divergence between these two procedures: in the former procedure, delay is a consequence of the subject's own choice, whereas in the later procedure, subjects are explicitly reinforced for delaying a response. In Experiment 1, 7 rats maintained at 80% of their free-feeding weights showed poorer efficiency of lever-pressing responses on a DRL 30-s schedule than when they were maintained at 90% of free-feeding weight. In Experiment 2, 16 rats were subjected to a concurrent chain schedule: the initial link was concurrent fixed ratio 1 fixed ratio 1, and each of these alternatives was followed by a short-DRL requirement with a one pellet reinforcer or a long-DRL requirement with a three pellet reinforcer. In one block of trials, rats were not allowed to choose between the two terminal links (forced-choice), whereas in the other block of trials rats were allowed to choose freely between the two terminal links (free-choice). Compared with rats maintained at 95% of their free-feeding weights, rats maintained at 80% of their free-feeding weights showed poorer efficiency in the terminal links’ DRL schedule performance (just as in Experiment 1), but this difference was shown only in the forced-choice blocks. These results indicate that motivational control of DRL schedule performance interacts with type of choice-making opportunity and highlight the direct comparison of motivational control of impulsive choice and DRL schedule performance.     

Variable-interval schedules of timeout from avoidance

Rats were trained on concurrent schedules in which pressing one lever postponed shock and pressing the other occasionally produced a 2-min timeout during which the shock-postponement schedule was suspended and its correlated stimuli were removed. Throughout, the shock-postponement schedule maintained proficient levels of avoidance. Nevertheless, in Experiment 1 responding on the timeout lever was established rapidly, was maintained at stable levels on variable-interval schedules, was extinguished by withholding timeout, was reestablished when timeout was reintroduced, and was brought under discriminative control with a multiple variable-interval extinction schedule of timeout. These results are in contrast with Verhave's (1962) conclusion that timeout is an ineffective reinforcer when presented to rats on intermittent schedules. In Experiment 2 the consequence of responding on the timeout lever was altered so that the shock-postponement schedule remained in effect even though the stimulus conditions associated with timeout were produced for 2 min. Responding extinguished, indicating that suspension of the shock-postponement schedule, not stimulus change, was the source of reinforcement. By establishing the reinforcing efficacy of timeout with standard variable-interval schedules, these experiments illustrate a procedure for studying negative reinforcement in the same way as positive reinforcement.     

Body weight and response acquisition with delayed reinforcement.

The relation between body weight and responding established with unsignaled delayed reinforcement was investigated. In three experiments, naive rats were deprived to either 70%, 80%, or 90% of ad libitum weight and were then exposed to tandem variable-interval 15-s differential-reinforcement-of-other-behavior 30-s schedules. The tandem schedule defined a resetting unsignaled delay-of-reinforcement procedure. In the first experiment, speed of magazine training, acquisition of lever pressing, and final rate of lever pressing were related to body weight. In the next experiment, lever pressing was established and maintained in rats that were magazine trained at 70% of ad libitum weight but that were then exposed to the delay procedure at 90% of ad libitum weight. Responding did not change consistently either across or within subjects in subsequent conditions in which body weight was manipulated. In the final experiment, lever pressing was established and maintained with delayed reinforcement in the absence of magazine training for each of 2 rats at 70% and for 1 of 2 rats at 90% of ad libitum weight. The results further illuminate the conditions under which responding can be established in the absence of training and when such responses are reinforced only following an unsignaled delay period.     

Suppression of operant behavior and schedule-induced licking in rats

The first experiment studied the effects of punishment on rats' lever pressing maintained by a fixed-interval schedule of food reinforcement and on the associated schedule-induced licking. When licking was followed by shock, licking was suppressed but lever pressing was largely unaffected. When lever pressing was followed by shock, lever pressing was suppressed but licking was unaffected. In both cases, the punished behavior recovered its previous unpunished level when the shocks were discontinued. In a second experiment, the rats' lever pressing was maintained by a variable-interval schedule of food reinforcement under which polydipsic licking also developed. Both lever pressing and licking were partially suppressed during a stimulus correlated with occasional unavoidable electric shocks. With a higher shock intensity, both behaviors were suppressed further. Both lever pressing and licking recovered their previous levels when shocks were discontinued. These results show that schedule-induced licking, which has been described as adjunctive behavior, can be suppressed by procedures that suppress reinforced lever pressing, an operant behavior.     

Temporal control in a complex environment: An analysis of schedule-related behavior

Three experiments conducted in an automated ten-compartment chamber recorded collateral activities of rats reinforced for lever pressing on differential-reinforcement-of-low-rate schedules. In Experiment 1, the rate of lever pressing increased when stimulus support for collateral activities was removed, thus confirming earlier findings. However, there were no temporal or sequential patterns of collateral activities that predicted operant responding. In Experiment 2, the rate of lever pressing increased only if (a) access to all stimulus support for collateral activities was simultaneously prevented, and (b) the rat was forced to remain in the presence of the lever and food tray. The availability of any of the stimuli related to collateral activity was sufficient to keep lever-pressing rates from increasing. Experiment 3 examined collateral activities under a signaled differential-reinforcement-of-low-rate schedule. Preventing access to stimuli supporting collateral activities had little effect on stable lever pressing when the signal was maintained. When the signal was removed, collateral activities continued, but lever-pressing rates increased in three of the four rats and rates of food presentation declined in all rats. Hypotheses that collateral activities have (a) a timekeeping or discriminative function, or (b) directly inhibit operant responding were not supported. The results suggest that collateral activities may facilitate operant responding by simply removing the subject from the presence of reinforcement-related stimuli.     

Signal modality and choice between signaled and unsignaled food

Choice between signaled and unsignaled response-independent food schedules was assessed in three experiments using a commitment procedure. In Experiment 1, subjects tested with a 5-s visual signal consistently changed from the signaled to the unsignaled schedule. Changing from the unsignaled to the signaled schedule was observed only occasionally and only at low levels. The same outcome was observed in Experiment 2 with different types of visual signals and with different stimulus combinations identifying the signal period, the signal-absent period, and the unsignaled schedule. In Experiment 3 the visual signal was replaced with an auditory signal for four of the subjects tested in Experiment 2. The subjects then changed from the unsignaled to the signaled schedule or showed a substantial reduction in choice for the unsignaled schedule. The data were assessed using a conditioned-reinforcement interpretation of choice.     

Variable-interval reinforcement schedule value influences responding following REM sleep deprivation

The effects of rapid-eye movement sleep deprivation (REMSD) in rats were studied in relation to variable-interval (VI) reinforcement schedule value. Initially, lever pressing was maintained on a VI 30-s schedule of food pellet delivery. After a baseline was established, rats were repeatedly exposed to 96 hr of REMSD and control conditions of an equivalent duration. Responding decreased following REMSD but not after exposure to control conditions. Lever pressing was then maintained on a VI 15-s schedule of food pellet delivery and exposure to the REMSD and control conditions was repeated. Under this condition following repeated REMSD exposures, rates of lever pressing became similar to baseline responding. A VI 30-s schedule of food pellet delivery was then reinstated and REMSD and control conditions were repeated. Lever pressing following exposure to the REMSD condition decreased for 3 of 4 rats. Results suggest that VI schedule value influences the effects of REMSD on responding.     

Extinction of responding maintained by timeout from avoidance

The resistance to extinction of lever pressing maintained by timeout from avoidance was examined. Rats were trained under a concurrent schedule in which responses on one lever postponed shock on a free-operant avoidance (Sidman) schedule (response-shock interval = 30 s) and responses on another lever produced 2 min of signaled timeout from avoidance on a variable-ratio 15 schedule. Following extended training (106 to 363 2-hr sessions), two experiments were conducted. In Experiment 1 two different methods of extinction were compared. In one session, all shocks were omitted, and there was some weakening of avoidance but little change in timeout responding. In another session, responding on the timeout lever was ineffective, and under these conditions timeout responding showed rapid extinction. The within-session patterns produced by extinction manipulations were different than the effects of drugs such as morphine, which also reduces timeout responding. In Experiment 2 shock was omitted for many consecutive sessions. Response rates on the avoidance lever declined relatively rapidly, with noticeable reductions within 5 to 10 sessions. Extinction of the timeout lever response was much slower than extinction of avoidance in all 4 rats, and 2 rats continued responding at baseline levels for more than 20 extinction sessions. These results show that lever pressing maintained by negative reinforcement can be highly resistant to extinction. The persistence of responding on the timeout lever after avoidance extinction is not readily explained by current theories.     

Response specificity in animal timing.

The stimuli that control responding in the peak procedure were investigated by training rats, in separate sessions, to make two different responses for food reinforcement. During one type of session, lever pressing was normally reinforced 32 s after the onset of a light. During the other type of session, chain pulling was normally reinforced either 8 s after the onset of one auditory cue or 128 s after the onset of a different auditory cue. For both types of sessions, only the appropriate manipulandum was available, and 20% of the trials lasted 240 s and involved no response-contingent consequences. Rats were then tested with the auditory cues in the presence of the lever and the light in the presence of the chain. If the time of reinforcement associated with each stimulus was learned, response rates should peak at these times during transfer testing. However, if a specific response pattern was learned for each stimulus, little transfer should occur. The results did not clearly support either prediction, leading to the conclusion that both a representation of the time of reinforcement and the rat's own behavior may control responding in this situation.     

Concurrent responses during multiple schedules in rats

Abstract.— Rats' lever pressing was reinforced with a sucrose solution. In the presence of a tone pressing one lever was reinforced on a FR 40 schedule, in the absence of the tone pressing another lever was reinforced on a FR 20 schedule. The components alternated every fourth minute. In both components, the duration of pauses in lever pressing after reinforcement was positively correlated with the duration of licking the empty sucrose-delivery mechanism. In the FR 40 component, the duration of pauses was also positively correlated with the number of presses on the non-reforced lever. Licking and non-reinforced lever pressing were decreased when the rate of reinforced lever pressing was high. The response interactions were similar to those described for concurrent reinforcement schedules.     

Choice of higher density signalled shock over lower density unsignalled shock

Unsignalled, inescapable shocks were presented to four albino rats in Experiment 1. By pressing a lever subjects could change the condition to signalled shock for 3-min periods after which unsignalled shock was automatically reinstated. All subjects changed from unsignalled to signalled shock when shock density was the same or when the density of signalled shock was two times greater than unsignalled shock. When the density of signalled shock was four times that of unsignalled shock, three subjects changed to the higher density schedule. One subject changed to a density of signalled shock eight times that of unsignalled shock. The second study showed that the two shock schedules most similar in Experiment 1 were discriminably different because subjects chose lower over higher shock densities when both densities were unsignalled. An analysis stressing safe (signal absent) and unsafe (signal present) periods was discussed.