Sequence schedules of reinforcement

The performance of pigeons was studied under a second-order schedule composed of fixed-interval components, each of which was associated with a different discriminative stimulus, the stimuli occurring in a fixed order. In one condition, food presentation followed the completion of the fourth component. This was designated a fixed-ratio sequence schedule. In another condition, responses in the first component completed after a fixed time were reinforced. This was designated a fixed-interval sequence schedule. Although the stimulus order and maximum reinforcement frequency were identical under the two schedules, considerably more responding occurred under the fixed-interval sequence schedule in all components. Relatively few food presentations occurred after responding during any but the terminal components of the fixed-interval sequence schedule, a feature independent of the parameter values investigated. In addition, while a pattern of increased responding between food presentations prevailed under both schedules, under the fixed-interval sequence schedule the rate in the terminal component was frequently less than in the penultimate component. The fixed-interval sequence schedule appeared to have several properties of simple fixed-interval schedules.     

Conditioned reinforcement in second-order schedules

Pigeons responded under a schedule in which food was presented only after a fixed number of fixed-interval components were completed. Two such second-order schedules were studied: under one, 30 consecutive 2-min fixed-interval components were required; under the other, 15 consecutive 4-min fixed-interval components were required. Under both schedules, when a 0.7-sec stimulus light was presented at completion of each fixed interval, positively accelerated responding developed in each component. When no stimulus change occurred at completion of each fixed interval, relatively low and constant rates of responding prevailed in each component; a similar result was obtained when a 0.7-sec stimulus change occurred at completion of each fixed interval except the one which terminated with primary reinforcement. The 0.7-sec stimulus correlated with food delivery was an effective conditioned reinforcer in maintaining patterns of responding in fixed-interval components despite low average frequencies of food reinforcement.     

Responding under sequence schedules of electric shock presentation

Lever pressing by squirrel monkeys was examined under second-order schedules of electric shock presentation in which different discriminative stimuli were associated with consecutive components (sequence schedules). Components were always two-minute fixed-interval schedules, and three different overall schedules were studied. Under an overall eight-minute fixed-interval schedule, the first component completion after at least eight minutes had elapsed produced electric shock. The number of components actually completed ranged from one to four; thus, different discriminative stimuli were occasionally associated with electric shock presentation. Under an overall “yoked” variable-ratio schedule, electric shock was presented after completion of a variable number of components; the required number and the distribution of components were matched to those obtained under the overall eight-minute fixed-interval schedule. Under an overall fixed-ratio schedule, electric shock was presented after completion of four components (chained schedule). Under all three sequence schedules, responding in early components was characterized by a pause followed by a single response after the end of the two-minute interval; responding in later components was characterized by a shorter pause followed by positively accelerated responding. Manipulation of overall schedules of shock presentation in these complex behavioral situations produced changes in responding comparable to those ordinarily obtained after similar manipulation of dependencies under both single and second-order schedules of food presentation. These experiments extend the range of conditions and levels of complexity under which responding can be maintained by presentation of electric shock.     

Second-order schedules: comparison of different procedures for scheduling paired and nonpaired brief stimuli

Pigeons performed on a second-order schedule in which fixed-interval components were maintained under a variable-interval schedule. Completion of each fixed-interval component resulted in a brief-stimulus presentation and/or food. The relation of the brief stimulus and food was varied across conditions. Under some conditions, the brief stimulus was never paired with food. Under other conditions, the brief stimulus was paired with food; three different pairing procedures were used: (a) a response produced the simultaneous onset of the stimulus and food; (b) a response produced the stimulus before food with the stimulus remaining on during food presentation; (c) a response produced the stimulus and the offset of the stimulus was simultaneous with the onset of the food cycle. The various pairing and nonpairing operations all produced similar effects on performance. Under all conditions, response rates were positively accelerated within fixed-interval components. Total response rates and Index of Curvature measures were similar across conditions. In one condition, a blackout was paired with food; with this different stimulus in effect, less curvature resulted. The results suggest that pairing of a stimulus is not a necessary condition for within-component patterning under some second-order schedules.     

Fixed-interval responding under second-order schedules of food presentation or cocaine injection.

Squirrel monkeys operated a key under second-order schedules in which every tenth completion of a 5-minute fixed interval resulted in either presentation of food or intravenous injection of cocaine. When a 2-second light was presented at the completion of the component fixed-interval schedules, positively accelerated responding developed and was maintained in each component. Over a tenfold range of doses of cocaine(30 to 300 microgram/kg/injection) and amounts of food (0.75 to 7.5 g/presentation); the second-order schedule of cocaine injection maintained higher average rates of responding than the second-order schedule of food presentation. Substituting saline for cocaine or eliminating food presentation decreased average rates of responding. When no stimulus change occurred at the completion of the first nine component fixed-interval schedules, but the 2-second light and food presentation or cocaine injection still occurred after the tenth component, only low and relatively constant rates of responding were maintained in each component. Patterns of responding characteristic of 5-minute fixed-interval schedules were maintained by the 2-second light paired with either cocaine injection or food presentation, though the maximum frequency of cocaine injection or food presentation was less than once per 50 minutes.     

Fixed versus variable sequences of food and stimulus presentation in second-order schedules

Three pigeons were exposed to a second-order schedule in which the behavior specified by a fixed-interval component schedule was reinforced according to a ratio overall schedule. The completion of components not followed by food was signalled by a brief stimulus never paired with food. Food and the stimulus occurred in a random sequence or in fixed alternation, but the overall schedules (variable ratio 2 or fixed ratio 2) ensured that an equal number of food and brief-stimulus presentations occurred in each session. The control exerted by the food and by the brief stimulus was measured by overall response rates, mean pauses, frequency distributions of pauses, and response patterning across components. In general, the stimulus controlled patterns of behavior more similar to those controlled by food when food and the stimulus occurred in a random sequence than when they occurred in fixed alternation.     

Second-order schedules and the problem of conditioned reinforcement

Thirteen pigeons were exposed to a variety of second-order schedules in which responding under a component schedule was reinforced according to a schedule of reinforcement. Under different conditions, completion of each component resulted in either (1) the brief presentation of a stimulus also present during reinforcement (pairing operation), (2) the brief presentation of a stimulus not present during reinforcement (nonpairing operation), or (3) no brief stimulus presentation (tandem). Brief-stimulus presentations engendered a pattern of responding within components similar to that engendered by food. Patterning was observed when fixed-interval and fixed-ratio components were maintained under fixed- and variable-ratio and fixed- and variable-interval schedules. There were no apparent differences in performance under pairing and nonpairing conditions in any study. The properties of the stimuli presented in brief-stimulus operations produced different effects on response patterning. In one study, similar effects on performance were found whether brief-stimulus presentations were response-produced or delivered independently of responding. Response patterning did not occur when the component schedule under which a nonpaired stimulus was produced occurred independently of the food schedule. The results suggest a reevaluation of the role of conditioned reinforcement in second-order schedule performance. The similarity of behavior under pairing and nonpairing operations is consistent with two hypotheses: (1) the major effect is due to the discriminative properties of the brief stimulus; (2) the scheduling operation under which the paired or nonpaired stimulus is presented can establish it as a reinforcer.     

Second-order schedules: discrimination of components

Pigeons were exposed to a series of second-order schedules in which the completion of a fixed number of fixed-interval components produced food. In Experiment 1, brief (2 sec) stimulus presentations occurred as each fixed-interval component was completed. During the brief-stimulus presentation terminating the last fixed-interval component, a response was required on a second key, the brief-stimulus key, to produce food. Responses on the brief-stimulus key before the last brief-stimulus presentation had no scheduled consequences, but served as a measure of the extent to which the final component was discriminated from preceding components. Whether there were one, two, four, or eight fixed-interval components, responses on the brief-stimulus key occurred during virtually every brief-stimulus presentation. In Experiment 2, an attempt was made to punish unnecessary responses on the brief-stimulus key, i.e., responses on the brief-stimulus key that occurred before the last component. None of the pigeons learned to withhold these responses, even though they produced a 15-sec timeout and loss of primary reinforcement. In Experiment 3, different key colors were associated with each component of a second-order schedule (a chain schedule). In contrast to Experiment 1, brief-stimulus key responses were confined to the last component. It was concluded that pigeons do not discriminate well between components of second-order schedules unless a unique exteroceptive cue is provided for each component. The relative discriminability of the components may account for the observed differences in initial-component response rates between comparable brief-stimulus, tandem, and chain schedules.     

Contiguity of briefly presented stimuli with food reinforcement

Pigeons performed on second-order schedules of reinforcement consisting of four fixed-interval components. Only the terminal component ended with food. Performance was studied both when a brief stimulus followed the completion of each of the first three fixed intervals (brief-stimulus schedule) and when the stimulus was omitted (tandem schedule). Variations in the temporal contiguity of the last presentation of the stimulus and the presentation of food indicated that the shorter the delay, the greater was the enhancement of rate of responding in comparison with tandem performance. A positively accelerated pattern of responding within fixed-interval components was a function of the contiguity of the brief stimulus and reinforcement; this pattern was absent for all tandem-schedule performance.     

Polydipsia induced in rats by second-order schedules of reinforcement.

Three rats were exposed to a second-order schedule in which fixed-interval components ended either with food or with a brief stimulus that was never paired with food. Food and the brief stimulus occurred in a random sequence (variable-ratio 2 overall schedule). Another three rats were exposed to a similar second-order schedule, the only difference being that the food or the stimulus was presented independently of operant behavior (fixed-time components). The three rats exposed to the fixed-interval components licked at a water spout after each food presentation. These rats also licked in the intervals after the brief stimulus. Although the discriminative properties of food and of the brief stimulus were identical in relation to subsequent reinforcement, licking after the stimulus was less than after food. The three rats exposed to the second-order schedules with fixed-time components also licked at the water spout after food, but these rats did not lick consistently after brief stimulus presentations.     

Responding in the pigeon under chained schedules of food presentation: the repetition of a stimulus during alternate components

Key pecking in the pigeon was maintained under chained schedules in which the completion of one schedule component initiated the next component, and food was presented upon completion of a sequence of components. Under the chained schedules studied, a particular key color appeared during more than one component, and different key colors appeared during the other components. When seven 1-min fixed-interval components comprised a chained schedule and the response key was the same color during the first, third, fifth, and terminal components, patterns of positively accelerated responding were maintained during all but the first two components of each sequence. In general, response rates were always lowest during the first one or two components and highest during the terminal component when as few as three and as many as eight components comprised a schedule. Increasing the number of components from three to eight showed that response rate during a component increased when it was no longer one of the initial two components of the schedule, even though its temporal relation to food presentation had not changed. Finally, when seven components comprised a schedule and the response key was one color during the first, third, and fifth and a different color during the last component, response rates were low during the first five components and high during the last two components preceding food presentation.     

Responding maintained under fixed-interval and fixed-time schedules of electric shock presentation

Following initial histories under a schedule of electric shock postponement, lever pressing in squirrel monkeys was maintained under fixed-interval and fixed-time schedules of electric shock presentation. No difference in either rate or pattern of responding was obtained when these schedules were presented as components of a multiple schedule. When they were presented singly for long periods of time, the fixed-interval schedule consistently maintained a higher response rate than the fixed-time schedule. The pattern of responding under both schedules was similar, typically consisting of a pause at the beginning of each interval followed by either a steady or a positively accelerating rate of responding. The results suggest that the response-shock dependency is of critical importance in the maintenance of high rates of responding under schedules of electric shock presentation, and support the general view that such responding may be conceptualized as operant behavior under control of many of the same variables that control responding under comparable schedules of food or water reinforcement.     

Effects of d-amphetamine on responding under second-order schedules of reinforcement with paired and nonpaired brief stimuli.

Three pigeons were studied under a multiple schedule in which pecks in each component were reinforced according to a variable-interval 120-s second-order schedule with fixed-interval 60-s units. In the first component of the multiple schedule, the completion of a fixed interval produced either food or a 4-s change in key color plus houselight illumination. In the second component an identical schedule was in effect, but the stimulus was a 0.3-s change in key color. Both long and short brief stimuli were not paired with food presentations in Conditions 1 and 3 and were paired with food in Condition 2. There were no consistent differences in response patterns under paired and nonpaired brief-stimulus conditions when the stimulus was a 4-s change in key color accompanied by houselight illumination. However, pairing the 0.3-s key-color change with food presentations resulted in higher indices of curvature and lower response rates in the early segments of the fixed interval than when the stimulus was not paired with food presentations. Low doses of d-amphetamine (0.3 and 1 mg/kg) produced small and inconsistent increases in overall response rates, and higher doses (3 and 10 mg/kg) decreased overall response rates. d-Amphetamine altered response patterns within fixed intervals by decreasing the indices of curvature and increasing response rates in the early segments of the fixed interval. Response rates and patterns under paired and nonpaired brief-stimulus conditions were not differentially affected by d-amphetamine. Thus, evidence for the enhancement of the conditioned reinforcement effects of psychomotor stimulant drugs was not found with the second-order schedules used in the present study.     

Responding under fixed-ratio and multiple fixed-interval fixed-ratio schedules of electric shock presentation

Squirrel monkeys, initially trained under a schedule of electric shock postponement and then under fixed-interval schedules of electric shock presentation, were studied under multiple fixed-interval fixed-ratio and under fixed-ratio schedules of shock presentation. Under the fixed-interval (10-min) component of the multiple schedule, a pause was followed by a gradual increase in responding to a rate maintained until shock presentation; under the fixed-ratio (3-, 10-, or 30-response) component of the multiple schedule, a brief pause was typically followed by a relatively high and uniform rate of responding until shock was presented. When the 60-sec timeout periods, which usually followed shock presentation, were eliminated from the multiple schedule for one monkey, responding was only transiently affected. In the one monkey studied, responding was maintained under a fixed-ratio schedule alone (with timeout periods), but rates of responding were lower than under the fixed-ratio component of the multiple schedule. Characteristic patterns of responding, similar to those engendered under schedules of food presentation or shock termination, can be maintained under fixed-ratio schedules of shock presentation; further, patterns of responding can be controlled by discriminative stimuli in multiple schedules.     

Responding in the cat maintained under response-independent electric shock and response-produced electric shock

Key-pressing responses in the cat were maintained under conditions in which brief electric shock was first postponed by responses (avoidance), then periodically presented independently of responses, and finally produced by responses on a fixed-interval schedule of 15 min (FI 15-min). A steady rate of responding occurred under shock avoidance and under response-independent shock; positively accelerated responding was engendered by the FI 15-min schedule. A second experiment studied responding under second-order schedules composed of three FI 5-min components. Responding was suppressed when a stimulus was presented briefly at completion of each FI 5-min component and a shock followed the brief stimulus at completion of the third component. Responding was maintained when each of the first two components was completed either with or without presentation of a brief stimulus and a shock alone was presented at completion of the third FI 5-min component.     

Fixed-interval stimulus control

Chlorpromazine was studied for its effects on responding under a second-order schedule in which food was presented following a sequence of 20 one-minute fixed-interval components. A brief visual stimulus occurred at the completion of each fixed interval including the one that terminated with food presentation. Chlorpromazine showed rate-dependent effects in that it increased low rates in the early components of the second-order schedule and, to a lesser extent, decreased high rates in the later components. Chlorpromazine also increased rates in the early quarters within the 1-min fixed-internal components and to a smaller extent decreased rates in the final quarter. The alteration in the patterns of responding within 1-min fixed-interval components terminating in a brief stimulus presentation was substantially less than that which occurred throughout the succession of 1-min fixed-interval components terminating in food presentation, thus suggesting that the presentation of the brief stimulus exerted more control over responding within components than did food presentation over the sequence of components. This result and others suggest that studies using drugs may be useful in elucidating the factors controlling patterns of responding in second-order schedules.     

A comparison of responding maintained under second-order schedules of intramuscular cocaine injection or food presentation in squirrel monkeys.

Key pressing by squirrel monkeys was maintained under second-order schedules of either intramuscular cocaine injection or food presentation. Under one schedule, each completion of a 10-response fixed-ratio unit produced a brief visual stimulus; the first fixed-ratio unit completed after 30 minutes elapsed produced the stimulus paired with either cocaine injection or food presentation. Generally, short pauses followed by high rates of responding were maintained within the fixed-ratio units, and responding was positively accelerated over the 30-minute interval. Under another schedule, each completion of a 3-minute fixed-interval unit produced the brief stimulus; completion of the 10th fixed-interval unit produced the stimulus paired with either cocaine injection or food presentation. Generally, short pauses followed by high rates of responding were maintained within the fixed-ratio units, and responding was positively accelerated over the 30-minute interval. Under another schedule, each completion of a 3-minute fixed-interval unit produced the brief stimulus; completion of the 10th fixed-interval unit produced the stimulus paired with either cocaine injection or food presentation. Rates of responding increased within the fixed-interval units, and to a greater extent over the entire 10 fixed-interval units. Patterns of responding depended more on the schedule of reinforcement than on whether cocaine or food maintained responding. Omitting the brief stimuli following all but the last fixed-ratio or fixed-interval units decreased average rates and altered the patterns of responding. Substituting a visual stimulus that was never paired with cocaine or food following all but the last fixed-ratio or fixed-interval units decreased response rates to a lesser extent and did not substantially alter patterns of responding. When the duration of the paired stimulus was varied from .3 to 30.0 seconds, the highest response rates occurred at intermediate durations (1.0 to 10.0 seconds). The manner in which the stimulus changes affected performances depended more on the schedule of reinforcement than on whether cocaine injection or food presentation maintained responding.     

Drugs and punished responding II: d-amphetamine-induced increases in punished responding

The effects of d-amphetamine on punished responding were studied in two experiments. In Experiment I, pigeons responded under a multiple fixed-ratio 30 response fixed-interval 5-min schedule of food presentation with 60-sec limited holds in both components. Each response was punished with electric shock, the intensity of which was varied systematically. In Experiment II, another group of pigeons responded under a multiple fixed-interval 5-min fixed-interval 5-min schedule of food presentation with 40-sec limited holds. Each response was punished with shock during one component, and every thirtieth response was punished in the other component. d-Amphetamine increased overall rates of punished responding only rarely under any of the punishment conditions; however, response rates within the fixed-interval when rates were low were increased by d-amphetamine when the shock intensity was low (Experiment I), or when responses produced shock intermittently (Experiment II). The data suggest that the effects of d-amphetamine on punished responding depend on the control rate of responding, the punishment intensity, the punishment frequency, and the schedule of food presentation.     

Conditioned suppression or facilitation as a function of the behavioral baseline

Rats were exposed to a multiple schedule of reinforcement. During one component, a bar-press was followed by reinforcement only if it occurred between 15 and 20 sec after the previous response. This differential-reinforcement-of-low-rate (DRL) schedule produced a typical slow rate of responding. During the other component, reinforcement followed the first response to be emitted during limited periods of time which occurred at fixed intervals. These fixed-interval schedules with a limited hold produced higher response rates, described as `interval' or `ratio-like' behavior. Responding during the DRL component increased in frequency during a tone which ended with an unavoidable shock of low intensity, but decreased during the tone when the shock intensity was raised. The `interval' and `ratio-like' responding decreased in frequency during the tone at all shock intensities. Initial acceleration of the DRL responding appeared to be due to adventitious punishment of collateral behavior which was observed between the bar-presses. The more severe conditioned suppression during the fixed-interval components might be the result of the lower probability of reinforcement after any single response.     

Relations between patterns of responding and the presentation of stimuli under second-order schedules

Key-pecking behavior in the pigeon was maintained under second-order schedules in which food was presented after a variable number of 2-min fixed-interval components were completed. When either the same stimulus (Exp. I) or different stimuli (Exp. II) appeared on the key during consecutive components, and a stimulus that was occasionally paired with food was presented briefly at completion of each component, (1) patterns of positively accelerated responding were maintained during the components, and, (2) mean response rates were generally as high during the initial components of a sequence as during the later components. In both experiments, when the food-paired stimulus was omitted and either no stimulus or a stimulus never paired with food was presented at completion of each component, mean rates of responding increased, but patterns of positively accelerated responding were not maintained during individual components. When a food-paired stimulus was not presented at completion of the components, mean response rates in Exp. I were low during the initial components of a sequence and gradually increased during subsequent components; in Exp. II mean response rates were variable, and pauses and abrupt changes in response rates were typical.