RESPONDING MAINTAINED UNDER INTERMITTENT SCHEDULES OF ELECTRIC-SHOCK PRESENTATION: “SAFETY” OR SCHEDULE EFFCTS?

Four experiments were conducted in which lever pressing by squirrel monkeys was maintained under multiple, mixed, or chained schedules of electric-shock presentation. In the first two experiments, a multiple schedule was employed in which a fixed-interval schedule of shock presentation alternated with a signaled two-minute component. Initially, no events were scheduled during the two-minute component (a safety period). In the first experiment, the safety period was “degraded” by introducing and systematically increasing the frequency of periodic shocks presented during that component. In the second experiment, the proportion of overall safe time to unsafe time was decreased by decreasing the value of the fixed-interval schedule while holding constant shock frequency during the two-minute component. In the third experiment, the overall arrangement was changed from a multiple to a mixed schedule in an attempt to determine whether fixed-interval responding would be maintained when a single exteroceptive stimulus was associated with both components. In the fourth experiment, the overall arrangement was changed from a multiple to a chained schedule in an effort to determine whether fixed-interval responding would be maintained when its consequence was presentation of a signaled “unsafe” period. Fixed-interval responding was well maintained under all experimental conditions; the varied relationships obtained lend more support to conceptualizations of shock-maintained behavior as exemplifying schedule-controlled behavior than to suggestions that such behavior may be readily accounted for by “safety theory.”     

Responding under fixed-ratio and multiple fixed-interval fixed-ratio schedules of electric shock presentation

Squirrel monkeys, initially trained under a schedule of electric shock postponement and then under fixed-interval schedules of electric shock presentation, were studied under multiple fixed-interval fixed-ratio and under fixed-ratio schedules of shock presentation. Under the fixed-interval (10-min) component of the multiple schedule, a pause was followed by a gradual increase in responding to a rate maintained until shock presentation; under the fixed-ratio (3-, 10-, or 30-response) component of the multiple schedule, a brief pause was typically followed by a relatively high and uniform rate of responding until shock was presented. When the 60-sec timeout periods, which usually followed shock presentation, were eliminated from the multiple schedule for one monkey, responding was only transiently affected. In the one monkey studied, responding was maintained under a fixed-ratio schedule alone (with timeout periods), but rates of responding were lower than under the fixed-ratio component of the multiple schedule. Characteristic patterns of responding, similar to those engendered under schedules of food presentation or shock termination, can be maintained under fixed-ratio schedules of shock presentation; further, patterns of responding can be controlled by discriminative stimuli in multiple schedules.     

Responding maintained under fixed-interval and fixed-time schedules of electric shock presentation

Following initial histories under a schedule of electric shock postponement, lever pressing in squirrel monkeys was maintained under fixed-interval and fixed-time schedules of electric shock presentation. No difference in either rate or pattern of responding was obtained when these schedules were presented as components of a multiple schedule. When they were presented singly for long periods of time, the fixed-interval schedule consistently maintained a higher response rate than the fixed-time schedule. The pattern of responding under both schedules was similar, typically consisting of a pause at the beginning of each interval followed by either a steady or a positively accelerating rate of responding. The results suggest that the response-shock dependency is of critical importance in the maintenance of high rates of responding under schedules of electric shock presentation, and support the general view that such responding may be conceptualized as operant behavior under control of many of the same variables that control responding under comparable schedules of food or water reinforcement.     

Sequence schedules of reinforcement

The performance of pigeons was studied under a second-order schedule composed of fixed-interval components, each of which was associated with a different discriminative stimulus, the stimuli occurring in a fixed order. In one condition, food presentation followed the completion of the fourth component. This was designated a fixed-ratio sequence schedule. In another condition, responses in the first component completed after a fixed time were reinforced. This was designated a fixed-interval sequence schedule. Although the stimulus order and maximum reinforcement frequency were identical under the two schedules, considerably more responding occurred under the fixed-interval sequence schedule in all components. Relatively few food presentations occurred after responding during any but the terminal components of the fixed-interval sequence schedule, a feature independent of the parameter values investigated. In addition, while a pattern of increased responding between food presentations prevailed under both schedules, under the fixed-interval sequence schedule the rate in the terminal component was frequently less than in the penultimate component. The fixed-interval sequence schedule appeared to have several properties of simple fixed-interval schedules.     

A comparison of responding maintained under second-order schedules of intramuscular cocaine injection or food presentation in squirrel monkeys.

Key pressing by squirrel monkeys was maintained under second-order schedules of either intramuscular cocaine injection or food presentation. Under one schedule, each completion of a 10-response fixed-ratio unit produced a brief visual stimulus; the first fixed-ratio unit completed after 30 minutes elapsed produced the stimulus paired with either cocaine injection or food presentation. Generally, short pauses followed by high rates of responding were maintained within the fixed-ratio units, and responding was positively accelerated over the 30-minute interval. Under another schedule, each completion of a 3-minute fixed-interval unit produced the brief stimulus; completion of the 10th fixed-interval unit produced the stimulus paired with either cocaine injection or food presentation. Generally, short pauses followed by high rates of responding were maintained within the fixed-ratio units, and responding was positively accelerated over the 30-minute interval. Under another schedule, each completion of a 3-minute fixed-interval unit produced the brief stimulus; completion of the 10th fixed-interval unit produced the stimulus paired with either cocaine injection or food presentation. Rates of responding increased within the fixed-interval units, and to a greater extent over the entire 10 fixed-interval units. Patterns of responding depended more on the schedule of reinforcement than on whether cocaine or food maintained responding. Omitting the brief stimuli following all but the last fixed-ratio or fixed-interval units decreased average rates and altered the patterns of responding. Substituting a visual stimulus that was never paired with cocaine or food following all but the last fixed-ratio or fixed-interval units decreased response rates to a lesser extent and did not substantially alter patterns of responding. When the duration of the paired stimulus was varied from .3 to 30.0 seconds, the highest response rates occurred at intermediate durations (1.0 to 10.0 seconds). The manner in which the stimulus changes affected performances depended more on the schedule of reinforcement than on whether cocaine injection or food presentation maintained responding.     

Effects of chlorpromazine on fixed-ratio responding: modification by fixed-interval discriminative stimuli.

Effects of chlorpromazine (1 to 100 mg/kg) were assessed on two pigeons' responding under various modifications of a multiple schedule of food delivery. During a fixed-interval component, the first response after 5 min produced food; during the subsequent, fixed-ratio component, the 30th response produced food. Modifications of the schedule entailed changes in stimulus conditions imposed during the fixed-ratio component that did not systematically alter characteristics of performance under non-drug conditions. In the first phase of the experiment, distinctive visual stimuli were correlated with each schedule component (conventional multiple schedule); chlorpromazine produced small decreases in fixed-ratio responding (20% at 30 mg/kg). When each response during the fixed-ratio component produced the stimulus correlated with the fixed-interval schedule (fixed-interval discriminative stimulus) for 1.2 s, effects of chlorpromazine were not different from those under the conventional multiple schedule. Chlorpromazine produced greater decreases in fixed-ratio responding (55% at 30 mg/kg) when either the first response of each fixed ratio changed the stimulus correlated with the fixed-ratio schedule to the fixed-interval discriminative stimulus for the remainder of the fixed-ratio component, or when the fixed-interval discriminative stimulus was presented independently of responding according to a matched temporal sequence. When the fixed-interval discriminative stimulus was present continuously during the fixed-ratio component (mixed schedule), chlorpromazine produced even more substantial decreases in fixed-ratio responding (greater than 80% at 30 mg/kg). Effects of chlorpromazine on fixed-interval responding were also modified by the schedules of fixed-interval discriminative stimulus presentation. The effects of chlorpromazine were a joint function of the stimuli prevailing during the multiple schedule and the degree to which responding influenced these stimuli.     

Fixed-interval responding under second-order schedules of food presentation or cocaine injection.

Squirrel monkeys operated a key under second-order schedules in which every tenth completion of a 5-minute fixed interval resulted in either presentation of food or intravenous injection of cocaine. When a 2-second light was presented at the completion of the component fixed-interval schedules, positively accelerated responding developed and was maintained in each component. Over a tenfold range of doses of cocaine(30 to 300 microgram/kg/injection) and amounts of food (0.75 to 7.5 g/presentation); the second-order schedule of cocaine injection maintained higher average rates of responding than the second-order schedule of food presentation. Substituting saline for cocaine or eliminating food presentation decreased average rates of responding. When no stimulus change occurred at the completion of the first nine component fixed-interval schedules, but the 2-second light and food presentation or cocaine injection still occurred after the tenth component, only low and relatively constant rates of responding were maintained in each component. Patterns of responding characteristic of 5-minute fixed-interval schedules were maintained by the 2-second light paired with either cocaine injection or food presentation, though the maximum frequency of cocaine injection or food presentation was less than once per 50 minutes.     

Behavior controlled by scheduled injections of cocaine in squirrel and rhesus monkeys.

Rates and patterns of key-press responding maintained under schedules in which responding resulted in intravenous injections of cocaine were studied in squirrel monkeys and rhesus monkeys. Each injection was followed by a 60- or 100-sec timeout period. Schedule-controlled behavior was obtained at appropriate cocaine doses in each species. Under FR 10 or FR 30 schedules, performance was characterized by high rates of responding (usually more than one response per second) in each ratio. Under FI 5-min schedules, performance was characterized by an initial pause, followed by acceleration of responding to a final rate that was maintained until the end of the interval. Under multiple fixed-ratio fixed-interval schedules, rates and patterns of responding appropriate to each schedule component were maintained. Responding seldom occurred during timeout periods under any schedule studied. At doses of cocaine above or below those that maintained characteristic schedule-controlled behavior, rates of responding were relatively low and patterns of responding were irregular. Characteristic fixed-interval responding was maintained over a wider range of cocaine doses than characteristic fixed-ratio responding. Complex patterns of responding controlled by discriminative stimuli under fixed-ratio or fixed-interval schedules can be maintained by cocaine injections in squirrel monkeys and rhesus monkeys.     

Reinforcement by an imprinting stimulus versus water on simple schedules in ducklings

Ducklings (5 to 28 days old) were trained to peck a pole on fixed-ratio, fixed-interval, and multiple schedules using brief presentation of an imprinting stimulus as the response-contingent event. Other ducklings of the same age were trained similarly except that reinforcement consisted of access to water. With water reinforcement the typical fixed-ratio (“break-run”), fixed-interval (“scallop”), and multiple schedule response patterns were readily established and consistently maintained. With the imprinting stimulus these schedule effects were inconsistent in some subjects and virtually nonexistent in others, despite extended training. Schedule control with the imprinting stimulus was not improved by the use of a reinforcement signaling procedure which enhances responding reinforced by electrical brain stimulation on intermittent schedules. However, the overall rates of responding and the extinction functions generated after reinforcement with water versus the imprinting stimulus were comparable. These findings imply that control by temporal and discriminative stimuli may be relatively weak when a young organism's behavior is reinforced by presentation of an imprinting stimulus.     

Second-order schedules: brief shock at the completion of each component

Pigeons worked on second-order schedules in which completion of fixed-interval component schedules was reinforced with food according to a variable-interval schedule of reinforcement. The completion of each fixed-interval component resulted in the presentation of a brief electric shock. In one condition (shock-paired), the completion of every fixed-interval component, including those that ended in food, resulted in the shock. In another condition (shock-nonpaired), completion resulted in shock except for those components that ended in food. Shock presentations resulted in a positively accelerated rate within fixed-interval components. This patterning within components was similar whether the shock was intermittently paired with food or not. Response rates tended to decrease as shock intensity increased. The characteristic fixed-interval response pattern within components did not occur when shock presentations were omitted at the end of each component (tandem schedule). When shocks were scheduled but food was no longer presented (extinction) response rates declined to a near-zero level. The performance under shock conditions is similar to that in other studies in which visual and auditory stimuli are presented at the completion of component schedules.     

Responding in the cat maintained under response-independent electric shock and response-produced electric shock

Key-pressing responses in the cat were maintained under conditions in which brief electric shock was first postponed by responses (avoidance), then periodically presented independently of responses, and finally produced by responses on a fixed-interval schedule of 15 min (FI 15-min). A steady rate of responding occurred under shock avoidance and under response-independent shock; positively accelerated responding was engendered by the FI 15-min schedule. A second experiment studied responding under second-order schedules composed of three FI 5-min components. Responding was suppressed when a stimulus was presented briefly at completion of each FI 5-min component and a shock followed the brief stimulus at completion of the third component. Responding was maintained when each of the first two components was completed either with or without presentation of a brief stimulus and a shock alone was presented at completion of the third FI 5-min component.     

Behavior simultaneously maintained by both presentation and termination of noxious stimuli

Lever pressing by two squirrel monkeys was maintained under a 3-minute variable-interval schedule of response-produced electric-shock presentation. At the same time, responding on a second lever was maintained under a 3-minute fixed-interval schedule of termination of the shock-presentation schedule and shock-correlated stimuli. Under the termination schedule, the first response after a 3-minute period produced a 1-minute timeout, during which no events occurred and responding had no scheduled consequence. Relatively high and constant rates of responding were maintained on the lever where responding produced shock. Lower rates and positively accelerated patterns of responding occurred on the lever where responding terminated the shock schedule. Thus, responding was simultaneously maintained by presentation of an event and by termination of a stimulus associated with that event. Rates and patterns of responding on each lever were reversed when the schedules arranged on each lever were reversed on two occasions. When shock intensity was increased from 0 to 10 mA, responding maintained both by presentation of shock and by termination of the shock schedule increased, but responding maintained by shock presentation increased to a greater extent. Positive and negative reinforcement, usually regarded as separate behavioral processes involving different events, can coexist when behavior is controlled by different contingencies involving the same event.     

Changing the response unit from a single peck to a fixed number of pecks in fixed-interval schedules

Each of three pigeons was studied first under a standard fixed-interval schedule. With the fixed interval held constant, the schedule was changed to a second-order schedule in which the response unit was the behavior on a small fixed-ratio schedule (first a fixed-ratio 10 and then a fixed-ratio 20 schedule). That is, every completion of the fixed-ratio schedule produced a 0.7-sec darkening of the key and reset the response count to zero for the next ratio. The first fixed-ratio completed after the fixed-interval schedule elapsed produced the 0.7-sec blackout followed immediately by food. These manipulations were carried out under two different fixed-interval durations for each bird ranging from 3 min to 12 min. The standard fixed-interval schedules produced the typical pause after reinforcement followed by responding at a moderate rate until the next reinforcement. The second-order schedules also engendered a pause after reinforcement, but responding occurred in bursts separated by brief pauses after each blackout. For a particular fixed-interval duration, post-reinforcement pauses increased slightly as the number of pecks in the response unit increased despite large differences in the rate and pattern of key pecking. Post-reinforcement pause increased with the fixed-interval duration under all response units. These data confirm that the allocation of time between pausing and responding is relatively independent of the rate and topography of responding after the pause.     

Second-order schedules and the problem of conditioned reinforcement

Thirteen pigeons were exposed to a variety of second-order schedules in which responding under a component schedule was reinforced according to a schedule of reinforcement. Under different conditions, completion of each component resulted in either (1) the brief presentation of a stimulus also present during reinforcement (pairing operation), (2) the brief presentation of a stimulus not present during reinforcement (nonpairing operation), or (3) no brief stimulus presentation (tandem). Brief-stimulus presentations engendered a pattern of responding within components similar to that engendered by food. Patterning was observed when fixed-interval and fixed-ratio components were maintained under fixed- and variable-ratio and fixed- and variable-interval schedules. There were no apparent differences in performance under pairing and nonpairing conditions in any study. The properties of the stimuli presented in brief-stimulus operations produced different effects on response patterning. In one study, similar effects on performance were found whether brief-stimulus presentations were response-produced or delivered independently of responding. Response patterning did not occur when the component schedule under which a nonpaired stimulus was produced occurred independently of the food schedule. The results suggest a reevaluation of the role of conditioned reinforcement in second-order schedule performance. The similarity of behavior under pairing and nonpairing operations is consistent with two hypotheses: (1) the major effect is due to the discriminative properties of the brief stimulus; (2) the scheduling operation under which the paired or nonpaired stimulus is presented can establish it as a reinforcer.     

Separating the reinforcing and discriminative properties of brief-stimulus presentations in second-order schedules.

Pigeons' responses were maintained under multiple schedules to study properties of briefly presented stimuli. Responses in one component produced food according to a second-order schedule with fixed-interval components in which food or a brief stimulus occurred with equal probability. In the second component responses produced only the brief stimulus under a fixed-ratio schedule. Under various conditions the brief stimulus in the first component was (a) paired with food, (b) not paired with food, (c) partially omitted, or (d) scheduled simultaneously with the second-order schedule under an independent variable-interval schedule. Paired and nonpaired brief stimuli maintained similar response patterning in the second-order schedule. However, only paired stimuli maintained responses in the second component. The data suggest that nonpaired brief stimuli engender response patterning in second-order schedules as a result of their discriminative properties. When the stimulus is paired with food, these discriminative properties sometime mask a reinforcement effect, and no change in response patterning is observed. When the discriminative properties of the brief stimulus are absent, the reinforcing effects of pairing the brief stimulus with food may be observed.     

Schedules using noxious stimuli. IV: An interlocking shock-postponement schedule in the squirrel monkey

Responding was studied under various schedules of electric shock postponement and presentatation in the squirrel monkey. Under an interlocking shock-postponement schedule, successive responses decreased the time by which a response postponed the next scheduled shock until a shock immediately followed the nth response. Some parameters of this schedule, which can be formally related to fixed-interval schedules, engendered a pattern of positively accelerated responding between shocks. This pattern did not occur under comparable parameter values of an alternative fixed-ratio, avoidance schedule under which each response postponed shock by a fixed duration and every nth response produced shock. Subsequently, performances were studied under schedules of shock presentation. Responding was never maintained under fixed-ratio schedules of shock presentation, but was maintained with a pattern of positive acceleration under an alternative fixed-ratio, fixed-interval schedule and under a fixed-interval schedule.     

Conditioned reinforcement in second-order schedules

Pigeons responded under a schedule in which food was presented only after a fixed number of fixed-interval components were completed. Two such second-order schedules were studied: under one, 30 consecutive 2-min fixed-interval components were required; under the other, 15 consecutive 4-min fixed-interval components were required. Under both schedules, when a 0.7-sec stimulus light was presented at completion of each fixed interval, positively accelerated responding developed in each component. When no stimulus change occurred at completion of each fixed interval, relatively low and constant rates of responding prevailed in each component; a similar result was obtained when a 0.7-sec stimulus change occurred at completion of each fixed interval except the one which terminated with primary reinforcement. The 0.7-sec stimulus correlated with food delivery was an effective conditioned reinforcer in maintaining patterns of responding in fixed-interval components despite low average frequencies of food reinforcement.     

Maintenance of responding by squirrel monkeys under a concurrent shock-postponement, fixed-interval shock-presentation schedule.

A chain-pulling response was initially developed under a shock-postponement (avoidance) schedule with two squirrel monkeys. Few responses occurred on a lever where responding initially had no scheduled consequence or, subsequently, when a 3-minute fixed-interval shock-presentation schedule was concurrently arranged for lever responses. Appropriate rates and patterns of lever responding developed and were later maintained under the fixed-interval 3-minute shock-presentation schedule alone when the chain and shock-postponement schedule were removed. When both the shock-postponement and shock-presentation schedules were again simultaneously in effect, steady rates of chain pulling were maintained by the shock-postponement schedule and positively accelerated rates and patterns were maintained on the lever by the shock-presentation schedule. Response rates under both schedules were directly related to shock intensity. A history of exposure to a shock-postponement schedule, even though with a topographically different response and manipulandum, was sufficient for the development and eventual maintenance of responding by the presentation of shock. Further, differential performances can be maintained simultaneously by the presentation and postponement of electric shock.     

Fixed-ratio and variable-ratio schedules of brief stimuli in second-order schedules of matching to sample

Eight pigeons matched to sample under second-order schedules of food reinforcement. Under fixed-interval unit schedules, the first correct match to occur after a given period of time was followed by the presentation of a brief stimulus. The termination of the last fixed-interval unit schedule was followed by food according to second-order fixed-ratio and variable-ratio schedules. In Experiment 1, as the number of fixed-interval unit schedules increased, long pauses occurred under the second-order fixed-ratio schedules, but not under the variable-ratio schedules. The similarity of performance measures such as local rate and accuracy indicated that the differences engendered by these two types of schedule are in the duration of the periods of not-responding. In Experiment 2, the addition of a brief stimulus at the end of each unit schedule in chained schedules that had different discriminative stimuli present for the duration of each unit did not substantially affect the performance, and long pauses continued to occur. However, few long pauses occurred under schedules with brief stimulus presentations alone. The most inaccurate performances were engendered by chained schedules without brief stimuli.     

Conjunctive schedules of reinforcement II: response requirements and stimulus effects

Responding of three pigeons was maintained under conjunctive fixed-ratio, fixed-interval schedules where a key peck produced food after both schedule requirements were completed. The individual schedule requirements were then successively removed and reinstated with responding maintained under the following conditions: conjunctive fixed-ratio, fixed-time; fixed-time; and fixed-interval schedules. Patterns of responding changed in accord with the successive removal of the schedule requirements. Compared to the conjunctive fixed-ratio, fixed-interval schedule, pause duration increased and response rate decreased under conjunctive fixed-ratio, fixed-time schedules and under fixed-time schedules alone. Overall mean rates of responding were highest and pause duration lowest under fixed-interval schedules. When changes in the keylight colors were correlated with completion of the fixed-ratio, the end of the fixed-interval, or both of these conditions, the pattern of responding was modified and indicated a greater degree of control by the individual schedules. Although two birds showed large increases in interreinforcement time when they were initially exposed to the conjunctive schedule, when responding stabilized this measure was largely invariant for all birds across most schedule conditions.