Behavior simultaneously maintained by both presentation and termination of noxious stimuli

Lever pressing by two squirrel monkeys was maintained under a 3-minute variable-interval schedule of response-produced electric-shock presentation. At the same time, responding on a second lever was maintained under a 3-minute fixed-interval schedule of termination of the shock-presentation schedule and shock-correlated stimuli. Under the termination schedule, the first response after a 3-minute period produced a 1-minute timeout, during which no events occurred and responding had no scheduled consequence. Relatively high and constant rates of responding were maintained on the lever where responding produced shock. Lower rates and positively accelerated patterns of responding occurred on the lever where responding terminated the shock schedule. Thus, responding was simultaneously maintained by presentation of an event and by termination of a stimulus associated with that event. Rates and patterns of responding on each lever were reversed when the schedules arranged on each lever were reversed on two occasions. When shock intensity was increased from 0 to 10 mA, responding maintained both by presentation of shock and by termination of the shock schedule increased, but responding maintained by shock presentation increased to a greater extent. Positive and negative reinforcement, usually regarded as separate behavioral processes involving different events, can coexist when behavior is controlled by different contingencies involving the same event.     

Effects of ratio contingencies on responding maintained by schedules of electric-shock presentation (response-produced shock)

Squirrel monkeys' lever pressing was established under fixed-interval schedules of electric-shock presentation (response-produced shock). After appropriate temporal patterns of lever pressing were engendered, either fixed-ratio schedules of shock presentation were added to the fixed interval, or yoked variable-ratio schedules were substituted for the fixed-interval schedules. When fixed-ratio schedules were added, there was an initial rise in response rate and schedule-appropriate patterns of responding developed. After many sessions, however, responding ceased abruptly, in some cases with remarkable quickness. When variable-ratio schedules were substituted, responded declined gradually and eventually was poorly maintained. Ratio contingencies may not support responding as well as interval contingencies when electric shock is the maintaining event.     

The puzzle of responding maintained by response-contingent shock.

Four squirrel monkeys were first exposed to a sequence of procedures that reliably generate responding maintained by brief response-contingent electric shocks arranged according to a fixed-interval schedule. After responding had become stable on the fixed-interval schedule, additional contingencies were added in tandem, whereby after completion of the interval, the spacing of responses affected shock delivery. In one procedure, responses had to be spaced more widely than their previous median value if shock were to be delivered. In the other procedure, responses had to be spaced more closely to produce shock. On the first of these procedures, decreased but stable responses rates would indicate that shock functioned as a positive reinforcer; on the second, increased response rates would indicate the positively reinforcing function. Instead, response rates accelerated on the procedure that targeted more widely spaced responses for shock delivery, and decelerated or ceased on the procedure that arranged for shocks to be produced by more closely spaced responses. Consistent with other recent findings, these results question the interpretation of performances maintained by response-contingent shock as engendered by positive reinforcement and are consistent with aversive-control interpretations. The details of that aversive control are not entirely clear, however, and these same procedures would be informative if applied to shock-maintained behavior that is generated in other ways.     

Responding maintained under fixed-interval and fixed-time schedules of electric shock presentation

Following initial histories under a schedule of electric shock postponement, lever pressing in squirrel monkeys was maintained under fixed-interval and fixed-time schedules of electric shock presentation. No difference in either rate or pattern of responding was obtained when these schedules were presented as components of a multiple schedule. When they were presented singly for long periods of time, the fixed-interval schedule consistently maintained a higher response rate than the fixed-time schedule. The pattern of responding under both schedules was similar, typically consisting of a pause at the beginning of each interval followed by either a steady or a positively accelerating rate of responding. The results suggest that the response-shock dependency is of critical importance in the maintenance of high rates of responding under schedules of electric shock presentation, and support the general view that such responding may be conceptualized as operant behavior under control of many of the same variables that control responding under comparable schedules of food or water reinforcement.     

Drug discrimination under a concurrent fixed-interval fixed-interval schedule.

Pigeons were trained to discriminate 5.0 mg/kg pentobarbital from saline under a concurrent fixed-interval (FI) FI schedule of food presentation on which, after pentobarbital administration, responses on one key were reinforced with food under an FI 60-s component and responses on the other key were reinforced under an FI 240-s component. After saline administration, the schedule contingencies on the two keys were reversed. After both pentobarbital and saline, pigeons responded more frequently on the key on which responses had been programmed to produce the reinforcer under the FI 60 component of the concurrent schedule. The schedule was changed to concurrent FI 150 FI 150 s for drug-substitution tests. In each bird, increasing doses of pentobarbital, ethanol, and chlordiazepoxide produced increases in the proportion of responses on the key on which responses had been reinforced under the FI 60 component after pentobarbital administration during training sessions. The proportion of responses on that key was slightly lower for ethanol than for chlordiazepoxide and pentobarbital. At a dose of pentobarbital higher than the training dose, responding decreased on the key that had been reinforced under the FI 60 component during training sessions. Phencyclidine produced less responding on the key programmed under the FI 60-s component than did pentobarbital. Methamphetamine produced responding primarily on the key on which responses had been reinforced under the FI 60-s component after saline administration.     

Schedules of food postponement: II. Maintenance of behavior by food postponement and effects of the schedule parameter

In Experiment I, food-deprived, feeder-trained squirrel monkeys pressed a lever to postpone brief electric shocks (Response-Shock=Shock-Shock interval=30 seconds). Forty-one three-hour sessions of shock postponement were followed by 120 sessions of concurrent shock and food postponement. The shock schedule was unchanged and the food schedule was Response-food interval–20 seconds, Food-food interval 10 seconds. After concurrent shock and food postponement, the shock schedule was discontinued and 40 sessions of food postponement ensued, followed by 53 sessions of extinction. After extinction, food postponement was resumed for 11 sessions. Stable responding with low food rates was maintained under food-postponement after the concurrent schedule. Responding decreased to low levels under extinction and recovered immediately to previous levels when the food-postponement schedule was re-instated. In Experiment II, a parameter of the food-postponement schedule was studied sequentially. Using the same subjects, the Response-food–Food-food interval was manipulated from four seconds to 80 seconds with several orders of presentation. Relations of response rates and food rates to the parameter were similar to those seen under shock postponement. Exposure to very short postponement times (four seconds), resulting in very high food rates, decreased but did not abolish subsequent responding at longer postponement times. Results are discussed from the point of view that reinforcing functions of stimuli consequent on responding depend on a prior history of scheduled contact with those stimuli.     

Effects of concurrent response-independent reinforcement on fixed-interval schedule performance

In three experiments, behavior maintained by fixed-interval schedules changed when response-independent reinforcement was delivered concurrently according to fixed- or variable-time schedules. In Experiment I, a pattern of positively accelerated responding during fixed interval was changed to a linear pattern when response-independent reinforcement occurred under a variable-time schedule. Overall response rates (total responses/total time) decreased as the frequency of response-independent reinforcement increased. Experiment II showed that the response-rate changes in the first experiment were controlled by the response-reinforcer relation, but the changes in patterns of responding were similar whether concurrently available reinforcement at varying times was response-dependent or response-independent. In the final experiment, the addition of response-independent reinforcement at fixed times to a fixed-interval schedule resulted in changes in both local and overall response rates and in the occurrence of positively accelerated responding between reinforcements. These results suggest that the temporal distribution of reinforcers determines response patterns and that both the response-reinforcement dependency and the schedule of reinforcement determine overall response rates during concurrently scheduled response-dependent and response-independent reinforcement.     

Preference for less frequent shock under fixed-interval schedules of electric-shock presentation.

Lever pressing by 2 squirrel monkeys was maintained under fixed-interval 6-min and fixed-interval 2-min schedules of electric-shock presentation. Preference for these schedules was assessed during three experimental phases. In all phases, responses on one lever produced shock according to one or the other fixed-interval schedule, and responses on a second, changeover, lever switched between schedules. The opportunity to change over was presented during separate choice periods (during which the fixed-interval schedules did not operate) that followed the first through fourth shocks in each schedule. If no changeover occurred during those choice periods, a changeover automatically occurred following the fifth shock. In Phase I, durations of the choice periods were fixed. In Phase II, the choice periods equaled a proportion of their respective fixed interval. During Phase III (completed with 1 monkey) a response on the changeover lever during a given choice period reinstated the most recent fixed interval, and a failure to respond resulted in a changeover. During each of these phases, distinct preferences developed for the 6-min schedule. These results suggest that the maintenance of lever pressing by fixed-interval presentation of electric shock may not be an example of positive reinforcement, and that the response-maintaining characteristics of shock presentation may derive from other properties of the schedule.     

Relative reinforcer magnitude under a nonindependent concurrent schedule of cocaine reinforcement in rhesus monkeys.

Lever pressing by three rhesus monkeys was maintained under a two-lever concurrent schedule of cocaine reinforcement. Responding on one lever (constant-dose lever) produced a constant dose of 0.05 or 0.1 mg/kg/injection arranged according to a variable-interval 1-min schedule. Responding on the other lever (variable-dose lever) produced a comparison dose of cocaine (0.013 to 0.8 mg/kg/injection), also under a variable-interval 1-min schedule. The two variable-interval schedules were made nonindependent by arranging that the assignment of a reinforcer by one schedule inactivated the second schedule until the assigned reinforcer had been obtained. This modification ensured that the two cocaine doses were obtained with approximately equal frequency, regardless of the distribution of the subject's responding. Preference, indicated by relative response frequency on the variable-dose lever, was almost always for the larger of the doses and was a monotonic function of the comparison dose, except at the highest doses. Preferences at the highest comparison doses may have resulted from the low overall response rates exhibited at these doses. Relative response frequencies on the variable-dose lever roughly matched relative reinforcer magnitude (mg/kg/injection available on the variable-dose lever divided by the sum of mg/kg/injections available on each lever).     

Intractable properties of responding under a fixed-interval schedule

The behavior engendered by the fixed-interval schedule is characterized by its variability within and across intervals. The present experiment was designed to assess further the magnitude of interval-to-interval dynamics and to explore conditions which might enhance control by response number for subsequent output. Pigeons were exposed to three experimental manipulations after responding had stabilized under a fixed-interval five-minute schedule. First, a discrete five-stimulus counter was added so that the key color changed after a fixed number of responses. Then additional grain presentations were made at the end of the interval so that high response output was differentially reinforced in the presence of the counter stimuli. Finally, the counter stimuli were presented as an irregular clock (i.e., independently of responding), but the durations were yoked to performance under the counter condition. The data show that response number can exert influence from one interval to the next, but this source of control is weak and not influenced by the experimental manipulations. Results from the clock arrangement indicate that behavior is controlled largely by the stimulus conditions prevailing at the time of interval onset.     

Schedule-induced biting under fixed-interval schedules of food or electric-shock presentation

Squirrel monkeys pressed a lever under fixed-interval schedules of food or of electric-shock presentation. Both schedules induced repeated biting on a latex hose. Whether lever pressing was controlled by food or by electric shock, a pattern of decreasing hose biting and increasing lever pressing occurred within fixed-interval cycles. As the fixed-interval duration was increased from 6 to 600 sec, average rates of lever pressing decreased under both schedules. Average rates of hose biting first increased with increasing parameter value, reaching a maximum at values that varied from 60 to 337 sec in different monkeys, and then declined at higher values. d-Amphetamine at appropriate doses increased overall rates of lever pressing maintained by food or by shock, but either did not affect or decreased overall rates of hose biting. When no timeout period occurred between fixed-interval cycles, the monkeys bit most frequently immediately after food or electric shock was presented. When there was a timeout period, hose biting began shortly after the start of the fixed-interval cycles, with little or no hose biting immediately after food or electric shock was presented. Most hose biting appeared to be schedule-induced rather than food- or shock-induced.     

A comparison of responding maintained under second-order schedules of intramuscular cocaine injection or food presentation in squirrel monkeys.

Key pressing by squirrel monkeys was maintained under second-order schedules of either intramuscular cocaine injection or food presentation. Under one schedule, each completion of a 10-response fixed-ratio unit produced a brief visual stimulus; the first fixed-ratio unit completed after 30 minutes elapsed produced the stimulus paired with either cocaine injection or food presentation. Generally, short pauses followed by high rates of responding were maintained within the fixed-ratio units, and responding was positively accelerated over the 30-minute interval. Under another schedule, each completion of a 3-minute fixed-interval unit produced the brief stimulus; completion of the 10th fixed-interval unit produced the stimulus paired with either cocaine injection or food presentation. Generally, short pauses followed by high rates of responding were maintained within the fixed-ratio units, and responding was positively accelerated over the 30-minute interval. Under another schedule, each completion of a 3-minute fixed-interval unit produced the brief stimulus; completion of the 10th fixed-interval unit produced the stimulus paired with either cocaine injection or food presentation. Rates of responding increased within the fixed-interval units, and to a greater extent over the entire 10 fixed-interval units. Patterns of responding depended more on the schedule of reinforcement than on whether cocaine or food maintained responding. Omitting the brief stimuli following all but the last fixed-ratio or fixed-interval units decreased average rates and altered the patterns of responding. Substituting a visual stimulus that was never paired with cocaine or food following all but the last fixed-ratio or fixed-interval units decreased response rates to a lesser extent and did not substantially alter patterns of responding. When the duration of the paired stimulus was varied from .3 to 30.0 seconds, the highest response rates occurred at intermediate durations (1.0 to 10.0 seconds). The manner in which the stimulus changes affected performances depended more on the schedule of reinforcement than on whether cocaine injection or food presentation maintained responding.     

Effects of primary reinforcement on pigeons' initial-link responding under a concurrent chains schedule with nondifferntial terminal links

The effect of primary reinforcement on initial-link responding under concurrent-chains schedules with nondifferential terminal links was assessed in 12 pigeons. The iniitial and terminal links were variable-interval schedules (always the same for both alternatives). The positions (left or right key) of the initial-link stimuli (red or green) were randomized while the correlation between color and food amount remained constant within each condition. The terminal-link stimuli were always presented on the center key. Except in two control groups and conditions, the terminal-link stimuli were the same color (nondifferential, blue or yellow). Over six conditions, the differences in food amont and the durations of the initial- and terminal-link schedules were manipulated. In 57 of 60 cases, birds generated choice proportions above .50 in favor of the initial-link stimlus that was correlated with the larger reinforcer. There was some indication that preference increased with shortened terminal-link durations. Because the terminal-link stimuli were nondifferential, differential responding in the initial links cannot be explained easily by conditioned reinforcement represented by the terminal-link stimuli. Thus, primiary reinforcement has a direct effect on initial-link responding in concurrent-chains schedules.     

Ratio responding as a function of concurrent avoidance schedules, yoked shocks, and ratio value

Fixed-ratio food-reinforced responding in rats was studied alone and with concurrent shock avoidance or with concurrent response-independent shocks matched to those that occurred in the avoidance condition. Under each condition, fixed-ratio size was increased over successive daily sessions. Fixed-ratio response rate generally passed through a maximum as a function of fixed-ratio size. Decreased fixed-ratio responding at values beyond the maximum occurred when (1) the time to complete a fixed ratio approximated the response-shock interval of the avoidance schedule, (2) the shock rate increased, and/or (3) the ratio requirements were so high that ratio strain occurred. Avoidance rates decreased slightly as fixed-ratio size increased.     

Neurochemical changes correlated with behavior maintained under fixed-interval and fixed-ratio schedules of reinforcement.

Key pecking of 4 pigeons was maintained under a multiple 3-min fixed-interval, 30-response fixed-ratio schedule of food presentation. Only one schedule was in effect during an experimental session, and each was correlated with a different keylight stimulus and location (left vs. right). The different schedule components alternated across days or weeks. Cerebrospinal fluid was collected from chronically implanted intracerebroventricular cannulae following sessions with the different schedules, as well as following sessions in which reinforcement was withheld (extinction), when response-independent food was delivered, and when the experimental chamber was dark and there were no scheduled events. Metabolites of the neurotransmitters serotonin, norepinephrine, and dopamine were assayed in cerebrospinal fluid using high-performance liquid chromatography with electrochemical detection. Compared to the fixed-ratio condition, responding maintained under the fixed-interval schedule resulted in consistently higher levels of the serotonin metabolite 5-hydroxyindoleacetic acid and of the dopamine metabolite homovanillic acid in all pigeons. Levels of 3-methoxy-4-hydroxyphenylethylene glycol, a metabolite of norepinephrine, and dihydroxyphenylacetic acid, another dopamine metabolite, were also higher in 3 of the 4 pigeons following exposure to the fixed-interval schedules when compared to levels of these metabolites after exposure to the fixed-ratio schedule. Extinction of fixed-ratio responding resulted in large increases in 5-hydroxyindoleacetic acid compared to levels of this metabolite under the fixed-ratio schedule, whereas this serotonin metabolite decreased during extinction of responding under the fixed-interval schedule. Control procedures suggested that the neurochemical changes were not related to the rate of responding but were a function of the specific experimental conditions. Distinctive neurochemical changes that accompany schedule-controlled responding show the sensitivity of the neurochemical environment to behavioral contingencies and demonstrate further the profound impact that such contingencies have on biobehavioral processes.     

Oral self-administration of pentobarbital by rhesus monkeys: relative reinforcing effects under concurrent fixed-ratio schedules.

During daily 3-hr sessions, orally delivered pentobarbital solutions and water, or two separate pentobarbital solutions, were concurrently available to rhesus monkeys according to fixed-ratio schedules of mouth contacts with a spout. First water, and then each of four "comparison-concentration" pentobarbital solutions (0.0625, 0.25, 1, and 4 mg/mL), was successively available from one spout for a block of sessions under a fixed-ratio-64 (three monkeys) or fixed-ratio-16 (one monkey) schedule. Under an identically sized fixed-ratio schedule, deliveries of a "standard-concentration" pentobarbital solution were concurrently available from a second spout. The concentration of the standard solution remained unchanged throughout testing of the series of comparison solutions. Each of three pentobarbital concentrations (4, 1, and 0.25 mg/mL) in turn served as the standard concentration. Within each pair of concurrently available solutions, the higher drug concentration maintained more behavior than the lower concentration. Thus when monkeys were provided with concurrent access to different pentobarbital concentrations, relative reinforcing effects were directly related to drug concentration. Further, the amount of behavior maintained by a particular drug concentration was dependent on the concentration of the concurrently available drug solution. Thus, the relative effectiveness of a reinforcer in maintaining behavior is a function of both the reinforcer's magnitude and the availability of alternative reinforcers in the environment.     

Brief exposure to contingent reinforcement produces a sustained increment in the strength of an elicited response.

Squirrel monkeys' lever pressing produced by response-independent shock was measured before, and for an extended period following, exposure to a shock-avoidance procedure. Following avoidance training, the frequency of responding increased by multiples of 10 to 50 across subjects and evidence little or no decrement for up to 6 months of postreinforcement testing. Manual responding produced by intense environmental stimulation may be substantially strengthened by a brief history of reinforcement, such that it is sustained over long periods without reinforcement.     

Concurrent avoidance of shocks by pigeons pecking a key

Three pigeons were studied on concurrent, unsignaled, avoidance schedules in a two-key procedure. Shock-shock intervals were two seconds in both schedules. The response-shock interval on one key was always 22 seconds, while the response-shock interval associated with the other key was varied from 7 to 52 seconds in different experimental conditions. Response rates on the key associated with the varied schedule tended to decrease when the response-shock interval length was increased. Responding on the key associated with the constant schedule was not systematically affected.     

Drug discrimination under a concurrent fixed-ratio fixed-ratio schedule.

Pigeons were trained to discriminate 5.0 mg/kg pentobarbital from saline under a two-key concurrent fixed-ratio 10 fixed-ratio 40 schedule of food presentation, in which the fixed-ratio component with the lower response requirement was programmed to reinforce responding on one key after drug administration (pentobarbital-biased key) and on the other key after saline administration (saline-biased key). After responding stabilized, pigeons averaged 98% of their responses on the pentobarbital-biased key during training sessions preceded by pentobarbital, and they averaged 90% of their responses on the saline-biased key during training sessions preceded by saline. In test sessions preceded by doses of pentobarbital, chlordiazepoxide, or ethanol, pigeons switched from responding on the saline-biased key at low doses to responding on the pentobarbital-biased key at higher doses (the dose-response curve was quantal). High doses of phencyclidine produced responding on both keys, whereas pigeons responded almost exclusively on the saline-biased key after all doses of methamphetamine. These and previous experiments using concurrent reinforcement schedules to study drug discrimination illustrate that the schedule of reinforcement is an important determinant of the shape of dose-effect curves in drug-discrimination experiments.