The effect of punishment shock intensity upon responding under multiple schedules

In the first of two experiments, responses of two pigeons were maintained by multiple variable-interval, variable-ratio schedules of food reinforcement. Concurrent punishment was introduced, which consisted of a brief electric shock after each tenth response. The initial punishment intensities had no lasting effect upon responding. Then, as shock intensity increased, variable-ratio response rates were suppressed more quickly than variable-interval response rates. When shock intensity decreased, variable-interval responding recovered more quickly, but the rates under both schedules eventually returned to their pre-punishment levels. In the second experiment, the following conditions were studied in three additional pigeons: (1) With each shock intensity in effect for a number of sessions, punishment shock intensity was gradually increased and decreased and responding was maintained by multiple variable-ratio, fixed-ratio schedules of food reinforcement; (2) Changes in punishment shock intensity as described above with responding maintained by either a variable-ratio or a fixed-ratio schedule, which were presented on alternate days; (3) Session-to-session changes in shock intensity with responding maintained by multiple variable-ratio, fixed-ratio schedules. Responding under the two schedules was suppressed to approximately the same extent by a particular shock intensity. Also, post-reinforcement pauses under the fixed-ratio schedule increased as response suppression increased.     

Elimination of reinforced behavior: intermittent schedules of not-responding

Pigeons' key pecking resulted in food according to either a variable-ratio or a variable-interval schedule. At the same time, food was available for not pecking for a specified time. The required time of not-pecking was segmented into not-responding units, and these units were followed by food according to a fixed-ratio schedule. Both unit duration and the number required were varied. In general, the shorter the time unit or the smaller the ratio, the lower was response rate. When total required not-responding time was constant, but changes in unit duration and the number required altered how the total was achieved, shorter units produced lower rates. Other conditions involved substitution of food delivered independent of responding for the not-responding schedule. With low and moderate total times to food presentation, the not-responding schedule produced lower rates; with the longest times, the response-independent schedule generated less responding. When considered in terms of relative frequency of food presentation available from a source other than pecking, the not-responding schedule reduced rate more effectively than did the response-independent schedule. Comparisons with other research suggested that food presented dependent on not responding compared favorably with punishment as a procedure for reducing response rate. Transient effects differed. Although punishment temporarily depresses rate when first imposed and temporarily enhances it when first removed, food given for not responding quickly generated steady-state rates.     

Behavior simultaneously maintained by both presentation and termination of noxious stimuli

Lever pressing by two squirrel monkeys was maintained under a 3-minute variable-interval schedule of response-produced electric-shock presentation. At the same time, responding on a second lever was maintained under a 3-minute fixed-interval schedule of termination of the shock-presentation schedule and shock-correlated stimuli. Under the termination schedule, the first response after a 3-minute period produced a 1-minute timeout, during which no events occurred and responding had no scheduled consequence. Relatively high and constant rates of responding were maintained on the lever where responding produced shock. Lower rates and positively accelerated patterns of responding occurred on the lever where responding terminated the shock schedule. Thus, responding was simultaneously maintained by presentation of an event and by termination of a stimulus associated with that event. Rates and patterns of responding on each lever were reversed when the schedules arranged on each lever were reversed on two occasions. When shock intensity was increased from 0 to 10 mA, responding maintained both by presentation of shock and by termination of the shock schedule increased, but responding maintained by shock presentation increased to a greater extent. Positive and negative reinforcement, usually regarded as separate behavioral processes involving different events, can coexist when behavior is controlled by different contingencies involving the same event.     

The influence of positive and negative reinforcement on selective attention in the rat

In Experiments 1 and 2 rats were trained under two multiple schedules of reinforcement. In one, bar pressing during a tone-light compound stimulus was reinforced under a variable-interval food reinforcement schedule. In the other multiple schedule, bar pressing avoided grid shock on a free-operant schedule. In both multiple schedules, a discrimination was maintained by an extinction schedule that was operative during the absence of the tone-light compound. In Experiments 1 and 2 the intensity of the tone-light compound was manipulated over three levels. Subsequent extinction tests revealed that light was attended to, almost exclusively of the tone, when food reinforcement had maintained bar pressing. On the other hand, the tone gained considerable attentional control under the shock avoidance schedule. This stimulus-reinforcer interaction was maintained for all three levels of the compound intensity. In Experiment 3 it was investigated whether this interaction was associative by presenting shock during the absence of the tone-light compound when food reinforcement maintained responding, and food during the absence of the compound when shock avoidance maintained responding. Since both food and shock were presented during a single session for both schedules, nonassociative effects of the reinforcing stimuli were equivalent across the schedules. Nevertheless, the stimulus-reinforcer interaction was maintained, indicating that the interaction was an associative effect.     

Spatiotemporal patterns of behavior produced by variable-interval schedules of reinforcement

The spatiotemporal patterns of behavior exhibited by two pigeons during a variable-interval 15-second schedule of food reinforcement, a variable-interval 5-minute schedule, and then extinction of key pecking were recorded using an apparatus that continuously tracked the position of the bird in the experimental chamber. The variable-interval 15-second schedule produced a close-to-key pattern between reinforcements with two types of regular excursions from the region of the key frequently occurring after reinforcement. Subsequent exposure to the variable-interval 5-minute schedule produced more extended and extremely regular patterns between responses. Reinstatement of the variable-interval 15-second schedule reestablished the close-to-key pattern with regular excursions frequently occurring after reinforcement. During extinction the spatiotemporal patterns that had developed during the variable-interval 5-minute schedule reappeared and gradually dissipated. These patterns may have been a form of superstitious behavior.     

Some effects of punishment shock intensity upon discriminative responding

Three pigeons received visual discrimination training under both multiple variable-ratio extinction and variable-interval extinction schedules. All birds developed nearly perfect discrimination. When punishment for every tenth response during food reinforcement was presented, responding decreased as shock intensity increased. At the same time, responding during extinction, which was not punished, increased at intermediate punishment intensities, but returned to low levels under severe punishment. A second procedure, in which punishment and no-punishment sessions alternated unsystematically, was employed with two of the birds. The results under this procedure essentially replicated the data obtained as punishment shock intensity increased gradually.     

Effects of chlordiazepoxide and d-amphetamine on responding suppressed by conditioned punishment

Key pecking by two pigeons, maintained under a variable-interval two-minute schedule of food presentation, was suppressed when each response produced a five-second visual stimulus that was occasionally paired with shock. Stimulus-shock pairings occurred independently of responding according to a variable-time six-minute schedule for one bird or once per session for the other bird. The effects of chlordiazepoxide and d-amphetamine were assessed on this baseline of behavior suppressed by conditioned punishment. Chlordiazepoxide produced dose-related increases in the rate of punished responding, whereas d-amphetamine produced only decreases. In addition, chlordiazepoxide produced dose-related increases in response rate during the response-produced and response-independent stimulus presentations; d-amphetamine, however, only decreased responding during stimulus presentations.     

Drug effects on responding maintained by stimulus-reinforcer and response-reinforcer contingencies.

The effects of pentobarbital and d-amphetamine were assessed on key pecking by pigeons under conventional single-key multiple schedules and under two-key multiple schedules in which discriminative stimuli appeared on one key (stimulus key) while pecks on a second key (constant key) produced food. Pecks on the stimulus key had no scheduled consequences. A 60-second variable-interval schedule operated in one component of each multiple schedule: either extinction or a 60-second variable-time schedule operated in the alternate component. When the alternate-component schedule was extinction, a high rate of responding was maintained in the variable-interval component of the single-key schedule; responding on both keys was maintained in the variable-interval component of the two-key schedule. Pentobarbital increased responding in the variable-interval component of the single-key schedule and increased stimulus-key, but not constant-key responding in that component of the two-key schedule. When the alternate-component schedule was changed to variable time, responding declined in the variable-interval component of the single-key schedule; stimulus-key responding was no longer maintained under the two-key schedule. Pentobarbital decreased responding in the variable-interval component of both schedules. With an exception, d-amphetamine only decreased responding in the variable-interval component of the single- and two-key schedules both when the alternate-component schedule was extinction and when it was variable time. The results suggest that the effects of pentobarbital, but not d-amphetamine, depend on the nature of the contingency (stimulus-reinforcer, response-reinforcer) that maintains responding.     

Two types of pigeon key pecking: suppression of long- but not short-duration key pecks by duration-dependent shock

The key pecking of eight pigeons was maintained on a variable-interval 1-minute schedule of food reinforcement. Sometimes, all responses between 35 and 50 milliseconds in duration produced a shock; sometimes, all responses between 10 and 25 milliseconds produced a shock; sometimes, shocks were produced by pecks without regard to duration (nondifferential punishment), and sometimes shocks were delivered independently of responding. Punishment of 35- to 50-millisecond responses selectively suppressed those responses, while punishment of 10- to 25-millisecond responses and nondifferential punishment suppressed responding overall but did not suppress responses of particular duration. Punishment of 35- to 50-millisecond responses suppressed key pecking slightly less than did nondifferential punishment. Punishment of 10- to 25-millisecond responses and response-independent shock produced roughly equal amounts of suppression, substantially less than the other punishment procedures. The data support the view that there are at least two kinds of key peck, identifiable on the basis of duration, one of which (short duration) is insensitive to its consequences.     

Positive interaction (induction) in multiple variable-interval, differential-reinforcement-of-high-rate schedules

The effect of increases in the rate of responding in one component of a multiple schedule upon the rate of responding in a second component was investigated. Pigeons were exposed to a multiple schedule where both components were initially variable-interval schedules having the same parameter value. After rate of key pecking stabilized, one component was changed to a schedule that differentially reinforced high rates of responding. Rate of reinforcement in this varied component was adjusted to remain equal to rate of reinforcement in the constant (variable-interval) component. Four of five pigeons showed a maintained increase in rate of responding during both the constant and varied components, even though rates of reinforcement did not change.     

A facilitative effect of punishment on unpunished behavior

The key pecking of two pigeons was reinforced on a variable-interval schedule of reinforcement during the presentation of each of two stimuli. In various phases of the experiment, punishment followed every response emitted in the presence of one of the stimuli. In general, when the rate of punished responding changed during the presentation of one stimulus, the rate of unpunished responding during the other stimulus changed in the opposite direction. This sort of change in rate is an example of behavioral contrast. When punishment was introduced, the rate of punished responding decreased and the rate of unpunished responding increased as functions of shock intensity. When the rate of previously punished responding increased after the termination of the shock, the rate of the always unpunished responding decreased. When the procedure correlated with a red key was changed from variable-interval reinforcement and punishment for each response to extinction and no punishment, the rate of reinforced responding during presentations of a green key decreased and then increased while the rate of the previously punished responding during red first increased and then decreased during extinction.     

Positive and negative conditioned suppression in the pigeon: Effects of the locus and modality of the CS

In Experiment I, four pigeons were exposed to trials in which a 12-sec key light illumination was followed by free food. These trials were superimposed upon a baseline of key pecking for food reinforcement on a variable-interval schedule. When the signal for food was on the operant key, response rate was substantially higher during the signal than during the baseline procedure. When the signal was on a second, signal key, operant responding was suppressed during the signal and substantial pecking of the signal key occurred. The sum of signal key and operant key pecks far exceeded the operant baseline rate of responding. An explanation of opposite results obtained with rats and pigeons as subjects in experiments of this type was suggested in terms of the spatial relation between the signal for free food and the operant target which usually characterizes these experiments. Experiment II assessed the importance of signal location when shock rather than food was the US. Suppression of operant key pecking was unaffected by signal location. Experiment III assessed the relative effectiveness of visual and auditory stimuli (clicks) as signals for food and shock, and found that all combinations of signal and US were equally effective in suppressing operant key responding. The three experiments together suggested that the identification of important effects of species—typical behavior in one experimental situation does not imply that there will be like effects in similar situations.     

Responding under schedules combining response-dependent and response-independent shock delivery

Lever pressing of three squirrel monkeys with experience under continuous avoidance schedules was maintained by response-produced shock under a 5-minute variable-interval schedule. Responding decreased when half of the scheduled shocks were delivered independently of lever pressing and decreased further when all shocks were independent of lever pressing. Responding was lowest when all shocks were eliminated. When the proportion of response-dependent shocks increased, responding increased. This relation occurred even though the frequency and temporal distribution of shock delivery remained the same. Responding of two monkeys increased in a graded fashion as the frequency of shock was increased by arranging variable-time 5-minute, 2-minute, and 1-minute schedules jointly with the variable-interval 5-minute schedule. Thus, increasing the proportion of response-independent shocks decreased responding when the overall frequency of shocks stayed the same, but increased responding when the overall frequency of shock delivery increased.     

Effects on responding of mixed and multiple schedules of signalled and unsignalled response-dependent electric-shock delivery

Responding in two rats was maintained under mixed and multiple variable-interval 35-sec variable-interval 35-sec food delivery schedules. Similar rates and patterns of responding occurred in each component of the two schedules. Mixed and multiple variable-interval 65-sec variable-interval 65-sec schedules of response-dependent shock delivery were super-imposed on the mixed and multiple baseline food schedules, respectively. In one component, a 5-sec stimulus was presented on the average of once every 65 sec. Offset of the stimulus arranged that the next response would produce shock. In the other component, no stimulus was presented during the 5-sec period. The mixed schedule of signalled and unsignalled dependent shock delivery yielded similar degrees of response suppression in each component, but the multiple schedule of shock delivery revealed differential degrees of response suppression. Considerably more suppression occurred in the component not associated with the preshock stimulus, thus implicating the discriminative functions of the correlated stimulus.     

Intermittent punishment of human responding maintained by intermittent reinforcement

To determine the effects of variable-interval shock punishment on behavior maintained by variable-interval and variable-ratio reinforcement, human subjects' key-pressing behavior was reinforced with money on a four-component multiple schedule. Components 1 and 2 were variable-interval 30-sec, and Components 3 and 4 were variable-ratio 210. After responding was stabilized, response-contingent electric shock was scheduled on a variable-interval 10-sec schedule during the second and fourth components of each cycle. Subjects instructed as to the reinforcement contingencies showed gradually increasing suppression of variable-interval responding at increasing shock intensities and either very high or very low rates of variable-ratio responding at higher intensities. Minimally instructed subjects showed suppression at higher shock intensities, but no clear differential suppression as a function of reinforcement schedule. Recovery from initial suppression was observed within sessions.     

Economic and biological influences on a pigeon's key peck

Pigeons were studied in a two-component multiple schedule. In the first phase of the experiment, key pecks were reinforced on a variable-interval 2-min schedule in both components and free food was delivered additionally during one component. When components alternated every 8 sec, all pigeons pecked at a much higher rate during the component with free food than during the other component. At a component duration of 16 min, the reverse was true: all pigeons pecked at a higher rate during the component without free food. In the second phase, the additional food during one component was made contingent on pecking. Responding during the component without the extra food remained essentially unchanged, as expected, since rate of reinforcement remained identical to that in the previous phase. However, rate of responding during the component with the extra food (now contingent on pecking) was elevated, compared to the rate in the first phase, and did not show the marked decline as component duration was increased.     

Schedules using noxious stimuli. III. Responding maintained with response-produced electric shocks

Responding was maintained in two squirrel monkeys under several variations of a 10-min fixed-interval schedule of electric shock presentation. The monkeys were first trained under a 2-min variable-interval schedule of food presentation, and then under a concurrent schedule of food presentation and shock presentation. In one monkey, when shocks (12.6 ma) followed each response during the last minute of an 11-min cycle ending with a timeout period, responding was increased during the first 10 min and suppressed during the last minute of each cycle. When the shock schedule was eliminated, both the enhancement and suppression disappeared, and a steady rate of responding was maintained under the variable-interval schedule. When the food schedule was eliminated, the shock schedule maintained a characteristic fixed-interval pattern of responding during the first 10 min, but suppressed responding during the last minute of each cycle. The fixed-interval pattern of responding was maintained when the timeout period was eliminated and when only one shock could occur at the end of the cycle. In the second monkey, responding under the concurrent food and shock schedule was suppressed when responses produced shocks after 3-min. Under an 11-min cycle, responding continued to be maintained at increasing shock intensities. When the food schedule was eliminated, a fixed-interval pattern of responding was maintained under a 10-min schedule of shock presentation (12.6 ma). Whether response-produced electric shocks suppressed responding or maintained responding depended on the schedule of shock presentation.     

Negatively reinforced key pecking

A reinforcement-switching procedure was used to produce negatively reinforced key pecking in pigeons. First, key pecking on a chain schedule (fixed-interval 10-sec variable-interval 60-sec) was conditioned using grain reinforcement. Second, intermittent shock in the initial link was introduced at a low intensity and gradually increased. Third, food reinforcement in the terminal link was eliminated. With shock at 90 V occurring on the average every 3 sec, initial-link pecking was maintained with no terminal-link food. Three of four pigeons responded consistently at shock intensities of 90, 70, and 50 V but not at 30 V. A fourth pigeon responded at but not below 90 V. Rate of response was directly related to shock frequency. Eliminating food deprivation did not affect the negatively reinforced performance.     

Information on response requirements compared with information on food density as a reinforcer of observing in pigeons

On a variable-interval schedule, pecking the key to the pigeon's right (observing response) produced red or green displays relating to the delivery of grain and its dependence on pecking the key to the left (food key). During various blocks of sessions, mixed (no stimulus change) schedules including the following pairs of components were temporarily converted by the observing response to their corresponding multiple (correlated stimuli) schedules: variable-interval 60-s, extinction; variable-interval 60-s, variable-time (response-independent) 60-s; extinction, variable-time 60-s. Differences in food delivery maintained substantial rates of responding on the observing key, without regard to pecking requirements on the food key. Although stimuli correlated with differences in the response requirement on the food key maintained higher observing rates than those maintained by uncorrelated stimuli, they were much lower than those based on food. The value of predictive stimuli as reinforcers is determined by the value of the events predicted. In particular, the cost of pecking appears to be low, and this may place limitations on the applicability of energy-based and economic models of behavior.     

Factors influencing inhibitory stimulus control: differential reinforcement of other behavior during discrimination training

Pigeons were trained to respond on a multiple variable-interval 1-min variable-interval 1-min schedule, then switched to a multiple variable-interval 1-min differential-reinforcement-of-other-behavior discrimination training. The rate of reinforcement was held constant during the shift from non-differential reinforcement to discrimination training. Behavioral contrast and post-discrimination inhibitory stimulus control followed the observation of a reduction in the rate of responding to the stimulus correlated with reinforcement for the non-occurrence of pecking.