SUPPRESSIVE AND FACILITATIVE EFFECTS OF SHOCK INTENSITY AND INTERRESPONSE TIMES FOLLOWED BY SHOCK

Although response‐dependent shock often suppresses responding, response facilitation can occur. In two experiments, we examined the suppressive and facilitative effects of shock by manipulating shock intensity and the interresponse times that produced shock. Rats' lever presses were reinforced on a variable‐interval 40‐s schedule of food presentation. Shock followed either long or short interresponse times. Shock intensity was raised from 0.05 mA to 0.4 mA or 0.8 mA. Overall, shock contingent on long interresponse times punished long interresponse times and increased response rates. Shock contingent on short interresponse times punished short interresponse times and decreased response rates. In Experiment 1, raising the range of interresponse times that produced shock enhanced these effects. In Experiment 2, the effects of shock intensity depended on the interresponse times that produced shock. When long interresponse times produced shock, low intensities increased response rates. High intensities decreased response rates. When short interresponse times produced shock, high shock intensities punished short interresponse times and decreased response rates more than low intensities. The results may explain why punishment procedures occasionally facilitate responding and establish parameters for future studies of punishment.     

Molecular contingencies in schedules of intermittent punishment

In two experiments, key pecking of pigeons was maintained by a variable-interval 180-s schedule of food presentation. Conjointly, a second schedule delivered response-dependent electric shock. In the first experiment, shocks were presented according to either a variable-interval or a nondifferential interval-percentile schedule. The variable-interval shock schedule differentially delivered shocks following long interresponse times. Although the nondifferential shock schedules delivered shocks less differentially with respect to interresponse times, the two shock schedules equally reduced the relative frequency of long interresponse times. The second experiment differentially shocked long or short interresponse times in different conditions, with resulting decreases in the relative frequency of the targeted interresponse times. These experiments highlight the importance of selecting the appropriate level of analysis for the interaction of behavior and environment. Orderly relations present at one level of analysis (e.g., interresponse times) may not be revealed at other levels of analysis (e.g., overall response rate).     

Interresponse-time punishment: a basis for shock-maintained behavior.

Lever pressing of squirrel monkeys postponed brief electric shock according to a free-operant shock-postponement procedure. Pressing also produced shock with a probability proportional to the duration of the current interresponse time in some conditions, or to the fifth ordinally-preceding interresponse time in others. These conditions provided equal frequencies and temporal distributions of response-produced shocks either contingent on or independent of the current interresponse-time duration, respectively. Shock delivered contingent on the current interresponse-time duration resulted in shorter mean interresponse times and higher overall response rates that shock delivered independent of the current interresponse time. In subsequent conditions, response-produced shocks were sufficient to maintain responding following suspension of the postponement procedure only when those shocks were contingent on the current interresponse time. Presenting shock independent of the current interresponse time, conversely, suppressed response rate and ultimately led to cessation of responding in the absence of a conjoint shock-postponement procedure. These results demonstrate interresponse-time punishment in the absence of any indirect avoidance contingencies based on overall shock-frequency reduction, and strongly support similar interpretation at the more local level of shock-frequency reduction correlated with particular interresponse times. Differential punishment of long interresponse times also provides both an a priori basis for predicting whether a schedule of shock presentation will maintain or suppress responding and a framework for interpreting many of the functional relations between overall response rate and parameters of consequent shock presentation. Finally, these results and others indicate the importance of response-consequence contiguity above and beyong any notion of noncontiguous contingency in the control of behavior.     

The concurrent reinforcement of two interresponse times: absolute rate of reinforcement

Three pigeons obtained food on a one-key schedule of reinforcement for two concurrent, discriminated interresponse times. The overall rate of reinforcement was determined by a family of variable-interval schedules and by a continuous reinforcement schedule. The average frequency of reinforcement varied from 1.1 to 300 reinforcements per hour; the relative frequency of reinforcement for each of the two interresponse times was 0.5 throughout the experiment. The number of responses per minute increased sharply as the number of reinforcements per hour increased from 1 to 20. Beyond 30 reinforcements per hour, the curve was approximately flat, although it sometimes decreased slightly at the highest reinforcement rates. The relative frequency of the shorter interresponse time also increased sharply as the number of reinforcements per hour increased from 1 to 20. The asymptote of the relative frequency function approximately equalled the relative reciprocal of the length of the shorter interresponse time for reinforcement rates greater than 30 or 40 reinforcements per hour. This approximation was obscured by the response-rate function.     

Two-key concurrent paced variable-interval paced variable-interval schedules of reinforcement

Nine pigeons were used in two experiments in which a response was reinforced if a variable-interval schedule had assigned a reinforcement and if the response terminated an interresponse time within a certain interval, or class, of interresponse times. One such class was scheduled on one key, and a second class was scheduled on a second key. The procedure was, therefore, a two-key concurrent paced variable-interval paced variable-interval schedule. In Exp. I, the lengths of the two reinforced interresponse times were varied. The relative frequency of responding on a key approximately equalled the relative reciprocal of the length of the interresponse time reinforced on that key. In Exp. II, the relative frequency and relative magnitude of reinforcement were varied. The relative frequency of responding on the key for which the shorter interresponse time was reinforced was a monotonically increasing, negatively accelerated function of the relative frequency of reinforcement on that key. The relative frequency of responding depended on the relative magnitude of reinforcement in approximately the same way as it depended on the relative frequency of reinforcement. The relative frequency of responding on the key for which the shorter interresponse time was reinforced depended on the lengths of the two reinforced interresponse times and on the relative frequency and relative magnitude of reinforcement in the same way as the relative frequency of the shorter interresponse time depended on these variables in previous one-key concurrent schedules of reinforcement for two interresponse times.     

Choice between response rates

Three pigeons were required to peck a single key at a higher and a lower rate, corresponding to two classes of shorter and longer concurrently reinforced interresponse times. Food reinforcers arranged by a single variable-interval schedule were randomly allocated to the two reinforced interresponse times. The absolute durations of reinforced interresponse times were varied while the total reinforcements per hour was held constant and the relative duration, i.e., the relative reciprocal, of the shorter reinforcer class was held constant at 0.70. Preference for the higher rate of responding, as measured by the relative frequency of responses terminating interresponse times in the shorter reinforced class, depended on the absolute reinforced response rates. Preference for the higher reinforced rate increased from a level of near-indifference (0.50) at high reinforced response rates, through the matching level (0.70) at intermediate reinforced response rates, to a virtually exclusive preference (>0.90) at low reinforced response rates. These results resemble corresponding preference functions obtained with two-key concurrent-chains schedules and thereby provide another sense in which it may be said that interresponse-time distributions from interval schedules estimate preference functions for the component response rates corresponding to different classes of reinforced interresponse times.     

Avoidance based on shock intensity reduction with no change in shock probability

Rats were trained on a free-operant avoidance procedure in which shock intensity was controlled by interresponse time. Shocks were random at a density of about 10 shocks per minute. Shock probability was response independent. As long as interresponse times remained less than the limit in effect, any shocks received were at the lower of two intensities (0.75 mA). Whenever interresponse times exceeded the limit, any shocks received were at the higher intensity (1.6 mA). The initial limit of 15 seconds was decreased in 3-second steps to either 6 or 3 seconds. All animals lever pressed to avoid higher intensity shock. As the interresponse time limit was reduced, the response rate during the lower intensity shock and the proportion of brief interresponse times increased. Substantial warmup effects were evident, particularly at the shorter interresponse-time limits. Shock intensity reduction without change in shock probability was effective in the acquisition and maintenance of avoidance responding, as well as in differentiation of interresponse times. This research suggests limitations on the generality of a safety signal interpretation of avoidance conditioning.     

An interresponse time analysis of variable-ratio punishment

An interresponse time analysis was used to study the effects of variable-ratio punishment schedules on the temporal pattern of reinforced responding. Twelve pigeons responded on a baseline variable-interval schedule of food reinforcement. A variable-ratio ten schedule of electric shock punishment was then introduced. The shock intensity was systematically increased to the highest intensity at which responding could be maintained. At this intensity, the mean variable-ratio value was increased and then decreased. Variable-ratio punishment resulted in an increased relative frequency of very short unreinforced interresponse times (response bursting). Increased response bursting accounted for instances of response rate facilitation. In addition, shock was followed by interresponse times of decreasing mean length over the first several responses after shock.     

Operant acceleration during a pre-reward stimulus

Stimuli of 20, 40, and 80 sec duration terminated with five non-response-contingent food pellets were superimposed upon lever pressing reinforced with single pellets on a DRL 30-sec schedule. Two rhesus monkeys served as subjects. No change in response frequency was observed during the 20- and 40-sec stimuli. During the 80-sec pre-food stimulus, overall response frequency increased to approximately 150% and 220% of pre-stimulus levels, and the temporal distributions of interresponse times shifted toward the shorter intervals. When the 80-sec stimulus was no longer terminated with food, the response frequency decreased and the temporal distributions of interresponse times gradually approached pre-stimulus levels. An increased frequency of short interresponse times and an increase in response rate was again observed when the pellet termination procedure was reinstituted with the 80-sec stimulus. No change in response frequency or interresponse times was observed in the absence of the conditioning stimulus, and performance efficiency, as reflected in the ratio of responses to reinforcements during non-stimulus periods, remained stable throughout the experiment.     

Effects of timeout on spaced responding in pigeons

Three pigeons were trained under a differential-reinforcement-of-low-rate schedule of 20 sec, and then exposed to a schedule under which responses terminating interresponse times less than 20 sec produced timeout and responses terminating interresponse times greater than 20 sec produced reinforcement. Response-produced timeouts selectively decreased the probability of short interresponse times and thereby produced a higher frequency of reinforcement. The suppressive effect of timeout was independent of timeout duration, with timeouts of 5, 10, or 20 sec. Similar effects were found when the minimum interresponse time that could be terminated by response-produced reinforcement was increased to 30 sec. The suppressive effects of timeout on responding maintained by these schedules were similar to previous reports in which responding was punished with electric shock.     

Signalled reinforcement in differential-reinforcement-of-low rate schedules

At several fixed and variable minimum reinforced interresponse times, a stimulus was added to differential-reinforcement-of-low-rate schedules to signal the availability or nonavailability of reinforcement. As the minimum reinforced interresponse time increased, the rate of unreinforced responding decreased. Changing from fixed to variable minimum interresponse time in the basic differential-reinforcement-of-low-rate schedule further decreased the rate of unreinforced responding. Both effects were to some degree reversible. For fixed minimum reinforced interresponse times of 30 sec or shorter, most unreinforced responses terminated interresponse times just short of that required for reinforcement. The minimum reinforced interresponse time and the number of short response latencies (≤0.5 sec) to the onset of the signal were negatively correlated. Both of these analyses suggested that at values of 30 sec or shorter, the subjects discriminated the availability of the reinforcer more on the basis of time than on the basis of presence or absence of the signal.     

The reinforcement of four interresponse times in a two-alternative situation

Pigeons pecked for food in a two-key procedure. A concurrent variable-interval variable-interval schedule of reinforcement for two classes of interresponse times was arranged on each key. A visual stimulus set the occasion for potential reinforcement of the four operant classes: shorter and longer interresponse times on left and right keys. In Exp. I, the relative frequency of respones on a key equalled the relative frequency of reinforcement on that key. In Exp. II, the relative frequency of an interresponse time equalled the relative reciprocal of its length. In Exp. III, the relative frequency of an interresponse time was a monotonically increasing function of its relative frequency of reinforcement. These functions relating the relative frequency of an interresponse time to its relative length and to its relative frequency of reinforcement were the same as if there had been no second key. Also, the distribution of responses between keys was independent of the relative frequency of an interresponse time on either key. Experiment IV replicated Exp. I except that choices between keys were controlled by a stimulus that signalled the availability of reinforcement on the right key. A comparison of Exp. I and IV suggested that the relative frequency of an interresponse time on one key generally was independent of behavior on the other key, but that the number of responses per minute on a key did depend on behavior on the other key.     

Choice and behavioral patterning

Ten pigeons pecked left and right keys in a discrete-trials experiment in which access to food was contingent upon changeovers to the right key after particular runs of left-key pecks. In each of three sets of conditions, two run lengths were reinforced according to a concurrent variable-interval schedule: reinforcement followed runs of either 1 or 2, 1 or 4, or 2 or 4 left-key pecks preceding changeovers. The intertrial interval separating successive pecks was varied from .5 to 10.0 sec, and the relative frequency of reinforcement for the shorter of the two reinforced runs was varied from 0 to .75. The contingencies established local behavioral patterning that roughly approximated that required for reinforcement. For a fixed pair of reinforced run lengths, preference for the shorter of the two frequently increased as the intertrial interval increased and therefore as the minimum temporal durations of both reinforced runs increased. Preference for the shorter of the two also increased as its corresponding relative frequency of reinforcement increased. Both of these effects on preference were qualitatively similar to corresponding effects in previous research with two different kinds of reinforced behavioral patterns, interresponse times and interchangeover times. In all these experiments, analytical units were found in the temporal patterns of behavior, not in the behavior immediately contiguous with a reinforcer. It is suggested that a particular local temporal pattern of behavior is established to the extent to which it is repeatedly remembered when reinforcers are delivered, regardless of whether the delivery of a reinforcer is explicitly contingent upon that pattern.     

Preference as a function of absolute response durations

The durations of 2 responses, 2 categories of reinforced nondiscriminated interresponse times, were varied while their relative durations were held approximately constant, with the longer about 2 1/2 times longer than the shorter. Three pigeons pecked for food. Reinforcers for the shorter and longer responses were arranged by a concurrent variable-interval, variable-interval schedule. Preference for the shorter response increased when both were lengthened. These results, taken together with previous results for discriminated interresponse times, show that preference for the shorter of 2 responses depends on their absolute durations, whether they are discriminated or not and regardless of autoshaped key pecks that may occur in the discriminated case. Time-allocation-matching was not generally obtained. The results qualitatively agree with an associative learner, a computational processing model derived from a molecular analysis of behavior.     

Selective punishment of interresponse times: The roles of shock intensity and scheduling

Two experiments investigated the roles of shock intensity and scheduling in selective punishment of interresponse times. In each experiment the punishment contingencies were imposed on a background of rats' responding maintained by a variable-interval schedule of food presentation. In Experiment 1 all interresponse times greater than 8 seconds produced shock. In Experiment 2 all interresponse times greater than 8 seconds but less than 12 seconds produced shock. In each experiment shock intensity was initially 0.3 milliamperes (mA) and then was varied through an ascending sequence ranging from 0.1 mA to 0.4 mA, in 0.1-mA increments. Experiment 1 produced response-rate increases at low intensities (0.1 and 0.2 mA) but eliminated responding at the remaining intensities. Experiment 2 produced response-rate increases only with 0.1-mA shock, although responding was maintained at all shock parameters investigated. Analysis of the interresponse times per opportunity showed differential suppression of the targeted responses in all cases except the high-intensity shock phases of Experiment 1. The current data support and extend previous studies of selective interresponse-time-dependent shock schedules but suggest that response-rate increases are not a necessary outcome of this type of procedure. The view that variable-interval schedules of shock presentation selectively target long interresponse times was also supported.     

Parametric manipulation of interresponse-time contingency independent of reinforcement rate

Pecking of pigeons was reinforced under a modified interval-percentile procedure that allowed independent manipulation of overall reinforcement rate and the degree to which reinforcement depended on interresponse-time duration. Increasing the contingency, as measured by the phi coefficient, between reinforcement and long interresponse times while controlling the overall rate of reinforcement systematically increased the frequency of those interresponse times and decreased response rate under both of the reinforcement rates studied. Increasing reinforcement rate also generally increased response rate, particularly under weaker interresponse-time contingencies. Random-interval schedules with comparable reinforcement rates generated response rates and interresponse-time distributions similar to those obtained with moderate-to-high interresponse-time reinforcement contingencies. These results suggest that interresponse-time reinforcement contingencies inherent in random-interval and constant-probability variable-interval schedules exercise substantial control over responding independent of overall reinforcement rate effects. The interresponse-time reinforcement contingencies inherent in these schedules may actually mask the effects of overall reinforcement rate; thus differences in response rate as a function of reinforcement rate when interresponse-time reinforcement is eliminated may be underestimated.     

Magnitude and frequency of reinforcement and frequencies of interresponse times

The relative magnitude and relative frequency of reinforcement for two concurrent interresponse times (1.5 to 2.5 sec and 3.5 to 4.5 sec) were simultaneously varied in an experiment in which pigeons obtained grain by pecking on a single key. Visual discriminative stimuli accompanied the two time intervals in which reinforcements were arranged by a one-minute variable-interval schedule. The resulting interresponse times of each of three pigeons fell into two groups; "short" (1.0 to 2.5 sec) and "long" (3.0 to 4.5 sec). Steady-state relative frequencies of these interresponse times were orderly functions of both reinforcement variables. The combined effects of both independent variables were well summarized by a linear function of one variable, relative access to food. Unlike corresponding two-key concurrent variable-interval schedules, the present schedule did not produce an equality between the relative frequency of an operant and either the relative magnitude or the relative frequency of reinforcement of that operant. A tentative account is provided for this difference between one-key and two-key functions.     

Preference for starting and finishing behavior patterns

Pigeon's key pecking was reinforced with food in two experiments in which the correspondence between preference for starting one of two reinforced behavior patterns and the likelihood of finishing it subsequently was examined. Reinforcers were scheduled according to concurrent schedules for two classes of interresponse times, modified such that reinforcers followed a center-key peck terminating either a shorter interresponse time started by a left-key peck or a longer interresponse time started by a right-key peck. In Experiment 1, the times when reinforcers potentially were available were not discriminated, whereas in Experiment 2 they were. Absolute reinforced pattern durations were varied. The relative frequency of starting a particular pattern was highly correlated with relative frequency of that completed pattern in both experiments. Other relations between starting and finishing a pattern depended on whether reinforced interresponse times were discriminated. For instance, preference for starting a pattern sometimes correlated negatively with the likelihood of subsequently completing it. The present experiments are described as capturing part of the ordinary language meaning of "intention," according to which an organism's behavior at one moment sets the occasion for an observer to say that the organism "intends" in the future to engage in one behavior rather than another.     

Synthetic variable-interval schedules of reinforcement

Three pigeons pecked for food on a synthetic variable-interval schedule of reinforcement that had two independent parts: a variable-interval schedule that arranged a distribution of interreinforcement intervals, and a device that randomly assigned each reinforcement to one of 10 classes of interresponse times. The frequencies of reinforcement for the 10 classes of interresponse times were systematically varied, while the overall frequency of reinforcement was held within a comparatively narrow range. The 10 classes extended either from 0.1 to 0.6 sec in 0.05-sec intervals, or from 1.0 to 6.0 sec in 0.5-sec intervals. In the former case, some control by reinforcement was obtained, but it was weak and no simple relationships were discernible. In the latter case, the relative frequency of an interresponse time was a generally increasing function of its relative frequency of reinforcement, and two simple controlling relationships were found. First, the function relating interresponse times per opportunity to reinforcements per opportunity was, over a restricted range, approximately linear with a slope of unity. Second, when all 10 classes of interresponse times were reinforced equally often, the relative frequency of an interresponse time approximately equalled the relative reciprocal of its length.     

DRL escape: effects of minimum duration and intensity of electric shock

Three dogs were exposed to a DRL-escape procedure that required them to endure a minimum duration of electric shock without responding in order for a response to terminate that shock. When this minimum duration increased from 0 to either 2.25 or 7.00 sec, response latencies increased proportionately. With the minimum duration held constant at 2.25 sec, a gradual increase in shock intensity to 5.0 ma had no systematic effect upon latencies. Even under the highest shock intensity, 5.0 ma, latency and interresponse-time distributions were unimodal with very few latencies and interresponse times less than the minimum duration. Three additional dogs were exposed to an escape procedure in which every response was immediately reinforced. For these subjects, the same increase in shock intensity to 5.0 ma was accompanied by a decrease in latencies. The precise temporal spacing of responses obtained with the DRL-escape procedure may in part be due to the fact that every response latency and interresponse time that did not meet the minimum duration was not only extinguished but was also punished.