Selective punishment of interresponse times

Lever pressing by two squirrel monkeys was maintained under a variable-interval 60-second schedule of food presentation. When response-dependent electric shock was made contingent on comparatively long interresponse times, response rate increased, and further increases were obtained when the minimum interresponse-time requirement was decreased. When an equal proportion of responses produced shock without regard to interresponse time, rates decreased. Thus, shock contingent on long interresponse times selectively decreased the relative frequency of those interresponse times, and increased the relative frequency of shorter interresponse times, whereas shock delivered independent of interresponse times decreased the relative frequency of shorter interresponse times while increasing the frequency of longer ones. The results provide preliminary evidence that interresponse times may be differentiated by punishment, further supporting the notion that interresponse times may be considered functional units of behavior.     

Avoidance based on shock intensity reduction with no change in shock probability

Rats were trained on a free-operant avoidance procedure in which shock intensity was controlled by interresponse time. Shocks were random at a density of about 10 shocks per minute. Shock probability was response independent. As long as interresponse times remained less than the limit in effect, any shocks received were at the lower of two intensities (0.75 mA). Whenever interresponse times exceeded the limit, any shocks received were at the higher intensity (1.6 mA). The initial limit of 15 seconds was decreased in 3-second steps to either 6 or 3 seconds. All animals lever pressed to avoid higher intensity shock. As the interresponse time limit was reduced, the response rate during the lower intensity shock and the proportion of brief interresponse times increased. Substantial warmup effects were evident, particularly at the shorter interresponse-time limits. Shock intensity reduction without change in shock probability was effective in the acquisition and maintenance of avoidance responding, as well as in differentiation of interresponse times. This research suggests limitations on the generality of a safety signal interpretation of avoidance conditioning.     

SUPPRESSIVE AND FACILITATIVE EFFECTS OF SHOCK INTENSITY AND INTERRESPONSE TIMES FOLLOWED BY SHOCK

Although response‐dependent shock often suppresses responding, response facilitation can occur. In two experiments, we examined the suppressive and facilitative effects of shock by manipulating shock intensity and the interresponse times that produced shock. Rats' lever presses were reinforced on a variable‐interval 40‐s schedule of food presentation. Shock followed either long or short interresponse times. Shock intensity was raised from 0.05 mA to 0.4 mA or 0.8 mA. Overall, shock contingent on long interresponse times punished long interresponse times and increased response rates. Shock contingent on short interresponse times punished short interresponse times and decreased response rates. In Experiment 1, raising the range of interresponse times that produced shock enhanced these effects. In Experiment 2, the effects of shock intensity depended on the interresponse times that produced shock. When long interresponse times produced shock, low intensities increased response rates. High intensities decreased response rates. When short interresponse times produced shock, high shock intensities punished short interresponse times and decreased response rates more than low intensities. The results may explain why punishment procedures occasionally facilitate responding and establish parameters for future studies of punishment.     

Parametric manipulation of interresponse-time contingency independent of reinforcement rate

Pecking of pigeons was reinforced under a modified interval-percentile procedure that allowed independent manipulation of overall reinforcement rate and the degree to which reinforcement depended on interresponse-time duration. Increasing the contingency, as measured by the phi coefficient, between reinforcement and long interresponse times while controlling the overall rate of reinforcement systematically increased the frequency of those interresponse times and decreased response rate under both of the reinforcement rates studied. Increasing reinforcement rate also generally increased response rate, particularly under weaker interresponse-time contingencies. Random-interval schedules with comparable reinforcement rates generated response rates and interresponse-time distributions similar to those obtained with moderate-to-high interresponse-time reinforcement contingencies. These results suggest that interresponse-time reinforcement contingencies inherent in random-interval and constant-probability variable-interval schedules exercise substantial control over responding independent of overall reinforcement rate effects. The interresponse-time reinforcement contingencies inherent in these schedules may actually mask the effects of overall reinforcement rate; thus differences in response rate as a function of reinforcement rate when interresponse-time reinforcement is eliminated may be underestimated.     

Selective punishment of interresponse times: The roles of shock intensity and scheduling

Two experiments investigated the roles of shock intensity and scheduling in selective punishment of interresponse times. In each experiment the punishment contingencies were imposed on a background of rats' responding maintained by a variable-interval schedule of food presentation. In Experiment 1 all interresponse times greater than 8 seconds produced shock. In Experiment 2 all interresponse times greater than 8 seconds but less than 12 seconds produced shock. In each experiment shock intensity was initially 0.3 milliamperes (mA) and then was varied through an ascending sequence ranging from 0.1 mA to 0.4 mA, in 0.1-mA increments. Experiment 1 produced response-rate increases at low intensities (0.1 and 0.2 mA) but eliminated responding at the remaining intensities. Experiment 2 produced response-rate increases only with 0.1-mA shock, although responding was maintained at all shock parameters investigated. Analysis of the interresponse times per opportunity showed differential suppression of the targeted responses in all cases except the high-intensity shock phases of Experiment 1. The current data support and extend previous studies of selective interresponse-time-dependent shock schedules but suggest that response-rate increases are not a necessary outcome of this type of procedure. The view that variable-interval schedules of shock presentation selectively target long interresponse times was also supported.     

Molecular contingencies in schedules of intermittent punishment

In two experiments, key pecking of pigeons was maintained by a variable-interval 180-s schedule of food presentation. Conjointly, a second schedule delivered response-dependent electric shock. In the first experiment, shocks were presented according to either a variable-interval or a nondifferential interval-percentile schedule. The variable-interval shock schedule differentially delivered shocks following long interresponse times. Although the nondifferential shock schedules delivered shocks less differentially with respect to interresponse times, the two shock schedules equally reduced the relative frequency of long interresponse times. The second experiment differentially shocked long or short interresponse times in different conditions, with resulting decreases in the relative frequency of the targeted interresponse times. These experiments highlight the importance of selecting the appropriate level of analysis for the interaction of behavior and environment. Orderly relations present at one level of analysis (e.g., interresponse times) may not be revealed at other levels of analysis (e.g., overall response rate).     

An interresponse time analysis of variable-ratio punishment

An interresponse time analysis was used to study the effects of variable-ratio punishment schedules on the temporal pattern of reinforced responding. Twelve pigeons responded on a baseline variable-interval schedule of food reinforcement. A variable-ratio ten schedule of electric shock punishment was then introduced. The shock intensity was systematically increased to the highest intensity at which responding could be maintained. At this intensity, the mean variable-ratio value was increased and then decreased. Variable-ratio punishment resulted in an increased relative frequency of very short unreinforced interresponse times (response bursting). Increased response bursting accounted for instances of response rate facilitation. In addition, shock was followed by interresponse times of decreasing mean length over the first several responses after shock.     

An interresponse-time analysis of responding maintained by schedules of response-produced electric shock

The present study investigated ratio contingencies to evaluate factors that may determine the maintenance of responding when electric shock is the consequent event. Initially, squirrel monkeys (Saimiri sciureus) were exposed to a continuous-avoidance schedule to initiate bar pressing. Subsequently, a multiple random-interval variable-ratio yoked schedule of response-produced shock was used to maintain and to compare interval and ratio performance. A microcomputer recorded and stored the number of responses and interresponse times occurring between successive shock presentations during a given random-interval component, and these numbers determined the ratio requirements during the subsequent ratio component. Responding was maintained for more than 80 sessions in two of three monkeys under the multiple schedule with the ratio yoked to the interval component. Responding during the ratio component persisted in only one monkey, however, when the components were no longer yoked. An analysis of the interresponse times immediately preceding shock under the multiple yoked schedule revealed that the terminal interresponse times were longer under the interval schedule than under the ratio contingency. The interresponse-time analysis indicated that differential interresponse-time relationships may be major determinants of the maintenance of behavior controlled by schedules of electric-shock presentation.     

Density and delay of punishment of free-operant avoidance

In two experiments, the free-operant shock-avoidance behavior of rats was punished by electric shock. Two aspects of the schedule of response-produced shock were varied: the frequency of punishment over time (punishment density) and the temporal interval between the punished response and the punishment (punishment delay). The general finding was that response-produced shock suppressed avoidance responding under most of the density-delay combinations studied, and suppression increased as a function of increases in density and decreases in delay. Rate increases of small magnitude also were observed, usually as an initial reaction to the lesser densities and longer delays. Response suppression, while decreasing the number of punishment shocks received, also increased the number of avoidance shocks, so that the total number of shocks received usually was greater than the minimal number possible. The results were discussed from the standpoint of similarities between the effects of punishing positively and negatively reinforced behavior. The finding that subjects did not minimize the total number of shocks suggested that when avoidance behavior is punished, responding is controlled more by the local consequences of responding than by overall shock frequencies during the course of the session.     

Effects of timeout on spaced responding in pigeons

Three pigeons were trained under a differential-reinforcement-of-low-rate schedule of 20 sec, and then exposed to a schedule under which responses terminating interresponse times less than 20 sec produced timeout and responses terminating interresponse times greater than 20 sec produced reinforcement. Response-produced timeouts selectively decreased the probability of short interresponse times and thereby produced a higher frequency of reinforcement. The suppressive effect of timeout was independent of timeout duration, with timeouts of 5, 10, or 20 sec. Similar effects were found when the minimum interresponse time that could be terminated by response-produced reinforcement was increased to 30 sec. The suppressive effects of timeout on responding maintained by these schedules were similar to previous reports in which responding was punished with electric shock.     

Two types of pigeon key pecking: suppression of long- but not short-duration key pecks by duration-dependent shock

The key pecking of eight pigeons was maintained on a variable-interval 1-minute schedule of food reinforcement. Sometimes, all responses between 35 and 50 milliseconds in duration produced a shock; sometimes, all responses between 10 and 25 milliseconds produced a shock; sometimes, shocks were produced by pecks without regard to duration (nondifferential punishment), and sometimes shocks were delivered independently of responding. Punishment of 35- to 50-millisecond responses selectively suppressed those responses, while punishment of 10- to 25-millisecond responses and nondifferential punishment suppressed responding overall but did not suppress responses of particular duration. Punishment of 35- to 50-millisecond responses suppressed key pecking slightly less than did nondifferential punishment. Punishment of 10- to 25-millisecond responses and response-independent shock produced roughly equal amounts of suppression, substantially less than the other punishment procedures. The data support the view that there are at least two kinds of key peck, identifiable on the basis of duration, one of which (short duration) is insensitive to its consequences.     

Reduction of shock duration as negative reinforcement in free-operant avoidance

Rats were trained on a free-operant procedure in which shock duration was controlled by responses within a limited range of interresponse times. Shocks of 1.6-mA intensity occurred randomly with average density of 10 shocks per minute. As long as interresponse times were 15 seconds or less, any shocks received were at the briefer of two durations (.3 second). Whenever interresponse times exceeded 15 seconds, any shocks received were at the longer duration (1.0 second). For six of eight animals, avoidance responding developed quickly and reached levels of better than 90%. Four yoked animals stopped responding within the first few sessions. Shock duration reduction without change in shock probability or intensity was sufficient for the acquisition and maintenance of avoidance responding.     

Responding under schedules combining response-dependent and response-independent shock delivery

Lever pressing of three squirrel monkeys with experience under continuous avoidance schedules was maintained by response-produced shock under a 5-minute variable-interval schedule. Responding decreased when half of the scheduled shocks were delivered independently of lever pressing and decreased further when all shocks were independent of lever pressing. Responding was lowest when all shocks were eliminated. When the proportion of response-dependent shocks increased, responding increased. This relation occurred even though the frequency and temporal distribution of shock delivery remained the same. Responding of two monkeys increased in a graded fashion as the frequency of shock was increased by arranging variable-time 5-minute, 2-minute, and 1-minute schedules jointly with the variable-interval 5-minute schedule. Thus, increasing the proportion of response-independent shocks decreased responding when the overall frequency of shocks stayed the same, but increased responding when the overall frequency of shock delivery increased.     

Integrated delays to shock as negative reinforcement

Rats were shocked at the rate of two per minute until they pressed a lever. In Experiment I, shocks were delivered at variable-time intervals averaging 30 sec; in Experiment II, shocks were delivered at fixed-time intervals of 30 sec. A response produced an alternate condition for a fixed-time period. The shock frequency following a response, calculated over the whole alternate condition, was two per minute. The pattern of shocks in the alternate condition was controlled so that the first shock occurred at the same time as it would have occurred had the response not been emitted; the remaining shocks were delayed until near the end of the alternate condition. Bar pressing was acquired in both experiments. This finding is not explained by two-factor theories of avoidance and is inconsistent with the notion that overall shock-frequency reduction is necessary for negative reinforcement. The data imply that responding is determined by the integrated delays to each shock following a response versus the integrated delays to shock in the absence of a response.     

Shock-duration reduction on the basis of interresponse times: the effect of the interresponse time limit

Rats were trained on a free-operant procedure in which the duration of randomly occurring shocks depended on the interresponse times of lever presses. Shocks of 1.6-mA intensity were delivered at random intervals with an average density of 10 shocks per min. Each shock that was delivered lasted 0.3 s as long as the interresponse times were within a preset limit. Whenever the interresponse time exceeded the limit, the shocks that were delivered lasted 1 s until the occurrence of a response that met the limit. The limit was reduced in 3-s steps from 15 s to either 6 s or 3 s, at which point 3 of the animals were exposed to an ascending series. The avoidance of long-duration shocks was highly efficient at the 15-s and 12-s limits, and it decreased at the 9-s limit. With the exception of one animal, performance was substantially worse at the 6-s limit and it deteriorated for all the animals that were exposed to the 3-s limit. The data suggest that shock-duration reduction is quite effective as negative reinforcement for avoidance but is perhaps less effective than shock-intensity reduction.     

Fixed-interval schedules of electric shock presentation: extinction and recovery of performance under different shock intensities and fixed-interval durations

In squirrel monkeys responding under a schedule in which responding postponed the delivery of electric shock, the presentation of response-dependent shock under a fixed-interval (FI) schedule increased the rate of responding. When the schedule of shock-postponement was eliminated, so that the only shocks delivered were those produced by responses under the FI schedule, a pattern of positively accelerated responding developed and was maintained over an extended period. When responses did not produce shocks (extinction), responding decreased. When shocks were again presented under the FI schedule, the previous pattern of responding quickly redeveloped. In general, response rates were directly related to the intensity of the shock presented, and inversely related to the duration of the fixed-interval. These results raise fundamental questions about the traditional classification of stimuli as reinforcers or punishers. The basic similarities among FI schedules of food presentation, shock termination, and shock presentation strengthen the conclusion that the schedule under which an event is presented and the characteristics of the behavior at the time the event is presented, are of overriding importance in determining the effect of that event on behavior.     

HUMAN RESPONDING ON RANDOM‐INTERVAL SCHEDULES OF RESPONSE‐COST PUNISHMENT: THE ROLE OF REDUCED REINFORCEMENT DENSITY

An experiment with adult humans investigated the effects of response‐contingent money loss (response‐cost punishment) on monetary‐reinforced responding. A yoked‐control procedure was used to separate the effects on responding of the response‐cost contingency from the effects of reduced reinforcement density. Eight adults pressed buttons for money on a three‐component multiple reinforcement schedule. During baseline, responding in all components produced money gains according to a random‐interval 20‐s schedule. During punishment conditions, responding during the punishment component conjointly produced money losses according to a random‐interval schedule. The value of the response‐cost schedule was manipulated across conditions to systematically evaluate the effects on responding of response‐cost frequency. Participants were assigned to one of two yoked‐control conditions. For participants in the Yoked Punishment group, during punishment conditions money losses were delivered in the yoked component response independently at the same intervals that money losses were produced in the punishment component. For participants in the Yoked Reinforcement group, responding in the yoked component produced the same net earnings as produced in the punishment component. In 6 of 8 participants, contingent response cost selectively decreased response rates in the punishment component and the magnitude of the decrease was directly related to the punishment schedule value. Under punishment conditions, for participants in the Yoked Punishment group response rates in the yoked component also decreased, but the decrease was less than that observed in the punishment component, whereas for participants in the Yoked Reinforcement group response rates in the yoked component remained similar to rates in the no‐punishment component. These results provide further evidence that contingent response cost functions similarly to noxious punishers in that it appears to suppress responding apart from its effects on reinforcement density.     

Interresponse-time analysis of stimulus control in multiple schedules

Interresponse-time distributions were recorded in two components of multiple variable-interval schedules that were varied over several conditions. Values of the exponent for power functions relating ratios of interresponse times emitted per opportunity to ratios of reinforcers obtained in the two components varied with interresponse-time class interval. The exponent (sensitivity to reinforcement) afforded a measure of stimulus control exerted by the discriminative stimuli. Exponents were near zero for short interresponse times, consistent with previous conclusions that responses following short interresponse times are controlled by response-produced or proprioceptive stimuli. Values of exponents increased with longer interresponse times, indicating strong control by exteroceptive stimuli over responses following interresponse times of approximately one second or longer.     

Conditioned suppression, punishment, and aversion

Three experiments were conducted to assess the aversive properties of a visual stimulus in the presence of which one group of birds received response-contingent shock (discriminated punishment) while a yoked group of birds received non-contingent shocks (conditioned suppression). In Experiment 1, presentation of the visual stimulus contingent on key pecking reduced the response rate (conditioned punishment effect) for birds under the conditioned suppression procedure but did not reduce the response rate of birds under the discriminative punishment procedure. Non-contingent shocks also produced greater suppression of responding maintained by positive reinforcement in the presence of a visual stimulus than did response-contingent shocks. In Experiment 2, a greater shock intensity (2 mA) was used. All the differences between the two groups found in Experiment 1 were also found in Experiment 2. Experiment 3 demonstrated that response-contingent shock did not result in a conditioned punishment effect even when positive reinforcers were unavailable during the discriminative punishment schedule. The exteroceptive stimulus that was paired with shock in the conditioned suppression procedure acquired the ability to punish behavior. The exteroceptive stimulus in the discriminative punishment schedule did not acquire this ability.     

Interresponse-time shaping by variable-interval-like interresponse-time reinforcement contingencies

The interresponse-time reinforcement contingencies and distributions of interreinforcement intervals characteristic of certain variable-interval schedules were mimicked by reinforcing each key peck with a probability equal to the duration of the interresponse time it terminated, divided by the scheduled mean interreinforcement interval. The interresponse-time reinforcement contingency was then eliminated by basing the probability of reinforcement on the fifth interresponse time preceding the key peck. Even though distributions of interreinforcement intervals were unaffected by this manipulation, response rates consistently increased. A second experiment replicated this effect and showed it to combine additively with that of mean reinforcement rate. These results provide strong support for the contention that current analyses of variable-interval response rates that ignore the inherent interresponse-time reinforcement contingency may be seriously in error.