Timeout postponement without increased reinforcement frequency

Three experiments were conducted to examine pigeons' postponement of signaled extinction periods (timeouts) from a schedule of food reinforcement when such responding neither decreased overall timeout frequency nor increased the overall frequency of food reinforcement. A discrete-trial procedure was used in which a response during the first 5 s of a trial postponed an otherwise immediate 60-s timeout to a later part of that same trial but had no effect on whether the timeout occurred. During time-in periods, responses on a second key produced food according to a random-interval 20-s schedule. In Experiment 1, the response-timeout interval was 45 s under postponement conditions and 0 s under extinction conditions (responses were ineffective in postponing timeouts). The percentage of trials with a response was consistently high when the timeout-postponement contingency was in effect and decreased to low levels when it was discontinued under extinction conditions. In Experiment 2, the response-timeout interval was also 45 s but postponement responses increased the duration of the timeout, which varied from 60 s to 105 s across conditions. Postponement responding was maintained, generally at high levels, at all timeout durations, despite sometimes large decreases in the overall frequency of food reinforcement. In Experiment 3, timeout duration was held constant at 60 s while the response-timeout interval was varied systematically across conditions from 0 s to 45 s. Postponement responding was maintained under all conditions in which the response-timeout interval exceeded 0 s (the timeout interval in the absence of a response). In some conditions of Experiment 3, which were designed to control for the immediacy of food reinforcement and food-correlated (time-in) stimuli, responding postponed timeout but the timeout was delayed whether a response occurred or not. Responding was maintained for 2 of 3 subjects, suggesting that behavior was negatively reinforced by timeout postponement rather than positively reinforced by the more immediate presentation of food or food-correlated (time-in) stimuli.     

Successive interresponse times in fixed-ratio and second-order fixed-ratio performance

Three rats were trained on a schedule in which every sixth response produced a timeout of 5 sec minimum duration, and food was delivered at the onset of timeout. Successive interresponse times were measured under these conditions, and also when behavior was maintained by second-order fixed-ratio and fixed-interval schedules. Under the second-order schedules, each six-response fixed-ratio component was followed by a timeout, and occasionally food was delivered at the onset of a timeout. In the fixed-ratio schedule, the successive interresponse times showed a decrease followed by an increase before food delivery, but this systematic variation in interresponse times was not found when the performance was under second-order reinforcement. Under both second-order schedules the latencies of successive components, and the successive interresponse times within each component, showed a decrease as food delivery was approached.     

Reinforcement contingencies maintaining collateral responding under a DRL schedule

Two-key conjunctive schedules were studied with one key (food key) under a differential-reinforcement-of-low-rate 20-sec schedule, while the consequences of responding on another key (collateral key) were varied. When food depended not only upon a food-key interresponse time in excess of 20 sec, but also upon the occurrence of one or more collateral-key responses during the food-key interresponse time, the rate of collateral-key responding was low and food-key interresponse times rarely exceeded 20 sec. When collateral-key responses could produce a discriminative stimulus correlated with the availability of food under the DRL schedule, the discriminative stimulus functioned as a conditioned reinforcer to maintain higher rates of collateral-key responding, and the spacing of food-key responses increased. If the occurrence of the discriminative stimulus was independent of collateral-key responses, the rate of collateral-key responding was again low, but the spacing of food-key responses was still controlled by the discriminative stimulus. Both the conditioned reinforcer and the explicit reinforcement contingency could maintain collateral-key responding, but the adventitious correlation between collateral-key responses and the delivery of food could not maintain very much collateral-key responding. The pattern of responding on the food-key was determined to a much greater extent by the correlation between the discriminative stimulus and the delivery of food than by the pattern of responding on the collateral key.     

Signalled reinforcement in differential-reinforcement-of-low rate schedules

At several fixed and variable minimum reinforced interresponse times, a stimulus was added to differential-reinforcement-of-low-rate schedules to signal the availability or nonavailability of reinforcement. As the minimum reinforced interresponse time increased, the rate of unreinforced responding decreased. Changing from fixed to variable minimum interresponse time in the basic differential-reinforcement-of-low-rate schedule further decreased the rate of unreinforced responding. Both effects were to some degree reversible. For fixed minimum reinforced interresponse times of 30 sec or shorter, most unreinforced responses terminated interresponse times just short of that required for reinforcement. The minimum reinforced interresponse time and the number of short response latencies (≤0.5 sec) to the onset of the signal were negatively correlated. Both of these analyses suggested that at values of 30 sec or shorter, the subjects discriminated the availability of the reinforcer more on the basis of time than on the basis of presence or absence of the signal.     

Effects of d-amphetamine and chlordiazepoxide on spaced responding in pigeons

The effects of d-amphetamine and chlordiazepoxide were studied in pigeons on performance (1) under a schedule that reinforced responses on a key (food key) if they were more than 20 sec apart, (2) under the same schedule when responses also were required on a collateral key during the interresponse time on the food key, and (3) under the same schedule when responses were required on a collateral key during the interresponse time on the food key and collateral-key responses could produce a stimulus correlated with the availability of food. Under all three spaced-responding schedules, d-amphetamine and chlordiazepoxide at low dose levels slightly increased the frequency of short interresponse times on the food key for about half the birds, and either did not affect the interresponse time patterns of the other birds, or lengthened the durations slightly. At higher dose levels, d-amphetamine and chlordiazepoxide increased the frequency of long interresponse times or abolished responding in all birds. Changes in the pattern of interresponse times on the food key did not seem to depend on changes in the rate or pattern of collateral-key responses.     

Studies on responding under fixed-interval schedules of reinforcement: the effects on the pattern of responding of changes in requirements at reinforcement

In pigeons responding under a 180-sec fixed-interval schedule of reinforcement, the frequency distribution of the duration of the final interresponse time before the reinforcer was compared with the distribution of the preceding two interresponse times. The results confirmed qualitatively and quantitatively the expected preferential reinforcement of longer interreinforcement times under fixed-interval reinforcement. Requirements at reinforcement were then changed to eliminate the preferential reinforcement of longer interresponse times. Local patterns and mean rate of responding could change, without the characteristic fixed-interval pattern of increasing responding through the interval (scalloping) being much affected. It is concluded that this characteristic pattern of fixed-interval responding does not depend crucially on effects of the reinforcer at the moment of reinforcement, but rather to effects extending over much longer periods of time than just the last interresponse time.     

Effects of lick-contingent timeout on schedule-induced polydipsia

Rats bar pressing on a 1-min fixed-interval schedule for 45-mg food pellets became polydipsic when water was concurrently available. They were then exposed to conditions in which each lick on the drinking tube produced a timeout period during which the food-schedule lever was retracted and the fixed-interval timer either did or did not continue to operate. Licks occurring within a timeout period extended its duration. As the duration of the lick-initiated timeout period was increased logarithmically through four values from 10 sec to 80 sec, lick rates as well as water intake rates generally decreased for all three subjects. As timeout duration was progressively increased, the rate of licks occurring in the absense of, but producing, timeouts decreased for all three rats, whereas the rate of licks occurring in the presence of timeout periods remained essentially constant. Water-intake rates and, with one exception, lick rates were suppressed more by timeout periods during which the fixed-interval timer did not continue to operate. These results indicate that lick-contingent timeout from positive reinforcement reduces schedule-induced drinking, and this suppressive effect is greater when the timeout period necessarily increases the interreinforcement interval beyond its minimum duration than when it does not.     

Sequential dependencies in free-responding

Three pigeons pecked for food in an experiment in which reinforcements were arranged for responses terminating sequences of interresponse times. Each reinforced interresponse time belonged to a class extending either from 1.0 to 2.0 sec (class A) or from 3.0 to 4.5 sec (class B). Reinforcements were arranged by a single variable-interval schedule and a random device that assigned each reinforcement to one of four sequences of two successive interresponse times: AA, AB, BA, or BB. Throughout the experiment, half of the reinforcements were delivered for interresponse times in class A and half for those in class B. Over conditions, the interresponse time preceding a reinforced interresponse time always, half of the time, or never, belonged to class A. The duration of the interresponse time preceding a reinforced one had a pronounced effect on response patterning. It also had a pronounced effect on the overall response probability, which was highest, intermediate, and lowest, when the interresponse time preceding a reinforced interresponse time always, half of the time, or never, belonged to class A, respectively. In no case were successive interresponse times independent, so that overall response probability was not representative of momentary response probabilities.     

Extinction of responding maintained by timeout from avoidance

The resistance to extinction of lever pressing maintained by timeout from avoidance was examined. Rats were trained under a concurrent schedule in which responses on one lever postponed shock on a free-operant avoidance (Sidman) schedule (response-shock interval = 30 s) and responses on another lever produced 2 min of signaled timeout from avoidance on a variable-ratio 15 schedule. Following extended training (106 to 363 2-hr sessions), two experiments were conducted. In Experiment 1 two different methods of extinction were compared. In one session, all shocks were omitted, and there was some weakening of avoidance but little change in timeout responding. In another session, responding on the timeout lever was ineffective, and under these conditions timeout responding showed rapid extinction. The within-session patterns produced by extinction manipulations were different than the effects of drugs such as morphine, which also reduces timeout responding. In Experiment 2 shock was omitted for many consecutive sessions. Response rates on the avoidance lever declined relatively rapidly, with noticeable reductions within 5 to 10 sessions. Extinction of the timeout lever response was much slower than extinction of avoidance in all 4 rats, and 2 rats continued responding at baseline levels for more than 20 extinction sessions. These results show that lever pressing maintained by negative reinforcement can be highly resistant to extinction. The persistence of responding on the timeout lever after avoidance extinction is not readily explained by current theories.     

Mirror pecking and timeout under a multiple fixed-ratio schedule of food delivery

Pigeons were trained to peck a key under a multiple fixed-ratio 25 fixed-ratio 175 schedule of food presentation. In the first condition, either a mirror or the opportunity to produce a 30-second timeout were available. In a second condition, mirror and timeout availability were reversed for the two groups. Following a return to the initial condition, mirror and timeout keys were presented together for all birds. Mirror and timeout responses occurred predominantly in the pause in the larger fixed-ratio component, regardless of whether the opportunities for the two responses were available singly or together. Mirror responding occurred in a greater proportion of the pauses than did timeouts. When the opportunities for both mirror pecking and timeout were available concurrently, they occurred with probabilities similar to those under the single conditions. Within the pause itself, mirror responses most frequently occurred immediately after reinforcement. Timeouts occurred most frequently toward the end of the pause, and some timeouts occurred in the early part of the run. Longer preratio pausing occurred in the larger fixed-ratio component in the conditions in which the mirror was present, whether or not any mirror pecks were recorded.     

Behavior under large values of the differential-reinforcement-of-low-rate schedule

Pigeons pecked a key and rats pressed a lever for food reinforcement under large values of the differential-reinforcement-of-low-rate schedule. Each subject was tested under 10 different schedule values ranging from 1 to 45 min and was exposed to each schedule value at least twice. The mean interresponse time and mean interreinforcement time increased with the schedule value according to power functions. Response-probability functions were computed for schedule values below 20 min and showed an increase in response probability as a function of time since the last response in most cases. Mean responses per reinforcer increased as a function of schedule value for the rats, but decreased as a function of schedule value for the pigeons. The proportion of responses with interresponse times shorter than 1 sec were an increasing function of schedule value for the pigeons, but did not vary as a function of schedule value for the rats.     

Self-imposed timeouts under increasing response requirements

Self-imposed timeouts by pigeons working under a progressive-ratio food schedule were studied under different conditions. The main findings were (1) continued production of timeouts over an extended series of sessions, (2) more frequent responding on the key with the timeout consequence than on a key having no consequence, (3) an inverse relationship between number of timeouts and level of body weight, (4) production of timeouts when the timeout duration was brief, lengthy, or controlled by the pigeon, and (5) dependence of self-imposed timeouts on variables controlling responding under the progressive-ratio schedule. Under all experimental conditions, with the exception of performances at the high body weight, timeouts were more frequent during the longer progressive-ratio steps and usually were localized in the post-reinforcement pause or the early part of the step. The timeout behavior could be interpreted as either an escape from aversive stimuli generated by the progressive-ratio schedule or as a response reinforced by the consequent stimulus change.     

The reinforcement of least-frequent interresponse times

A new schedule of reinforcement was used to maintain key-pecking by pigeons. The schedule reinforced only pecks terminating interresponse times which occurred least often relative to the exponential distribution of interresponse times to be expected from an ideal random generator. Two schedule parameters were varied: (1) the rate constant of the controlling exponential distribution and (2) the probability that a response would be reinforced, given that it met the interresponse-time contingency. Response rate changed quickly and markedly with changes in the rate constant; it changed only slightly with a fourfold change in the reinforcement probability. The schedule produced stable rates and high intra- and inter-subject reliability, yet interresponse time distributions were approximately exponential. Such local interresponse time variability in the context of good overall control suggests that the schedule may be used to generate stable, predictable, yet sensitive baseline rates. Implications for the measurement of rate are discussed.     

SUPPRESSIVE AND FACILITATIVE EFFECTS OF SHOCK INTENSITY AND INTERRESPONSE TIMES FOLLOWED BY SHOCK

Although response‐dependent shock often suppresses responding, response facilitation can occur. In two experiments, we examined the suppressive and facilitative effects of shock by manipulating shock intensity and the interresponse times that produced shock. Rats' lever presses were reinforced on a variable‐interval 40‐s schedule of food presentation. Shock followed either long or short interresponse times. Shock intensity was raised from 0.05 mA to 0.4 mA or 0.8 mA. Overall, shock contingent on long interresponse times punished long interresponse times and increased response rates. Shock contingent on short interresponse times punished short interresponse times and decreased response rates. In Experiment 1, raising the range of interresponse times that produced shock enhanced these effects. In Experiment 2, the effects of shock intensity depended on the interresponse times that produced shock. When long interresponse times produced shock, low intensities increased response rates. High intensities decreased response rates. When short interresponse times produced shock, high shock intensities punished short interresponse times and decreased response rates more than low intensities. The results may explain why punishment procedures occasionally facilitate responding and establish parameters for future studies of punishment.     

Two-key concurrent paced variable-interval paced variable-interval schedules of reinforcement

Nine pigeons were used in two experiments in which a response was reinforced if a variable-interval schedule had assigned a reinforcement and if the response terminated an interresponse time within a certain interval, or class, of interresponse times. One such class was scheduled on one key, and a second class was scheduled on a second key. The procedure was, therefore, a two-key concurrent paced variable-interval paced variable-interval schedule. In Exp. I, the lengths of the two reinforced interresponse times were varied. The relative frequency of responding on a key approximately equalled the relative reciprocal of the length of the interresponse time reinforced on that key. In Exp. II, the relative frequency and relative magnitude of reinforcement were varied. The relative frequency of responding on the key for which the shorter interresponse time was reinforced was a monotonically increasing, negatively accelerated function of the relative frequency of reinforcement on that key. The relative frequency of responding depended on the relative magnitude of reinforcement in approximately the same way as it depended on the relative frequency of reinforcement. The relative frequency of responding on the key for which the shorter interresponse time was reinforced depended on the lengths of the two reinforced interresponse times and on the relative frequency and relative magnitude of reinforcement in the same way as the relative frequency of the shorter interresponse time depended on these variables in previous one-key concurrent schedules of reinforcement for two interresponse times.     

Avoidance of timeout from response-independent reinforcement

Responses on a lever by rats postponed scheduled timeouts, or periods during which the delivery of response-independent food was withheld. The effects of a number of experimental variables were examined and the conclusions drawn are that the functional relations describing free-operant avoidance of timeout from response-independent reinforcement are similar to those for avoidance of electric shock and that both phenomena are sensitive to the same parametric manipulations. Results suggest that high frequency of food delivery in timein maintains a higher rate of timeout avoidance than low frequency. The evidence argues against an interpretation in terms of adventitious food-reinforcement of the timeout avoidance response. Finally, the effects of scheduling timeouts independently of responding and of omitting timeouts confirm the view that timeouts can be aversive and may act as punishment for responding.     

Molar And Molecular Control In Variable-interval And Variable-ratio Schedules

Response rates are typically higher under variable-ratio than under variable-interval schedules of reinforcement, perhaps because of differences in the dependence of reinforcement rate on response rate or because of differences in the reinforcement of long interresponse times. A variable-interval-with-added-linear-feedback schedule is a variable-interval schedule that provides a response rate/reinforcement rate correlation by permitting the minimum interfood interval to decrease with rapid responding. Four rats were exposed to variable-ratio 15, 30, and 60 food reinforcement schedules, variable-interval 15-, 30-, and 60-s food reinforcement schedules, and two versions of variable-interval-with-added-linear-feedback 15-, 30-, and 60-s food reinforcement schedules. Response rates on the variable-interval-with-added-linear-feedback schedule were similar to those on the variable-interval schedule; all three schedules led to lower response rates than those on the variable-ratio schedules, especially when the schedule values were 30. Also, reinforced interresponse times on the variable-interval-with-added-linear-feedback schedule were similar to those on variable interval and much longer than those produced by variable ratio. The results were interpreted as supporting the hypothesis that response rates on variable-interval schedules in rats are lower than those on comparable variable-ratio schedules, primarily because the former schedules reinforce long interresponse times.     

An interresponse time analysis of variable-ratio punishment

An interresponse time analysis was used to study the effects of variable-ratio punishment schedules on the temporal pattern of reinforced responding. Twelve pigeons responded on a baseline variable-interval schedule of food reinforcement. A variable-ratio ten schedule of electric shock punishment was then introduced. The shock intensity was systematically increased to the highest intensity at which responding could be maintained. At this intensity, the mean variable-ratio value was increased and then decreased. Variable-ratio punishment resulted in an increased relative frequency of very short unreinforced interresponse times (response bursting). Increased response bursting accounted for instances of response rate facilitation. In addition, shock was followed by interresponse times of decreasing mean length over the first several responses after shock.     

Timeout from a stimulus correlated with the extinction component of a multiple schedule

Four groups of pigeons were trained to discriminate between green and red. Pecks on a second key produced a timeout from the schedule in effect for 30 sec. For two of the groups, this timeout response turned off all the lights in the chamber (Blackout), while for the other two, only the key-lights were turned off (No Blackout). For one of the Blackout groups and one of the No Blackout groups, responses on the discrimination key during the extinction component (S^?) also resulted in a mild electric shock. Blackout groups produced more timeouts during S^? than did No Blackout groups, but electric shock punishment suppressed, rather than enhanced, timeout responding. These findings suggest a need for reevaluation of the hypothesis that the timeout response is an escape from an aversive S^?.     

Response-produced timeouts under a progressive-ratio schedule with a punished reset option

Three behavioral options were available to food-deprived pigeons: (1) pecking one key resulted in food reinforcement according to a 50-response progressive-ratio schedule, (2) pecking a second key reset the progressive-ratio schedule to the initial progressive-ratio step, and (3) pecking a third key produced a 3-min timeout period. Pecks on the reset key were shocked. Under low and intermediate shock intensities, timeouts were not produced; under high shock levels, timeouts were produced regularly. Timeouts occurred during the initial period of a progressive-ratio step and were more frequent during the longer steps of the progressive-ratio schedule. Response-produced timeouts under these experimental conditions could be interpreted either as an escape from aversive behavioral options or as a low-probability behavior emerging when the food reinforcement schedule exerted weaker control.