Effects of timeout on spaced responding in pigeons

Three pigeons were trained under a differential-reinforcement-of-low-rate schedule of 20 sec, and then exposed to a schedule under which responses terminating interresponse times less than 20 sec produced timeout and responses terminating interresponse times greater than 20 sec produced reinforcement. Response-produced timeouts selectively decreased the probability of short interresponse times and thereby produced a higher frequency of reinforcement. The suppressive effect of timeout was independent of timeout duration, with timeouts of 5, 10, or 20 sec. Similar effects were found when the minimum interresponse time that could be terminated by response-produced reinforcement was increased to 30 sec. The suppressive effects of timeout on responding maintained by these schedules were similar to previous reports in which responding was punished with electric shock.     

Successive interresponse times in fixed-ratio and second-order fixed-ratio performance

Three rats were trained on a schedule in which every sixth response produced a timeout of 5 sec minimum duration, and food was delivered at the onset of timeout. Successive interresponse times were measured under these conditions, and also when behavior was maintained by second-order fixed-ratio and fixed-interval schedules. Under the second-order schedules, each six-response fixed-ratio component was followed by a timeout, and occasionally food was delivered at the onset of a timeout. In the fixed-ratio schedule, the successive interresponse times showed a decrease followed by an increase before food delivery, but this systematic variation in interresponse times was not found when the performance was under second-order reinforcement. Under both second-order schedules the latencies of successive components, and the successive interresponse times within each component, showed a decrease as food delivery was approached.     

The effect of foreperiod duration on reaction time and its relation to interval schedules of reinforcement

Two groups of pigeons were exposed to a simple reaction-time procedure in which mean foreperiod duration was 5, 10, or 20 seconds. For one group, the foreperiods had an arithmetic, or rectangular, distribution; for the second group, they had a constant-probability, or Bernoulli, distribution. Under both distributions, mean response latency was an increasing, negatively accelerated function of mean foreperiod duration. On a given trial, response latency was a function of its associated foreperiod duration: latency was a decreasing function of foreperiod duration in the arithmetic distribution, and an increasing function of foreperiod duration in the constant-probability distribution. Examination of the distribution of latencies revealed a harmonic structure reminiscent of distributions of interresponse times under variable-interval schedules of reinforcement. Taken together, the results confirm and extend previous findings with human subjects, and also suggest numerous similarities to behavior maintained by variable-interval schedules.     

Punishment: the interactive effects of delay and intensity of shock

A discrete-trial punishment procedure, with rats, was used to examine how delay-of-shock intervals of 0 to 28 sec and shock intensity interact to decrease the frequency and increase the latency of a positively reinforced response. For delay-of-shock intervals of 0, 7, 14, and 28 sec, there was a range of shock intensities, for some subjects, over which the punishing effect of shock was an increasing, monotonic function of shock intensity. For other subjects this transition was abrupt. Functions relating response frequency and latency measures to shock intensity were displaced toward higher values on the shock intensity axis with an increase in delay-of-shock interval. The effects of “gradual” and “abrupt” introduction to “severe” shock, as well as re-exposure to previously used shock intensities, were examined under both the immediate and delay-of-shock conditions. With delay-of-shock intervals of 7, 14, or 28 sec, shock intensities of approximately 0.50 milliamperes or greater were necessary to decrease substantially the number and increase the latency of the lever-pressing response. For the immediate punishment group this intensity was approximately 0.20 ma. These facts were related to Annau and Kamin's (1961) conditioned emotional response experiment in which a shock intensity of 0.49 ma or greater was required to suppress the rate of a positively reinforced response.     

Signalled reinforcement in differential-reinforcement-of-low rate schedules

At several fixed and variable minimum reinforced interresponse times, a stimulus was added to differential-reinforcement-of-low-rate schedules to signal the availability or nonavailability of reinforcement. As the minimum reinforced interresponse time increased, the rate of unreinforced responding decreased. Changing from fixed to variable minimum interresponse time in the basic differential-reinforcement-of-low-rate schedule further decreased the rate of unreinforced responding. Both effects were to some degree reversible. For fixed minimum reinforced interresponse times of 30 sec or shorter, most unreinforced responses terminated interresponse times just short of that required for reinforcement. The minimum reinforced interresponse time and the number of short response latencies (≤0.5 sec) to the onset of the signal were negatively correlated. Both of these analyses suggested that at values of 30 sec or shorter, the subjects discriminated the availability of the reinforcer more on the basis of time than on the basis of presence or absence of the signal.     

Shock-duration reduction on the basis of interresponse times: the effect of the interresponse time limit

Rats were trained on a free-operant procedure in which the duration of randomly occurring shocks depended on the interresponse times of lever presses. Shocks of 1.6-mA intensity were delivered at random intervals with an average density of 10 shocks per min. Each shock that was delivered lasted 0.3 s as long as the interresponse times were within a preset limit. Whenever the interresponse time exceeded the limit, the shocks that were delivered lasted 1 s until the occurrence of a response that met the limit. The limit was reduced in 3-s steps from 15 s to either 6 s or 3 s, at which point 3 of the animals were exposed to an ascending series. The avoidance of long-duration shocks was highly efficient at the 15-s and 12-s limits, and it decreased at the 9-s limit. With the exception of one animal, performance was substantially worse at the 6-s limit and it deteriorated for all the animals that were exposed to the 3-s limit. The data suggest that shock-duration reduction is quite effective as negative reinforcement for avoidance but is perhaps less effective than shock-intensity reduction.     

Interresponse-time punishment: a basis for shock-maintained behavior.

Lever pressing of squirrel monkeys postponed brief electric shock according to a free-operant shock-postponement procedure. Pressing also produced shock with a probability proportional to the duration of the current interresponse time in some conditions, or to the fifth ordinally-preceding interresponse time in others. These conditions provided equal frequencies and temporal distributions of response-produced shocks either contingent on or independent of the current interresponse-time duration, respectively. Shock delivered contingent on the current interresponse-time duration resulted in shorter mean interresponse times and higher overall response rates that shock delivered independent of the current interresponse time. In subsequent conditions, response-produced shocks were sufficient to maintain responding following suspension of the postponement procedure only when those shocks were contingent on the current interresponse time. Presenting shock independent of the current interresponse time, conversely, suppressed response rate and ultimately led to cessation of responding in the absence of a conjoint shock-postponement procedure. These results demonstrate interresponse-time punishment in the absence of any indirect avoidance contingencies based on overall shock-frequency reduction, and strongly support similar interpretation at the more local level of shock-frequency reduction correlated with particular interresponse times. Differential punishment of long interresponse times also provides both an a priori basis for predicting whether a schedule of shock presentation will maintain or suppress responding and a framework for interpreting many of the functional relations between overall response rate and parameters of consequent shock presentation. Finally, these results and others indicate the importance of response-consequence contiguity above and beyong any notion of noncontiguous contingency in the control of behavior.     

Shock intensity and duration interactions on free-operant avoidance behavior

Shock intensities (1 to 4 mA) and shock durations (0.3 to 0.75 sec) were concurrently varied over a range commonly used in free-operant avoidance studies using a lever-press response. Response rates were a positive linear function of the log of the product of intensity times duration. Shock rates were a negative linear function of that log. The increase in response rates was primarily due to a selective increase in the conditional probability of making responses with long interresponse times. The disproportionality of receiving shocks early in the session (warm-up) was also a linear function of the log of the intensity-duration product, with increasing disproportionality as the value of the intensity-duration product was increased. Thus, with all measures of the avoidance performance, shock intensity and shock duration combine in a multiplicative fashion to determine the avoidance performance.     

Avoidance based on shock intensity reduction with no change in shock probability

Rats were trained on a free-operant avoidance procedure in which shock intensity was controlled by interresponse time. Shocks were random at a density of about 10 shocks per minute. Shock probability was response independent. As long as interresponse times remained less than the limit in effect, any shocks received were at the lower of two intensities (0.75 mA). Whenever interresponse times exceeded the limit, any shocks received were at the higher intensity (1.6 mA). The initial limit of 15 seconds was decreased in 3-second steps to either 6 or 3 seconds. All animals lever pressed to avoid higher intensity shock. As the interresponse time limit was reduced, the response rate during the lower intensity shock and the proportion of brief interresponse times increased. Substantial warmup effects were evident, particularly at the shorter interresponse-time limits. Shock intensity reduction without change in shock probability was effective in the acquisition and maintenance of avoidance responding, as well as in differentiation of interresponse times. This research suggests limitations on the generality of a safety signal interpretation of avoidance conditioning.     

Reduction of shock duration as negative reinforcement in free-operant avoidance

Rats were trained on a free-operant procedure in which shock duration was controlled by responses within a limited range of interresponse times. Shocks of 1.6-mA intensity occurred randomly with average density of 10 shocks per minute. As long as interresponse times were 15 seconds or less, any shocks received were at the briefer of two durations (.3 second). Whenever interresponse times exceeded 15 seconds, any shocks received were at the longer duration (1.0 second). For six of eight animals, avoidance responding developed quickly and reached levels of better than 90%. Four yoked animals stopped responding within the first few sessions. Shock duration reduction without change in shock probability or intensity was sufficient for the acquisition and maintenance of avoidance responding.     

SUPPRESSIVE AND FACILITATIVE EFFECTS OF SHOCK INTENSITY AND INTERRESPONSE TIMES FOLLOWED BY SHOCK

Although response‐dependent shock often suppresses responding, response facilitation can occur. In two experiments, we examined the suppressive and facilitative effects of shock by manipulating shock intensity and the interresponse times that produced shock. Rats' lever presses were reinforced on a variable‐interval 40‐s schedule of food presentation. Shock followed either long or short interresponse times. Shock intensity was raised from 0.05 mA to 0.4 mA or 0.8 mA. Overall, shock contingent on long interresponse times punished long interresponse times and increased response rates. Shock contingent on short interresponse times punished short interresponse times and decreased response rates. In Experiment 1, raising the range of interresponse times that produced shock enhanced these effects. In Experiment 2, the effects of shock intensity depended on the interresponse times that produced shock. When long interresponse times produced shock, low intensities increased response rates. High intensities decreased response rates. When short interresponse times produced shock, high shock intensities punished short interresponse times and decreased response rates more than low intensities. The results may explain why punishment procedures occasionally facilitate responding and establish parameters for future studies of punishment.     

Sequential dependencies in free-responding

Three pigeons pecked for food in an experiment in which reinforcements were arranged for responses terminating sequences of interresponse times. Each reinforced interresponse time belonged to a class extending either from 1.0 to 2.0 sec (class A) or from 3.0 to 4.5 sec (class B). Reinforcements were arranged by a single variable-interval schedule and a random device that assigned each reinforcement to one of four sequences of two successive interresponse times: AA, AB, BA, or BB. Throughout the experiment, half of the reinforcements were delivered for interresponse times in class A and half for those in class B. Over conditions, the interresponse time preceding a reinforced interresponse time always, half of the time, or never, belonged to class A. The duration of the interresponse time preceding a reinforced one had a pronounced effect on response patterning. It also had a pronounced effect on the overall response probability, which was highest, intermediate, and lowest, when the interresponse time preceding a reinforced interresponse time always, half of the time, or never, belonged to class A, respectively. In no case were successive interresponse times independent, so that overall response probability was not representative of momentary response probabilities.     

Rats' lever-press durations as psychophysical judgments of time

Hungry rats received food following lever-press durations exceeding a minimum value, which ranged from 0 to 6.4 sec. When no intertrial intervals separated successive presses, modal press durations remained at very short values as the minimum value required for food was increased. This was particularly true immediately after a food presentation. When an 8-sec intertrial interval followed each lever release, modal press durations were always at or beyond the minimum value required for food, and outcome of the preceding press had no effect on press duration. Possible reasons for the effects of intertrial intervals included punishment of short presses, increased delay of reinforcement of short presses, and reduced density of reinforcement. In addition, functions relating discrete-trials lever-press duration to minimum duration required for food were found to be qualitatively and quantitatively similar to the power functions recently proposed by Catania (1970) for interresponse time and response latency. This similarity was taken as support for a general psychophysical law of temporal judgments.     

A local-rate-of-response and interresponse-time analysis of behavioral contrast

Three pigeons were exposed to a two-component multiple schedule in which a variable-interval 3-min schedule was always in effect in one component. The schedule in the other component was either variable-interval 3-min or extinction in alternate blocks of sessions. When the schedule was changed from multiple variable-interval 3-min variable-interval 3-min to multiple variable-interval 3-min extinction in the second and fourth phases of the experiment, overall response rates in the unchanged variable-interval 3-min component increased in two pigeons. Response rate declined when the schedule was changed to multiple variable-interval 3-min variable-interval 3-min again. Correlated with increases in overall response rate in the unchanged component were increases in local response rates at the beginning of the unchanged component and immediately after food presentation. Local rates 40 sec after food presentation did not increase greatly in the presence of the multiple variable-interval 3-min extinction schedule. An interresponse time analysis of three local rate samples showed small increases in the relative frequency of short-duration interresponse times at the beginning of the unchanged component and immediately after food presentation. Neither the postreinforcement pause nor the latency to the first response in the unchanged component changed systematically.     

Selective punishment of interresponse times

Lever pressing by two squirrel monkeys was maintained under a variable-interval 60-second schedule of food presentation. When response-dependent electric shock was made contingent on comparatively long interresponse times, response rate increased, and further increases were obtained when the minimum interresponse-time requirement was decreased. When an equal proportion of responses produced shock without regard to interresponse time, rates decreased. Thus, shock contingent on long interresponse times selectively decreased the relative frequency of those interresponse times, and increased the relative frequency of shorter interresponse times, whereas shock delivered independent of interresponse times decreased the relative frequency of shorter interresponse times while increasing the frequency of longer ones. The results provide preliminary evidence that interresponse times may be differentiated by punishment, further supporting the notion that interresponse times may be considered functional units of behavior.     

Free operant avoidance in rats under increased nitrogen pressures.

Rats performed on a free operant avoidance schedule with a response-shock interval of 20 sec. and a shock-shock interval of 2 sec. Avoidance response rates increased and shock frequency decreased when the rats were exposed to elevated pressures of both air and a nitrogen-oxygen mixture in a hyperbaric chamber. Increases in response rates were related to raised partial pressures of nitrogen at 89.0 psi and 111.3 psi. Conditional probabilities of interresponse times indicated that increases in response rates were not due to disruption of temporal discrimination. Increased avoidance rates under pressure suggested direct excitatory effects of high pressures of nitrogen.     

An interresponse time analysis of variable-ratio punishment

An interresponse time analysis was used to study the effects of variable-ratio punishment schedules on the temporal pattern of reinforced responding. Twelve pigeons responded on a baseline variable-interval schedule of food reinforcement. A variable-ratio ten schedule of electric shock punishment was then introduced. The shock intensity was systematically increased to the highest intensity at which responding could be maintained. At this intensity, the mean variable-ratio value was increased and then decreased. Variable-ratio punishment resulted in an increased relative frequency of very short unreinforced interresponse times (response bursting). Increased response bursting accounted for instances of response rate facilitation. In addition, shock was followed by interresponse times of decreasing mean length over the first several responses after shock.     

Time limits for completing fixed ratios. IV. Components of the ratio

Pigeons received food after completing a fixed ratio if the temporal properties of responding exceeded minimum duration requirements. In one set of conditions, a minimum time had to elapse before the first response of the ratio (the initial pause). In another set, the minimum duration was the time between the first and last response of the ratio. Obtained times increased as a power function of required times in both conditions. The power function resembled that occurring in experiments involving temporal differentiation of individual responses, interresponse times, latencies, and entire fixed-ratio sequences. Moreover, in all of these experiments individual performances could be described as a function of the base duration (the duration occurring in the absence of temporal requirements) and the specific time requirement. Control conditions indicated that the effects resulted from temporal requirements and not from reinforcer intermittency.     

Selective punishment of interresponse times: The roles of shock intensity and scheduling

Two experiments investigated the roles of shock intensity and scheduling in selective punishment of interresponse times. In each experiment the punishment contingencies were imposed on a background of rats' responding maintained by a variable-interval schedule of food presentation. In Experiment 1 all interresponse times greater than 8 seconds produced shock. In Experiment 2 all interresponse times greater than 8 seconds but less than 12 seconds produced shock. In each experiment shock intensity was initially 0.3 milliamperes (mA) and then was varied through an ascending sequence ranging from 0.1 mA to 0.4 mA, in 0.1-mA increments. Experiment 1 produced response-rate increases at low intensities (0.1 and 0.2 mA) but eliminated responding at the remaining intensities. Experiment 2 produced response-rate increases only with 0.1-mA shock, although responding was maintained at all shock parameters investigated. Analysis of the interresponse times per opportunity showed differential suppression of the targeted responses in all cases except the high-intensity shock phases of Experiment 1. The current data support and extend previous studies of selective interresponse-time-dependent shock schedules but suggest that response-rate increases are not a necessary outcome of this type of procedure. The view that variable-interval schedules of shock presentation selectively target long interresponse times was also supported.     

Effects of response-shock interval and shock intensity on free-operant avoidance responding in the pigeon

Two experiments investigated free-operant avoidance responding with pigeons using a treadle-pressing response. In Experiment I, pigeons were initially trained on a free-operant avoidance schedule with a response-shock interval of 32 sec and a shock-shock interval of 10 sec, and were subsequently exposed to 10 values of the response-shock parameter ranging from 2.5 to 150 sec. The functions relating response rate to response-shock interval were similar to the ones reported by Sidman in his 1953 studies employing rats, and were independent of the order of presentation of the response-shock values. Shock rates decreased as response-shock duration increased. In Experiment II, a free-operant avoidance schedule with a response-shock interval of 20 sec and a shock-shock interval of 5 sec was used, and shock intensities were varied over five values ranging from 2 to 32 mA. Response rates increased markedly as shock intensity increased from 2 to 8 mA, but rates changed little with further increases in shock intensity. Shock rates decreased as intensity increased from 2 to 8 mA, and showed little change as intensity increased from 8 to 32 mA.