The reinforcement of least-frequent interresponse times

A new schedule of reinforcement was used to maintain key-pecking by pigeons. The schedule reinforced only pecks terminating interresponse times which occurred least often relative to the exponential distribution of interresponse times to be expected from an ideal random generator. Two schedule parameters were varied: (1) the rate constant of the controlling exponential distribution and (2) the probability that a response would be reinforced, given that it met the interresponse-time contingency. Response rate changed quickly and markedly with changes in the rate constant; it changed only slightly with a fourfold change in the reinforcement probability. The schedule produced stable rates and high intra- and inter-subject reliability, yet interresponse time distributions were approximately exponential. Such local interresponse time variability in the context of good overall control suggests that the schedule may be used to generate stable, predictable, yet sensitive baseline rates. Implications for the measurement of rate are discussed.     

The concurrent reinforcement of two interresponse times: absolute rate of reinforcement

Three pigeons obtained food on a one-key schedule of reinforcement for two concurrent, discriminated interresponse times. The overall rate of reinforcement was determined by a family of variable-interval schedules and by a continuous reinforcement schedule. The average frequency of reinforcement varied from 1.1 to 300 reinforcements per hour; the relative frequency of reinforcement for each of the two interresponse times was 0.5 throughout the experiment. The number of responses per minute increased sharply as the number of reinforcements per hour increased from 1 to 20. Beyond 30 reinforcements per hour, the curve was approximately flat, although it sometimes decreased slightly at the highest reinforcement rates. The relative frequency of the shorter interresponse time also increased sharply as the number of reinforcements per hour increased from 1 to 20. The asymptote of the relative frequency function approximately equalled the relative reciprocal of the length of the shorter interresponse time for reinforcement rates greater than 30 or 40 reinforcements per hour. This approximation was obscured by the response-rate function.     

The reinforcement of four interresponse times in a two-alternative situation

Pigeons pecked for food in a two-key procedure. A concurrent variable-interval variable-interval schedule of reinforcement for two classes of interresponse times was arranged on each key. A visual stimulus set the occasion for potential reinforcement of the four operant classes: shorter and longer interresponse times on left and right keys. In Exp. I, the relative frequency of respones on a key equalled the relative frequency of reinforcement on that key. In Exp. II, the relative frequency of an interresponse time equalled the relative reciprocal of its length. In Exp. III, the relative frequency of an interresponse time was a monotonically increasing function of its relative frequency of reinforcement. These functions relating the relative frequency of an interresponse time to its relative length and to its relative frequency of reinforcement were the same as if there had been no second key. Also, the distribution of responses between keys was independent of the relative frequency of an interresponse time on either key. Experiment IV replicated Exp. I except that choices between keys were controlled by a stimulus that signalled the availability of reinforcement on the right key. A comparison of Exp. I and IV suggested that the relative frequency of an interresponse time on one key generally was independent of behavior on the other key, but that the number of responses per minute on a key did depend on behavior on the other key.     

The distribution of interresponse times in the pigeon during variable-interval reinforcement

Three pigeons' pecks were reinforced on 1- and 2-min variable-interval schedules, and frequency distributions of their interresponse times (IRTs) were recorded. The conditional probability that a response would fall into any IRT category was estimated by the interresponse-times-per-opportunity transformation (IRTs/op). The resulting functions were notable chiefly for the relatively low probability of IRTs in the 0.2- to 0.3-sec range; in other respects they varied within and between subjects. The overall level of the curves generally rose over the course of 32 experimental hours, but their shapes changed unsystematically. The shape of the IRT distribution was much the same for VI 1-min and VI 2-min. The variability of these distributions supports the notion that the VI schedule only loosely controls response rate, permitting wide latitude to adventitious effects. There was no systematic evidence that curves changed over sessions to conform to the distribution of reinforcements by IRT.     

Differential reinforcement of interresponse times with controlled probability of reinforcement per response

Pigeons were presented food after interresponse times (IRTs) longer or shorter than a fixed percentage of their most recent IRTs. This procedure controlled probability of reinforcement per response while still allowing different classes of IRTs to be reinforced differentially. Support was found for IRT-reinforcement theory in that response rates were determined by the degree and direction of differential reinforcement of IRTs, but were relatively independent of probability of reinforcement per response and of the length of the control system's IRT memory. Stimulus control of these differential response rates was also demonstrated.     

Magnitude and frequency of reinforcement and frequencies of interresponse times

The relative magnitude and relative frequency of reinforcement for two concurrent interresponse times (1.5 to 2.5 sec and 3.5 to 4.5 sec) were simultaneously varied in an experiment in which pigeons obtained grain by pecking on a single key. Visual discriminative stimuli accompanied the two time intervals in which reinforcements were arranged by a one-minute variable-interval schedule. The resulting interresponse times of each of three pigeons fell into two groups; "short" (1.0 to 2.5 sec) and "long" (3.0 to 4.5 sec). Steady-state relative frequencies of these interresponse times were orderly functions of both reinforcement variables. The combined effects of both independent variables were well summarized by a linear function of one variable, relative access to food. Unlike corresponding two-key concurrent variable-interval schedules, the present schedule did not produce an equality between the relative frequency of an operant and either the relative magnitude or the relative frequency of reinforcement of that operant. A tentative account is provided for this difference between one-key and two-key functions.     

Frequency distribution of interresponse times during VI and VR reinforcement

A detailed analysis was made of the interresponse times (IRTs) of two rats under both a VI 40-sec and a VR 15-sec schedule. Except for the latency of the first response after a reinforcement, the mean IRTs of all further responses differed little. Similarly, the frequency distributions of the successive IRTs did not vary greatly, but were of no simple form. Sequential dependencies between successive IRTs were small, never accounting for more than 1% of the variance.     

The concurrent reinforcement of two interresponse times: the relative frequency of an interresponse time equals its relative harmonic length

The relative lengths of two concurrently reinforced interresponse times were varied in an experiment in which three pigeons obtained food by pecking on a single key. Visual discriminative stimuli accompanied the two time intervals in which reinforcements were scheduled according to a one-minute variable-interval. The steady-state relative frequency of an interresponse time approximately equalled the complement of its relative length, that is, its relative harmonic length. Thus, lengths of interresponse times and delays of reinforcement have the same effect on the relative frequencies of interresponse times and choices in one-key and two-key concurrent variable-interval schedules, respectively. A second experiment generalized further the functional equivalence between the effects of these one-key and two-key concurrent schedules by revealing that the usual matching-to-relative-immediacy in two-key concurrent schedules is undisturbed if reinforcement depends upon the occurrence of a response at the end of the delay interval, as it does in the one-key schedules. The results of both experiments are consistent with a quantitative theory of concurrent operant behavior.     

Selective punishment of interresponse times

Lever pressing by two squirrel monkeys was maintained under a variable-interval 60-second schedule of food presentation. When response-dependent electric shock was made contingent on comparatively long interresponse times, response rate increased, and further increases were obtained when the minimum interresponse-time requirement was decreased. When an equal proportion of responses produced shock without regard to interresponse time, rates decreased. Thus, shock contingent on long interresponse times selectively decreased the relative frequency of those interresponse times, and increased the relative frequency of shorter interresponse times, whereas shock delivered independent of interresponse times decreased the relative frequency of shorter interresponse times while increasing the frequency of longer ones. The results provide preliminary evidence that interresponse times may be differentiated by punishment, further supporting the notion that interresponse times may be considered functional units of behavior.     

Intermittent reinforcement of an interresponse time

Rats were exposed to schedules in which reinforcement was contingent upon the emission of a 1.0- to 2.0-sec interresponse time. The rate of emission and the temporal distribution of this interresponse time was recorded. Several different contingencies between the emission of the interresponse time and reinforcement were examined. Both the rate of emission and the temporal distribution of the 1.0- to 2.0-sec interresponse time varied as a function of the schedule on which it was reinforced. This finding, which suggests that an interresponse time behaves as other operants, has implications for the analysis of conventional reinforcement schedules in terms of the differential reinforcement of interresponse times.     

Concurrent schedules of interresponse time reinforcement: probability of reinforcement and the lower bounds of the reinforced interresponse time intervals

Data were obtained with rats on the effects of interresponse time contingent reinforcement of the lever press response using schedules in which interresponse times falling within either of two temporal intervals could be reinforced. Some of the findings were (a) the mode of the interresponse time distribution generally occurred near the first lower bound when the maximum reinforcement rate for the two lower bounds was equal; this also frequently occurred even when the reinforcement rate was less for the first lower bound; (b) as is the case with schedules using a single interval of reinforced interresponse times the values of the lower bounds partially determined the location and spread of the distributions; but the particular pair of values used did not seem to influence the effects of the probabilities of reinforcement; (c) although the modal interresponse time was usually at the lower bound of one of the two intervals of reinforced interresponse times, no simple relation existed between either the probability or rate of reinforcement of interresponse times in these two intervals and the location of this mode.     

Briefly delayed reinforcement: An interresponse time analysis

Key-peck responding of pigeons was compared under VI or DRL schedules arranging immediate reinforcement and briefly (.5 sec) delayed reinforcement. Delays were either signaled by a blackout in the chamber, unsignaled, or unsignaled with an additional requirement that responding not occur during the .5 sec interval immediately preceding reinforcement (response delay). Relative to the immediate reinforcement condition, response rates increased during the unsignaled delay, decreased during the signaled delay, and were inconsistent during the response delay condition. An analysis of interresponse times (IRTs) under the different conditions revealed a substantial increase in the frequency of short (0 to .5 sec) IRTs during the unsignaled condition and generally during the response delay conditions compared to that during the immediate reinforcement baseline. Signaled delays decreased the frequency of short (0 to .5 sec) IRTs relative to the immediate reinforcement condition. The results suggest that brief unsignaled delays and, in many instances, response delays increase the frequency of short IRTs by eliminating constraints on responding.     

The effects of morphine on the production and discrimination of interresponse times

Recent experiments suggest that the effects of drugs of abuse on the discrimination of the passage of time may differ for experimenter-imposed and subject-produced events. The current experiment examined this suggestion by determining the effects of morphine on the discrimination of interresponse times (IRTs). Pigeons pecked a center key on a random-interval 20-s schedule of matching-to-sample trials. Once the interval had timed out, a choice trial randomly followed either a short (2- to 3-s) or long (6- to 9-s) IRT on the center key. Pecking the side key lit one color produced food after a short IRT, and pecking the side key lit the other color produced food after a long IRT. Two experimental phases differed in the functional role of the different key colors. Under control conditions, the IRT distributions had two modes, one at the lower bound of the short category and a smaller one at the lower bound of the long category. Pigeons accurately categorized the duration of the IRTs: One key color was pecked following short IRTs and the other key color was pecked following long IRTs. Morphine flattened the IRT distribution and reduced the accuracy of categorizing IRTs. Categorization of long IRTs was particularly disrupted. Morphine did not produce overestimation of time as assessed by the production or categorization of IRTs. These results are similar to those obtained previously for the effects of morphine on the discrimination of the duration of experimenter-imposed events.     

Shock-duration reduction on the basis of interresponse times: the effect of the interresponse time limit

Rats were trained on a free-operant procedure in which the duration of randomly occurring shocks depended on the interresponse times of lever presses. Shocks of 1.6-mA intensity were delivered at random intervals with an average density of 10 shocks per min. Each shock that was delivered lasted 0.3 s as long as the interresponse times were within a preset limit. Whenever the interresponse time exceeded the limit, the shocks that were delivered lasted 1 s until the occurrence of a response that met the limit. The limit was reduced in 3-s steps from 15 s to either 6 s or 3 s, at which point 3 of the animals were exposed to an ascending series. The avoidance of long-duration shocks was highly efficient at the 15-s and 12-s limits, and it decreased at the 9-s limit. With the exception of one animal, performance was substantially worse at the 6-s limit and it deteriorated for all the animals that were exposed to the 3-s limit. The data suggest that shock-duration reduction is quite effective as negative reinforcement for avoidance but is perhaps less effective than shock-intensity reduction.     

Successive interresponse times in fixed-ratio and second-order fixed-ratio performance

Three rats were trained on a schedule in which every sixth response produced a timeout of 5 sec minimum duration, and food was delivered at the onset of timeout. Successive interresponse times were measured under these conditions, and also when behavior was maintained by second-order fixed-ratio and fixed-interval schedules. Under the second-order schedules, each six-response fixed-ratio component was followed by a timeout, and occasionally food was delivered at the onset of a timeout. In the fixed-ratio schedule, the successive interresponse times showed a decrease followed by an increase before food delivery, but this systematic variation in interresponse times was not found when the performance was under second-order reinforcement. Under both second-order schedules the latencies of successive components, and the successive interresponse times within each component, showed a decrease as food delivery was approached.     

Discriminated interresponse times: role of autoshaped responses.

When discriminated interresponse-time (IRT) procedures have been used to assess preference relations among temporally extended operants, deviations from matching have been obtained. Using a yoked-control procedure, the present study found that key pecking in a discriminated IRT procedure has two sources of strength--that arising from the response-reinforcer contingency that is explicitly arranged, and that arising from a stimulus-reinforcer contingency that is a by-product of the explicitly arranged contingency. The key pecking of all lead birds, and that of 3 of the 4 birds exposed to a yoked autoshaping procedure, was controlled by the keylight that signaled the lead birds' criterion IRTs. Because stimulus control of key pecking by the keylight, whether autoshaped or discriminative, fosters deviations from matching, the discriminated IRT procedure does not provide an appropriate basis for conclusions about preference relations among IRTs.     

Selective punishment of interresponse times: The roles of shock intensity and scheduling

Two experiments investigated the roles of shock intensity and scheduling in selective punishment of interresponse times. In each experiment the punishment contingencies were imposed on a background of rats' responding maintained by a variable-interval schedule of food presentation. In Experiment 1 all interresponse times greater than 8 seconds produced shock. In Experiment 2 all interresponse times greater than 8 seconds but less than 12 seconds produced shock. In each experiment shock intensity was initially 0.3 milliamperes (mA) and then was varied through an ascending sequence ranging from 0.1 mA to 0.4 mA, in 0.1-mA increments. Experiment 1 produced response-rate increases at low intensities (0.1 and 0.2 mA) but eliminated responding at the remaining intensities. Experiment 2 produced response-rate increases only with 0.1-mA shock, although responding was maintained at all shock parameters investigated. Analysis of the interresponse times per opportunity showed differential suppression of the targeted responses in all cases except the high-intensity shock phases of Experiment 1. The current data support and extend previous studies of selective interresponse-time-dependent shock schedules but suggest that response-rate increases are not a necessary outcome of this type of procedure. The view that variable-interval schedules of shock presentation selectively target long interresponse times was also supported.     

A mathematical model of learning under schedules of interresponse time reinforcement

A model is proposed for free responding under schedules of interresponse time reinforcement. Each response generates a pattern of stimuli that changes in an orderly way over time. At any time the changing pattern may be sampled, and the state of conditioning of the sampled pattern determines whether or not a response actually occurs. The prediction of this model for the asymptotic distribution of interresponse times is derived. The predictions are tested against data from ratio, interval, and DRL schedules of reinforcement. The fit of the model is sufficiently good to make it worth-while continuing investigation of the model.     

Effects of class-interval size upon certain frequency distributions of interresponse times

Inter-response time distributions with class intervals of 0.05 and 0.04 sec revealed characteristics not observable with class intervals of 0.5 sec or greater. Bimodalities were clearly evident in the inter-response time distributions for two out of three pigeons. These modes seemed to correspond to two discrete response topographies.