The concurrent reinforcement of two interresponse times: the relative frequency of an interresponse time equals its relative harmonic length

The relative lengths of two concurrently reinforced interresponse times were varied in an experiment in which three pigeons obtained food by pecking on a single key. Visual discriminative stimuli accompanied the two time intervals in which reinforcements were scheduled according to a one-minute variable-interval. The steady-state relative frequency of an interresponse time approximately equalled the complement of its relative length, that is, its relative harmonic length. Thus, lengths of interresponse times and delays of reinforcement have the same effect on the relative frequencies of interresponse times and choices in one-key and two-key concurrent variable-interval schedules, respectively. A second experiment generalized further the functional equivalence between the effects of these one-key and two-key concurrent schedules by revealing that the usual matching-to-relative-immediacy in two-key concurrent schedules is undisturbed if reinforcement depends upon the occurrence of a response at the end of the delay interval, as it does in the one-key schedules. The results of both experiments are consistent with a quantitative theory of concurrent operant behavior.     

The reinforcement of least-frequent interresponse times

A new schedule of reinforcement was used to maintain key-pecking by pigeons. The schedule reinforced only pecks terminating interresponse times which occurred least often relative to the exponential distribution of interresponse times to be expected from an ideal random generator. Two schedule parameters were varied: (1) the rate constant of the controlling exponential distribution and (2) the probability that a response would be reinforced, given that it met the interresponse-time contingency. Response rate changed quickly and markedly with changes in the rate constant; it changed only slightly with a fourfold change in the reinforcement probability. The schedule produced stable rates and high intra- and inter-subject reliability, yet interresponse time distributions were approximately exponential. Such local interresponse time variability in the context of good overall control suggests that the schedule may be used to generate stable, predictable, yet sensitive baseline rates. Implications for the measurement of rate are discussed.     

The reinforcement of short interresponse times

Five contingencies were superimposed successively on a variable-interval schedule of reinforcement. In each of the resulting conditions, a different short, interresponse time was reinforced and an interresponse-time distribution was obtained from each of three pigeons. The lower bound of the reinforced interresponse times ranged from 0.3 to 2.4 sec. The resulting distributions were combined, according to a rationale based upon concurrent operants, induction, and a property of variable-interval schedules, to describe the interresponse-time distributions from a variable-interval schedule.     

The reinforcement of four interresponse times in a two-alternative situation

Pigeons pecked for food in a two-key procedure. A concurrent variable-interval variable-interval schedule of reinforcement for two classes of interresponse times was arranged on each key. A visual stimulus set the occasion for potential reinforcement of the four operant classes: shorter and longer interresponse times on left and right keys. In Exp. I, the relative frequency of respones on a key equalled the relative frequency of reinforcement on that key. In Exp. II, the relative frequency of an interresponse time equalled the relative reciprocal of its length. In Exp. III, the relative frequency of an interresponse time was a monotonically increasing function of its relative frequency of reinforcement. These functions relating the relative frequency of an interresponse time to its relative length and to its relative frequency of reinforcement were the same as if there had been no second key. Also, the distribution of responses between keys was independent of the relative frequency of an interresponse time on either key. Experiment IV replicated Exp. I except that choices between keys were controlled by a stimulus that signalled the availability of reinforcement on the right key. A comparison of Exp. I and IV suggested that the relative frequency of an interresponse time on one key generally was independent of behavior on the other key, but that the number of responses per minute on a key did depend on behavior on the other key.     

Differential reinforcement of interresponse times with controlled probability of reinforcement per response

Pigeons were presented food after interresponse times (IRTs) longer or shorter than a fixed percentage of their most recent IRTs. This procedure controlled probability of reinforcement per response while still allowing different classes of IRTs to be reinforced differentially. Support was found for IRT-reinforcement theory in that response rates were determined by the degree and direction of differential reinforcement of IRTs, but were relatively independent of probability of reinforcement per response and of the length of the control system's IRT memory. Stimulus control of these differential response rates was also demonstrated.     

The distribution of interresponse times in the pigeon during variable-interval reinforcement

Three pigeons' pecks were reinforced on 1- and 2-min variable-interval schedules, and frequency distributions of their interresponse times (IRTs) were recorded. The conditional probability that a response would fall into any IRT category was estimated by the interresponse-times-per-opportunity transformation (IRTs/op). The resulting functions were notable chiefly for the relatively low probability of IRTs in the 0.2- to 0.3-sec range; in other respects they varied within and between subjects. The overall level of the curves generally rose over the course of 32 experimental hours, but their shapes changed unsystematically. The shape of the IRT distribution was much the same for VI 1-min and VI 2-min. The variability of these distributions supports the notion that the VI schedule only loosely controls response rate, permitting wide latitude to adventitious effects. There was no systematic evidence that curves changed over sessions to conform to the distribution of reinforcements by IRT.     

Magnitude and frequency of reinforcement and frequencies of interresponse times

The relative magnitude and relative frequency of reinforcement for two concurrent interresponse times (1.5 to 2.5 sec and 3.5 to 4.5 sec) were simultaneously varied in an experiment in which pigeons obtained grain by pecking on a single key. Visual discriminative stimuli accompanied the two time intervals in which reinforcements were arranged by a one-minute variable-interval schedule. The resulting interresponse times of each of three pigeons fell into two groups; "short" (1.0 to 2.5 sec) and "long" (3.0 to 4.5 sec). Steady-state relative frequencies of these interresponse times were orderly functions of both reinforcement variables. The combined effects of both independent variables were well summarized by a linear function of one variable, relative access to food. Unlike corresponding two-key concurrent variable-interval schedules, the present schedule did not produce an equality between the relative frequency of an operant and either the relative magnitude or the relative frequency of reinforcement of that operant. A tentative account is provided for this difference between one-key and two-key functions.     

Frequency distribution of interresponse times during VI and VR reinforcement

A detailed analysis was made of the interresponse times (IRTs) of two rats under both a VI 40-sec and a VR 15-sec schedule. Except for the latency of the first response after a reinforcement, the mean IRTs of all further responses differed little. Similarly, the frequency distributions of the successive IRTs did not vary greatly, but were of no simple form. Sequential dependencies between successive IRTs were small, never accounting for more than 1% of the variance.     

The maximization of overall reinforcement rate on concurrent chains

We model behavioral allocation on concurrent chains in which the initial links are independent variable-interval schedules. We also quantify the relationship between behavior during the initial links and the probability of entering a terminal link. The behavior that maximizes overall reinforcement rate is then considered and compared with published experimental data. Although all the trends in the data are predicted by rate maximization, there are considerable deviations from the predictions of rate maximization when reward magnitudes are unequal. We argue from our results that optimal allocation on concurrent chains, and prey choice as used in the theory of optimal diets, are distinct concepts. We show that the maximization of overall rate can lead to apparent violations of stochastic transitivity.     

Intermittent reinforcement of an interresponse time

Rats were exposed to schedules in which reinforcement was contingent upon the emission of a 1.0- to 2.0-sec interresponse time. The rate of emission and the temporal distribution of this interresponse time was recorded. Several different contingencies between the emission of the interresponse time and reinforcement were examined. Both the rate of emission and the temporal distribution of the 1.0- to 2.0-sec interresponse time varied as a function of the schedule on which it was reinforced. This finding, which suggests that an interresponse time behaves as other operants, has implications for the analysis of conventional reinforcement schedules in terms of the differential reinforcement of interresponse times.     

Selective punishment of interresponse times

Lever pressing by two squirrel monkeys was maintained under a variable-interval 60-second schedule of food presentation. When response-dependent electric shock was made contingent on comparatively long interresponse times, response rate increased, and further increases were obtained when the minimum interresponse-time requirement was decreased. When an equal proportion of responses produced shock without regard to interresponse time, rates decreased. Thus, shock contingent on long interresponse times selectively decreased the relative frequency of those interresponse times, and increased the relative frequency of shorter interresponse times, whereas shock delivered independent of interresponse times decreased the relative frequency of shorter interresponse times while increasing the frequency of longer ones. The results provide preliminary evidence that interresponse times may be differentiated by punishment, further supporting the notion that interresponse times may be considered functional units of behavior.     

Concurrent schedules of interresponse time reinforcement: probability of reinforcement and the lower bounds of the reinforced interresponse time intervals

Data were obtained with rats on the effects of interresponse time contingent reinforcement of the lever press response using schedules in which interresponse times falling within either of two temporal intervals could be reinforced. Some of the findings were (a) the mode of the interresponse time distribution generally occurred near the first lower bound when the maximum reinforcement rate for the two lower bounds was equal; this also frequently occurred even when the reinforcement rate was less for the first lower bound; (b) as is the case with schedules using a single interval of reinforced interresponse times the values of the lower bounds partially determined the location and spread of the distributions; but the particular pair of values used did not seem to influence the effects of the probabilities of reinforcement; (c) although the modal interresponse time was usually at the lower bound of one of the two intervals of reinforced interresponse times, no simple relation existed between either the probability or rate of reinforcement of interresponse times in these two intervals and the location of this mode.     

Shock-duration reduction on the basis of interresponse times: the effect of the interresponse time limit

Rats were trained on a free-operant procedure in which the duration of randomly occurring shocks depended on the interresponse times of lever presses. Shocks of 1.6-mA intensity were delivered at random intervals with an average density of 10 shocks per min. Each shock that was delivered lasted 0.3 s as long as the interresponse times were within a preset limit. Whenever the interresponse time exceeded the limit, the shocks that were delivered lasted 1 s until the occurrence of a response that met the limit. The limit was reduced in 3-s steps from 15 s to either 6 s or 3 s, at which point 3 of the animals were exposed to an ascending series. The avoidance of long-duration shocks was highly efficient at the 15-s and 12-s limits, and it decreased at the 9-s limit. With the exception of one animal, performance was substantially worse at the 6-s limit and it deteriorated for all the animals that were exposed to the 3-s limit. The data suggest that shock-duration reduction is quite effective as negative reinforcement for avoidance but is perhaps less effective than shock-intensity reduction.     

Selective punishment of interresponse times: The roles of shock intensity and scheduling

Two experiments investigated the roles of shock intensity and scheduling in selective punishment of interresponse times. In each experiment the punishment contingencies were imposed on a background of rats' responding maintained by a variable-interval schedule of food presentation. In Experiment 1 all interresponse times greater than 8 seconds produced shock. In Experiment 2 all interresponse times greater than 8 seconds but less than 12 seconds produced shock. In each experiment shock intensity was initially 0.3 milliamperes (mA) and then was varied through an ascending sequence ranging from 0.1 mA to 0.4 mA, in 0.1-mA increments. Experiment 1 produced response-rate increases at low intensities (0.1 and 0.2 mA) but eliminated responding at the remaining intensities. Experiment 2 produced response-rate increases only with 0.1-mA shock, although responding was maintained at all shock parameters investigated. Analysis of the interresponse times per opportunity showed differential suppression of the targeted responses in all cases except the high-intensity shock phases of Experiment 1. The current data support and extend previous studies of selective interresponse-time-dependent shock schedules but suggest that response-rate increases are not a necessary outcome of this type of procedure. The view that variable-interval schedules of shock presentation selectively target long interresponse times was also supported.     

Duration and rate of reinforcement as determinants of concurrent responding

The duration and frequency of food presentation were varied in concurrent variable-interval variable-interval schedules of reinforcement. In the first experiment, in which pigeons were exposed to a succession of eight different schedules, neither relative duration nor relative frequency of reinforcement had as great an effect on response distribution as they have when they are manipulated separately. These results supported those previously reported by Todorov (1973) and Schneider (1973). In a second experiment, each of seven pigeons was exposed to only one concurrent schedule in which the frequency and/or duration of reinforcement differed on the two keys. Under these conditions, each pigeon's relative rate of response closely matched the relative total access to food that each schedule provided. This result suggests that previous failures to obtain matching may be due to factors such as an insufficient length of exposure to each schedule or to the pigeons' repeated exposure to different concurrent schedules.     

Some effects of relative reinforcement rate and changeover delay in response-independent concurrent schedules of reinforcement

Reinforcements were arranged independently of the pigeon's behavior by concurrent variable-interval schedules. The reinforcements arranged by one of the schedules occurred when the chamber was illuminated with amber light, and the reinforcements arranged by the other schedule occurred when the chamber was illuminated with blue light. Both schedules functioned concurrently, but reinforcers were delivered by each only in the presence of the appropriate stimulus condition. A response on a white key, the only key in the chamber, alternated the stimulus condition and the effective schedule. The results of this procedure were similar to those obtained with concurrent response-dependent variable-interval schedules of reinforcement. The proportion of the total session time spent in the presence of a schedule component approximated the proportion of the total number of reinforcements in the component. Changeover rate was a decreasing function of the changeover delay and of the difference between the relative rates of reinforcement for each pair of concurrent schedules.     

Relative and absolute reinforcement frequency as determinants of choice in concurrent variable interval schedules

Rats were trained under concurrent schedules consisting of two equal variable interval component shedules providing sucrose solutions of different concentrations (0.6 M, 0.2 M; 50 μl in each case) as the reinforcers. The mean interreinforcement interval specified by the schedules was varied from 10 to 640 sec. Absolute response rate in each component was an increasing hyperbolic function of reinforcement frequency. Relative response rate and relative time allocation revealed a consistent preference for the more concentrated solution; neither measure of preference was systematically related to reinforcement frequency. The results are consistent with Baum and Rachlin's (1969) extension of the matching law, and with a derivation of the matching law from the hyperbolic relation between absolute response rate and reinforcement frequency in variable interval schedules (Herrnstein, 1970).     

Independence of response force and reinforcement rate on concurrent variable-interval schedule performance

Five pigeons were trained over 43 experimental conditions on a variety of concurrent variable-interval schedules on which the forces required on the response keys were varied. The results were well described by the generalized matching law with log reinforcement ratios and log force ratios exerting independent (noninteractive) effects on preference. A further analysis using the Akaike criterion, an information-theoretic measure of the efficiency of a model, showed that overall reinforcement rate and overall force requirement did not affect preference. Unlike reinforcement rate changes, force requirement increases did not change the response rate on the alternate key, and an extension of Herrnstein's absolute response rate function for force variation on a single variable-interval schedule is suggested.     

A mathematical model of learning under schedules of interresponse time reinforcement

A model is proposed for free responding under schedules of interresponse time reinforcement. Each response generates a pattern of stimuli that changes in an orderly way over time. At any time the changing pattern may be sampled, and the state of conditioning of the sampled pattern determines whether or not a response actually occurs. The prediction of this model for the asymptotic distribution of interresponse times is derived. The predictions are tested against data from ratio, interval, and DRL schedules of reinforcement. The fit of the model is sufficiently good to make it worth-while continuing investigation of the model.