Stimulus control of differential-reinforcement-of-low-rate responding

Five pigeons were given single-stimulus training on an 8-sec differential-reinforcement-of-low-rate schedule followed by steady-state generalization training using 12 wavelength stimuli. Three birds had a high percentage of reinforced responses on the training schedule and flat generalization gradients of total responses. The birds with fewer reinforced responses had much steeper generalization gradients. Generalization gradients plotted as a function of both stimulus wavelength and interresponse time showed that for most birds, stimulus control was restricted to responses with long interresponse times. Responses with very short interresponse times were not under stimulus control and there was some evidence of inhibitory control of short interresponse times. Interresponse-times-per-opportunity functions, plotted as a function of stimulus wavelength, showed that stimulus wavelength controlled the temporal distribution of responses, rather than the overall rate of response. The data indicate that the differential-reinforcement-of-low-rate schedule generates several response categories that are controlled in different ways by wavelength and time-correlated stimuli, and that averaging responses regardless of interresponse-time length obscures this control.     

An interresponse time analysis of variable-ratio punishment

An interresponse time analysis was used to study the effects of variable-ratio punishment schedules on the temporal pattern of reinforced responding. Twelve pigeons responded on a baseline variable-interval schedule of food reinforcement. A variable-ratio ten schedule of electric shock punishment was then introduced. The shock intensity was systematically increased to the highest intensity at which responding could be maintained. At this intensity, the mean variable-ratio value was increased and then decreased. Variable-ratio punishment resulted in an increased relative frequency of very short unreinforced interresponse times (response bursting). Increased response bursting accounted for instances of response rate facilitation. In addition, shock was followed by interresponse times of decreasing mean length over the first several responses after shock.     

Collateral responding during differential reinforcement of low rates

Two pigeons were trained to peck either of two response keys for food, under two different variable-interval schedules. When responding stabilized, the schedule on the left key (reinforcement-key) was changed to a differential-reinforcement-of-low-rates schedule, and responses on the right key (extinction-key) were no longer reinforced. The mean interresponse time of responses on the reinforcement-key approximated the temporal requirement of the reinforcement schedule on that key. Collateral responding on the extinction-key was maintained by one of the birds. A “run” of these collateral responses was defined as a sequence of responses on the extinction-key occurring between two responses on the reinforcement-key. For this one bird, collateral behavior, measured by mean time per run and mean number of responses per run, was an increasing function of the temporal requirements of the reinforcement schedule on the reinforcement key, and it was strongly positively correlated with the mean interresponse time of responses on the reinforcement-key. However, from an analysis of the results, the collateral behavior did not appear to have mediated the temporal spacing of responses on the reinforcement-key.     

A test of the effectiveness of the differential-reinforcement-of-low-rate schedule

Pigeons and rats were used in a yoked-control design that equated the reinforcement distributions of differential-reinforcement-of-low-rate and variable-interval schedules. Both a between-subjects design and a within-subjects design found response rate higher for the variable-interval schedule than for the differential-reinforcement-of-low-rate schedule, thus demonstrating the effectiveness of the differential-reinforcement-of-low-rate contingency. The interresponse-time distributions were unimodal for all subjects under the variable-interval schedule and bimodal for pigeons under the differential-reinforcement-of-low-rate schedule. The interresponse-time distributions for rats under the differential-reinforcement-of-low-rate schedule were also bimodal in three of four cases but the height of the modes at the shorter interresponse times were small in both absolute value and in relation to the height of the modes at the shorter interresponse times of the pigeons' distributions.     

Bouts of responding: the relation between bout rate and the rate of variable-interval reinforcement

By nose poking a lighted key, rats obtained food pellets on either a variable-interval schedule of reinforcement or a schedule that required an average of four additional responses after the end of tile variable-interval component (a tandem variable-interval variable-ratio 4 schedule). With both schedule types, the mean variable interval was varied between blocks of sessions from 16 min to 0.25 min. Total rate of key poking increased similarly as a function of the reinforcer rate for the two schedule types, but response rate was higher with than without the four-response requirement. Analysis of log survivor plots of interresponse times showed that key poking occurred in bouts. The rate of initiating bouts increased as a function of reinforcer rate but was either unaffected or was decreased by adding the four-response requirement. Within-bout response rate was insensitive to reinforcer rate and only inconsistently affected by the four-response requirement. For both kinds of schedule, the ratio of bout time to between-bout pause time was approximately a power function of reinforcer rate, with exponents above and below 1.0.     

Effects of d-amphetamine and chlordiazepoxide on spaced responding in pigeons

The effects of d-amphetamine and chlordiazepoxide were studied in pigeons on performance (1) under a schedule that reinforced responses on a key (food key) if they were more than 20 sec apart, (2) under the same schedule when responses also were required on a collateral key during the interresponse time on the food key, and (3) under the same schedule when responses were required on a collateral key during the interresponse time on the food key and collateral-key responses could produce a stimulus correlated with the availability of food. Under all three spaced-responding schedules, d-amphetamine and chlordiazepoxide at low dose levels slightly increased the frequency of short interresponse times on the food key for about half the birds, and either did not affect the interresponse time patterns of the other birds, or lengthened the durations slightly. At higher dose levels, d-amphetamine and chlordiazepoxide increased the frequency of long interresponse times or abolished responding in all birds. Changes in the pattern of interresponse times on the food key did not seem to depend on changes in the rate or pattern of collateral-key responses.     

The concurrent reinforcement of two interresponse times: absolute rate of reinforcement

Three pigeons obtained food on a one-key schedule of reinforcement for two concurrent, discriminated interresponse times. The overall rate of reinforcement was determined by a family of variable-interval schedules and by a continuous reinforcement schedule. The average frequency of reinforcement varied from 1.1 to 300 reinforcements per hour; the relative frequency of reinforcement for each of the two interresponse times was 0.5 throughout the experiment. The number of responses per minute increased sharply as the number of reinforcements per hour increased from 1 to 20. Beyond 30 reinforcements per hour, the curve was approximately flat, although it sometimes decreased slightly at the highest reinforcement rates. The relative frequency of the shorter interresponse time also increased sharply as the number of reinforcements per hour increased from 1 to 20. The asymptote of the relative frequency function approximately equalled the relative reciprocal of the length of the shorter interresponse time for reinforcement rates greater than 30 or 40 reinforcements per hour. This approximation was obscured by the response-rate function.     

Post-reinforcement pauses and response rate of monkeys on a two-hand fixed-ratio schedule

Fixed-ratio behavior of monkeys was analyzed separately for two hands. While one hand responded on the fixed-ratio schedule the other performed a holding response and the function of the hands changed in alternate ratio runs. After performance was stable on the fixed ratio (70 responses, two monkeys; 100 responses, two monkeys, 120 responses, two monkeys) 90 sessions of further training equalized post-reinforcement pauses and the mean interresponse time of the two hands. Hand preference in reaching for food remained unchanged. Then, the fixed-ratio requirement was changed (a) in small sequential steps, (b) in two large steps, and, (c) within sessions alternating two runs at a high ratio with two runs at a low ratio. The mean duration of post-reinforcement pauses was correlated with a fixed ratio maintained throughout a session but single pauses were neither controlled by the immediately preceding nor by the following ratio run when a cue to its length was available. The mean interresponse time was insensitive to changes in fixed ratio. The fixed-ratio performance was generally similar to that of pigeons and rats.     

Interresponse time duration in fixed-interval schedules of reinforcement: control by ordinal position and time since reinforcement

The times between each of the first thirteen responses after reinforcement (the first twelve interresponse times) were determined for two pigeons whose pecking was reinforced on fixed-interval schedules of food reinforcement ranging from 0.5 min to 5 min. These interresponse times were classified with respect to their ordinal position in the sequence of responses and with respect to the time since the preceding reinforcement at which the initiating response occurred. The median interresponse time durations were essentially constant after the sixth response after reinforcement regardless of the time at which the interresponse time was initiated. The durations of the first few interresponse times after reinforcement decreased as the number of preceding responses increased and as the time since the preceding reinforcement increased.     

Effects of timeout on spaced responding in pigeons

Three pigeons were trained under a differential-reinforcement-of-low-rate schedule of 20 sec, and then exposed to a schedule under which responses terminating interresponse times less than 20 sec produced timeout and responses terminating interresponse times greater than 20 sec produced reinforcement. Response-produced timeouts selectively decreased the probability of short interresponse times and thereby produced a higher frequency of reinforcement. The suppressive effect of timeout was independent of timeout duration, with timeouts of 5, 10, or 20 sec. Similar effects were found when the minimum interresponse time that could be terminated by response-produced reinforcement was increased to 30 sec. The suppressive effects of timeout on responding maintained by these schedules were similar to previous reports in which responding was punished with electric shock.     

Conjunctive schedules of reinforcement: III. A fixed-interval adjusting fixed-ratio schedule

Key pecking of three pigeons was studied under a conjunctive schedule that specified both a fixed-interval and an adjusting fixed-ratio requirement. The fixed-interval schedule was 6 min for one pigeon and 3 min for the other two. The size of the ratio requirement was determined within each cycle of the fixed interval by the duration of the pause before responding began. The fixed-ratio value was at maximum at the start of each fixed interval and decreased linearly until the first response occurred (adjusting fixed-ratio schedule). A peck produced food when the number of responses remaining on the fixed-ratio schedule was completed and when the fixed interval had elapsed. If no response occurred during the interval, the fixed-ratio requirement decreased to one and a single response after the interval elapsed produced food. The initial value of the adjusting fixed-ratio schedule was studied over a range of 0 to 900. Increases in the adjusting fixed-ratio schedule to about 300 responses increased both pause duration and running response rate and also modified the pattern of responding from that obtained under the fixed-interval schedule. Higher values of the adjusting fixed ratio generally decreased pause duration and running response rate and also disrupted responding. Interreinforcement time under the conjunctive schedule was increased substantially when the adjusting fixed-ratio size exceeded 300 responses.     

Fixed-ratio reinforcement of spaced responding

Responses by rats were reinforced with food under a second-order schedule involving fixed-ratio reinforcement of temporally spaced responses. Requirements of 20, 8, and 3 responses were examined. The typical characteristic of spaced responding was maintained under the ratio schedules: interresponse time distributions were similar to those typically seen, and were not noticeably affected by the ratio value. Comparison of total response rate, correct response rate, and accuracy showed correct response rate to be the most consistently affected by changes in the ratio value. Substantial evidence of schedule control was seen only for correct responses. Incorrect response records were erratic, but rates generally declined as reinforcement was approached. Correct response records were characterized by increasing rate as reinforcement was approached. It was suggested that the pattern of fixed-ratio performance revealed may be affected by the behavioral unit examined.     

The effect of informative feedback on temporal tracking in the pigeon

Pigeons emitted interresponse times that were reinforced if they fell between an upper and a lower bound (t相似文献   

Choice between response rates

Three pigeons were required to peck a single key at a higher and a lower rate, corresponding to two classes of shorter and longer concurrently reinforced interresponse times. Food reinforcers arranged by a single variable-interval schedule were randomly allocated to the two reinforced interresponse times. The absolute durations of reinforced interresponse times were varied while the total reinforcements per hour was held constant and the relative duration, i.e., the relative reciprocal, of the shorter reinforcer class was held constant at 0.70. Preference for the higher rate of responding, as measured by the relative frequency of responses terminating interresponse times in the shorter reinforced class, depended on the absolute reinforced response rates. Preference for the higher reinforced rate increased from a level of near-indifference (0.50) at high reinforced response rates, through the matching level (0.70) at intermediate reinforced response rates, to a virtually exclusive preference (>0.90) at low reinforced response rates. These results resemble corresponding preference functions obtained with two-key concurrent-chains schedules and thereby provide another sense in which it may be said that interresponse-time distributions from interval schedules estimate preference functions for the component response rates corresponding to different classes of reinforced interresponse times.     

Magnitude and frequency of reinforcement and frequencies of interresponse times

The relative magnitude and relative frequency of reinforcement for two concurrent interresponse times (1.5 to 2.5 sec and 3.5 to 4.5 sec) were simultaneously varied in an experiment in which pigeons obtained grain by pecking on a single key. Visual discriminative stimuli accompanied the two time intervals in which reinforcements were arranged by a one-minute variable-interval schedule. The resulting interresponse times of each of three pigeons fell into two groups; "short" (1.0 to 2.5 sec) and "long" (3.0 to 4.5 sec). Steady-state relative frequencies of these interresponse times were orderly functions of both reinforcement variables. The combined effects of both independent variables were well summarized by a linear function of one variable, relative access to food. Unlike corresponding two-key concurrent variable-interval schedules, the present schedule did not produce an equality between the relative frequency of an operant and either the relative magnitude or the relative frequency of reinforcement of that operant. A tentative account is provided for this difference between one-key and two-key functions.     

Some properties of spaced responding in pigeons

Pigeons exposed to a schedule which reinforces interresponse times (IRTs) longer than a given value (DRL schedule) eventually reach a stable pattern of responding which is shown to be a function both of the DRL value and of previous experience with other DRL values. On any given DRL schedule, the stable performance of most pigeons which have been previously exposed to a variety of such schedules, shows an IRT distribution with median equal to the DRL value. For DRL values longer than about 30 sec, however, the median IRT falls short of the DRL value; this failure of adjustment to longer values appears to be a species characteristic of pigeons. The function relating reinforcement rate to 1/DRL value is also shown to be approximately linear over the same range, with variable slope (less than 45°) and a downturn in the vicinity of DRL 30.     

Studies on responding under fixed-interval schedules of reinforcement: the effects on the pattern of responding of changes in requirements at reinforcement

In pigeons responding under a 180-sec fixed-interval schedule of reinforcement, the frequency distribution of the duration of the final interresponse time before the reinforcer was compared with the distribution of the preceding two interresponse times. The results confirmed qualitatively and quantitatively the expected preferential reinforcement of longer interreinforcement times under fixed-interval reinforcement. Requirements at reinforcement were then changed to eliminate the preferential reinforcement of longer interresponse times. Local patterns and mean rate of responding could change, without the characteristic fixed-interval pattern of increasing responding through the interval (scalloping) being much affected. It is concluded that this characteristic pattern of fixed-interval responding does not depend crucially on effects of the reinforcer at the moment of reinforcement, but rather to effects extending over much longer periods of time than just the last interresponse time.     

The reinforcement of least-frequent interresponse times

A new schedule of reinforcement was used to maintain key-pecking by pigeons. The schedule reinforced only pecks terminating interresponse times which occurred least often relative to the exponential distribution of interresponse times to be expected from an ideal random generator. Two schedule parameters were varied: (1) the rate constant of the controlling exponential distribution and (2) the probability that a response would be reinforced, given that it met the interresponse-time contingency. Response rate changed quickly and markedly with changes in the rate constant; it changed only slightly with a fourfold change in the reinforcement probability. The schedule produced stable rates and high intra- and inter-subject reliability, yet interresponse time distributions were approximately exponential. Such local interresponse time variability in the context of good overall control suggests that the schedule may be used to generate stable, predictable, yet sensitive baseline rates. Implications for the measurement of rate are discussed.     

A quantitative interresponse-time analysis of DRL performance differentiates similar effects of the antidepressant desipramine and the novel anxiolytic gepirone.

We describe an interresponse-time analysis of performance on a differential-reinforcement-of-low-rate 72-s schedule. This analysis compares the obtained interresponse-time distribution of individual rats to a corresponding random interresponse-time distribution. The random interresponse-time distribution is a negative exponential probability function; it predicts the relative distribution of interresponse times if the rat emitted the same number of responses randomly (i.e., with a constant probability) with respect to time. The analysis provides quantitative measures of peak location and dispersion of the interresponse times toward random performance. In Experiment 1, an unexpected outcome of this analysis was that the rats would have obtained more reinforcers had they responded at the same rate but randomly. Based on the interresponse-time analysis in Experiment 1, it was shown that rats trained on the differential-reinforcement-of-low-rate 72-s schedule could increase the number of reinforcers obtained in two ways: first, by a coherent shift of the interresponse-time distribution toward longer durations and, second, by dispersal of the interresponse times toward a random interresponse-time distribution. Experiment 2 applied the analysis described in Experiment 1 to the effects of desipramine and gepirone. Both drugs decreased response rate and increased reinforcement rate, but their effects on the distribution of interresponse times were different. The increase in reinforcement rate observed with desipramine was accompanied by a coherent shift of the reinforcement rate observed with gepirone was accompanied by dispersal of the interresponse-time distribution toward the random negative exponential prediction.     

Sequential dependencies in free-responding

Three pigeons pecked for food in an experiment in which reinforcements were arranged for responses terminating sequences of interresponse times. Each reinforced interresponse time belonged to a class extending either from 1.0 to 2.0 sec (class A) or from 3.0 to 4.5 sec (class B). Reinforcements were arranged by a single variable-interval schedule and a random device that assigned each reinforcement to one of four sequences of two successive interresponse times: AA, AB, BA, or BB. Throughout the experiment, half of the reinforcements were delivered for interresponse times in class A and half for those in class B. Over conditions, the interresponse time preceding a reinforced interresponse time always, half of the time, or never, belonged to class A. The duration of the interresponse time preceding a reinforced one had a pronounced effect on response patterning. It also had a pronounced effect on the overall response probability, which was highest, intermediate, and lowest, when the interresponse time preceding a reinforced interresponse time always, half of the time, or never, belonged to class A, respectively. In no case were successive interresponse times independent, so that overall response probability was not representative of momentary response probabilities.