Choice between response rates

Three pigeons were required to peck a single key at a higher and a lower rate, corresponding to two classes of shorter and longer concurrently reinforced interresponse times. Food reinforcers arranged by a single variable-interval schedule were randomly allocated to the two reinforced interresponse times. The absolute durations of reinforced interresponse times were varied while the total reinforcements per hour was held constant and the relative duration, i.e., the relative reciprocal, of the shorter reinforcer class was held constant at 0.70. Preference for the higher rate of responding, as measured by the relative frequency of responses terminating interresponse times in the shorter reinforced class, depended on the absolute reinforced response rates. Preference for the higher reinforced rate increased from a level of near-indifference (0.50) at high reinforced response rates, through the matching level (0.70) at intermediate reinforced response rates, to a virtually exclusive preference (>0.90) at low reinforced response rates. These results resemble corresponding preference functions obtained with two-key concurrent-chains schedules and thereby provide another sense in which it may be said that interresponse-time distributions from interval schedules estimate preference functions for the component response rates corresponding to different classes of reinforced interresponse times.     

Selective punishment of interresponse times

Lever pressing by two squirrel monkeys was maintained under a variable-interval 60-second schedule of food presentation. When response-dependent electric shock was made contingent on comparatively long interresponse times, response rate increased, and further increases were obtained when the minimum interresponse-time requirement was decreased. When an equal proportion of responses produced shock without regard to interresponse time, rates decreased. Thus, shock contingent on long interresponse times selectively decreased the relative frequency of those interresponse times, and increased the relative frequency of shorter interresponse times, whereas shock delivered independent of interresponse times decreased the relative frequency of shorter interresponse times while increasing the frequency of longer ones. The results provide preliminary evidence that interresponse times may be differentiated by punishment, further supporting the notion that interresponse times may be considered functional units of behavior.     

INTERRESPONSE TIME STRUCTURES IN VARIABLE-RATIO AND VARIABLE-INTERVAL SCHEDULES

The interresponse-time structures of pigeon key pecking were examined under variable-ratio, variable-interval, and variable-interval plus linear feedback schedules. Whereas the variable-ratio and variable-interval plus linear feedback schedules generally resulted in a distinct group of short interresponse times and a broad distribution of longer interresponse times, the variable-interval schedules generally showed a much more continuous distribution of interresponse times. The results were taken to indicate that a log survivor analysis or double exponential fit of interresponse times may not be universally applicable to the task of demonstrating that operant behavior can be dichotomized into bouts of engagement and periods of disengagement.     

Preference for starting and finishing behavior patterns

Pigeon's key pecking was reinforced with food in two experiments in which the correspondence between preference for starting one of two reinforced behavior patterns and the likelihood of finishing it subsequently was examined. Reinforcers were scheduled according to concurrent schedules for two classes of interresponse times, modified such that reinforcers followed a center-key peck terminating either a shorter interresponse time started by a left-key peck or a longer interresponse time started by a right-key peck. In Experiment 1, the times when reinforcers potentially were available were not discriminated, whereas in Experiment 2 they were. Absolute reinforced pattern durations were varied. The relative frequency of starting a particular pattern was highly correlated with relative frequency of that completed pattern in both experiments. Other relations between starting and finishing a pattern depended on whether reinforced interresponse times were discriminated. For instance, preference for starting a pattern sometimes correlated negatively with the likelihood of subsequently completing it. The present experiments are described as capturing part of the ordinary language meaning of "intention," according to which an organism's behavior at one moment sets the occasion for an observer to say that the organism "intends" in the future to engage in one behavior rather than another.     

Interresponse-time analysis of stimulus control in multiple schedules

Interresponse-time distributions were recorded in two components of multiple variable-interval schedules that were varied over several conditions. Values of the exponent for power functions relating ratios of interresponse times emitted per opportunity to ratios of reinforcers obtained in the two components varied with interresponse-time class interval. The exponent (sensitivity to reinforcement) afforded a measure of stimulus control exerted by the discriminative stimuli. Exponents were near zero for short interresponse times, consistent with previous conclusions that responses following short interresponse times are controlled by response-produced or proprioceptive stimuli. Values of exponents increased with longer interresponse times, indicating strong control by exteroceptive stimuli over responses following interresponse times of approximately one second or longer.     

The concurrent reinforcement of two interresponse times: absolute rate of reinforcement

Three pigeons obtained food on a one-key schedule of reinforcement for two concurrent, discriminated interresponse times. The overall rate of reinforcement was determined by a family of variable-interval schedules and by a continuous reinforcement schedule. The average frequency of reinforcement varied from 1.1 to 300 reinforcements per hour; the relative frequency of reinforcement for each of the two interresponse times was 0.5 throughout the experiment. The number of responses per minute increased sharply as the number of reinforcements per hour increased from 1 to 20. Beyond 30 reinforcements per hour, the curve was approximately flat, although it sometimes decreased slightly at the highest reinforcement rates. The relative frequency of the shorter interresponse time also increased sharply as the number of reinforcements per hour increased from 1 to 20. The asymptote of the relative frequency function approximately equalled the relative reciprocal of the length of the shorter interresponse time for reinforcement rates greater than 30 or 40 reinforcements per hour. This approximation was obscured by the response-rate function.     

Response-rate differences in variable-interval and variable-ratio schedules: An old problem revisited

In Experiment 1, a variable-ratio 10 schedule became, successively, a variable-interval schedule with only the minimum interreinforcement intervals yoked to the variable ratio, or a variable-interval schedule with both interreinforcement intervals and reinforced interresponse times yoked to the variable ratio. Response rates in the variable-interval schedule with both interreinforcement interval and reinforced interresponse time yoking fell between the higher rates maintained by the variable-ratio schedule and the lower rates maintained by the variable-interval schedule with only interreinforcement interval yoking. In Experiment 2, a tandem variable-interval 15-s variable-ratio 5 schedule became a yoked tandem variable-ratio 5 variable-interval x-s schedule, and a tandem variable-interval 30-s variable-ratio 10 schedule became a yoked tandem variable-ratio 10 variable-interval x-s schedule. In the yoked tandem schedules, the minimum interreinforcement intervals in the variable-interval components were those that equated overall interreinforcement times in the two phases. Response rates did not decline in the yoked schedules even when the reinforced interresponse times became longer. Experiment 1 suggests that both reinforced interresponse times and response rate–reinforcement rate correlations determine response-rate differences in variable-ratio 10 and yoked variable-interval schedules in rats. Experiment 2 suggests a minimal role for the reinforced interresponse time in determining response rates on tandem variable-interval 30-s variable-ratio 10 and yoked tandem variable-ratio 10 variable-interval x-s schedules in rats.     

The reinforcement of four interresponse times in a two-alternative situation

Pigeons pecked for food in a two-key procedure. A concurrent variable-interval variable-interval schedule of reinforcement for two classes of interresponse times was arranged on each key. A visual stimulus set the occasion for potential reinforcement of the four operant classes: shorter and longer interresponse times on left and right keys. In Exp. I, the relative frequency of respones on a key equalled the relative frequency of reinforcement on that key. In Exp. II, the relative frequency of an interresponse time equalled the relative reciprocal of its length. In Exp. III, the relative frequency of an interresponse time was a monotonically increasing function of its relative frequency of reinforcement. These functions relating the relative frequency of an interresponse time to its relative length and to its relative frequency of reinforcement were the same as if there had been no second key. Also, the distribution of responses between keys was independent of the relative frequency of an interresponse time on either key. Experiment IV replicated Exp. I except that choices between keys were controlled by a stimulus that signalled the availability of reinforcement on the right key. A comparison of Exp. I and IV suggested that the relative frequency of an interresponse time on one key generally was independent of behavior on the other key, but that the number of responses per minute on a key did depend on behavior on the other key.     

Two-key concurrent paced variable-interval paced variable-interval schedules of reinforcement

Nine pigeons were used in two experiments in which a response was reinforced if a variable-interval schedule had assigned a reinforcement and if the response terminated an interresponse time within a certain interval, or class, of interresponse times. One such class was scheduled on one key, and a second class was scheduled on a second key. The procedure was, therefore, a two-key concurrent paced variable-interval paced variable-interval schedule. In Exp. I, the lengths of the two reinforced interresponse times were varied. The relative frequency of responding on a key approximately equalled the relative reciprocal of the length of the interresponse time reinforced on that key. In Exp. II, the relative frequency and relative magnitude of reinforcement were varied. The relative frequency of responding on the key for which the shorter interresponse time was reinforced was a monotonically increasing, negatively accelerated function of the relative frequency of reinforcement on that key. The relative frequency of responding depended on the relative magnitude of reinforcement in approximately the same way as it depended on the relative frequency of reinforcement. The relative frequency of responding on the key for which the shorter interresponse time was reinforced depended on the lengths of the two reinforced interresponse times and on the relative frequency and relative magnitude of reinforcement in the same way as the relative frequency of the shorter interresponse time depended on these variables in previous one-key concurrent schedules of reinforcement for two interresponse times.     

A test of the effectiveness of the differential-reinforcement-of-low-rate schedule

Pigeons and rats were used in a yoked-control design that equated the reinforcement distributions of differential-reinforcement-of-low-rate and variable-interval schedules. Both a between-subjects design and a within-subjects design found response rate higher for the variable-interval schedule than for the differential-reinforcement-of-low-rate schedule, thus demonstrating the effectiveness of the differential-reinforcement-of-low-rate contingency. The interresponse-time distributions were unimodal for all subjects under the variable-interval schedule and bimodal for pigeons under the differential-reinforcement-of-low-rate schedule. The interresponse-time distributions for rats under the differential-reinforcement-of-low-rate schedule were also bimodal in three of four cases but the height of the modes at the shorter interresponse times were small in both absolute value and in relation to the height of the modes at the shorter interresponse times of the pigeons' distributions.     

Molar And Molecular Control In Variable-interval And Variable-ratio Schedules

Response rates are typically higher under variable-ratio than under variable-interval schedules of reinforcement, perhaps because of differences in the dependence of reinforcement rate on response rate or because of differences in the reinforcement of long interresponse times. A variable-interval-with-added-linear-feedback schedule is a variable-interval schedule that provides a response rate/reinforcement rate correlation by permitting the minimum interfood interval to decrease with rapid responding. Four rats were exposed to variable-ratio 15, 30, and 60 food reinforcement schedules, variable-interval 15-, 30-, and 60-s food reinforcement schedules, and two versions of variable-interval-with-added-linear-feedback 15-, 30-, and 60-s food reinforcement schedules. Response rates on the variable-interval-with-added-linear-feedback schedule were similar to those on the variable-interval schedule; all three schedules led to lower response rates than those on the variable-ratio schedules, especially when the schedule values were 30. Also, reinforced interresponse times on the variable-interval-with-added-linear-feedback schedule were similar to those on variable interval and much longer than those produced by variable ratio. The results were interpreted as supporting the hypothesis that response rates on variable-interval schedules in rats are lower than those on comparable variable-ratio schedules, primarily because the former schedules reinforce long interresponse times.     

Interresponse time duration in fixed-interval schedules of reinforcement: control by ordinal position and time since reinforcement

The times between each of the first thirteen responses after reinforcement (the first twelve interresponse times) were determined for two pigeons whose pecking was reinforced on fixed-interval schedules of food reinforcement ranging from 0.5 min to 5 min. These interresponse times were classified with respect to their ordinal position in the sequence of responses and with respect to the time since the preceding reinforcement at which the initiating response occurred. The median interresponse time durations were essentially constant after the sixth response after reinforcement regardless of the time at which the interresponse time was initiated. The durations of the first few interresponse times after reinforcement decreased as the number of preceding responses increased and as the time since the preceding reinforcement increased.     

Signalled reinforcement in differential-reinforcement-of-low rate schedules

At several fixed and variable minimum reinforced interresponse times, a stimulus was added to differential-reinforcement-of-low-rate schedules to signal the availability or nonavailability of reinforcement. As the minimum reinforced interresponse time increased, the rate of unreinforced responding decreased. Changing from fixed to variable minimum interresponse time in the basic differential-reinforcement-of-low-rate schedule further decreased the rate of unreinforced responding. Both effects were to some degree reversible. For fixed minimum reinforced interresponse times of 30 sec or shorter, most unreinforced responses terminated interresponse times just short of that required for reinforcement. The minimum reinforced interresponse time and the number of short response latencies (≤0.5 sec) to the onset of the signal were negatively correlated. Both of these analyses suggested that at values of 30 sec or shorter, the subjects discriminated the availability of the reinforcer more on the basis of time than on the basis of presence or absence of the signal.     

Local patterns of responding maintained by concurrent and multiple schedules

Local patterns of responding were studied when pigeons pecked for food in concurrent variable-interval schedules (Experiment I) and in multiple variable-interval schedules (Experiment II). In Experiment I, similarities in the distribution of interresponse times on the two keys provided further evidence that responding on concurrent schedules is determined more by allocation of time than by changes in local pattern of responding. Relative responding in local intervals since a preceding reinforcement showed consistent deviations from matching between relative responding and relative reinforcement in various postreinforcement intervals. Response rates in local intervals since a preceding changeover showed that rate of responding is not the same on both keys in all postchangeover intervals. The relative amount of time consumed by interchangeover times of a given duration approximately matched relative frequency of reinforced interchangeover times of that duration. However, computer simulation showed that this matching was probably a necessary artifact of concurrent schedules. In Experiment II, when component durations were 180 sec, the relationship between distribution of interresponse times and rate of reinforcement in the component showed that responding was determined by local pattern of responding in the components. Since responding on concurrent schedules appears to be determined by time allocation, this result would establish a behavioral difference between multiple and concurrent schedules. However, when component durations were 5 sec, local pattern of responding in a component (defined by interresponse times) was less important in determining responding than was amount of time spent responding in a component (defined by latencies). In fact, with 5-sec component durations, the relative amount of time spent responding in a component approximately matched relative frequency of reinforcement in the component. Thus, as component durations in multiple schedules decrease, multiple schedules become more like concurrent schedules, in the sense that responding is affected by allocation of time rather than by local pattern of responding.     

Molecular contingencies in schedules of intermittent punishment

In two experiments, key pecking of pigeons was maintained by a variable-interval 180-s schedule of food presentation. Conjointly, a second schedule delivered response-dependent electric shock. In the first experiment, shocks were presented according to either a variable-interval or a nondifferential interval-percentile schedule. The variable-interval shock schedule differentially delivered shocks following long interresponse times. Although the nondifferential shock schedules delivered shocks less differentially with respect to interresponse times, the two shock schedules equally reduced the relative frequency of long interresponse times. The second experiment differentially shocked long or short interresponse times in different conditions, with resulting decreases in the relative frequency of the targeted interresponse times. These experiments highlight the importance of selecting the appropriate level of analysis for the interaction of behavior and environment. Orderly relations present at one level of analysis (e.g., interresponse times) may not be revealed at other levels of analysis (e.g., overall response rate).     

THE LOCAL ORGANIZATION OF BEHAVIOR: DISCRIMINATION OF AND MEMORY FOR SIMPLE BEHAVIORAL PATTERNS

A procedure was developed to enable nonverbal organisms to report what they remember of the temporal organization of their recent behavior. A baseline behavior with known temporal structure was established by a concurrent variable-interval variable-interval schedule for two temporal patterns of behavior (two different classes of reinforced interresponse times). The five pigeon subjects emitted these two temporal patterns on a center key and were occasionally given a short-term memory probe for their most-recently-emitted pattern. The probes consisted of symbolic delayed matching-to-sample tests, in which a response on a green side key was reinforced if the most recent pattern belonged to the shorter reinforced class, and a response to a red side key was reinforced if the most recent pattern belonged to the longer reinforced class. All subjects could report with over ninety percent accuracy what their most recently emitted behavioral pattern was when a retention interval separating the pattern from the memory probe was only .1 seconds. The retention interval was then manipulated, and it was found that recall for a pattern was frequently above chance after a delay of as much as eight seconds. Thus, pigeons can remember their most recent interresponse time not only right after it is emitted, but for several seconds thereafter. In other conditions, the patterns themselves were manipulated. It was found that as the patterns became more similar, discrimination became poorer. These results agree with the view that reinforcement tends to organize and integrate the local structure of behavior to the extent to which that structure is remembered.     

The effect upon simple animal behavior of different frequencies of reinforcement, Part II: separate control of the reinforcement of different IRTs

Rats' responding was stabilized for over 35 days on 4-min variable-interval reinforcement. Reinforcements per hour for 4-sec wide classes of interresponse times were then separately controlled by adjusting those for each class to the variable-interval values that had just prevailed. This produced little or no change in interresponse times, indicating that the new procedure was substantially equivalent to a variable-interval schedule. The variable-interval schedule produced a high and stable conditional probability of interresponse times in the 0- to 4-sec class, associated with a peak in reinforcements per hour for this class. Reducing the reinforcements per hour for this class while raising that for another class (by 3.3 reinforcements per hour) significantly reduced the conditional probability of 0- to 4-sec interresponse times. Restoring the 3.3 reinforcements per hour to the 0- to 4-sec class significantly elevated the conditional probability of interresponse times in this class. Hence, it is concluded that the distribution of interresponse times produced by a subject during some variable-interval schedules is determined partly by the relative reinforcement of different interresponse times that the variable-interval schedule provided.Reprinted from Part II of the Final Report of Research under Contract DA-49-007-MD-408 with the Medical Research and Development Board, Office of the Surgeon General, Department of the Army, 31 December 1954. Edwin B. Newman, Responsible Investigator; Douglas Anger, Research Assistant and author of report. Experimental work done in the Psychological Laboratories of Harvard University.     

Effects of d-amphetamine and chlordiazepoxide on spaced responding in pigeons

The effects of d-amphetamine and chlordiazepoxide were studied in pigeons on performance (1) under a schedule that reinforced responses on a key (food key) if they were more than 20 sec apart, (2) under the same schedule when responses also were required on a collateral key during the interresponse time on the food key, and (3) under the same schedule when responses were required on a collateral key during the interresponse time on the food key and collateral-key responses could produce a stimulus correlated with the availability of food. Under all three spaced-responding schedules, d-amphetamine and chlordiazepoxide at low dose levels slightly increased the frequency of short interresponse times on the food key for about half the birds, and either did not affect the interresponse time patterns of the other birds, or lengthened the durations slightly. At higher dose levels, d-amphetamine and chlordiazepoxide increased the frequency of long interresponse times or abolished responding in all birds. Changes in the pattern of interresponse times on the food key did not seem to depend on changes in the rate or pattern of collateral-key responses.     

Sequential dependencies in free-responding

Three pigeons pecked for food in an experiment in which reinforcements were arranged for responses terminating sequences of interresponse times. Each reinforced interresponse time belonged to a class extending either from 1.0 to 2.0 sec (class A) or from 3.0 to 4.5 sec (class B). Reinforcements were arranged by a single variable-interval schedule and a random device that assigned each reinforcement to one of four sequences of two successive interresponse times: AA, AB, BA, or BB. Throughout the experiment, half of the reinforcements were delivered for interresponse times in class A and half for those in class B. Over conditions, the interresponse time preceding a reinforced interresponse time always, half of the time, or never, belonged to class A. The duration of the interresponse time preceding a reinforced one had a pronounced effect on response patterning. It also had a pronounced effect on the overall response probability, which was highest, intermediate, and lowest, when the interresponse time preceding a reinforced interresponse time always, half of the time, or never, belonged to class A, respectively. In no case were successive interresponse times independent, so that overall response probability was not representative of momentary response probabilities.     

Changes in functional response units with briefly delayed reinforcement

In two experiments, key-peck responding of pigeons was compared under variable-interval schedules that arranged immediate reinforcement and ones that arranged unsignaled delays of reinforcement. Responses during the nominal unsignaled delay periods had no effect on the reinforcer presentations. In Experiment 1, the unsignaled delays were studied using variable-interval schedules as baselines. Relative to the immediate reinforcement condition, 0.5-s unsignaled delays decreased the duration of the reinforced interresponse times and increased the overall frequency of short (<0.5-s) interresponse times. Longer, 5.0-s unsignaled delays increased the duration of the reinforced interresponse times and decreased the overall frequency of the short interresponse times. In Experiment 2, similar effects to those of Experiment 1 were obtained when the 0.5-s unsignaled delays were imposed upon a baseline schedule that explicitly arranged reinforcement of short interresponse times and therefore already generated a large number of short interresponse times. The results support earlier suggestions that the unsignaled 0.5-s delays change the functional response unit from a single key peck to a multiple key-peck unit. These findings are discussed in terms of the mechanisms by which contingencies control response structure in the absence of specific structural requirements.