Reinforcement frequency and contingency as factors in fixed-ratio behavior

Two variables often confounded in fixed-ratio schedules are reinforcement frequency and response requirement. These variables were isolated by a technique that yoked the distributions of reinforcements in time for one group of pigeons to those of pigeons responding on various fixed-ratio schedules. The contingencies for the yoked birds were then manipulated by adding various tandem fixed-ratio requirements to their schedules. Post-reinforcement pause was approximately equal for the yoked and ratio pigeons, and was relatively insensitive to changes in the tandem requirement. Terminal response rate increased with increases in the tandem requirement, even though reinforcement rate was invariant. This increase was attributed to the progressive interference of the tandem requirement with the differential reinforcement of long interresponse times.     

Successive interresponse times in fixed-ratio and second-order fixed-ratio performance

Three rats were trained on a schedule in which every sixth response produced a timeout of 5 sec minimum duration, and food was delivered at the onset of timeout. Successive interresponse times were measured under these conditions, and also when behavior was maintained by second-order fixed-ratio and fixed-interval schedules. Under the second-order schedules, each six-response fixed-ratio component was followed by a timeout, and occasionally food was delivered at the onset of a timeout. In the fixed-ratio schedule, the successive interresponse times showed a decrease followed by an increase before food delivery, but this systematic variation in interresponse times was not found when the performance was under second-order reinforcement. Under both second-order schedules the latencies of successive components, and the successive interresponse times within each component, showed a decrease as food delivery was approached.     

Post-reinforcement pauses and response rate of monkeys on a two-hand fixed-ratio schedule

Fixed-ratio behavior of monkeys was analyzed separately for two hands. While one hand responded on the fixed-ratio schedule the other performed a holding response and the function of the hands changed in alternate ratio runs. After performance was stable on the fixed ratio (70 responses, two monkeys; 100 responses, two monkeys, 120 responses, two monkeys) 90 sessions of further training equalized post-reinforcement pauses and the mean interresponse time of the two hands. Hand preference in reaching for food remained unchanged. Then, the fixed-ratio requirement was changed (a) in small sequential steps, (b) in two large steps, and, (c) within sessions alternating two runs at a high ratio with two runs at a low ratio. The mean duration of post-reinforcement pauses was correlated with a fixed ratio maintained throughout a session but single pauses were neither controlled by the immediately preceding nor by the following ratio run when a cue to its length was available. The mean interresponse time was insensitive to changes in fixed ratio. The fixed-ratio performance was generally similar to that of pigeons and rats.     

Stimulus control of differential-reinforcement-of-low-rate responding

Five pigeons were given single-stimulus training on an 8-sec differential-reinforcement-of-low-rate schedule followed by steady-state generalization training using 12 wavelength stimuli. Three birds had a high percentage of reinforced responses on the training schedule and flat generalization gradients of total responses. The birds with fewer reinforced responses had much steeper generalization gradients. Generalization gradients plotted as a function of both stimulus wavelength and interresponse time showed that for most birds, stimulus control was restricted to responses with long interresponse times. Responses with very short interresponse times were not under stimulus control and there was some evidence of inhibitory control of short interresponse times. Interresponse-times-per-opportunity functions, plotted as a function of stimulus wavelength, showed that stimulus wavelength controlled the temporal distribution of responses, rather than the overall rate of response. The data indicate that the differential-reinforcement-of-low-rate schedule generates several response categories that are controlled in different ways by wavelength and time-correlated stimuli, and that averaging responses regardless of interresponse-time length obscures this control.     

Two-key concurrent paced variable-interval paced variable-interval schedules of reinforcement

Nine pigeons were used in two experiments in which a response was reinforced if a variable-interval schedule had assigned a reinforcement and if the response terminated an interresponse time within a certain interval, or class, of interresponse times. One such class was scheduled on one key, and a second class was scheduled on a second key. The procedure was, therefore, a two-key concurrent paced variable-interval paced variable-interval schedule. In Exp. I, the lengths of the two reinforced interresponse times were varied. The relative frequency of responding on a key approximately equalled the relative reciprocal of the length of the interresponse time reinforced on that key. In Exp. II, the relative frequency and relative magnitude of reinforcement were varied. The relative frequency of responding on the key for which the shorter interresponse time was reinforced was a monotonically increasing, negatively accelerated function of the relative frequency of reinforcement on that key. The relative frequency of responding depended on the relative magnitude of reinforcement in approximately the same way as it depended on the relative frequency of reinforcement. The relative frequency of responding on the key for which the shorter interresponse time was reinforced depended on the lengths of the two reinforced interresponse times and on the relative frequency and relative magnitude of reinforcement in the same way as the relative frequency of the shorter interresponse time depended on these variables in previous one-key concurrent schedules of reinforcement for two interresponse times.     

Comments on Shimp's (1983) double dissociation between knowledge and tacit knowledge

Shimp (1983) found in five pigeons “double dissociation” between the distribution of pairs of long and short reinforced interresponse times and “self-reports,” obtained by symbolic matching to sample, concerning these interresponse times. This result can be explained by assuming that the birds used a fixed temporal interval as matching criterion, independently of which pattern of interresponse times was reinforced.     

SINGLE‐SAMPLE DISCRIMINATION OF DIFFERENT SCHEDULES' REINFORCED INTERRESPONSE TIMES

Food‐deprived rats in Experiment 1 responded to one of two tandem schedules that were, with equal probability, associated with a sample lever. The tandem schedules' initial links were different random‐interval schedules. Their values were adjusted to approximate equality in time to completing each tandem schedule's response requirements. The tandem schedules differed in their terminal links: One reinforced short interresponse times; the other reinforced long ones. Tandem‐schedule completion presented two comparison levers, one of which was associated with each tandem schedule. Pressing the lever associated with the sample‐lever tandem schedule produced a food pellet. Pressing the other produced a blackout. The difference between terminal‐link reinforced interresponse times varied across 10‐trial blocks within a session. Conditional‐discrimination accuracy increased with the size of the temporal difference between terminal‐link reinforced interresponse times. In Experiment 2, one tandem schedule was replaced by a random ratio, while the comparison schedule was either a tandem schedule that only reinforced long interresponse times or a random‐interval schedule. Again, conditional‐discrimination accuracy increased with the temporal difference between the two schedules' reinforced interresponse times. Most rats mastered the discrimination between random ratio and random interval, showing that the interresponse times reinforced by these schedules can serve to discriminate between these schedules.     

Sequential dependencies in free-responding

Three pigeons pecked for food in an experiment in which reinforcements were arranged for responses terminating sequences of interresponse times. Each reinforced interresponse time belonged to a class extending either from 1.0 to 2.0 sec (class A) or from 3.0 to 4.5 sec (class B). Reinforcements were arranged by a single variable-interval schedule and a random device that assigned each reinforcement to one of four sequences of two successive interresponse times: AA, AB, BA, or BB. Throughout the experiment, half of the reinforcements were delivered for interresponse times in class A and half for those in class B. Over conditions, the interresponse time preceding a reinforced interresponse time always, half of the time, or never, belonged to class A. The duration of the interresponse time preceding a reinforced one had a pronounced effect on response patterning. It also had a pronounced effect on the overall response probability, which was highest, intermediate, and lowest, when the interresponse time preceding a reinforced interresponse time always, half of the time, or never, belonged to class A, respectively. In no case were successive interresponse times independent, so that overall response probability was not representative of momentary response probabilities.     

Choice between response rates

Three pigeons were required to peck a single key at a higher and a lower rate, corresponding to two classes of shorter and longer concurrently reinforced interresponse times. Food reinforcers arranged by a single variable-interval schedule were randomly allocated to the two reinforced interresponse times. The absolute durations of reinforced interresponse times were varied while the total reinforcements per hour was held constant and the relative duration, i.e., the relative reciprocal, of the shorter reinforcer class was held constant at 0.70. Preference for the higher rate of responding, as measured by the relative frequency of responses terminating interresponse times in the shorter reinforced class, depended on the absolute reinforced response rates. Preference for the higher reinforced rate increased from a level of near-indifference (0.50) at high reinforced response rates, through the matching level (0.70) at intermediate reinforced response rates, to a virtually exclusive preference (>0.90) at low reinforced response rates. These results resemble corresponding preference functions obtained with two-key concurrent-chains schedules and thereby provide another sense in which it may be said that interresponse-time distributions from interval schedules estimate preference functions for the component response rates corresponding to different classes of reinforced interresponse times.     

Temporal patterns of responding in small fixed-ratio schedules

Pigeons were exposed to an ascending series of small fixed-ratio schedules from fixed-ratio 1 to 7. Two of those pigeons were later placed on a fixed-ratio 30 schedule. The two primary dependent variables were the postreinforcement pause and the interresponse time. Changes in these variables under small fixed ratios were sometimes opposite to changes reported with large fixed ratios. For example, postreinforcement pauses decreased in length as the fixed-ratio requirement increased from fixed-ratio 1 to fixed-ratio 3. Also, the interresponse times early in the small fixed-ratio schedule were shorter than those immediately preceding reinforcement. These findings question the role of interresponse-time reinforcement in determining temporal patterns of responding under small fixed-ratio schedules. They also suggest that there may be a limited region in which the independent variable, fixed-ratio size, does not operate as previously described.     

Food-deprivation level alters the effects of morphine on pigeons'' key pecking.

Four pigeons pecked response keys under a multiple fixed-ratio 30 fixed-interval 5-min schedule of food presentation. Components alternated separated by 15-s timeouts; each was presented six times. Pigeons were maintained at 70%, 85%, and greater than 90% of their free-feeding weights across experimental conditions. When response rates were stable, the effects of morphine (0.56 to 10.0 mg/kg) and saline were investigated. Morphine reduced response rates in a dose-dependent manner under the fixed-ratio schedule and at high doses under the fixed-interval schedule. In some cases, low doses of morphine increased rates under the fixed-interval schedule. When pigeons were less food deprived, reductions in pecking rates occurred at lower doses under both schedules for 3 of 4 birds compared to when they were more food deprived. When pigeons were more food deprived, low doses of morphine increased rates of pecking in the initial portions of fixed intervals by a greater magnitude. Thus, food-deprivation levels altered both the rate-decreasing and rate-increasing effects of morphine. These effects may share a common mechanism with increased locomotor activity produced by drugs and with increased drug self-administration under conditions of more severe food deprivation.     

Interresponse time duration in fixed-interval schedules of reinforcement: control by ordinal position and time since reinforcement

The times between each of the first thirteen responses after reinforcement (the first twelve interresponse times) were determined for two pigeons whose pecking was reinforced on fixed-interval schedules of food reinforcement ranging from 0.5 min to 5 min. These interresponse times were classified with respect to their ordinal position in the sequence of responses and with respect to the time since the preceding reinforcement at which the initiating response occurred. The median interresponse time durations were essentially constant after the sixth response after reinforcement regardless of the time at which the interresponse time was initiated. The durations of the first few interresponse times after reinforcement decreased as the number of preceding responses increased and as the time since the preceding reinforcement increased.     

Drug discrimination under a concurrent fixed-ratio fixed-ratio schedule.

Pigeons were trained to discriminate 5.0 mg/kg pentobarbital from saline under a two-key concurrent fixed-ratio 10 fixed-ratio 40 schedule of food presentation, in which the fixed-ratio component with the lower response requirement was programmed to reinforce responding on one key after drug administration (pentobarbital-biased key) and on the other key after saline administration (saline-biased key). After responding stabilized, pigeons averaged 98% of their responses on the pentobarbital-biased key during training sessions preceded by pentobarbital, and they averaged 90% of their responses on the saline-biased key during training sessions preceded by saline. In test sessions preceded by doses of pentobarbital, chlordiazepoxide, or ethanol, pigeons switched from responding on the saline-biased key at low doses to responding on the pentobarbital-biased key at higher doses (the dose-response curve was quantal). High doses of phencyclidine produced responding on both keys, whereas pigeons responded almost exclusively on the saline-biased key after all doses of methamphetamine. These and previous experiments using concurrent reinforcement schedules to study drug discrimination illustrate that the schedule of reinforcement is an important determinant of the shape of dose-effect curves in drug-discrimination experiments.     

The reinforcement of least-frequent interresponse times

A new schedule of reinforcement was used to maintain key-pecking by pigeons. The schedule reinforced only pecks terminating interresponse times which occurred least often relative to the exponential distribution of interresponse times to be expected from an ideal random generator. Two schedule parameters were varied: (1) the rate constant of the controlling exponential distribution and (2) the probability that a response would be reinforced, given that it met the interresponse-time contingency. Response rate changed quickly and markedly with changes in the rate constant; it changed only slightly with a fourfold change in the reinforcement probability. The schedule produced stable rates and high intra- and inter-subject reliability, yet interresponse time distributions were approximately exponential. Such local interresponse time variability in the context of good overall control suggests that the schedule may be used to generate stable, predictable, yet sensitive baseline rates. Implications for the measurement of rate are discussed.     

SUPPRESSIVE AND FACILITATIVE EFFECTS OF SHOCK INTENSITY AND INTERRESPONSE TIMES FOLLOWED BY SHOCK

Although response‐dependent shock often suppresses responding, response facilitation can occur. In two experiments, we examined the suppressive and facilitative effects of shock by manipulating shock intensity and the interresponse times that produced shock. Rats' lever presses were reinforced on a variable‐interval 40‐s schedule of food presentation. Shock followed either long or short interresponse times. Shock intensity was raised from 0.05 mA to 0.4 mA or 0.8 mA. Overall, shock contingent on long interresponse times punished long interresponse times and increased response rates. Shock contingent on short interresponse times punished short interresponse times and decreased response rates. In Experiment 1, raising the range of interresponse times that produced shock enhanced these effects. In Experiment 2, the effects of shock intensity depended on the interresponse times that produced shock. When long interresponse times produced shock, low intensities increased response rates. High intensities decreased response rates. When short interresponse times produced shock, high shock intensities punished short interresponse times and decreased response rates more than low intensities. The results may explain why punishment procedures occasionally facilitate responding and establish parameters for future studies of punishment.     

Changes in functional response units with briefly delayed reinforcement

In two experiments, key-peck responding of pigeons was compared under variable-interval schedules that arranged immediate reinforcement and ones that arranged unsignaled delays of reinforcement. Responses during the nominal unsignaled delay periods had no effect on the reinforcer presentations. In Experiment 1, the unsignaled delays were studied using variable-interval schedules as baselines. Relative to the immediate reinforcement condition, 0.5-s unsignaled delays decreased the duration of the reinforced interresponse times and increased the overall frequency of short (<0.5-s) interresponse times. Longer, 5.0-s unsignaled delays increased the duration of the reinforced interresponse times and decreased the overall frequency of the short interresponse times. In Experiment 2, similar effects to those of Experiment 1 were obtained when the 0.5-s unsignaled delays were imposed upon a baseline schedule that explicitly arranged reinforcement of short interresponse times and therefore already generated a large number of short interresponse times. The results support earlier suggestions that the unsignaled 0.5-s delays change the functional response unit from a single key peck to a multiple key-peck unit. These findings are discussed in terms of the mechanisms by which contingencies control response structure in the absence of specific structural requirements.     

An interresponse time analysis of variable-ratio punishment

An interresponse time analysis was used to study the effects of variable-ratio punishment schedules on the temporal pattern of reinforced responding. Twelve pigeons responded on a baseline variable-interval schedule of food reinforcement. A variable-ratio ten schedule of electric shock punishment was then introduced. The shock intensity was systematically increased to the highest intensity at which responding could be maintained. At this intensity, the mean variable-ratio value was increased and then decreased. Variable-ratio punishment resulted in an increased relative frequency of very short unreinforced interresponse times (response bursting). Increased response bursting accounted for instances of response rate facilitation. In addition, shock was followed by interresponse times of decreasing mean length over the first several responses after shock.     

Molecular contingencies in schedules of intermittent punishment

In two experiments, key pecking of pigeons was maintained by a variable-interval 180-s schedule of food presentation. Conjointly, a second schedule delivered response-dependent electric shock. In the first experiment, shocks were presented according to either a variable-interval or a nondifferential interval-percentile schedule. The variable-interval shock schedule differentially delivered shocks following long interresponse times. Although the nondifferential shock schedules delivered shocks less differentially with respect to interresponse times, the two shock schedules equally reduced the relative frequency of long interresponse times. The second experiment differentially shocked long or short interresponse times in different conditions, with resulting decreases in the relative frequency of the targeted interresponse times. These experiments highlight the importance of selecting the appropriate level of analysis for the interaction of behavior and environment. Orderly relations present at one level of analysis (e.g., interresponse times) may not be revealed at other levels of analysis (e.g., overall response rate).     

Steady-state performance on fixed-, mixed-, and random-ratio schedules

Three groups of rats pressed a lever for milk reinforcers on various simple reinforcement schedules (one schedule per condition). In Group M, each pair of conditions included a mixed-ratio schedule and a fixed-ratio schedule with equal average response:reinforcer ratios. On mixed-ratio schedules, reinforcement occurred with equal probability after a small or a large response requirement was met. In Group R, fixed-ratio and random-ratio schedules were compared in each pair of conditions. For all subjects in these two groups, the frequency distributions of interresponse times of less than one second were very similar on all ratio schedules, exhibiting a peak at about .2 seconds. For comparison, subjects in Group V responded on variable-interval schedules, and few interresponse times as short as .2 seconds were recorded. The results suggest that the rate of continuous responding is the same on all ratio schedules, and what varies among ratio schedules is the frequency, location, and duration of pauses. Preratio pauses were longer on fixed-ratio schedules than on mixed-ratio or random-ratio schedules, but there was more within-ratio pausing on mixed-ratio and random-ratio schedules. Across a single trial, the probability of an interruption in responding decreased on fixed-ratio schedules, was roughly constant on random-ratio schedules, and often increased and then decreased on mixed-ratio schedules. These response patterns provided partial support for Mazur's (1982) theory that the probability of instrumental responding is directly related to the probability of reinforcement and the proximity of reinforcement.     

Parametric manipulation of interresponse-time contingency independent of reinforcement rate

Pecking of pigeons was reinforced under a modified interval-percentile procedure that allowed independent manipulation of overall reinforcement rate and the degree to which reinforcement depended on interresponse-time duration. Increasing the contingency, as measured by the phi coefficient, between reinforcement and long interresponse times while controlling the overall rate of reinforcement systematically increased the frequency of those interresponse times and decreased response rate under both of the reinforcement rates studied. Increasing reinforcement rate also generally increased response rate, particularly under weaker interresponse-time contingencies. Random-interval schedules with comparable reinforcement rates generated response rates and interresponse-time distributions similar to those obtained with moderate-to-high interresponse-time reinforcement contingencies. These results suggest that interresponse-time reinforcement contingencies inherent in random-interval and constant-probability variable-interval schedules exercise substantial control over responding independent of overall reinforcement rate effects. The interresponse-time reinforcement contingencies inherent in these schedules may actually mask the effects of overall reinforcement rate; thus differences in response rate as a function of reinforcement rate when interresponse-time reinforcement is eliminated may be underestimated.