Pausing under variable-ratio schedules: Interaction of reinforcer magnitude, variable-ratio size, and lowest ratio

Pigeons pecked a key under two-component multiple variable-ratio schedules that offered 8-s or 2-s access to grain. Postreinforcement pausing and the rates of responding following the pause (run rates) in each component were measured as a function of variable-ratio size and the size of the lowest ratio in the configuration of ratios comprising each schedule. In one group of subjects, variable-ratio size was varied while the size of the lowest ratio was held constant. In a second group, the size of the lowest ratio was varied while variable-ratio size was held constant. For all subjects, the mean duration of postreinforcement pausing increased in the 2-s component but not in the 8-s component. Postreinforcement pauses increased with increases in variable-ratio size (Group 1) and with increases in the lowest ratio (Group 2). In both groups, run rates were slightly higher in the 8-s component than in the 2-s component. Run rates decreased slightly as variable-ratio size increased, but were unaffected by increases in the size of the lowest ratio. These results suggest that variable-ratio size, the size of the lowest ratio, and reinforcer magnitude interact to determine the duration of postreinforcement pauses.     

Determinants of pausing under variable-ratio schedules: Reinforcer magnitude, ratio size, and schedule configuration

Pigeons pecked a key under two-component multiple variable-ratio schedules that offered 8-s or 2-s access to grain. Phase 1 assessed the effects of differences in reinforcer magnitude on postreinforcement pausing, as a function of ratio size. In Phase 2, postreinforcement pausing and the first five interresponse times in each ratio were measured as a function of differences in reinforcer magnitude under equal variable-ratio schedules consisting of different configurations of individual ratios. Rates were also calculated exclusive of postreinforcement pause times in both phases. The results from Phase 1 showed that as ratio size increased, the differences in pausing educed by unequal reinforcer magnitudes also increased. The results of Phase 2 showed that the effects of reinforcer magnitude on pausing and IRT durations were a function of schedule configuration. Under one configuration, in which the smallest ratio was a fixed-ratio 1, pauses were unaffected by magnitude but the first five interresponse times were affected. Under the other configuration, in which the smallest ratio was a fixed-ratio 7, pauses were affected by reinforcer magnitude but the first five interresponse times were not. The effect of each configuration seemed to be determined by the value of the smallest individual ratio. Rates calculated exclusive of postreinforcement pause times were, in general, directly related to reinforcer magnitude, and the relation was shown to be a function of schedule configuration.     

Temporal patterns of responding in small fixed-ratio schedules

Pigeons were exposed to an ascending series of small fixed-ratio schedules from fixed-ratio 1 to 7. Two of those pigeons were later placed on a fixed-ratio 30 schedule. The two primary dependent variables were the postreinforcement pause and the interresponse time. Changes in these variables under small fixed ratios were sometimes opposite to changes reported with large fixed ratios. For example, postreinforcement pauses decreased in length as the fixed-ratio requirement increased from fixed-ratio 1 to fixed-ratio 3. Also, the interresponse times early in the small fixed-ratio schedule were shorter than those immediately preceding reinforcement. These findings question the role of interresponse-time reinforcement in determining temporal patterns of responding under small fixed-ratio schedules. They also suggest that there may be a limited region in which the independent variable, fixed-ratio size, does not operate as previously described.     

A comparison of ratio and interval reinforcement schedules with comparable interreinforcement times

Pigeons were trained to peck keys on fixed-ratio and fixed-interval schedules of food reinforcement. Both schedules produced a pattern of behavior characterized as pause and run, but the relation of pausing to time between reinforcers differed for the two schedules even when mean time between reinforcers was the same. Pausing in the fixed ratio occupied less of the time between reinforcers for shorter interreinforcer times. For two of three birds, the relation was reversed at longer interreinforcer times. As an interreinforcer time elapsed, there was an increasing tendency to return to responding for the fixed interval, but a roughly constant tendency to return to responding for the fixed-ratio schedule. In Experiment 1 these observations were made for both single-reinforcement schedules and multiple schedules of fixed-ratio and fixed-interval reinforcement. In Experiment 2 the observations were extended to a comparison of fixed-ratio versus variable-interval reinforcement schedules, where the distribution of interreinforcement times in the variable interval approximated that for the fixed ratio.     

Determinants of pigeons' waiting time: Effects of interreinforcement interval and food delay

Four pigeons performed on three types of schedules at short (i.e., 10, 30, or 60 s) interreinforcement intervals: (a) a delay-dependent schedule where interreinforcement interval was held constant (i.e., increases in waiting time decreased food delay), (b) an interreinforcement-interval-dependent schedule where food delay was held constant (i.e., increases in waiting time increased interreinforcement interval), and (c) a both-dependent schedule where increases in waiting time produced increases in interreinforcement interval but decreases in food delay. Waiting times were typically longer under the delay-dependent schedules than under the interreinforcement-interval-dependent schedules. Those under both-dependent schedules for 1 subject were intermediate between those under the other two schedule types, whereas for the other subjects waiting times under the both-dependent procedure were similar either to those under the delay-dependent schedule or to those under the interreinforcement-interval-dependent schedule, depending both on the subject and the interreinforcement interval. These results indicate that neither the interreinforcement interval nor food delay is the primary variable controlling waiting time, but rather that the two interact in a complex manner to determine waiting times.     

Fixed-ratio pausing: Joint effects of past reinforcer magnitude and stimuli correlated with upcoming magnitude

Pigeons responded on fixed-ratio schedules ending in small or large reinforcers (grain presentations of different duration) interspersed within each session. In mixed-schedule conditions, the response key was lit with a single color throughout the session, and pausing was directly related to the past reinforcer (longer pauses after large reinforcers than after small ones). In multiple-schedule conditions, different colors accompanied the ratios ending in small and large reinforcers, and pausing was affected by the upcoming reinforcer as well as the past one. Pauses were shorter before large reinforcers than before small ones, but they continued to be longer after large reinforcers than after small ones. The influence of the past reinforcer was modulated by the magnitude of the upcoming reinforcer; in the presence of the stimulus before the small reinforcer, the effect of the past reinforcer was enhanced relative to its effect in the stimulus before the large reinforcer. These results show that pausing between ratios is jointly determined by two competing factors: past conditions of reinforcement and stimuli correlated with upcoming conditions.     

Analysis of fixed-ratio behavior maintained by drug reinforcers.

Behavior maintained by intravenously delivered alfentanil, cocaine, or ketamine was assessed using a fixed-ratio schedule of reinforcement. As the dose of each drug was increased, rate of responding also increased up to a maximum. Further increases in dose resulted in decreased response rates (inverted U-shaped curve). An analysis of postreinforcement-pause-time and run-time measures for the ascending limb of the inverted U-shaped functions revealed that behavior was characterized by systematic decreases in both pause time and run time as dose and rate increased. An examination of the descending limb of the dose-response functions revealed that lowered response rates for cocaine and ketamine were correlated with increases in run time and small and inconsistent effects on postreinforcement pause time. Behavior maintained by rate-reducing doses of alfentanil was characterized by lengthened postreinforcement pauses with small increases in run time. These data suggest that at larger doses, drug reinforcers may have unconditioned or direct effects on the behavior that the drug is maintaining, and more important, that the nature of these unconditioned effects depends on the drug that is maintaining behavior.     

Species differences in temporal control of behavior

Temporal control of rats' and pigeons' responding was analyzed and compared in detail on fixed-interval and fixed-time schedules with parameters of 30, 60, and 120 seconds. On fixed-time schedules, rats' responding decreased greatly or ceased, whereas pigeons continued to respond, especially on low schedule values. The running rate of responses (calculated by excluding the postreinforcement pause) was related to the duration of the preceding postreinforcement pause for rats but not for pigeons. Changes in response rate in successive segments of the interval were best described by normal curves. The relationship between midpoints of the normal curves and schedule value was a power function, with an exponent of less than one for pigeons but greater than one for rats. These differences could be explained in terms of a basic difference between the key-peck and lever-press responses, the two being differently affected by the response-eliciting properties of food.     

Effects of d-amphetamine and cocaine on strained ratio behavior in a repeated-acquisition task.

Pigeons acquired a different four-response chain each session by responding sequentially on three keys in the presence of four colors. When the fixed-ratio requirement for food presentation was five completions of the chain, d-amphetamine and cocaine disrupted the behavior. As the dose of each drug was increased, the overall response rate decreased, the overall accuracy was impaired (i,e., percent errors increased), and there was less within-session error reduction (acquisition). In contrast, when the fixed-ratio requirement was either 20 or 50 completions of the chain, certain doses of both drugs produced large increases in the overall response rate by eliminating the extended pausing (ratio strain) that was characteristic of the control sessions. These rate-increasing effects were accompanied by error-decreasing effects, both during acquisition and after the response chain had been acquired. Taken together, the results show that the effects of d-amphetamine and cocaine on behavior in a repeated-acquisition task can be modulated by manipulating the value of the fixed-ratio schedule maintaining the behavior.     

The effects of number of responses on the postreinforcement pause in fixed-interval schedules

The present study manipulated the number of responses in a modified fixed-interval schedule by imposing a blackout after each unreinforced response during the interval. The blackout duration was varied, and the duration of the fixed interval was held constant. The subjects were initially exposed to a fixed-interval 300-sec schedule. Blackout durations of 0, 10, and 50 sec were used. Following this, a fixed-interval 30-sec schedule was used with blackout durations of 0, 1, and 5 sec. Under the fixed-interval 300-sec schedule, the number of interreinforcement responses varied over a wider range than occurred under the fixed-interval 30-sec schedule. The duration of the postreinforcement pause decreased as blackout durations were increased and number of responses decreased on the fixed-interval 300-sec schedule, but pause length did not vary with changes in blackout duration and number of responses for the fixed-interval 30-sec schedule. The differences in the effects of blackout duration and response manipulation on the two fixed-interval schedules were attributed to relatively greater changes in the number of interreinforcement responses for the fixed-interval 300-sec schedule.     

Psychophysics of key-peck duration in the pigeon

The duration of the pigeon's key peck was differentially reinforced in either a trials or a free-operant procedure. Mean emitted peck duration was a power function of the duration required for food delivery to occur. The exponents of the power function differed considerably from those observed in earlier research involving longer duration responses in pigeons and other species. The coefficients of variation also did not correspond with those of the earlier research on other responses, nor did consideration of the durations actually reinforced resolve the differences. Duration was neither a function of response rate nor of intermittency of reinforcement. Key-peck duration was changed in an orderly way by differential reinforcement. However, it appeared to be more strongly determined by its duration in the absence of differential reinforcement than were longer duration responses.     

Conjunctive schedules of reinforcement: III. A fixed-interval adjusting fixed-ratio schedule

Key pecking of three pigeons was studied under a conjunctive schedule that specified both a fixed-interval and an adjusting fixed-ratio requirement. The fixed-interval schedule was 6 min for one pigeon and 3 min for the other two. The size of the ratio requirement was determined within each cycle of the fixed interval by the duration of the pause before responding began. The fixed-ratio value was at maximum at the start of each fixed interval and decreased linearly until the first response occurred (adjusting fixed-ratio schedule). A peck produced food when the number of responses remaining on the fixed-ratio schedule was completed and when the fixed interval had elapsed. If no response occurred during the interval, the fixed-ratio requirement decreased to one and a single response after the interval elapsed produced food. The initial value of the adjusting fixed-ratio schedule was studied over a range of 0 to 900. Increases in the adjusting fixed-ratio schedule to about 300 responses increased both pause duration and running response rate and also modified the pattern of responding from that obtained under the fixed-interval schedule. Higher values of the adjusting fixed ratio generally decreased pause duration and running response rate and also disrupted responding. Interreinforcement time under the conjunctive schedule was increased substantially when the adjusting fixed-ratio size exceeded 300 responses.     

Performances on ratio and interval schedules of reinforcement: Data and theory

Two differences between ratio and interval performance are well known: (a) Higher rates occur on ratio schedules, and (b) ratio schedules are unable to maintain responding at low rates of reinforcement (ratio “strain”). A third phenomenon, a downturn in response rate at the highest rates of reinforcement, is well documented for ratio schedules and is predicted for interval schedules. Pigeons were exposed to multiple variable-ratio variable-interval schedules in which the intervals generated in the variable-ratio component were programmed in the variable-interval component, thereby “yoking” or approximately matching reinforcement in the two components. The full range of ratio performances was studied, from strained to continuous reinforcement. In addition to the expected phenomena, a new phenomenon was observed: an upturn in variable-interval response rate in the midrange of rates of reinforcement that brought response rates on the two schedules to equality before the downturn at the highest rates of reinforcement. When the average response rate was corrected by eliminating pausing after reinforcement, the downturn in response rate vanished, leaving a strictly monotonic performance curve. This apparent functional independence of the postreinforcement pause and the qualitative shift in response implied by the upturn in variable-interval response rate suggest that theoretical accounts will require thinking of behavior as partitioned among at least three categories, and probably four: postreinforcement activity, other unprogrammed activity, ratio-typical operant behavior, and interval-typical operant behavior.     

Choice between sequences of fixed-ratio schedules: effects of ratio values and probability of food delivery.

Pigeons were exposed to schedules of food delivery that consisted of two sequential fixed ratios. When alternative sequences provided two food deliveries per 50 responses, the schedule with the shorter initial fixed-ratio value was consistently preferred. Progressively reducing from 1.0 to .25 the probability of food delivery following completion of the second fixed ratio of the sequence with the shorter initial fixed ratio did not reduce preference for this sequence. Moreover, the sequence with the shorter initial fixed ratio also was preferred when the probability of food delivery following completion of the initial ratio in that sequence was progressively reduced from 1.0 to .5, although preference shifted to the alternative when the probability was reduced to 0. These findings suggest that the length of the initial fixed ratio was a primary determinant of choice. Subsequent manipulations demonstrated, however, that when the initial fixed ratios of the two alternatives were equal, changes in the ratio value and probability of food delivery following completion of the second fixed ratio lawfully affected choice.     

Comparison of drug effects on fixed-ratio performance and chain performance maintained under a second-order fixed-ratio schedule.

In one component of a multiple schedule, pigeons were required to complete the same four-response chain each session by responding sequentially on three identically lighted keys in the presence of four successively presented colors (chain performance). Food presentation occurred after five completions of the chain (i.e., after 20 correct responses). Errors, such as responding on the center or right key when the left was designated correct, produced a brief timeout but did not reset the chain. In the other component, responding on a single key (lighted white) was maintained by food presentation under a fixed-ratio 20 schedule. In general, phencyclidine and d-amphetamine produced dose-dependent decreases in the overall response rates in both components. With pentobarbital, overall rate in each component generally increased at intermediate doses and decreased at higher doses. All three drugs produced dose-dependent disruptive effects on chain-performance accuracy. Phencyclidine and pentobarbital increased percent errors at doses that had little or no rate-decreasing effects, whereas d-amphetamine generally increased percent errors only at doses that substantially decreased overall rate. At high doses, all three drugs produced greater disruption of chain performance than of fixed-ratio performance, as indicated by a slower return to control responding, although the effects of d-amphetamine were less selective than those of phencyclidine or pentobarbital.     

Time allocation in concurrent schedules: the effect of signalled reinforcement

The responses of five pigeons were reinforced on concurrent variable-interval variable-interval reinforcement schedules in which changeover key responses changed the stimulus and reinforcement schedules associated with the food key. While the reinforcement availability in one component remained unchanged throughout the experiment, the reinforcement availability in the other component was, during several conditions, signalled by the onset of an additional discriminative stimulus. During unsignalled conditions, both the relative frequency of responding and the relative time spent in each component approximated the obtained relative reinforcement frequency in each component. The effect of signalling reinforcer availability in one component was to (1) reduce responding in the signalled component to near-zero levels, and (2) increase the relative time in the unsignalled component, without a corresponding increase in the obtained relative reinforcement frequency. The magnitude of the increase in relative time in the unsignalled component decreased as the overall frequency of reinforcement increased. This deviation in the matching relation between relative time and the obtained relative reinforcement frequency was eliminated if the overall reinforcement frequency was increased before the signal was introduced and then, without removing the signal, gradually reduced.     

Tests of transitivity in choices between fixed and variable reinforcer delays.

This experiment tested for transitivity in pigeons' choices between variable-time (VT) and fixed-time (FT) schedules. In a discrete-trials procedure, a subject chose between two alternatives by making a single key peck. Each choice was between a "standard alternative," which was the same schedule throughout a condition, and an "adjusting alternative," in which the delay to reinforcement was systematically increased or decreased many times a session. These adjustments enabled an approximate indifference point to be identified--the value of the adjusting delay at which the subject chose each alternative about equally often. Each test of transitivity involved four conditions. In one, the standard alternative was a variable-time schedule with a 2-s reinforcer, and the adjusting alternative also delivered a 2-s reinforcer. A second condition was similar except that the adjusting alternative delivered a 5-s reinforcer. The indifference point from each of these conditions was then converted to a fixed-time schedule for subsequent comparisons in the third and fourth conditions, respectively. Each of these last two conditions compared one of the fixed-time schedules (based upon the previous conditions and including their different reinforcer durations) with an adjusting schedule that delivered the alternative reinforcer duration, to determine whether the obtained indifference points would be those predicted from the prior alternative-duration comparisons with the VT schedule. There was little evidence for intransitivity of choice: Averaged across subjects and replications, the obtained indifference points deviated from perfect transitivity by less than 8%, and these deviations were not statistically significant. These results contrast with those of Navarick and Fantino (1972), who found frequent violations of transitivity between periodic and aperiodic schedules using a concurrent-chains procedure with variable-interval schedules in the initial links.     

Behavioral contrast for key pecking as a function of component duration when only one component varies

Pigeons pecked keys for food reinforcers delivered by multiple variable-interval 2-min variable-interval 2-min schedules. Positive behavioral contrast was created by changing one component to extinction; negative contrast was achieved by changing one component to a variable-interval 15-s schedule. The duration of each component was varied independently of the other from 5 to 960 s. The size of positive contrast was greatest when the extinction component was 30 or 60 s long. It did not change significantly with changes in the duration of the variable-interval 2-min component. The absolute size of negative contrast decreased with increases in the duration of the variable-interval 2-min component. It did not change significantly with changes in the duration of the variable-interval 15-s component. These results show that the size of contrast is determined primarily by the duration of the component that provides the less favorable conditions of reinforcement. These results are not predicted by current theories.     

Selective antagonism of the error-increasing effect of morphine by naloxone in a repeated-acquisition task.

Pigeons acquired a different four-response chain each session by responding sequentially on three keys in the presence of four colors. The response chain was maintained by food presentation under a fixed-ratio schedule. Errors produced a brief timeout but did not reset the chain. When either morphine or naloxone was administered alone, the overall response rate decreased with increasing doses. The rate-decreasing effect was accompanied by an increase in percent errors with morphine but not with naloxone. Both effects of morphine were antagonized by doses of naloxone that were ineffective when given alone. The antagonism was selective in that naloxone (3 mg/kg) completely blocked the error-increasing effect but not the rate-decreasing effect of the higher doses of morphine. The view that naloxone is a specific narcotic antagonist was supported by the finding that naloxone failed to antagonize the rate-decreasing and error-increasing effects of d-amphetamine, pentobarbital, and phencyclidine.