Responding under chained and tandem fixed-ratio schedules

The role of stimuli in chained fixed-ratio schedules of reinforcement was examined. At various ratio values, responding on schedules consisting of three or five equal components, with a different colored light in each component (“block counter”) was compared with responding on tandem or simple fixed-ratio schedules having the same color present throughout the entire ratio. At all ratio values except the smallest, the chain stimuli resulted in longer pauses after reinforcement. The magnitude of this effect became greater as the size of the ratio was increased. Post-reinforcement pause durations were longer under five-component schedules than under three-component schedules. Running rates in the first component were lower on the chained schedules than on the tandem schedules; on both kinds of schedule, rates were lower in the first component than in the rest of the ratio. When the sequence of stimuli was reversed, the duration of the post-reinforcement pause dropped markedly and the running rate in the initial component increased, but these effects gradually disappeared after the first reversal session. When the final chain stimulus was substituted for the first component stimulus but continued to appear in the final chain component as well, the pause duration dropped and remained at this lower level during subsequent sessions.     

Fixed-ratio and variable-ratio schedules of brief stimuli in second-order schedules of matching to sample

Eight pigeons matched to sample under second-order schedules of food reinforcement. Under fixed-interval unit schedules, the first correct match to occur after a given period of time was followed by the presentation of a brief stimulus. The termination of the last fixed-interval unit schedule was followed by food according to second-order fixed-ratio and variable-ratio schedules. In Experiment 1, as the number of fixed-interval unit schedules increased, long pauses occurred under the second-order fixed-ratio schedules, but not under the variable-ratio schedules. The similarity of performance measures such as local rate and accuracy indicated that the differences engendered by these two types of schedule are in the duration of the periods of not-responding. In Experiment 2, the addition of a brief stimulus at the end of each unit schedule in chained schedules that had different discriminative stimuli present for the duration of each unit did not substantially affect the performance, and long pauses continued to occur. However, few long pauses occurred under schedules with brief stimulus presentations alone. The most inaccurate performances were engendered by chained schedules without brief stimuli.     

Adjusting fixed-ratio schedules in the squirrel monkey

On an adjusting schedule of reinforcement, a parameter of the schedule is varied as a function of some characteristic of the animal's performance. In Experiment I, the fixed-ratio response requirement was varied as a function of the time that elapsed before the animal started responding in each fixed-ratio (initial pause). When initial pauses were shorter than a specified duration, the response requirement was increased; when they were longer than the specified duration, the response requirement was decreased. Specified durations of 1, 2, 4, 8, and 15 min were studied. The average response requirement maintained by each monkey was directly related to the length of the specified duration of initial pause. In Experiment II, the fixed-ratio response requirement was constant, but reinforcement occurred only when the initial pause was longer than a specified duration. The average durations of initial pauses were directly related to the length of the specified duration and to the response requirement. Meprobamate consistently decreased the average durations of initial pauses.     

Effects of schedule of reinforcement on a pentobarbital discrimination in rats.

The purpose of this study was to determine the effects of the schedule of reinforcement on a pentobarbital discrimination in rats. Five rats were trained to discriminate 10 mg/kg pentobarbital from saline under a multiple fixed-interval 180-s fixed-ratio 20 schedule of reinforcement. During both saline and pentobarbital training sessions, subjects emitted a higher percentage of correct responses under the fixed-ratio component as compared to the fixed-interval component of the multiple schedule. Determination of the pentobarbital dose-response curve under the fixed-ratio component resulted in a steep curve characterized by responding on the saline lever at low doses and on the drug lever at higher doses. Under the fixed-interval component, a graded dose-effect curve was produced, with considerable responding on both levers after intermediate doses of pentobarbital. The administration of phencyclidine and MK-801 resulted in an intermediate level of drug-lever responding for some subjects. Administration of d-amphetamine resulted in saline (nondrug) appropriate responding. The results of this study demonstrate that the schedule of reinforcement is a determinant of drug stimulus control, just as it is a determinant of other drug effects.     

Changing the response unit from a single peck to a fixed number of pecks in fixed-interval schedules

Each of three pigeons was studied first under a standard fixed-interval schedule. With the fixed interval held constant, the schedule was changed to a second-order schedule in which the response unit was the behavior on a small fixed-ratio schedule (first a fixed-ratio 10 and then a fixed-ratio 20 schedule). That is, every completion of the fixed-ratio schedule produced a 0.7-sec darkening of the key and reset the response count to zero for the next ratio. The first fixed-ratio completed after the fixed-interval schedule elapsed produced the 0.7-sec blackout followed immediately by food. These manipulations were carried out under two different fixed-interval durations for each bird ranging from 3 min to 12 min. The standard fixed-interval schedules produced the typical pause after reinforcement followed by responding at a moderate rate until the next reinforcement. The second-order schedules also engendered a pause after reinforcement, but responding occurred in bursts separated by brief pauses after each blackout. For a particular fixed-interval duration, post-reinforcement pauses increased slightly as the number of pecks in the response unit increased despite large differences in the rate and pattern of key pecking. Post-reinforcement pause increased with the fixed-interval duration under all response units. These data confirm that the allocation of time between pausing and responding is relatively independent of the rate and topography of responding after the pause.     

Altering the proportion of components in a mixed fixed-ratio schedule

Pigeons were trained under a schedule consisting of a number of fixed-ratio 100 components followed by a single fixed-ratio 10 component. The proportion of fixed-ratio 100 to fixed-ratio 10 components was varied according to several ascending and descending series within the range of 99:1 to 1:1. When this proportion was reduced to about 20:1 and below, the pause following each fixed-ratio 100 gradually decreased in length. Primes, a burst of responses at the start of the fixed-ratio 100 component, increased in frequency, and then decreased when the proportion became extremely low. Also, when the relative frequency of fixed-ratio 10 components was very high, primes were seldom observed in the first fixed-ratio 100 component following a fixed-ratio 10 component, but were distributed evenly throughout the remaining fixed-ratio 100 components.     

Second-order schedules of token reinforcement: effects of varying the schedule of food presentation

In the initial link of a complex schedule, one discriminative stimulus was presented and lever pressing produced tokens on fixed-ratio schedules. In the terminal link, signalled by a second discriminative stimulus, deposits of the tokens produced food. With two rats, the terminal link was presented after each sixth component schedule of token reinforcement was completed. With the other two rats, the terminal link was presented following the first component schedule completed after a fixed interval. During the terminal link, each token deposit initially produced food. The schedule of food presentation was subsequently increased such that an increasing number of token deposits in the terminal link was required for each food presentation. Rates of lever pressing in the initial link were inversely related to the schedule of food presentation in the terminal link. These results are similar to those of experiments that have varied schedules of food presentation in chained schedules. Rates and patterns of responding controlled throughout the initial link were more similar to those ordinarily controlled by second-order brief-stimulus schedules than to those controlled by comparable extended chained schedules.     

Responding under fixed-ratio and multiple fixed-interval fixed-ratio schedules of electric shock presentation

Squirrel monkeys, initially trained under a schedule of electric shock postponement and then under fixed-interval schedules of electric shock presentation, were studied under multiple fixed-interval fixed-ratio and under fixed-ratio schedules of shock presentation. Under the fixed-interval (10-min) component of the multiple schedule, a pause was followed by a gradual increase in responding to a rate maintained until shock presentation; under the fixed-ratio (3-, 10-, or 30-response) component of the multiple schedule, a brief pause was typically followed by a relatively high and uniform rate of responding until shock was presented. When the 60-sec timeout periods, which usually followed shock presentation, were eliminated from the multiple schedule for one monkey, responding was only transiently affected. In the one monkey studied, responding was maintained under a fixed-ratio schedule alone (with timeout periods), but rates of responding were lower than under the fixed-ratio component of the multiple schedule. Characteristic patterns of responding, similar to those engendered under schedules of food presentation or shock termination, can be maintained under fixed-ratio schedules of shock presentation; further, patterns of responding can be controlled by discriminative stimuli in multiple schedules.     

Contrast effects in multiple fixed-interval reinforcement schedules

Pigeons were exposed to a multiple fixed-interval one-minute fixed-interval three-minute schedule of reinforcement following training on either a multiple fixed-interval one-minute fixed-interval one-minute schedule or a multiple fixed-interval three-minute fixed-interval three-minute schedule. For all birds, large negative local contrast effects developed during the first of four three-minute intervals in a component; response rate was depressed and postreinforcement pause lengthened in this interval. Positive local contrast effects were evident during the first of 12 one-minute intervals in a component for five of six birds; at asymptote, the pause was very short and response rate slightly elevated during this interval. Overall positive contrast was generally transient and varied considerably across subjects, while overall negative contrast effects, if they occurred, appeared only after a large number of sessions.     

Differential reinstatement predicted by preextinction response rate

Reinstatement refers to the recovery of previously extinguished responding by the responseindependent delivery of a stimulus that was a reinforcer in training. Two experiments were conducted to examine relative reinstatement following the training of differential preextinction response rates, either with equal (Experiment 1) or unequal (Experiment 2) preextinction reinforcement rates. In Experiment 1, each of 3 pigeons first pecked at relatively high rates in the tandem variable-time 117-sec fixed-interval 3-sec component of a multiple schedule and at lower rates in a separate tandem variableinterval 117-sec fixed-time 3-sec component. Reinforcement rates were equal between components. Pecking then was extinguished in each component, before being reinstated under a multiple variabletime 120-sec variable-time 120-sec schedule. Greater reinstatement occurred in the component previously correlated with higher rates of pecking. In Experiment 2, in an initial condition, the mean rate of lever pressing for one group of 8 rats was significantly higher under a fixed-ratio 3 schedule than for another group of 8 rats under a fixed-ratio 1 schedule. Mean reinforcement rate was significantly higher for the group exposed to the fixed-ratio 1 schedule. For each group, lever pressing then was extinguished, before being reinstated under a variable-time 30-sec schedule. Significantly greater mean reinstatement occurred for the group previously exposed to the fixed-ratio 3 schedule. These results suggest that differential reinstatement may be predicted by preextinction response rate, perhaps independently of preextinction reinforcement rate.     

Conjunctive schedules of reinforcement II: response requirements and stimulus effects

Responding of three pigeons was maintained under conjunctive fixed-ratio, fixed-interval schedules where a key peck produced food after both schedule requirements were completed. The individual schedule requirements were then successively removed and reinstated with responding maintained under the following conditions: conjunctive fixed-ratio, fixed-time; fixed-time; and fixed-interval schedules. Patterns of responding changed in accord with the successive removal of the schedule requirements. Compared to the conjunctive fixed-ratio, fixed-interval schedule, pause duration increased and response rate decreased under conjunctive fixed-ratio, fixed-time schedules and under fixed-time schedules alone. Overall mean rates of responding were highest and pause duration lowest under fixed-interval schedules. When changes in the keylight colors were correlated with completion of the fixed-ratio, the end of the fixed-interval, or both of these conditions, the pattern of responding was modified and indicated a greater degree of control by the individual schedules. Although two birds showed large increases in interreinforcement time when they were initially exposed to the conjunctive schedule, when responding stabilized this measure was largely invariant for all birds across most schedule conditions.     

Effects of chlorpromazine on fixed-ratio responding: modification by fixed-interval discriminative stimuli.

Effects of chlorpromazine (1 to 100 mg/kg) were assessed on two pigeons' responding under various modifications of a multiple schedule of food delivery. During a fixed-interval component, the first response after 5 min produced food; during the subsequent, fixed-ratio component, the 30th response produced food. Modifications of the schedule entailed changes in stimulus conditions imposed during the fixed-ratio component that did not systematically alter characteristics of performance under non-drug conditions. In the first phase of the experiment, distinctive visual stimuli were correlated with each schedule component (conventional multiple schedule); chlorpromazine produced small decreases in fixed-ratio responding (20% at 30 mg/kg). When each response during the fixed-ratio component produced the stimulus correlated with the fixed-interval schedule (fixed-interval discriminative stimulus) for 1.2 s, effects of chlorpromazine were not different from those under the conventional multiple schedule. Chlorpromazine produced greater decreases in fixed-ratio responding (55% at 30 mg/kg) when either the first response of each fixed ratio changed the stimulus correlated with the fixed-ratio schedule to the fixed-interval discriminative stimulus for the remainder of the fixed-ratio component, or when the fixed-interval discriminative stimulus was presented independently of responding according to a matched temporal sequence. When the fixed-interval discriminative stimulus was present continuously during the fixed-ratio component (mixed schedule), chlorpromazine produced even more substantial decreases in fixed-ratio responding (greater than 80% at 30 mg/kg). Effects of chlorpromazine on fixed-interval responding were also modified by the schedules of fixed-interval discriminative stimulus presentation. The effects of chlorpromazine were a joint function of the stimuli prevailing during the multiple schedule and the degree to which responding influenced these stimuli.     

Quantification of rats' behavior during reinforcement periods

What is treated as a single unit of reinforcement often involves what could be called a reinforcement period during which two or more acts of ingestion may occur, and each of these may have associated with it a series of responses, some reflexive, some learned, that lead up to ingestion. Food-tray presentation to a pigeon is an example of such a “reinforcement period.” In order to quantify this behavior, a continuous-reinforcement schedule was used as the reinforcement period and was chained to a fixed-ratio schedule. Both fixed-ratio size and reinforcement-period duration were manipulated. Rats were used as subjects, food as reinforcement, and a lever press as the operant. Major findings included (a) a rapid decline in response rates during the first 15 to 20 seconds of the reinforcement periods, and (b) a strong positive relationship between these response rates and the size of the fixed ratio. Also revealed was a short scallop not normally found in fixed-ratio response patterns, whose length was a function of fixed-ratio size and reinforcement-period duration. It is suggested that rapidly fluctuating excitatory processes can account for many of these findings and that such processes are functionally significant in terms of behavioral compensation.     

A comparison of responding maintained under second-order schedules of intramuscular cocaine injection or food presentation in squirrel monkeys.

Key pressing by squirrel monkeys was maintained under second-order schedules of either intramuscular cocaine injection or food presentation. Under one schedule, each completion of a 10-response fixed-ratio unit produced a brief visual stimulus; the first fixed-ratio unit completed after 30 minutes elapsed produced the stimulus paired with either cocaine injection or food presentation. Generally, short pauses followed by high rates of responding were maintained within the fixed-ratio units, and responding was positively accelerated over the 30-minute interval. Under another schedule, each completion of a 3-minute fixed-interval unit produced the brief stimulus; completion of the 10th fixed-interval unit produced the stimulus paired with either cocaine injection or food presentation. Generally, short pauses followed by high rates of responding were maintained within the fixed-ratio units, and responding was positively accelerated over the 30-minute interval. Under another schedule, each completion of a 3-minute fixed-interval unit produced the brief stimulus; completion of the 10th fixed-interval unit produced the stimulus paired with either cocaine injection or food presentation. Rates of responding increased within the fixed-interval units, and to a greater extent over the entire 10 fixed-interval units. Patterns of responding depended more on the schedule of reinforcement than on whether cocaine or food maintained responding. Omitting the brief stimuli following all but the last fixed-ratio or fixed-interval units decreased average rates and altered the patterns of responding. Substituting a visual stimulus that was never paired with cocaine or food following all but the last fixed-ratio or fixed-interval units decreased response rates to a lesser extent and did not substantially alter patterns of responding. When the duration of the paired stimulus was varied from .3 to 30.0 seconds, the highest response rates occurred at intermediate durations (1.0 to 10.0 seconds). The manner in which the stimulus changes affected performances depended more on the schedule of reinforcement than on whether cocaine injection or food presentation maintained responding.     

Response-dependent shock in second-order fixed-ratio schedules of food presentation

Three pigeons key pecked under second-order schedules in which the completion of two successive fixed-ratio 50 components constituted a reinforcement cycle. Tandem, chained, and brief-stimulus second-order schedules were studied when completion of the initial fixed-ratio 50 component delivered brief intense electric shock in every nth reinforcement cycle and n assumed values between one and nine. During sessions without shock, the brief-stimulus (unpaired with food) schedule generated higher rates of responding in the initial component than did the tandem schedule. Electric shock engendered increased time to the fifth response and a repeated pause-run pattern of not responding and responding, particuuarly in the initial component, even with shock scheduled in every ninth reinforcement cycle. The results were consistent with those reported for shock of a shorter duration scheduled in every reinforcement cycle. The overall rate of responding decreased as a function of increasing shock density and was lower in brief-stimulus than in tandem schedules.     

The contribution of an added counter to a fixed-ratio schedule

Although previous research showed that a visual counter increased the rate of responding on a large fixed-ratio schedule, a theoretical analysis of the factors responsible for fixed-ratio performance suggests that the primary control by number of responses since reinforcement is to weaken the performance. The present experiment employed a multiple schedule in which the same fixed-ratio value alternated with and without an added counter. It tested the hypothesis that the differential reinforcement of high-rate responding masked the attenuation of the fixed-ratio performance from the unoptimal discriminative control produced by the fixed relation between number of responses and reinforcement. In the present experiment the postreinforcement pause was consistently longer in the components with the added counter, while running rates remained comparable between the components of the multiple schedule. Both components of the multiple schedule involved differential reinforcement of high-rate responding while only the components with the added counter amplified the discriminative control by number of pecks since reinforcement.     

Mirror pecking and timeout under a multiple fixed-ratio schedule of food delivery

Pigeons were trained to peck a key under a multiple fixed-ratio 25 fixed-ratio 175 schedule of food presentation. In the first condition, either a mirror or the opportunity to produce a 30-second timeout were available. In a second condition, mirror and timeout availability were reversed for the two groups. Following a return to the initial condition, mirror and timeout keys were presented together for all birds. Mirror and timeout responses occurred predominantly in the pause in the larger fixed-ratio component, regardless of whether the opportunities for the two responses were available singly or together. Mirror responding occurred in a greater proportion of the pauses than did timeouts. When the opportunities for both mirror pecking and timeout were available concurrently, they occurred with probabilities similar to those under the single conditions. Within the pause itself, mirror responses most frequently occurred immediately after reinforcement. Timeouts occurred most frequently toward the end of the pause, and some timeouts occurred in the early part of the run. Longer preratio pausing occurred in the larger fixed-ratio component in the conditions in which the mirror was present, whether or not any mirror pecks were recorded.     

Schedule-induced mirror responding in the pigeon

Two pigeons that were previously exposed to a multiple schedule of reinforcement in the presence of a stuffed and a live pigeon, and two of three naive pigeons, responded on a mirror during exposure to multiple fixed-ratio, fixed-ratio schedules of reinforcement for key pecking. Both the topography and temporal pattern of mirror responding were comparable to schedule-induced “attack” on live and stuffed targets. Rate of target responding was reduced when either the mirror was covered with paper or when the multiple schedule was removed. A reversal in the relationship between reinforcement schedules and discriminative stimuli demonstrated that mirror responding was controlled by the stimulus correlated with the higher fixed-ratio schedule. With one component of the multiple schedule held constant at fixed ratio 25 and the ratio requirement of the other component varying from 25 to 150, there was an inverted U-shaped relationship between rate of mirror responding and fixed-ratio schedule in the varied component. As in Flory's study (1969b) there was an inverted U-shaped relationship between target responding and inter-food intervals. The combined results of these studies suggest that the relationship between rate of target responding and reinforcement schedules is controlled primarily by the inter-food intervals resulting from the schedules.     

Effects of multiple versus chained schedules on stereotypy and item engagement

We evaluated rates of automatically reinforced stereotypy and item engagement for 2 children with autism under multiple and chained schedules in a multielement design. Each schedule included components during which stereotypy was blocked (S–) or allowed (S+), and we used colored cards as schedule‐correlated stimuli. We report rates of stereotypy and item engagement during S– and S+ components, as well as the percentage of component time that elapsed before the first instances of stereotypy and item engagement. We observed less stereotypy and more consistent item engagement during chained‐schedule sessions, and stimulus control of stereotypy and item engagement was established with the chained schedule. A subsequent concurrent‐chains analysis revealed participant preference for the chained schedule. These results highlight the importance of contingent access to stereotypy when therapists attempt to gain stimulus control of stereotypy and increase functional item engagement.     

Contrast, component duration, and the following schedule of reinforcement

Experiment 1 investigated component duration in a four-component multiple schedule designed to separate the effects of the preceding schedule of reinforcement from those of the following schedule of reinforcement. The preceding schedule in the preceding component had no consistent effect regardless of component duration. The following schedule was a powerful determinant of behavior, however, with higher response rates resulting from a following period of extinction. Moreover, the effect of the following schedule was greater with short component durations, which strongly suggests that component-duration effects previously found with multiple schedules are due generally to variation in the degree of control by the following schedule. Experiments 2--4 investigated the basis of the effects of the following schedule. In some situations differential responding in the following component was the controlling variable, but in others it was differential reinforcement in the following component. The results are consistent with the view that response rate during a stimulus is inversely related to the "value" of the following component, where the calculation of value must include both primary and conditioned reinforcement.