Effects of ratio reinforcement schedules on discrimination performance by Japanese monkeys

In Experiment 1, Japanese monkeys were trained on three conditional position-discrimination problems with colors as the conditional cues. Within each session, each problem was presented for two blocks of ten reinforcements; correct responses were reinforced under continuous-reinforcement, fixed-ratio 5, and variable-ratio 5 schedules, each assigned to one of the three problems. The assignment of schedules to problems was rotated a total of three times (15 sessions per assignment) after 30 sessions of acquisition training. Accuracy of discrimination increased to a moderate level with fewer trials under CRF than under ratio schedules. In contrast, the two ratio schedules, fixed and variable, were more effective in maintaining accurate discrimination than was CRF. With further training, as asymptotes were reached, accuracy was less affected by the schedule differences. These results demonstrated an interaction between the effects of reinforcement schedules and the level of acquisition. In Experiment 2, ratio sizes were gradually increased to 30. Discrimination accuracy was maintained until the ratio reached 20; ratio 30 strained the performance. Under FR conditions, accuracy increased as correct choice responses cumulated after reinforcement.     

Second-order schedules of token reinforcement with pigeons: effects of fixed- and variable-ratio exchange schedules

Pigeons' key pecks produced food under second-order schedules of token reinforcement, with light-emitting diodes serving as token reinforcers. In Experiment 1, tokens were earned according to a fixed-ratio 50 schedule and were exchanged for food according to either fixed-ratio or variable-ratio exchange schedules, with schedule type varied across conditions. In Experiment 2, schedule type was varied within sessions using a multiple schedule. In one component, tokens were earned according to a fixed-ratio 50 schedule and exchanged according to a variable-ratio schedule. In the other component, tokens were earned according to a variable-ratio 50 schedule and exchanged according to a fixed-ratio schedule. In both experiments, the number of responses per exchange was varied parametrically across conditions, ranging from 50 to 400 responses. Response rates decreased systematically with increases in the fixed-ratio exchange schedules, but were much less affected by changes in the variable-ratio exchange schedules. Response rates were consistently higher under variable-ratio exchange schedules than tinder comparable fixed-ratio exchange schedules, especially at higher exchange ratios. These response-rate differences were due both to greater pre-ratio pausing and to lower local rates tinder the fixed-ratio exchange schedules. Local response rates increased with proximity to food under the higher fixed-ratio exchange schedules, indicative of discriminative control by the tokens.     

A comparison of ratio and interval reinforcement schedules with comparable interreinforcement times

Pigeons were trained to peck keys on fixed-ratio and fixed-interval schedules of food reinforcement. Both schedules produced a pattern of behavior characterized as pause and run, but the relation of pausing to time between reinforcers differed for the two schedules even when mean time between reinforcers was the same. Pausing in the fixed ratio occupied less of the time between reinforcers for shorter interreinforcer times. For two of three birds, the relation was reversed at longer interreinforcer times. As an interreinforcer time elapsed, there was an increasing tendency to return to responding for the fixed interval, but a roughly constant tendency to return to responding for the fixed-ratio schedule. In Experiment 1 these observations were made for both single-reinforcement schedules and multiple schedules of fixed-ratio and fixed-interval reinforcement. In Experiment 2 the observations were extended to a comparison of fixed-ratio versus variable-interval reinforcement schedules, where the distribution of interreinforcement times in the variable interval approximated that for the fixed ratio.     

Tests of behavior momentum in simple and multiple schedules with rats and pigeons.

Four experiments examined the relationship between rate of reinforcement and resistance to change in rats' and pigeons' responses under simple and multiple schedules of reinforcement. In Experiment 1, 28 rats responded under either simple fixed-ratio, variable-ratio, fixed-interval, or variable-interval schedules; in Experiment 2, 3 pigeons responded under simple fixed-ratio schedules. Under each schedule, rate of reinforcement varied across four successive conditions. In Experiment 3, 14 rats responded under either a multiple fixed-ratio schedule or a multiple fixed-interval schedule, each with two components that differed in rate of reinforcement. In Experiment 4, 7 pigeons responded under either a multiple fixed-ratio or a multiple fixed-interval schedule, each with three components that also differed in rate of reinforcement. Under each condition of each experiment, resistance to change was studied by measuring schedule-controlled performance under conditions with prefeeding, response-independent food during the schedule or during timeouts that separated components of the multiple schedules, and by measuring behavior under extinction. There were no consistent differences between rats and pigeons. There was no direct relationship between rates of reinforcement and resistance to change when rates of reinforcement varied across successive conditions in the simple schedules. By comparison, in the multiple schedules there was a direct relationship between rates of reinforcement and resistance to change during most tests of resistance to change. The major exception was delivering response-independent food during the schedule; this disrupted responding, but there was no direct relationship between rates of reinforcement and resistance to change in simple- or multiple-schedule contexts. The data suggest that rate of reinforcement determines resistance to change in multiple schedules, but that this relationship does not hold under simple schedules.     

Interaction of methadone, reinforcement history, and variable-interval performance.

In the present study, we examined how a reinforcement schedule history that generated high or low rates of responding influenced the effects of acute (Experiment 1) and chronic (Experiment 2) methadone administration. Initially, key-peck responses of pigeons were maintained under a variable-interval 90-s schedule of food presentation, and a methadone dose-response curve was determined with doses of 0.6, 1.2, and 2.4 mg/kg. The pigeons were then exposed, for at least 40 sessions, to either a fixed-ratio 50 schedule or a differential-reinforcement-of-low-rate 10-s schedule, or were given continued exposure to the variable-interval schedule. The methadone dose-response curve was redetermined after all pigeons again were responding under the variable-interval schedule. The effects of two different daily methadone doses (9.0 and 12.0 mg/kg/day) and withdrawal precipitated by naloxone also were assessed. Experience with a fixed-ratio or differential reinforcement of low rate schedule did not result in significantly different response rates under the variable-interval schedule and, in general, the acute effects of methadone did not have differential effects correlated with schedule history. However, for 2 of 4 subjects the rate-decreasing effects of methadone on rates of key pecking were greater following a history of low-rate responding, suggesting a possible interaction between schedule history and effects of methadone. Daily methadone administration under the variable-interval schedule revealed that pigeons with experience under the differential reinforcement of low rate schedule developed more rapid and complete tolerance to the rate-decreasing effects of methadone. Three of the 4 subjects in this group showed rate increases above drug-free baselines during chronic methadone dosing. Pigeons with a history of fixed-ratio responding also developed tolerance to the rate-decreasing effects of methadone but without the subsequent rate increases seen by subjects with low-rate histories. No subjects with variable-interval histories showed complete recovery of drug-free baselines, suggesting that interpolated training under other schedules may attenuate the rate-altering effects of chronically administered drugs. Naloxone (1.0 mg/kg), administered during the chronic methadone phase, resulted in greater disruption of responding by pigeons with a history of low-rate responding, as compared to subjects in the other two groups.(ABSTRACT TRUNCATED AT 400 WORDS)     

Factors influencing responding under multiple schedules of conditioned and unconditioned reinforcement

Two experiments examined pigeons' responses under multiple schedules of conditioned and unconditioned reinforcement. In one component, responses produced food according to a fixed-interval schedule; in a second component, responses produced brief stimuli according to a fixed-ratio schedule. When brief-stimulus presentations were paired with food in the first component, rates in the second component were usually higher than 10 responses per minute. When pairing in the first component was eliminated, responding continued to be maintained in the second component. Elimination of food presentation from the first component substantially decreased responding in the second component, even though the brief stimulus had not been paired with food. Experiment II demonstrated that response rate was affected by the duration of both the second component and the brief stimulus. The results suggest that three conditions are important in maintaining responding with brief-stimulus presentations: (1) pairing the brief stimulus, at least initially, with food, (2) maintaining unconditioned reinforcement in one component, and (3) employing optimal brief-stimulus and component durations.     

Choice for aperiodic versus periodic ratio schedules: A comparison of concurrent and concurrent-chain procedures

Choice between mixed-ratio schedules, consisting of equiprobable ratios of 1 and 99 responses per reinforcement, and fixed-ratio schedules of food reinforcement was assessed by two commonly used procedures: concurrent schedules and concurrent-chains schedules. Rats were trained under concurrent fixed-ratio mixed-ratio schedules, in which both ratio schedules were simultaneously available, and under a concurrent-chains schedule, in which access to one of the mutually exclusive ratio schedules comprising the terminal links was contingent on a single “choice” response. The distribution of responses between the two ratio schedules was taken as the choice proportion under the concurrent procedure, and the distribution of “choice” responses was taken as the choice proportion under the concurrent-chains procedure. Seven of eight rats displayed systematic choice; of those, each displayed nearly exclusive choice for fixed-ratio 35 to the mixed-ratio schedule under the concurrent procedure, but each displayed nearly exclusive choice for the mixed-ratio schedule to fixed-ratio 35 under the concurrent-chains procedure. Thus, preference for a fixed or a mixed schedule of reinforcement depended on the procedure used to assess preference.     

Schedule-induced defecation by rats during ratio and interval schedules of food reinforcement.

Lever pressing in rats was maintained by continuous and intermittent schedules of food while defecation was monitored. In Experiment 1, reinforcement densities were matched across variable-ratio and variable-interval schedules for three pairs of rats. Defecation occurred in all 3 rats on the variable-ratio schedule and in all 3 rats on the yoked variable-interval schedule. In Experiment 2, fixed-ratio and fixed-interval schedules with similar reinforcement densities maintained lever pressing. Defecation occurred in 3 of 4 rats on the fixed-ratio schedule and in 4 of 4 rats on the fixed-interval schedule. Almost no defecation occurred during continuous reinforcement in either experiment. These results demonstrate that defecation may occur during both ratio and interval schedules and that the inter-reinforcement interval is more important than the behavioral requirements of the schedule in generating schedule-induced defecation.     

Concurrent ratio schedules: Fixed vs. variable response requirements

Rats were trained on concurrent fixed-ratio variable-ratio or concurrent fixed-ratio mixed-ratio schedules of food reinforcement. The variable-ratio schedule was composed of an arithmetic sequence of 11 ratios that averaged 50; the mixed-ratio schedule consisted of equiprobable ratios of 1 and 99. Fixed-ratio values, varied over experimental conditions, included 25, 35, 50, 60, and 99. The proportion of responses and time allocated to the variable- or mixed-ratio schedule increased as the size of the fixed ratio increased. For most subjects, higher proportions of responses and time were maintained on the fixed-ratio schedule at fixed-ratio values of 25 and 35; higher proportions of responses and time were maintained on the variable- or mixed-ratio schedule at fixed-ratio values of 50 or higher. On concurrent variable-ratio fixed-ratio schedules, the tendency for responding to be maintained exclusively by one schedule was related to the difference in local reinforcement rates obtained from those schedules. Exclusive responding was approximated when the difference in local reinforcement rates obtained from those schedules was large; responding was more evenly distributed between the schedules as the difference in the rates at which reinforcement was obtained from each decreased.     

Effects of reinforcement amount on attack induced under a fixed-interval schedule in pigeons

Key pecking by pigeons was maintained on a chained fixed-interval 4-min (12-min for 1 subject) fixed-ratio 1 schedule of food presentation. Attacks toward a restrained and protected conspecific were recorded. In the first experiment, the amount of food presented per interval was manipulated across phases by varying the number of fixed ratios required in the terminal link of the chain. Measures of attack for all pigeons during the fixed-interval component increased monotonically as a function of food amount. In the second experiment, two different food amounts alternated within each experimental session under a multiple schedule. For both pigeons in this experiment, measures of attack were higher during the component that delivered the larger food amount per interval. The differences in levels of attack induced by the two food amounts in Experiment 2, however, were not as great as in Experiment 1; apparently this was because attack during the first interval of each component was controlled in part (P-5626) or entirely (P-7848) by the reinforcement amount delivered at the end of the previous component. Attack was also a function of the location of the interfood interval within the session. For both pigeons, attack tended to decrease throughout the session. The results of both experiments suggest that attack is an increasing function of reinforcement amount under fixed-interval schedules, but that this function may be influenced by the manner in which reinforcement amount is manipulated, by the duration of the interfood interval, and by the location of the interfood interval within the experimental session. In general, these results are compatible with theories of induced attack and other schedule-induced behavior that emphasize aversive after-effects of reinforcement presentation.     

Food deliveries during the pause on fixed-interval schedules

Pigeons were trained on fixed-interval schedules of food delivery. In Experiments I and II, the fixed interval was initiated by the previous fixed-interval reinforcer; in Experiment III, the fixed interval was initiated by the first key peck following the preceding fixed-interval reinforcer (a chain fixed-ratio one, fixed-interval schedule). During the postreinforcement pause, variable-time schedules delivered food independent of any specific response. Rate of food delivery during the pause had only small effects on pause duration in Experiments I and II. In Experiment III, however, pause duration increased systematically with the rate of food delivery during the pause. These data suggest that the momentary proximity to reinforcement delivered via the fixed-interval schedule exerts potent control over pause termination. Additional analysis revealed that pause termination was unaffected by the intermittent delivery of food during the pause. Such data suggest that the temporal control by fixed-interval schedules is highly resistant to interference.     

Response requirements as constraints on output

Two experiments studied how added response requirements affected fixed-interval schedule performance. Experiment 1 involved tandem fixed-interval fixed-ratio schedules, and Experiment 2 studied conjunctive fixed-interval fixed-ratio schedules. In both, pigeons' output, defined as overall response rate or as responses during the interval, first increased and then decreased as the ratio was raised. With small ratio requirements, the frequency of reinforcement in time either did not change or decreased slightly. With progressively larger ratios, reinforcement frequency decreased consistently. Alternative explanations were discussed. The first, a reinforcement theory account, was that response strength is an increasing monotonic function of both the response requirement and reinforcement frequency, and the bitonic output function represents interacting effects. Increases in the response requirement accompanied by small changes in reinforcement frequency enhance output, but further increases result in large enough decrements in reinforcement frequency so that output is lowered. The second explanation does not view reinforcement as a basic process but, instead, derives from concepts of economics and conservation. Organisms allocate their behavior among alternatives so as to maximize value, where value is a function of the responses that can occur in a given situation under the set of restrictions imposed by particular schedules. One form of this theory explicitly predicts that output is a bitonic function of ratio requirements in simple ratio schedules. However, it was not clear that this model could explain the present effects involving joint ratio and interval schedule restrictions.     

The contribution of an added counter to a fixed-ratio schedule

Although previous research showed that a visual counter increased the rate of responding on a large fixed-ratio schedule, a theoretical analysis of the factors responsible for fixed-ratio performance suggests that the primary control by number of responses since reinforcement is to weaken the performance. The present experiment employed a multiple schedule in which the same fixed-ratio value alternated with and without an added counter. It tested the hypothesis that the differential reinforcement of high-rate responding masked the attenuation of the fixed-ratio performance from the unoptimal discriminative control produced by the fixed relation between number of responses and reinforcement. In the present experiment the postreinforcement pause was consistently longer in the components with the added counter, while running rates remained comparable between the components of the multiple schedule. Both components of the multiple schedule involved differential reinforcement of high-rate responding while only the components with the added counter amplified the discriminative control by number of pecks since reinforcement.     

Differential reinstatement predicted by preextinction response rate

Reinstatement refers to the recovery of previously extinguished responding by the responseindependent delivery of a stimulus that was a reinforcer in training. Two experiments were conducted to examine relative reinstatement following the training of differential preextinction response rates, either with equal (Experiment 1) or unequal (Experiment 2) preextinction reinforcement rates. In Experiment 1, each of 3 pigeons first pecked at relatively high rates in the tandem variable-time 117-sec fixed-interval 3-sec component of a multiple schedule and at lower rates in a separate tandem variableinterval 117-sec fixed-time 3-sec component. Reinforcement rates were equal between components. Pecking then was extinguished in each component, before being reinstated under a multiple variabletime 120-sec variable-time 120-sec schedule. Greater reinstatement occurred in the component previously correlated with higher rates of pecking. In Experiment 2, in an initial condition, the mean rate of lever pressing for one group of 8 rats was significantly higher under a fixed-ratio 3 schedule than for another group of 8 rats under a fixed-ratio 1 schedule. Mean reinforcement rate was significantly higher for the group exposed to the fixed-ratio 1 schedule. For each group, lever pressing then was extinguished, before being reinstated under a variable-time 30-sec schedule. Significantly greater mean reinstatement occurred for the group previously exposed to the fixed-ratio 3 schedule. These results suggest that differential reinstatement may be predicted by preextinction response rate, perhaps independently of preextinction reinforcement rate.     

Preference in concurrent variable-interval fixed-ratio schedules

Five pigeons were trained on concurrent variable-interval fixed-ratio schedules in three experiments. Experiment 1 used two variable-interval schedules and one fixed-ratio schedule, and the ratio requirement was varied. Using the generalized matching law, sensitivity to reinforcement was close to 1.0, but performance was biased toward the variable-interval schedule with the lower reinforcement rate. In Experiment 2, which used one variable-interval and one fixed-ratio schedule, the interval schedule was varied. All birds showed sensitivities to reinforcement of less than 1.0 and of less than the values obtained in Experiment 1. The performance was also biased toward the fixed-ratio schedule. Because the generalized matching law could not account for the differences in the data from Experiments 1 and 2, an extension of this law was suggested and successfully tested in Experiment 3. The proposed dual-sensitivity model was also shown to clarify some previously reported results.     

Effects of reinforcement magnitude and ratio values on behaviour maintained by a cyclic ratio schedule of reinforcement

In Experiments 1 and 2, lever pressing by rats was reinforced on a cyclic ratio schedule of food reinforcement, comprising a repeated sequence of fixed-ratio component schedules. Reinforcement magnitude was varied, on occasional sessions in Experiment 1 and across blocks of sessions in Experiment 2, from one to two or three 45-mg food pellets. In the one-pellet condition, post-reinforcement pauses increased with component schedule value. At higher magnitudes, post-reinforcement pauses increased, and overall response rates declined. Response rate on component schedules was a decreasing linear function of the obtained rate of reinforcement in all conditions. Plotted against component schedule value, response rate increased exponentially to an asymptote that decreased when reinforcement magnitude increased. These findings are consistent with regulatory accounts of food reinforced behaviour. In Experiment 3, rats were trained under a cyclic ratio schedule comprising fixed-ratio components including higher values, and some inverted U-shaped response functions were obtained. Those rats that did not showthis relationship were trained on cyclic ratios with even higher values, and all showed inverted U-shaped response functions. This suggests that behaviour on cyclic ratio schedules can reflect activating of reinforcement as well as the satiating effects seen in Experiments 1 and 2.     

Preference between variable-ratio and fixed-ratio schedules: local and extended relations.

Although it has repeatedly been demonstrated that pigeons, as well as other species, will often choose a variable schedule of reinforcement over an equivalent (or even richer) fixed schedule, the exact nature of that controlling relation has yet to be fully assessed. In this study pigeons were given repeated choices between concurrently available fixed-ratio and variable-ratio schedules. The fixed-ratio requirement (30 responses) was constant throughout the experiment, whereas the distribution of individual ratios making up the variable-ratio schedule changed across phases: The smallest and largest of these components were varied gradually, with the mean variable-ratio requirement constant at 60 responses. The birds' choices of the variable-ratio schedule tracked the size of the smallest variable-ratio component. A minimum variable-ratio component at or near 1 produced strong preference for the variable-ratio schedule, whereas increases in the minimum variable-ratio component resulted in reduced preference for the variable-ratio schedule. The birds' behavior was qualitatively consistent with Mazur's (1984) hyperbolic model of delayed reinforcement and could be described as approximate maximizing with respect to reinforcement value.     

Competitive fixed-interval performance in humans

Two persons responded in the same session in separate cubicles, but under a single schedule of reinforcement. Each time reinforcement was programmed, only the first response to occur, that is, the response of only one of the subjects, was reinforced. “Competitive” behavior that developed under these conditions was examined in three experiments. In Experiment 1 subjects responded under fixed-interval (FI) 30-s, 60-s, and 90-s schedules of reinforcement. Under the competition condition, relative to baseline conditions, the response rates were higher and the pattern was “break-and-run.” In Experiment 2, subjects were exposed first to a conventional FI schedule and then to an FI competition schedule. Next, they were trained to respond under either a differential-reinforcement-of-low-rate (DRL) or fixed-ratio (FR) schedule, and finally, the initial FI competition condition was reinstated. In this second exposure to the FI competition procedure, DRL subjects responded at lower rates than were emitted during the initial exposure to that condition and FR subjects responded at higher rates. For all subjects, however, responding gradually returned to the break-and-run pattern that had occurred during the first FI competition condition. Experiment 3 assessed potential variables contributing to the effects of the competitive FI contingencies during Experiments 1 and 2. Subjects were exposed to FI schedules where (a) probability of reinforcement at completion of each fixed interval was varied, or (b) a limited hold was in effect for reinforcement. Only under the limited hold was responding similar to that observed in previous experiments.     

The long-term effect of high- and low-rate responding histories on fixed-interval responding in rats

Tell rats were given extended lever-press training on a fixed-interval (FI) 30-s food reinforcement schedule from the outset or following exposure to one or two previous reinforcement schedules. For 4 rats the previots schedule was either fixed-ratio 20, which generated high response rates, or differential-reinforcement-of-low-rate 20 s, which produced low response rates. For 4 additional rats the extended training on FI 30 s was preceded by experience with two schedules: fixed-ratio 20 followed by differential-reinforcement-of-low-rate 20 s; or the same two schedules in the reverse order. Fixed-interval response rates were initially affected by the immediately preceding schedule, but after 80 to 100 sessions, all traces of prior schedule history had disappeared. The results also showed no long-term effect of schedule history on the interfood-interval patterns of responding on the FI 30-s schedule. These results support one of the most central tenets of the experimental analysis of behavior: control by the immediate consequences of behavior.     

Fixed-ratio and variable-ratio schedules of brief stimuli in second-order schedules of matching to sample

Eight pigeons matched to sample under second-order schedules of food reinforcement. Under fixed-interval unit schedules, the first correct match to occur after a given period of time was followed by the presentation of a brief stimulus. The termination of the last fixed-interval unit schedule was followed by food according to second-order fixed-ratio and variable-ratio schedules. In Experiment 1, as the number of fixed-interval unit schedules increased, long pauses occurred under the second-order fixed-ratio schedules, but not under the variable-ratio schedules. The similarity of performance measures such as local rate and accuracy indicated that the differences engendered by these two types of schedule are in the duration of the periods of not-responding. In Experiment 2, the addition of a brief stimulus at the end of each unit schedule in chained schedules that had different discriminative stimuli present for the duration of each unit did not substantially affect the performance, and long pauses continued to occur. However, few long pauses occurred under schedules with brief stimulus presentations alone. The most inaccurate performances were engendered by chained schedules without brief stimuli.