The effects of reinforcer magnitude in the preceding and upcoming ratios on between‐ratio pausing in multiple,mixed, and single fixed‐ratio schedules

Hens responded under multiple fixed‐ratio schedules with equal response requirements and either a 1‐s or a 6‐s reinforcer. Upcoming reinforcer size was indicated by key color. Components were presented in a quasirandom series so that all four component transitions occurred. Postreinforcement pauses were affected by the upcoming and preceding reinforcer size, with longer pauses after large reinforcers followed by small reinforcers than when followed by large ones, and longer pauses after small reinforcers that were followed by small reinforcers rather than large ones. Pauses increased with fixed‐ratio size and the effects of reinforcer size were larger the larger the ratio. When reinforcer size was not signaled—mixed fixed‐ratio schedules—pauses were shorter after small than after large reinforcers. Signalling the upcoming reinforcer attenuated the effect of the previous reinforcer size on pause duration when small was followed by small and when either small or large by large, but enhanced the effect when large was followed by small. There was no effect of reinforcer size on pause duration when single fixed‐ratio schedules were arranged. The effects of reinforcer size on pauses depends on the size and range of the fixed ratios as well as the exact procedures used in the study.     

Assessing functions of stimuli associated with rich‐to‐lean transitions using a choice procedure

Under fixed‐ratio schedules, transitions from more to less favorable conditions of reinforcement (rich‐to‐lean transitions) usually generate extended pausing. One possible explanation for this effect is that stimuli associated with rich‐to‐lean transitions are aversive and, thus, extended pausing functions as escape. The purpose of this study was to characterize further the aversive function of different transitions, and the stimuli associated with them, by allowing pigeons to choose to complete select ratios in the presence of either a mixed‐schedule stimulus or a transition‐specific multiple‐schedule stimulus. The mixed schedule was preferred during transitions that signaled an upcoming lean reinforcer (rich‐to‐lean and lean‐to‐lean), whereas the multiple schedule was preferred during transitions that signaled an upcoming rich reinforcer (lean‐to‐rich and rich‐to‐rich). These findings support the notion that stimuli associated with rich‐to‐lean (and to some extent lean‐to‐lean) transitions can function aversively; whereas stimuli associated with other transitions (e.g., lean‐to‐rich and rich‐to‐rich) can function as conditioned reinforcers. When the opportunity to choose between schedule‐correlated stimuli was available, however, choice latency was controlled exclusively by the multiple‐schedule stimulus. That is, the opportunity to select the mixed schedule did not attenuate rich‐to‐lean pauses, suggesting that extended pausing may be more than simply escape.     

Reinforcement magnitude and pausing on progressive-ratio schedules

Rats responded under progressive-ratio schedules for sweetened milk reinforcers; each session ended when responding ceased for 10 min. Experiment 1 varied the concentration of milk and the duration of postreinforcement timeouts. Postreinforcement pausing increased as a positively accelerated function of the size of the ratio, and the rate of increase was reduced as a function of concentration and by timeouts of 10 s or longer. Experiment 2 varied reinforcement magnitude within sessions (number of dipper operations per reinforcer) in conjunction with stimuli correlated with the upcoming magnitude. In the absence of discriminative stimuli, pausing was longer following a large reinforcer than following a small one. Pauses were reduced by a stimulus signaling a large upcoming reinforcer, particularly at the highest ratios, and the animals tended to quit responding when the past reinforcer was large and the stimulus signaled that the next one would be small. Results of both experiments revealed parallels between responding under progressive-ratio schedules and other schedules containing ratio contingencies. Relationships between pausing and magnitude suggest that ratio pausing is under the joint control of inhibitory properties of the past reinforcer and excitatory properties of stimuli correlated with the upcoming reinforcer, rather than under the exclusive control of either factor alone.     

AGGRESSION AS POSITIVE REINFORCEMENT IN MICE UNDER VARIOUS RATIO‐ AND TIME‐BASED REINFORCEMENT SCHEDULES

There is evidence suggesting aggression may be a positive reinforcer in many species. However, only a few studies have examined the characteristics of aggression as a positive reinforcer in mice. Four types of reinforcement schedules were examined in the current experiment using male Swiss CFW albino mice in a resident—intruder model of aggression as a positive reinforcer. A nose poke response on an operant conditioning panel was reinforced under fixed‐ratio (FR 8), fixed‐interval (FI 5‐min), progressive ratio (PR 2), or differential reinforcement of low rate behavior reinforcement schedules (DRL 40‐s and DRL 80‐s). In the FR conditions, nose pokes were maintained by aggression and extinguished when the aggression contingency was removed. There were long postreinforcement pauses followed by bursts of responses with short interresponse times (IRTs). In the FI conditions, nose pokes were maintained by aggression, occurred more frequently as the interval elapsed, and extinguished when the contingency was removed. In the PR conditions, nose pokes were maintained by aggression, postreinforcement pauses increased as the ratio requirement increased, and responding was extinguished when the aggression contingency was removed. In the DRL conditions, the nose poke rate decreased, while the proportional distributions of IRTs and postreinforcement pauses shifted toward longer durations as the DRL interval increased. However, most responses occurred before the minimum IRT interval elapsed, suggesting weak temporal control of behavior. Overall, the findings suggest aggression can be a positive reinforcer for nose poke responses in mice on ratio‐ and time‐based reinforcement schedules.     

Extended pausing by humans on multiple fixed-ratio schedules with varied reinforcer magnitude and response requirements

We conducted three experiments to reproduce and extend Perone and Courtney's (1992) study of pausing at the beginning of fixed-ratio schedules. In a multiple schedule with unequal amounts of food across two components, they found that pigeons paused longest in the component associated with the smaller amount of food (the lean component), but only when it was preceded by the rich component. In our studies, adults with mild intellectual disabilities responded on a touch-sensitive computer monitor to produce money. In Experiment 1, the multiple-schedule components differed in both response requirement and reinforcer magnitude (i.e., the rich component required fewer responses and produced more money than the lean component). Effects shown with pigeons were reproduced in all 7 participants. In Experiment 2, we removed the stimuli that signaled the two schedule components, and participants' extended pausing was eliminated. In Experiment 3, to assess sensitivity to reinforcer magnitude versus fixed-ratio size, we presented conditions with equal ratio sizes but disparate magnitudes and conditions with equal magnitudes but disparate ratio sizes. Sensitivity to these manipulations was idiosyncratic. The present experiments obtained schedule control in verbally competent human participants and, despite procedural differences, we reproduced findings with animal participants. We showed that pausing is jointly determined by past conditions of reinforcement and stimuli correlated with upcoming conditions.     

Fixed-ratio pausing: Joint effects of past reinforcer magnitude and stimuli correlated with upcoming magnitude

Pigeons responded on fixed-ratio schedules ending in small or large reinforcers (grain presentations of different duration) interspersed within each session. In mixed-schedule conditions, the response key was lit with a single color throughout the session, and pausing was directly related to the past reinforcer (longer pauses after large reinforcers than after small ones). In multiple-schedule conditions, different colors accompanied the ratios ending in small and large reinforcers, and pausing was affected by the upcoming reinforcer as well as the past one. Pauses were shorter before large reinforcers than before small ones, but they continued to be longer after large reinforcers than after small ones. The influence of the past reinforcer was modulated by the magnitude of the upcoming reinforcer; in the presence of the stimulus before the small reinforcer, the effect of the past reinforcer was enhanced relative to its effect in the stimulus before the large reinforcer. These results show that pausing between ratios is jointly determined by two competing factors: past conditions of reinforcement and stimuli correlated with upcoming conditions.     

Escape from rich‐to‐lean transitions: Stimulus change and timeout

Extended pausing during discriminable transitions from rich‐to‐lean conditions can be viewed as escape (i.e., rich‐to‐lean transitions function aversively). In the current experiments, pigeons’ key pecking was maintained by a multiple fixed‐ratio fixed‐ratio schedule of rich or lean reinforcers. Pigeons then were provided with another, explicit, mechanism of escape by changing the stimulus from the transition‐specific stimulus used in the multiple schedule to a mixed‐schedule stimulus (Experiment 1) or by producing a period of timeout in which the stimulus was turned off and the schedule was suspended (Experiment 2). Overall, escape was under joint control of past and upcoming reinforcer magnitudes, such that responses on the escape key were most likely during rich‐to‐lean transitions, and second‐most likely during lean‐to‐lean transitions. Even though pigeons pecked the escape key, they paused before doing so, and the latency to begin the fixed ratio (i.e., the pause) remained extended during rich‐to‐lean transitions. These findings suggest that although the stimulus associated with rich‐to‐lean transitions functioned aversively, pausing is more than simply escape responding from the stimulus.     

Tolerance to effects of cocaine on behavior under a response-initiated fixed-interval schedule

Tolerance to effects of cocaine can be modulated by schedules of reinforcement. With multiple ratio schedules, research has shown an inverse relationship between ratio requirement and amount of tolerance that resulted from daily administration of the drug. In contrast, tolerance to the effects of cocaine on behavior under multiple interval schedules generally has developed regardless of interval value. Under interval schedules reinforcement depends on the animal making one response following a time interval. Thus, as time to respond increases, the time to reinforcement decreases. On the other hand, fixed ratio schedules require a specified number of responses to be made prior to reinforcement. Therefore, delaying the initiation of responding does not coincide with a significant decrease in the time to reinforcement. In the current experiment, 6 pigeons were trained to respond under a three-component multiple schedule, with a different tandem fixed-ratio 1 fixed-interval schedule in each component. The multiple schedule required one response, which was followed by one of three fixed-interval values (5, 15, or 60 s). Thus, the multiple schedule was interval-like because after the fixed-ratio 1, only one more response was required for reinforcement, but it was also ratio-like because the length of the pause at the beginning of each interreinforcer interval affected the time until the next reinforcer. Acute administration of cocaine generally resulted in dose-dependent decreases in responding. Chronic (i.e., daily) administration of a rate-decreasing dose resulted in tolerance patterns similar to those usually obtained with multiple ratio schedules. That is, the magnitude of tolerance was related inversely to schedule size. These results suggest that delay to reinforcement from the initial response may play a role in the development of schedule-parameter-related tolerance.     

Variable-ratio versus variable-interval schedules: response rate, resistance to change, and preference

Two experiments asked whether resistance to change depended on variable-ratio as opposed to variable-interval contingencies of reinforcement and the different response rates they establish. In Experiment 1, pigeons were trained on multiple random-ratio random-interval schedules with equated reinforcer rates. Baseline response rates were disrupted by intercomponent food, extinction, and prefeeding. Resistance to change relative to baseline was greater in the interval component, and the difference was correlated with the extent to which baseline response rates were higher in the ratio component. In Experiment 2, pigeons were trained on multiple variable-ratio variable-interval schedules in one half of each session and on concurrent chains in the other half in which the terminal links corresponded to the multiple-schedule components. The schedules were varied over six conditions, including two with equated reinforcer rates. In concurrent chains, preference strongly overmatched the ratio of obtained reinforcer rates. In multiple schedules, relative resistance to response-independent food during intercomponent intervals, extinction, and intercomponent food plus extinction depended on the ratio of obtained reinforcer rates but was less sensitive than was preference. When reinforcer rates were similar, both preference and relative resistance were greater for the variable-interval schedule, and the differences were correlated with the extent to which baseline response rates were higher on the variable-ratio schedule, confirming the results of Experiment 1. These results demonstrate that resistance to change and preference depend in part on response rate as well as obtained reinforcer rate, and challenge the independence of resistance to change and preference with respect to response rate proposed by behavioral momentum theory.     

Preratio pausing: effects of an alternative reinforcer on fixed- and variable-ratio responding

Seven rats responding under fixed-ratio or variable-ratio schedules of food reinforcement had continuous access to a drinking tube inserted into the operant chamber. Under different conditions they could drink either tap water or one of two saccharin solutions. In a baseline condition, the drinking bottle was empty. Preratio pausing was observed with both schedules, more so with the fixed-ratio than the variable-ratio schedule, and increasing the concentration of the saccharin solution increased the duration of pausing. Comparisons with baseline performances revealed that the additional pausing was largely, but not entirely, spent drinking. The results support the view that pausing under ratio schedules is a consequence of competition between the scheduled reinforcer and alternative reinforcers that also are available within the experimental environment.     

Progressive-ratio schedules: effects of later schedule requirements on earlier performances

Four rats were studied with variants of a progressive-ratio schedule with a step size of 6 in which different terminal components followed completion of the 20th ratio: (a) a reversal of the progression, (b) a fixed-ratio 6 schedule, or (c) extinction. Responding in the progressive-ratio components of these schedules was compared to performances under conventional progressive-ratio baselines. Under baseline conditions, postreinforcement pauses increased exponentially as a function of increasing ratio size, whereas running rates showed modest declines. The procedure of linking the progressive-ratio schedule to the reversed progression or to the fixed-ratio component resulted in decreased pausing. Linking the progressive-ratio schedule to the extinction component had the opposite effect, that of producing weakened progressive-ratio performances as evidenced by increased pausing. Subjects whose responses were reinforced on half of the ratios also showed exponential increases; however, pauses were substantially shorter following ratios on which the reinforcer was omitted. The results suggested that progressive-ratio pausing reflects the influence of remote as well as local contingencies.     

Effects of shifts in food reinforcement context on rats’ consumption of concurrently available water or sucrose solution

The purpose of this study was to investigate the effects of signaled transitions from relatively rich to lean conditions of food reinforcement on drinking concurrently available water or sucrose‐sweetened water in rats. Past research demonstrated that these negative incentive shifts produce behavioral disruption in the form of extended pausing on fixed‐ratio schedules. Four male Long‐Evans rats operated on a two‐component multiple fixed‐ratio fixed‐ratio schedule. In one manipulation, the ratio was held constant and the components arranged either a large six‐pellet reinforcer (rich) or small one‐pellet reinforcer (lean). In a second manipulation, the components both produced a one‐pellet reinforcer but differed in terms of the ratio requirement, with the rich and lean conditions corresponding to relatively small and large ratios. In both manipulations, components were pseudorandomly presented to arrange four transitions signaled by retractable levers: lean‐to‐lean, lean‐to‐rich, rich‐to‐rich, and rich‐to‐lean (the negative incentive shift). During experimental conditions, a bottle with lickometer was inserted in the chamber, providing concurrent access either to tap water or a 10% sucrose solution. The negative incentive shift produced considerably more drinking than the other transitions in all rats during both manipulations. The level of drinking was not polydipsic; rather, it appears that the negative incentive shift enhanced the value of concurrently available reinforcers relative to food reinforcement.     

Fixed-interval performance and self-control in children.

Operant responses of 16 children (mean age 6 years and 1 month) were reinforced according to different fixed-interval schedules (with interreinforcer intervals of 20, 30, or 40 s) in which the reinforcers were either 20-s or 40-s presentations of a cartoon. In another procedure, they received training on a self-control paradigm in which both reinforcer delay (0.5 s or 40 s) and reinforcer duration (20 s or 40 s of cartoons) varied, and subjects were offered a choice between various combinations of delay and duration. Individual differences in behavior under the self-control procedure were precisely mirrored by individual differences under the fixed-interval schedule. Children who chose the smaller immediate reinforcer on the self-control procedure (impulsive) produced short postreinforcement pauses and high response rates in the fixed-interval conditions, and both measures changed little with changes in fixed-interval value. Conversely, children who chose the larger delayed reinforcer in the self-control condition (the self-controlled subjects) exhibited lower response rates and long postreinforcement pauses, which changed systematically with changes in the interval, in their fixed-interval performances.     

Response-reinforcer relations and the maintenance of behavior

The effects on pigeons' key pecking of unsignaled delays of reinforcement and response-independent reinforcement were compared after either variable-interval or differential-reinforcement-of-low-rate baseline schedules. One 30-min session arranging delayed reinforcement and one 30-min session arranging response-independent reinforcement were conducted daily, 6 hr apart. A within-subject yoked-control procedure equated reinforcer frequency and distribution across the two sessions. Response rates usually were reduced more by response-independent than by delayed but response-contingent delivery of reinforcers. Under both schedules, response rates were lower when obtained delays were greater. These results bear upon methodological and conceptual issues regarding comparisons of contingencies that change the temporal response–reinforcer relations.     

The reductive effects of noncontingent reinforcement fixed-time versus variable-time schedules.

The effects of fixed‐time (FT) and variable‐time (VT) schedules on responding were evaluated with 2 adults with mental retardation. Multielement and reversal designs were used to compare the effects of FT and VT schedules in reducing responses previously maintained on variable‐ratio reinforcement schedules. The schedules were equally effective in reducing the target behavior.     

Unit price and choice in a token-reinforcement context

Pigeons were exposed to multiple and concurrent second-order schedules of token reinforcement, with stimulus lights serving as token reinforcers. Tokens were produced and exchanged for food according to various fixed-ratio schedules, yielding equal and unequal unit prices (responses per unit food delivery). On one schedule (termed the standard schedule), the unit price was held constant across conditions. On a second schedule (the alternative schedule), the unit price was either the same or different from the standard. Under conditions with unequal unit prices, near-exclusive preference for the lower unit price was obtained. Under conditions with equal unit prices, the direction and degree of preference depended on ratio size (number of responses per exchange period). When this ratio differed, strong preferences for the smaller ratio were observed. When this ratio was equal, preferences were nearer indifference. Response rates on the multiple schedule were generally consistent with the preference data in showing sensitivity to ratio size. Results are discussed in terms of a unit-price model that includes handling and reinforcer immediacy as additional costs. On the whole, results show that preferences were determined primarily by delay to the exchange period.     

Exclusive preference develops less readily on concurrent ratio schedules with wheel-running than with sucrose reinforcement

Previous research suggested that allocation of responses on concurrent schedules of wheel‐running reinforcement was less sensitive to schedule differences than typically observed with more conventional reinforcers. To assess this possibility, 16 female Long Evans rats were exposed to concurrent FR FR schedules of reinforcement and the schedule value on one alternative was systematically increased. In one condition, the reinforcer on both alternatives was .1 ml of 7.5% sucrose solution; in the other, it was a 30‐s opportunity to run in a wheel. Results showed that the average ratio at which greater than 90% of responses were allocated to the unchanged alternative was higher with wheel‐running reinforcement. As the ratio requirement was initially increased, responding strongly shifted toward the unchanged alternative with sucrose, but not with wheel running. Instead, responding initially increased on both alternatives, then subsequently shifted toward the unchanged alternative. Furthermore, changeover responses as a percentage of total responses decreased with sucrose, but not wheel‐running reinforcement. Finally, for some animals, responding on the increasing ratio alternative decreased as the ratio requirement increased, but then stopped and did not decline with further increments. The implications of these results for theories of choice are discussed.     

Contrast and reallocation of extraneous reinforcers as a function of component duration and baseline rate of reinforcement

Four pigeons responded on multiple schedules arranged on a “main” key in a two-key experimental chamber. A constant schedule component was alternated with another component that was varied over conditions. On an extra response key, conjoint schedules of reinforcement that operated in both components were arranged concurrently with the multiple schedule on the main key. On the main key, changes in reinforcement rate in the varied component were inversely related to changes in response rates in the constant component (behavioral contrast). On the extra key, some reinforcers were reallocated between components, depending on the schedules in effect on the main key in the varied component. In the varied component, the obtained rates of reinforcement on the extra key were inversely related to main-key reinforcement rate. In the constant component, extra-key reinforcer rates were positively related to main-key reinforcer rates obtained in the varied component, and were not a function of response rates on the extra key. In two comparisons, the rate at which components alternated and the value of the main-key schedule in the constant component were varied. Consistent with earlier work, long components reduced the extent of contrast. Reductions in contrast as a function of component duration were accompanied by similar reductions in the extent of reinforcer reallocation on the extra key. In the second comparison, lowering the rate of reinforcement in the constant component increased the rate at which extra-key reinforcers were obtained, reduced the extent of reinforcer reallocation, and reduced contrast. Overall, the results are consistent with the suggestion that some contrast effects are due to the changes in extraneous reinforcement during the constant component, and that manipulations of component duration, and manipulations of the rate of reinforcement in the constant component, affect contrast because they influence the extent of extraneous reinforcer real-location.     

Cooperative responding in rats maintained by fixed‐ and variable‐ratio schedules

The present study investigated the effects of fixed‐ratio (FR) and variable‐ratio (VR) reinforcement schedules on patterns of cooperative responding in pairs of rats. Experiment 1 arranged FR 1, FR 10, and VR 10 schedules to establish cooperative responding (water delivery depended on the joint responding of two rats). Cooperative response rates and proportions were higher under intermittent schedules than under continuous reinforcement. The FR 10 schedule generated a break‐and‐run pattern, whereas the VR 10 schedule generated a relatively high and constant rate pattern. Experiment 2 evaluated the effects of parametric manipulations of FR and VR schedules on cooperative responding. Rates and proportions of cooperative responding generally increased between ratio sizes of 1 and 5 but showed no consistent trend as the ratio increased from 5 to 10. Experiment 3 contrasted cooperative responding between an FR6 schedule and a yoked control schedule. Coordinated behavior occurred at a higher rate under the former schedule. The present study showed that external consequences and the schedules under which the delivery of these consequences are based, select patterns of coordinated behavior.     

Negative reinforcement and choice in humans

Participants chose between reinforcement schedules differing in delay and/or duration of noise offset. In Experiment 1 it was found that (1) immediate reinforcement was preferred to delayed reinforcement when amounts (durations) of reinforcement were equal; (2) a relatively large reinforcer was preferred to a smaller one when both reinforcers were obtained immediately; and (3) preference for an immediate, small reinforcer versus a delayed, large reinforcer increased as the delay preceding the large reinforcer increased, a sign of “impulsivity”. In Experiment 2, the schedules differed in amount or delay and equal intervals were added either to the constant parameter or the varied parameter. A shift from virtually exclusive preference to indifference occurred in the latter case but not the former, a result supporting a model of self-control that assumes that the value of a schedule depends on the ratio of amount and delay, and that choice between schedules depends on the ratio of these values.