The effects of reinforcer magnitude in the preceding and upcoming ratios on between‐ratio pausing in multiple,mixed, and single fixed‐ratio schedules

Hens responded under multiple fixed‐ratio schedules with equal response requirements and either a 1‐s or a 6‐s reinforcer. Upcoming reinforcer size was indicated by key color. Components were presented in a quasirandom series so that all four component transitions occurred. Postreinforcement pauses were affected by the upcoming and preceding reinforcer size, with longer pauses after large reinforcers followed by small reinforcers than when followed by large ones, and longer pauses after small reinforcers that were followed by small reinforcers rather than large ones. Pauses increased with fixed‐ratio size and the effects of reinforcer size were larger the larger the ratio. When reinforcer size was not signaled—mixed fixed‐ratio schedules—pauses were shorter after small than after large reinforcers. Signalling the upcoming reinforcer attenuated the effect of the previous reinforcer size on pause duration when small was followed by small and when either small or large by large, but enhanced the effect when large was followed by small. There was no effect of reinforcer size on pause duration when single fixed‐ratio schedules were arranged. The effects of reinforcer size on pauses depends on the size and range of the fixed ratios as well as the exact procedures used in the study.     

Exclusive preference develops less readily on concurrent ratio schedules with wheel-running than with sucrose reinforcement

Previous research suggested that allocation of responses on concurrent schedules of wheel‐running reinforcement was less sensitive to schedule differences than typically observed with more conventional reinforcers. To assess this possibility, 16 female Long Evans rats were exposed to concurrent FR FR schedules of reinforcement and the schedule value on one alternative was systematically increased. In one condition, the reinforcer on both alternatives was .1 ml of 7.5% sucrose solution; in the other, it was a 30‐s opportunity to run in a wheel. Results showed that the average ratio at which greater than 90% of responses were allocated to the unchanged alternative was higher with wheel‐running reinforcement. As the ratio requirement was initially increased, responding strongly shifted toward the unchanged alternative with sucrose, but not with wheel running. Instead, responding initially increased on both alternatives, then subsequently shifted toward the unchanged alternative. Furthermore, changeover responses as a percentage of total responses decreased with sucrose, but not wheel‐running reinforcement. Finally, for some animals, responding on the increasing ratio alternative decreased as the ratio requirement increased, but then stopped and did not decline with further increments. The implications of these results for theories of choice are discussed.     

Choice dynamics in concurrent ratio schedules of reinforcement

The generalized matching law predicts performance on concurrent schedules when variable-interval schedules are programmed but is trivially applicable when independent ratio schedules are used. Responding usually is exclusive to the schedule with the lowest response requirement. Determining a method to program concurrent ratio schedules such that matching analyses can be usefully employed would extend the generality of matching research and lead to new avenues of research. In the present experiments, ratio schedules were programmed dependently such that responses to either of the two options progressed the requirement on both schedules. Responding is not exclusive because the probability of reinforcement increases on both schedules as responses are allocated to either schedule. In Experiment 1, performance on concurrent variable-ratio schedules was assessed, and reinforcer ratios were varied across conditions to investigate changes in sensitivity. Additionally, the length of a changeover delay was manipulated. In Experiment 2, performance was compared under concurrently available, dependently programmed variable-ratio and fixed-ratio schedules. Performance was well described by the generalized matching law. Increases in the changeover delay decreased sensitivity, whereas sensitivity was higher when variable-ratio schedules were employed, compared with fixed-ratio schedules. Concurrent ratio schedules can be a viable approach to studying functional differences between ratio and interval schedules.     

Unit price and choice in a token-reinforcement context

Pigeons were exposed to multiple and concurrent second-order schedules of token reinforcement, with stimulus lights serving as token reinforcers. Tokens were produced and exchanged for food according to various fixed-ratio schedules, yielding equal and unequal unit prices (responses per unit food delivery). On one schedule (termed the standard schedule), the unit price was held constant across conditions. On a second schedule (the alternative schedule), the unit price was either the same or different from the standard. Under conditions with unequal unit prices, near-exclusive preference for the lower unit price was obtained. Under conditions with equal unit prices, the direction and degree of preference depended on ratio size (number of responses per exchange period). When this ratio differed, strong preferences for the smaller ratio were observed. When this ratio was equal, preferences were nearer indifference. Response rates on the multiple schedule were generally consistent with the preference data in showing sensitivity to ratio size. Results are discussed in terms of a unit-price model that includes handling and reinforcer immediacy as additional costs. On the whole, results show that preferences were determined primarily by delay to the exchange period.     

EFFECTS OF POINT‐LOSS PUNISHERS ON HUMAN SIGNAL‐DETECTION PERFORMANCE

Three experiments using human participants varied the distribution of point‐gain reinforcers or point‐loss punishers in two‐alternative signal‐detection procedures. Experiment 1 varied the distribution of point‐gain reinforcers for correct responses (Group A) and point‐loss punishers for errors (Group B) across conditions. Response bias varied systematically as a function of the relative reinforcer or punisher frequencies. Experiment 2 arranged two conditions — one where an unequal ratio of reinforcement (5:1 or 1:5) was presented without punishment (R‐only), and another where the same reinforcer ratio was presented with an equal distribution of point‐loss punishers (R+P). Response bias was significantly greater in the R‐only condition than the R+P condition, supporting a subtractive model of punishment. Experiment 3 varied the distribution of point‐gain reinforcers for correct responses across four unequal reinforcer ratios (5:1, 2:1, 1:2, 1:5) both without (R‐only) and with (R+P) an equal distribution of point‐loss punishers for errors. Response bias varied systematically with changes in relative reinforcer frequency for both R‐only and R+P conditions, with 5 out of 8 participants showing increases in sensitivity estimates from R‐only to R+P conditions. Overall, the results indicated that punishers have similar but opposite effects to reinforcers in detection procedures and that combined reinforcer and punisher effects might be better modeled by a subtractive punishment model than an additive punishment model, consistent with research using concurrent‐schedule choice procedures.     

Response-rate invariance in concurrent schedules: effects of different changeover contingencies

In a two-key chamber, one key (the food key) was either red or green with different variable-interval schedules operating concurrently in each color and a second key (the changeover key) served to change the food-key color. Three pigeons were trained with either a 2-sec changeover delay or a 0-sec changeover delay and three birds with a fixed-ratio 2 on the changeover key instead of a changeover delay. The proportion of time spent in red approximated the proportion of reinforcers delivered in red for all birds. When the procedure was changed so that reinforcers were signalled in the green schedule, rates of reinforcement were unaltered, but the pigeons spent virtually the whole session in red. Changeovers to green were allowed only when a reinforcer was assigned by the schedule associated with green. For all pigeons with the fixed–ratio requirement on the changeover key or with a 0-sec changeover delay, the overall rate of red-key responses was higher during the signalling condition than during unsignalled, or baseline, condition. The present data question the generality of previous reports that the rate of one response is independent of the amount of time allocated to the alternative response.     

Choosing among multiple alternatives: Relative and overall reinforcer rates

Choice behavior among two alternatives has been widely researched, but fewer studies have examined the effect of multiple (more than two) alternatives on choice. Two experiments investigated whether changing the overall reinforcer rate affected preference among three and four concurrently scheduled alternatives. Experiment 1 trained six pigeons on concurrent schedules with three alternatives available simultaneously. These alternatives arranged reinforcers in a ratio of 9:3:1 with the configuration counterbalanced across pigeons. The overall rate of reinforcement was varied across conditions. Preference between the pair of keys arranging the 9:3 reinforcer ratio was less extreme than the pair arranging the 3:1 reinforcer ratio regardless of overall reinforcer rate. This difference was attributable to the richer alternative receiving fewer responses per reinforcer than the other alternatives. Experiment 2 trained pigeons on concurrent schedules with four alternatives available simultaneously. These alternatives arranged reinforcers in a ratio of 8:4:2:1, and the overall reinforcer rate was varied. Next, two of the alternatives were put into extinction and the random interval duration was changed from 60 s to 5 s. The ratio of absolute response rates was independent of interval length across all conditions. In both experiments, an analysis of sequences of visits following each reinforcer showed that the pigeons typically made their first response to the richer alternative irrespective of which alternative was just reinforced. Performance on these three‐ and four‐alternative concurrent schedules is not easily extrapolated from corresponding research using two‐alternative concurrent schedules.     

Changeover ratio effects on concurrent variable-interval performance

Rats' bar-pressing was maintained by concurrent variable-interval schedules of reinforcement. A fixed-ratio of pulls on a chain (the changeover ratio) was required for switching between schedules. The first experiment employed equal variable-interval schedules and symmetrical changeover ratios. Increasing these ratios resulted in a decrease in the rate of switching between schedules and an increase in local response rate. In the second experiment, a range of asymmetrical changeover ratios was used with equal variable-interval schedules, and a preference was found for the schedule associated with the larger switching-into ratio. Both the distributions of responses and time between the two schedules deviated from those expected on the basis of obtained reinforcers. In the third experiment, the switching-out-of ratio was dependent on the amount of time spent in a variable-interval 2-minute schedule; a constant ratio permitted switching out of the alternative variable-interval 1-minute schedule. A strong preference was shown for the variable-interval 2-minute schedule. The fourth experiment used equal variable-interval schedules; one changeover ratio was varied while the second remained constant. The results failed to show systematic differences in local response rates immediately after a changeover.     

Concurrent-schedule performance: Effects of relative and overall reinforcer rate

Six pigeons were trained to respond on two keys, each of which provided reinforcers on an arithmetic variable-interval schedule. These concurrent schedules ran nonindependently with a 2-s changeover delay. Six sets of conditions were conducted. Within each set of conditions the ratio of reinforcers available on the two alternatives was varied, but the arranged overall reinforcer rate remained constant. Each set of conditions used a different overall reinforcer rate, ranging from 0.22 reinforcers per minute to 10 reinforcers per minute. The generalized matching law fit the data from each set of conditions, but sensitivity to reinforcer frequency (a) decreased as the overall reinforcer rate decreased for both time allocation and response allocation based analyses of the data. Overall response rates did not vary with changes in relative reinforcer rate, but decreased with decreases in overall reinforcer rate. Changeover rates varied as a function of both relative and overall reinforcer rates. However, as explanations based on changeover rate seem unable to deal with the changes in generalized matching sensitivity, discrimination accounts of choice may offer a more promising interpretation.     

A comparison of delays and ratio requirements in self-control choice.

In a discrete-trial procedure, pigeons could choose between 2-s and 6-s access to grain by making a single key peck. In Phase 1, the pigeons obtained both reinforcers by responding on fixed-ratio schedules. In Phase 2, they received both reinforcers after simple delays, arranged by fixed-time schedules, during which no responses were required. In Phase 3, the 2-s reinforcer was available through a fixed-time schedule and the 6-s reinforcer was available through a fixed-ratio schedule. In all conditions, the size of the delay or ratio leading to the 6-s reinforcer was systematically increased or decreased several times each session, permitting estimation of an "indifference point," the schedule size at which a subject chose each alternative equally often. By varying the size of the schedule for the 2-s reinforcer across conditions, several such indifference points were obtained from both fixed-time conditions and fixed-ratio conditions. The resulting "indifference curves" from fixed-time conditions and from fixed-ratio conditions were similar in shape, and they suggested that a hyperbolic equation describes the relation between ratio size and reinforcement value as well as the relation between reinforcer delay and its reinforcement value. The results from Phase 3 showed that subjects chose fixed-time schedules over fixed-ratio schedules that generated the same average times between a choice response and reinforcement.     

Choice, changeover, and travel: A quantitative model

Six pigeons were trained on concurrent variable-interval schedules in which responding on fixed-interval schedules was required to give access to the alternate schedule. Responding on the concurrent schedules was not allowed, after changing over had commenced, until the changeover schedule had been completed. In Parts 1 to 3 of the experiment, the changeover fixed-interval schedules were equal and were 0 s, 10 s, and 20 s, respectively. In each part, the relative frequency of reinforcement obtained on the concurrent schedules was varied over at least five conditions. In Part 4, the concurrent schedules were equal, and one changeover fixed-interval schedule was twice the other. Under these conditions, the absolute sizes of the changeover schedules were varied. Increasing the changeover requirement from 0 s to 10 s (Parts 1 and 2) resulted in increases in the sensitivity of behavior allocation to reinforcers obtained, but no further increase was obtained when the changeover schedules were increased to 20 s (Part 3). In Part 4, performance was biased towards the concurrent schedule that took less time to enter. These results are consistent with a subtractive punishment model of travel in which the degree of punishment is measured by the number of reinforcers apparently lost from a schedule when the subject changes to that schedule. Absolute times spent on the main keys could be accurately described by a previous model of changeover performance.     

Changeover behavior under pairs of fixed-ratio and variable-ratio schedules of reinforcement

Three pigeons were studied under a pair of equal fixed-ratio schedules and a pair of equal variable-ratio schedules. Each pair was arranged as independent, concurrent schedules and also in a non-independent relation where each peck in a schedule counted toward the response requirement of both schedules. The non-independent pair of variable-ratio schedules maintained much higher changeover rates than any of the other three arrangements. Thus, two factors seemed necessary for generating high changeover rates. Responding on a schedule had to count toward the response requirement of both schedules, and the component schedules had to be variable. These data are consistent with the hypothesis that changeovers are at least partly controlled by the probability of reinforcement following a changeover.     

Choice between single and multiple reinforcers in concurrent-chains schedules

Pigeons responded on concurrent-chains schedules with equal variable-interval schedules as initial links. One terminal link delivered a single reinforcer after a fixed delay, and the other terminal link delivered either three or five reinforcers, each preceded by a fixed delay. Some conditions included a postreinforcer delay after the single reinforcer to equate the total durations of the two terminal links, but other conditions did not include such a postreinforcer delay. With short initial links, preference for the single-reinforcer alternative decreased when a postreinforcer delay was present, but with long initial links, the postreinforcer delays had no significant effect on preference. In conditions with a postreinforcer delay, preference for the single-reinforcer alternative frequently switched from above 50% to below 50% as the initial links were lengthened. This pattern of results was consistent with delay-reduction theory (Squires & Fantino, 1971), but not with the contextual-choice model (Grace, 1994) or the hyperbolic value-added model (Mazur, 2001) as they have usually been applied. However, the hyperbolic value-added model could account for the results if its calculations were expanded to include reinforcers delivered in later terminal links. The implications of these findings for models of concurrent-chains performance are discussed.     

Response–reinforcer dependency and resistance to change

The effects of the response–reinforcer dependency on resistance to change were studied in three experiments with rats. In Experiment 1, lever pressing produced reinforcers at similar rates after variable interreinforcer intervals in each component of a two‐component multiple schedule. Across conditions, in the fixed component, all reinforcers were response‐dependent; in the alternative component, the percentage of response‐dependent reinforcers was 100, 50 (i.e., 50% response‐dependent and 50% response‐independent) or 10% (i.e., 10% response‐dependent and 90% response‐independent). Resistance to extinction was greater in the alternative than in the fixed component when the dependency in the former was 10%, but was similar between components when this dependency was 100 or 50%. In Experiment 2, a three‐component multiple schedule was used. The dependency was 100% in one component and 10% in the other two. The 10% components differed on how reinforcers were programmed. In one component, as in Experiment 1, a reinforcer had to be collected before the scheduling of other response‐dependent or independent reinforcers. In the other component, response‐dependent and ‐independent reinforcers were programmed by superimposing a variable‐time schedule on an independent variable‐interval schedule. Regardless of the procedure used to program the dependency, resistance to extinction was greater in the 10% components than in the 100% component. These results were replicated in Experiment 3 in which, instead of extinction, VT schedules replaced the baseline schedules in each multiple‐schedule component during the test. We argue that the relative change in dependency from Baseline to Test, which is greater when baseline dependencies are high rather than low, could account for the differential resistance to change in the present experiments. The inconsistencies in results across the present and previous experiments suggest that the effects of dependency on resistance to change are not well understood. Additional systematic analyses are important to further understand the effects of the response–reinforcer relation on resistance to change and to the development of a more comprehensive theory of behavioral persistence.     

Fix and sample with rats in the dynamics of choice

The generality of the molar view of behavior was extended to the study of choice with rats, showing the usefulness of studying order at various levels of extendedness. Rats' presses on two levers produced food according to concurrent variable-interval variable-interval schedules. Seven different reinforcer ratios were arranged within each session, without cues identifying them, and separated by blackouts. To alternate between levers, rats pressed on a third changeover lever. Choice changed rapidly with changes in component reinforcer ratio, and more presses occurred on the lever with the higher reinforcer rate. With continuing reinforcers, choice shifted progressively in the direction of the reinforced lever, but shifted more slowly with each new reinforcer. Sensitivity to reinforcer ratio, as estimated by the generalized matching law, reached an average of 0.9 and exceeded that documented in previous studies with pigeons. Visits to the more-reinforced lever preceded by a reinforcer from that lever increased in duration, while all visits to the less-reinforced lever decreased in duration. Thus, the rats' performances moved faster toward fix and sample than did pigeons' performances in previous studies. Analysis of the effects of sequences of reinforcer sources indicated that sequences of five to seven reinforcers might have sufficed for studying local effects of reinforcers with rats. This study supports the idea that reinforcer sequences control choice between reinforcers, pulses in preference, and visits following reinforcers.     

How do reinforcers affect choice? Preference pulses after responses and reinforcers

In concurrent schedules, reinforcers are often followed by a brief period of heightened preference for the just‐productive alternative. Such ‘preference pulses’ may reflect local effects of reinforcers on choice. However, similar pulses may occur after nonreinforced responses, suggesting that pulses after reinforcers are partly unrelated to reinforcer effects. McLean, Grace, Pitts, and Hughes (2014) recommended subtracting preference pulses after responses from preference pulses after reinforcers, to construct residual pulses that represent only reinforcer effects. Thus, a reanalysis of existing choice data is necessary to determine whether changes in choice after reinforcers in previous experiments were actually related to reinforcers. In the present paper, we reanalyzed data from choice experiments in which reinforcers served different functions. We compared local choice, mean visit length, and visit‐length distributions after reinforcers and after nonreinforced responses. Our reanalysis demonstrated the utility of McLean et al.'s preference‐pulse correction for determining the effects of reinforcers on choice. However, visit analyses revealed that residual pulses may not accurately represent reinforcer effects, and reinforcer effects were clearer in visit analyses than in local‐choice analyses. The best way to determine the effects of reinforcers on choice may be to conduct visit analyses in addition to local‐choice analyses.     

Choice, relative reinforcer duration, and the changeover ratio

Relative reinforcer duration was varied in concurrent schedules with a fixed-ratio four changeover requirement. The schedule in effect after each reinforcer was randomly chosen. For all three pigeons, relative response rates overmatched relative reinforcer durations. Time allocation was less extreme and, on the average, matched relative reinforcer duration. In a subsequent manipulation, the level of preference was shown to depend on the size of the changeover requirement. These results are similar to those from related unequal reinforcement-frequency procedures.     

Effects of intertrial reinforcers on self-control choice.

In three experiments, pigeons chose between a small amount of food delivered after a short delay and a larger amount delivered after a longer delay. A discrete-trial adjusting-delay procedure was used to estimate indifference points--pairs of delay-amount combinations that were chosen about equally often. In Experiment 1, when additional reinforcers were available during intertrial intervals on a variable-interval schedule, preference for the smaller, more immediate reinforcer increased. Experiment 2 found that this shift in preference occurred partly because the variable-interval schedule started sooner after the smaller, more immediate reinforcer, but there was still a small shift in preference when the durations and temporal locations of the variable-interval schedules were identical for both alternatives. Experiment 3 found greater increases in preference for the smaller, more immediate reinforcer with a variable-interval 15-s schedule than with a variable-interval 90-s schedule. The results were generally consistent with a model that states that the impact of any event that follows a choice response declines according to a hyperbolic function with increasing time since the moment of choice.     

CORRESPONDENCE BETWEEN SINGLE VERSUS DAILY PREFERENCE ASSESSMENT OUTCOMES AND REINFORCER EFFICACY UNDER PROGRESSIVE‐RATIO SCHEDULES

Research has suggested that a daily multiple‐stimulus‐without‐replacement (MSWO) preference assessment may be more sensitive to changes in preference than other assessment formats, thereby resulting in greater correspondence with reinforcer efficacy over time ( DeLeon et al., 2001 ). However, most prior studies have measured reinforcer efficacy using rate of responding under single‐operant arrangements and dense schedules or under concurrent‐operants arrangements. An alternative measure of reinforcer efficacy involves the evaluation of responding under progressive‐ratio (PR) schedules. In the present study, 7 participants were given a single paired‐stimulus (PS) preference assessment followed by daily MSWO preference assessments. After each daily MSWO, participants responded for each stimulus on a PR schedule. The correspondence between break points and preferences, as assessed by the 2 assessment formats, was examined. Results demonstrated that both preference assessments did equally well at predicting reinforcer efficacy, although the PS more consistently identified the most effective reinforcer.     

EFFECT OF REINFORCER MAGNITUDE ON PERFORMANCE MAINTAINED BY PROGRESSIVE‐RATIO SCHEDULES

This experiment examined the relationship between reinforcer magnitude and quantitative measures of performance on progressive‐ratio schedules. Fifteen rats were trained under a progressive‐ratio schedule in seven phases of the experiment in which the volume of a 0.6‐M sucrose solution reinforcer was varied within the range 6–300 μl. Overall response rates in successive ratios conformed to a bitonic equation derived from Killeen's (1994) Mathematical Principles of Reinforcement. The “specific activation” parameter, a, which is presumed to reflect the incentive value of the reinforcer, was a monotonically increasing function of reinforcer volume; the “response time” parameter, δ, which defines the minimum response time, increased as a function of reinforcer volume; the “currency” parameter, b, which is presumed to reflect the coupling of responses to the reinforcer, declined as a function of volume. Running response rate (response rate calculated after exclusion of the postreinforcement pause) decayed monotonically as a function of ratio size; the index of curvature of this function increased as a function of reinforcer volume. Postreinforcement pause increased as a function of ratio size. Estimates of a derived from overall response rates and postreinforcement pauses showed a modest positive correlation across conditions and between animals. Implications of the results for the quantification of reinforcer value and for the use of progressive‐ratio schedules in behavioral neuroscience are discussed.