Choice dynamics in concurrent ratio schedules of reinforcement

The generalized matching law predicts performance on concurrent schedules when variable-interval schedules are programmed but is trivially applicable when independent ratio schedules are used. Responding usually is exclusive to the schedule with the lowest response requirement. Determining a method to program concurrent ratio schedules such that matching analyses can be usefully employed would extend the generality of matching research and lead to new avenues of research. In the present experiments, ratio schedules were programmed dependently such that responses to either of the two options progressed the requirement on both schedules. Responding is not exclusive because the probability of reinforcement increases on both schedules as responses are allocated to either schedule. In Experiment 1, performance on concurrent variable-ratio schedules was assessed, and reinforcer ratios were varied across conditions to investigate changes in sensitivity. Additionally, the length of a changeover delay was manipulated. In Experiment 2, performance was compared under concurrently available, dependently programmed variable-ratio and fixed-ratio schedules. Performance was well described by the generalized matching law. Increases in the changeover delay decreased sensitivity, whereas sensitivity was higher when variable-ratio schedules were employed, compared with fixed-ratio schedules. Concurrent ratio schedules can be a viable approach to studying functional differences between ratio and interval schedules.     

Preferences for token exchange‐production schedules: Effects of task difficulty and token‐production schedules

Previous research suggests that accumulated exchange‐production schedules promote increased work completion and are more preferred than distributed exchange‐production schedules. The purpose of this study was to evaluate whether the response effort or token‐production schedules associated with token delivery influenced preferences for exchange‐production schedules. Tokens exchanged under accumulated schedules were more preferred, relative to distributed schedules, when tokens were earned for completing easy tasks. When participants earned tokens for completing difficult tasks, two of three participants preferred accumulated exchange‐production schedules (Experiment 1). Under dense token‐production schedules, accumulated exchange‐production schedules were preferred, but participant's preferences switched to distributed schedules under increasing token‐production (i.e., leaner) schedules (Experiment 2).     

Within-session changes in responding during several simple schedules

Pigeons' key pecking was reinforced by food delivered by several fixed-interval, variable-ratio, and differential-reinforcement-of-low-rate schedules. Rate of responding, number of responses per reinforcer, length of postreinforcement pause, running response rate, and the time required to collect an available reinforcer changed systematically within sessions when the schedules provided high rates of reinforcement, but usually not when they provided low rates. These results suggest that the factors that produce within-session changes in responding are generally similar for different schedules of reinforcement. However, a separate factor may also contribute during variable-ratio schedules. The results question explanations for within-session changes that are related solely to the passage of time, to responding, and to one interpretation of attention. They support the idea that one or more factors related to reinforcement play a role.     

Rate Dependence and Token Reinforcement? A Preliminary Analysis

The Psychological Record - Changes in token exchange-production schedules may alter behavior when token-production schedules and token-exchange schedules are held constant. Although research...     

Performance in concurrent interval schedules: a systematic replication

Five pigeons were trained on a variety of concurrent interval schedules that arranged reinforcements at either fixed or variable times after the last reinforcement. Two measures were obtained: the number of responses on each schedule, and the time spent responding on each schedule. Ratios of response rates on the two schedules did not equal ratios of reinforcement rates when both schedules were variable nor when one was variable and the other fixed. Ratios of times spent responding approximately equalled ratios of reinforcement rates when both schedules were variable, but did not do so when one was fixed.     

Molar And Molecular Control In Variable-interval And Variable-ratio Schedules

Response rates are typically higher under variable-ratio than under variable-interval schedules of reinforcement, perhaps because of differences in the dependence of reinforcement rate on response rate or because of differences in the reinforcement of long interresponse times. A variable-interval-with-added-linear-feedback schedule is a variable-interval schedule that provides a response rate/reinforcement rate correlation by permitting the minimum interfood interval to decrease with rapid responding. Four rats were exposed to variable-ratio 15, 30, and 60 food reinforcement schedules, variable-interval 15-, 30-, and 60-s food reinforcement schedules, and two versions of variable-interval-with-added-linear-feedback 15-, 30-, and 60-s food reinforcement schedules. Response rates on the variable-interval-with-added-linear-feedback schedule were similar to those on the variable-interval schedule; all three schedules led to lower response rates than those on the variable-ratio schedules, especially when the schedule values were 30. Also, reinforced interresponse times on the variable-interval-with-added-linear-feedback schedule were similar to those on variable interval and much longer than those produced by variable ratio. The results were interpreted as supporting the hypothesis that response rates on variable-interval schedules in rats are lower than those on comparable variable-ratio schedules, primarily because the former schedules reinforce long interresponse times.     

FIXED-TIME SCHEDULES ATTENUATE EXTINCTION-INDUCED PHENOMENA IN THE TREATMENT OF SEVERE ABERRANT BEHAVIOR

We compared the effects of extinction (EXT) and fixed-time (FT) schedules as treatment for severe problem behavior displayed by 3 individuals with developmental disabilities. First, functional analyses identified the reinforcers maintaining aberrant behavior for all 3 individuals. Next, EXT and FT schedules were compared using a multielement design. During EXT, the reinforcer maintaining problem behavior was withheld. During FT, the reinforcers were presented response independently at preset intervals. Results showed that FT schedules were generally more effective than EXT schedules in reducing aberrant behavior. FT schedules may be used in situations when extinction-induced phenomena are problematic.     

Preference for unsegmented interreinforcement intervals in concurrent chains

Five pigeons were trained under concurrent-chain schedules in which a pair of independent, concurrent variable-interval 60-s schedules were presented in the initial link and either both variable-interval or both fixed-interval schedules were presented in the terminal link. Except for the baseline, one of the terminal-link schedules was always a two-component chained schedule and the other was either a simple or a tandem schedule of equal mean interreinforcement interval. The values of the fixed-interval schedules were either 15 s or 60 s; that of the variable-interval schedules was always 60 s. A 1.5-s changeover delay operated during the initial link in some conditions. The pigeons preferred a simple or a tandem schedule to a chain. For the fixed-interval schedules, this preference was greater when the fixed interval was 60 s than when it was 15 s. For the variable-interval schedules, the preferences were less pronounced and occurred only when the changeover delay was in effect. For a given type of schedule and interreinforcement interval, similar preferences were obtained whether the nonchained schedule was a tandem or simple schedule. The changeover delay generally inflated preference and lowered the changeover rate, especially when the terminal-link schedules were either short (15 s) or aperiodic (variable-interval). The results were consistent with the notion that segmenting the interreinforcement interval of a schedule into a chain lowers the preference for it.     

Waiting in pigeons: the effects of daily intercalation on temporal discrimination.

Pigeons trained on cyclic-interval schedules adjust their postfood pause from interval to interval within each experimental session. But on regular fixed-interval schedules, many sessions at a given parameter value are usually necessary before the typical fixed-interval "scallop" appears. In the first case, temporal control appears to act from one interfood interval to the next; in the second, it appears to act over hundreds of interfood intervals. The present experiments look at the intermediate case: daily variation in schedule parameters. In Experiments 1 and 2 we show that pauses proportional to interfood interval develop on short-valued response-initiated-delay schedules when parameters are changed daily, that additional experience under this regimen leads to little further improvement, and that pauses usually change as soon as the schedule parameter is changed. Experiment 3 demonstrates identical waiting behavior on fixed-interval and response-initiated-delay schedules when the food delays are short (less than 20 s) and conditions are changed daily. In Experiment 4 we show that daily intercalation prevents temporal control when interfood intervals are longer (25 to 60 s). The results of Experiment 5 suggest that downshifts in interfood interval produce more rapid waiting-time adjustments than upshifts. These and other results suggest that the effects of short interfood intervals seem to be more persistent than those of long intervals.     

Sensitivity to reinforcement in concurrent arithmetic and exponential schedules

The generalized matching law states that the logarithm of the ratio of responses emitted or time spent responding in concurrent variable-interval schedules is a linear function of the logarithm of the ratio of reinforcements obtained. The slope of this relation, sensitivity to reinforcement, varies about 1.0 but has been shown to be different when obtained in different laboratories. The present paper analyzed the results from 18 experiments on concurrent variable-interval schedule performance and showed that response-allocation sensitivity to reinforcement was significantly smaller when arithmetic, rather than exponential, progressions were used to produce variable-interval schedules. There were no differences in time-allocation sensitivity between the two methods of constructing variable-interval schedules. Since the two laboratories have consistently used different methods for constructing variable-interval schedules, the differences in obtained sensitivities to reinforcement are explained. The reanalysis suggests that animals may be sensitive to differences in the distribution of reinforcements in time.     

Choice, changeover, and travel: A quantitative model

Six pigeons were trained on concurrent variable-interval schedules in which responding on fixed-interval schedules was required to give access to the alternate schedule. Responding on the concurrent schedules was not allowed, after changing over had commenced, until the changeover schedule had been completed. In Parts 1 to 3 of the experiment, the changeover fixed-interval schedules were equal and were 0 s, 10 s, and 20 s, respectively. In each part, the relative frequency of reinforcement obtained on the concurrent schedules was varied over at least five conditions. In Part 4, the concurrent schedules were equal, and one changeover fixed-interval schedule was twice the other. Under these conditions, the absolute sizes of the changeover schedules were varied. Increasing the changeover requirement from 0 s to 10 s (Parts 1 and 2) resulted in increases in the sensitivity of behavior allocation to reinforcers obtained, but no further increase was obtained when the changeover schedules were increased to 20 s (Part 3). In Part 4, performance was biased towards the concurrent schedule that took less time to enter. These results are consistent with a subtractive punishment model of travel in which the degree of punishment is measured by the number of reinforcers apparently lost from a schedule when the subject changes to that schedule. Absolute times spent on the main keys could be accurately described by a previous model of changeover performance.     

DOES SENSITIVITY TO MAGNITUDE DEPEND ON THE TEMPORAL DISTRIBUTION OF REINFORCEMENT?

Our research addressed the question of whether sensitivity to relative reinforcer magnitude in concurrent chains depends on the distribution of reinforcer delays when the terminal-link schedules are equal. In Experiment 1, 12 pigeons responded in a two-component procedure. In both components, the initial links were concurrent variable-interval 40-s variable-interval 40-s, and the terminal links were both 20-s interval schedules in which responses were reinforced by either 4-s of grain in one, or 2-s of grain in the other. The only difference between the components was whether the terminal-link schedules were fixed interval or variable intervals. For all subjects, the relative rate of responding in the initial links for the terminal link that produced the 4-s reinforcer was greater when the terminal links were fixed-interval schedules than when they were variable-interval schedules. This result is contrary to the prediction of Grace's (1994) contextual choice model, but is consistent with both Mazur's (2001) hyperbolic value-added model and Killeen's (1985) incentive theory. In Experiment 2, 4 pigeons responded in a concurrent-chains procedure in which 4-s or 2-s reinforcers were provided independently of responding according to equal fixed-time or mixed-time schedules. Preference for the 4-s reinforcer increased as the variability of the intervals comprising the mixed-time schedules was decreased. Generalized-matching sensitivity of initial-link response allocation to relative reinforcer magnitude was proportional to the geometric mean of the terminal-link delays.     

Choice between delayed reinforcers and fixed-ratio schedules requiring forceful responding.

This experiment measured pigeons' choices between delayed reinforcers and fixed-ratio schedules in which a force of approximately 0.48 N was needed to operate the response key. In ratio-delay conditions, subjects chose between a fixed-ratio schedule and an adjusting delay. The delay was increased or decreased several times a session in order to estimate an indifference point--a delay duration at which the two alternatives were chosen about equally often. Each ratio-delay condition was followed by a delay-delay condition in which subjects chose between the adjusting delay and a variable-time schedule, with the components of this schedule selected to match the ratio completion times of the preceding ratio-delay condition. The adjusting delays at the indifference point were longer when the alternative was a fixed-ratio schedule than when it was a matched variable-time schedule, which indicated a preference for the matched variable-time schedules over the fixed-ratio schedules. This preference increased in a nonlinear manner with increasing ratio size. This nonlinearity was inconsistent with a theory that states that indifference points for both time and ratio schedules can be predicted by multiplying the choice response-reinforcer intervals of the two types of schedules by different multiplicative constants. Two other theories, which predict nonlinear increases in preference for the matched variable-time schedules, are discussed.     

Acquisition of preference in concurrent chains: comparing linear-operator and memory-representational models

Two experiments were conducted to test predictions of 2 models for acquisition of preference in concurrent chains. Pigeons were trained with pairs of fixed-interval and variable-interval terminal-link schedules in a successive-reversal design. Results showed that acquisition rate was faster when fixed-interval schedules preceded and when variable-interval schedules followed the reversal and was approximately constant when the same pair of schedules was used. These results were predicted by both linear-operator and memory-representational models. However, only the linear-operator model predicted effects of schedule duration. Overall, these results demonstrated that a simple linear-operator model can account for the major features of preference acquisition in concurrent chains and suggest that choice in transition and steady state may provide convergent validation of a single delay-discounting function.     

Performance of rats under concurrent variable-interval schedules of negative reinforcement

The behavior of rats under concurrent variable-interval schedules of negative reinforcement was examined. A single one-minute variable-interval programmer determined the availability of 30-second timeouts from electric shock. These were assigned to one or the other of the two component schedules with a probability of 0 to 1.0. The response requirement for the component schedules was standing to the right or left of the center of the experimental chamber. With a six-second changeover delay, the relative time spent under one component schedule varied directly and linearly with the relative number of timeouts earned under that component schedule. The absolute number of changeovers was highest when a similar number of timeouts was earned under each component schedule, and lowest when all or nearly all timeouts were earned under one component schedule. In general, these relations are similar to those reported with concurrent variable-interval schedules of positive reinforcement.     

The effect of multiple SΔ periods on responding on a fixed-interval schedule: V. Effect of periods of complete darkness and of occasional omissions of food presentations

In pigeons under fixed-interval schedules of reinforcement, responding during most of the interval can be suppressed by stimulus conditions never present when a response is promptly followed by reinforcing stimuli. When the external stimuli obtaining immediately before reinforcement are presented during brief probe periods in the course of the interval, the rate of responding in the probe depends on the temporal position of the probe during the interval; the rate of responding is lower during a probe early in the interval than during one late in the interval. The present experiments show that the temporal dependency still holds (1) in birds with no experience under unmodified fixed-interval schedules, (2) when the time between probes is spent in complete darkness, and (3) when food presentations are omitted at the end of 50% of intervals. The results strengthen and extend the conclusion from previous studies that the time relations themselves are the primary control of rate of responding under fixed-interval schedules of reinforcement.     

The role of signals in two variations of differential‐reinforcement‐of‐low‐rate procedures

Differential‐reinforcement‐of‐low‐rate (DRL) schedules are used to decrease the overall rate of, but not eliminate, a target response. Two variations of DRL, spaced‐responding and full‐session, exist. Preliminary comparative analyses suggest that the two schedules function differently when unsignaled. We compared response rates under these two DRL variations with and without signals. In Experiment 1, five preschool students played a game in which points were earned under DRL schedules. In some sessions, a stimulus signaled when responses would be reinforced (S+) or not reinforced (S‐). In others, only an S‐ was present. Signals (S+/S‐) facilitated and maintained responding in both types of DRL schedules. In Experiment 2, we modified the signals with five different preschoolers. Instead of an S‐ only, we did not present any signals. Elimination and high variability of the target response were observed with the S‐ only and absence of S+/S‐, respectively. Signaled DRL schedules are recommended for application.     

Are behaviors at one alternative in concurrent schedules independent of contingencies at the other alternative?

Some have reported changing the schedule at one alternative of a concurrent schedule changed responding at the other alternative (Catania, 1969), which seems odd because no contingencies were changed there. When concurrent schedules are programmed using two schedules, one associated with each alternative that operate continuously, changing the schedule at one alternative also changes the switch schedule at the other alternative. Thus, changes in responding at the constant alternative could be due to the change in the switch schedule. To assess this possibility, six rats were exposed to a series of conditions that alternated between pairs of interval schedules at both alternatives and a pair of interval schedules at one, constant, alternative and a pair of extinction schedules at the other alternative. Comparing run lengths, visit durations and response rates at the constant alternative in the alternating conditions did not show consistent increases and decreases when a strict criterion for changes was used. Using a less stringent definition (any change in mean values) showed changes. The stay/switch analysis suggests it may be inaccurate to apply behavioral contrast to procedures that change from concurrent variable‐interval variable‐interval schedules to concurrent variable‐interval extinction schedules because the contingencies in neither alternative are constant.     

Choice and conditioned reinforcement.

A potential weakness of one formulation of delay-reduction theory is its failure to include a term for rate of conditioned reinforcement, that is, the rate at which the terminal-link stimuli occur in concurrent-chains schedules. The present studies assessed whether or not rate of conditioned reinforcement has an independent effect upon choice. Pigeons responded on either modified concurrent-chains schedules or on comparable concurrent-tandem schedules. The initial link was shortened on only one of two concurrent-chains schedules and on only one of two corresponding concurrent-tandem schedules. This manipulation increased rate of conditioned reinforcement sharply in the chain but not in the tandem schedule. According to a formulation of delay-reduction theory, when the outcomes chosen (the terminal links) are equal, as in Experiment 1, choice should depend only on rate of primary reinforcement; thus, choice should be equivalent for the tandem and chain schedules despite a large difference in rate of conditioned reinforcement. When the outcomes chosen are unequal, however, as in Experiment 2, choice should depend upon both rate of primary reinforcement and relative signaled delay reduction; thus, larger preferences should occur in the chain than in the tandem schedules. These predictions were confirmed, suggesting that increasing the rate of conditioned reinforcement on concurrent-chains schedules may have no independent effect on choice.     

Partial Reinforcement in Appetitive Pavlovian Conditioning with Rats

Four experiments examined the effects of a partial reinforcement schedule on extinction using appetitive Pavlovian conditioning. Extinction was slower after partial than after continuous reinforcement when the schedules were administered to different groups (Experiment 1). The opposite result was found in Experiments 2 and 3 when both schedules were presented to the same group in the same context. When the schedules were presented to the same group in different contexts, then extinction was again slower after partial reinforcement (Experiment 3). Experiment 4 demonstrated that a change of context facilitates extinction to a greater extent after conditioning with a partial reinforcement schedule than with a continuous one. The results are explained by assuming that the nonreinforced trials of a partial reinforcement schedule create an internal state that serves as a contextual cue.