THE EFFECT OF CHANGES IN CRITERION VALUE ON DIFFERENTIAL REINFORCEMENT OF LOW RATE SCHEDULE PERFORMANCE

The differential reinforcement of low rate (DRL) schedule is commonly used to assess impulsivity, hyperactivity, and the cognitive effects of pharmacological treatments on performance. A DRL schedule requires subjects to wait a certain minimum amount of time between successive responses to receive reinforcement. The DRL criterion value, which specifies the minimum wait time between responses, is often shifted towards increasingly longer values over the course of training. However, the process invoked by shifting DRL values is poorly understood. Experiment 1 compared performance on a DRL 30‐s schedule versus a DRL 15‐s schedule that was later shifted to a DRL 30‐s schedule. Dependent measures assessing interresponse time (IRT) production and reward‐earning efficiency showed significant detrimental effects following a DRL schedule transition in comparison with the performance on a maintained DRL 30‐s schedule. Experiments 2a and 2b assessed the effects of small incremental changes vs. a sudden large shift in the DRL criterion on performance. The incremental changes produced little to no disruption in performance compared to a sudden large shift. The results indicate that the common practice of incrementing the DRL criterion over sessions may be an inefficient means of training stable DRL performance.     

Temporally spaced responding by pigeons: development and effects of deprivation and extinction

In the first five or six sessions on a DRL 20-sec schedule of reinforcement there developed a stable performance characterized by a relatively constant conditional probability of occurrence (IRTs/op) of interresponse times (IRTs) of durations greater than 5 or 6 sec. Extinction and the level of deprivation changed both the overall rate of responding and the form of the function relating the duration of an IRT to its value of IRTs/op. The value of IRTs/op decreased more rapidly for short than for longer IRTs, resulting in the emergence of a finer discrimination of IRT duration.     

Discrimination and emission of temporal intervals by pigeons

Because the frequency distribution of IRTs showed little or no control by a DRL schedule, the schedule was modified so that the pigeon's behavior after each IRT would indicate whether or not it had discriminated the duration of the IRT. After every two pecks on a red key, the key changed to blue for 30 sec. Then it automatically became red again. Pecks on the blue key were reinforced with food on a VI schedule only when the preceding IRT on the red key had been longer than 18 sec. The birds did not selectively emit longer IRTs on the red key: the value of IRTs/op did not increase with IRT duration. However, they did discriminate the duration of the IRT emitted on the red key: the rate of pecking on the blue key was an increasing function of the duration of the preceding IRT on the red key.     

Stimulus generalization of behavioral history: Interspecies generality and persistence of generalization gradients

Four pigeons were exposed to a tandem variable-interval (VI) fixed-ratio (FR) schedule in the presence of a 50-pixel (about 15 mm) square or an 80-pixel (about 24 mm) square and to a tandem VI differential-reinforcement-of-low-rate (DRL) schedule when a second 80-pixel or 50-pixel square was present. The values of the VI and FR schedules were adjusted to equate reinforcement rates in the two tandem schedules. Following this, a square-size continuum generalization test was administered under a fixed-interval (FI) schedule or extinction. In the first testing session, response frequency was a graded function of the similarity of the test stimuli to the training stimuli for all pigeons. These systematic generalization gradients persisted longer under the FI schedule than under extinction.     

Effects of reinforcer magnitude on responding under differential-reinforcement-of-low-rate schedules of rats and pigeons

Experiment I investigated the effects of reinforcer magnitude on differential-reinforcement-of-low-rate (DRL) schedule performance in three phases. In Phase 1, two groups of rats (n = 6 and 5) responded under a DRI. 72-s schedule with reinforcer magnitudes of either 30 or 300 microl of water. After acquisition, the water amounts were reversed for each rat. In Phase 2, the effects of the same reinforcer magnitudes on DRL 18-s schedule performance were examined across conditions. In Phase 3, each rat responded unider a DR1. 18-s schedule in which the water amotnts alternated between 30 and 300 microl daily. Throughout each phase of Experiment 1, the larger reinforcer magnitude resulted in higher response rates and lower reinforcement rates. The peak of the interresponse-time distributions was at a lower value tinder the larger reinforcer magnitude. In Experiment 2, 3 pigeons responded under a DRL 20-s schedule in which reinforcer magnitude (1-s or 6-s access to grain) varied iron session to session. Higher response rates and lower reinforcement rates occurred tinder the longer hopper duration. These results demonstrate that larger reinforcer magnitudes engender less efficient DRL schedule performance in both rats and pigeons, and when reinforcer magnitude was held constant between sessions or was varied daily. The present results are consistent with previous research demonstrating a decrease in efficiency as a function of increased reinforcer magnituide tinder procedures that require a period of time without a specified response. These findings also support the claim that DRI. schedule performance is not governed solely by a timing process.     

Temporal regulation of children with autism spectrum disorder exposed to a differential-reinforcement-of low-rates schedule

This study investigated temporal adjustment of children with autism spectrum disorder under a differential-reinforcement-of-low-rates (DRL) schedule. Sixteen participants, aged 3.2 to 7 years, were exposed to two conditions, DRL 5 s and DRL 20 s. Children participated in 7 sessions in each condition, except for 1 participant who attained the adjustment criteria in the DRL 5-s schedule. Temporal adjustment was measured with the proportion of reinforced interresponse times (IRTs) and the mean IRT. The operant response was a press on a touch screen and the reinforcers were cartoons. IQ and receptive language were measured prior to the DRL sessions. Results showed that the mean proportion of reinforced IRTs was slightly higher in the DRL 5-s schedule. The mean IRT was above the IRT requirement in both conditions. However, substantial individual variability was observed. Children with higher IQ and receptive language scores presented a greater proportion of reinforced IRTs in both conditions. Moreover, participants who adjusted their responses to the DRL 5-s schedule were more likely to adjust responding to the DRL 20-s schedule. This suggests that some children might be more sensitive to reinforcement contingencies than others. This study points at future research in the field of timing in children.     

Sequential response effects in the white rat during conditioning and extinction on a DRL schedule,

Sequential IRT data were obtained for three rats on a DRL 60-sec reinforcement schedule. It was found that first-order sequential dependencies exist under this schedule, including the partial dependence of the length of any given IRT on the length of the preceding IRT. The sequential analysis also served to extend the finding in the literature, based on frequency distributions, that the likelihood of a reinforced IRT is greater after a reinforced IRT than a non-reinforced IRT. Rapid extinction and reconditioning were obtained.     

Effects of a DRL contingency added to a fixed-interval reinforcement schedule

Following 30 days of reinforcement for the bar press response of two white rats on 30-sec fixed-interval (FI), a DRL component was added so that a minimal interresponse time (IRT) for the reinforced response, in addition to the FI variable, was necessary for reinforcement. Marked control over response rate by the superimposed DRL requirement was demonstrated by an inverse hyperbolic function as the DRL component was increased from 1 to 24 sec within the constant 30-sec FI interval. Interresponse time and post-reinforcement (post-S^R) “break” distributions taken at one experimental point (DRL = 24 sec) suggested that a more precise temporal discrimination was initiated by an S^R than by a response, since the relative frequency of a sequence of two reinforced responses appeared greater than that of a sequence of a non-reinforced response followed by a reinforced one. This latter finding was confirmed with new animals in a follow-up experiment employing a conventional 24-sec DRL schedule.     

Behavior under large values of the differential-reinforcement-of-low-rate schedule

Pigeons pecked a key and rats pressed a lever for food reinforcement under large values of the differential-reinforcement-of-low-rate schedule. Each subject was tested under 10 different schedule values ranging from 1 to 45 min and was exposed to each schedule value at least twice. The mean interresponse time and mean interreinforcement time increased with the schedule value according to power functions. Response-probability functions were computed for schedule values below 20 min and showed an increase in response probability as a function of time since the last response in most cases. Mean responses per reinforcer increased as a function of schedule value for the rats, but decreased as a function of schedule value for the pigeons. The proportion of responses with interresponse times shorter than 1 sec were an increasing function of schedule value for the pigeons, but did not vary as a function of schedule value for the rats.     

Choice between response units: The rate constancy model

In a conjoint schedule, reinforcement is available simultaneously on two or more schedules for the same response. The present experiments provided food for key pecking on both a random-interval and a differential-reinforcement-of-low-rate (DRL) schedule. Experiment 1 involved ordinary DRL schedules; Experiment 2 added an external stimulus to indicate when the required interresponse time had elapsed. In both experiments, the potential reinforcer frequency from each component was varied by means of a second-order fixed-ratio schedule, and the DRL time parameter was changed as well. Response rates were described by a model stating that time allocation to each component matches the relative frequency of reinforcement for that component. When spending time in a given component, the subject is assumed to respond at the rate characteristic of baseline performance. This model appeared preferable to the absolute-rate version of the matching law. The model was shown to be applicable to multiple-response concurrent schedules as well as to conjoint schedules, and it described some of the necessary conditions for response matching, undermatching, and bias. In addition, the pigeons did not optimize reinforcer frequency.     

The effect of physical restraint on behavior under the differential-reinforcement-of-low-rate schedule

Previous studies have identified and manipulated collateral behavior to assess the effect of collateral behavior on performance under the differential-reinforcement-of-low-rate (DRL) schedule. However, conclusions could not be applied to subjects not observed to engage in collateral behavior. The present study used a technique that prevented the occurrence of the types of collateral behavior typically observed in the pigeon. This technique did not require the identification of collateral behavior in the subjects. The exclusion of the types of collateral behavior typically observed in pigeons resulted in higher response rates and lower reinforcement rates under large DRL values but had no effect at lower DRL values. It was concluded that collateral behavior is necessary for low response rates and high reinforcement rates under large DRL values.     

Conditioned reinforcement value and choice.

The delay-reduction hypothesis of conditioned reinforcement states that the reinforcing value of a food-associated stimulus is determined by the delay to primary reinforcement signaled by the onset of the stimulus relative to the average delay to primary reinforcement in the conditioning situation. In contrast, most contemporary models of conditioned reinforcement strength posit that the reinforcing strength of a stimulus is some simple function only of the delay to primary reinforcement in the presence of stimulus. The delay-reduction hypothesis diverges from other conditioned reinforcement models in that it predicts that a fixed-duration food-paired stimulus will have different reinforcing values depending on the frequency of its presentation. In Experiment 1, pigeons' key pecks were reinforced according to concurrent-chains schedules with variable-interval 10-second and variable-interval 20-second terminal-link schedules. The initial-link schedule preceding the shorter terminal link was always variable-interval 60 seconds, and the initial-link schedule requirement preceding the longer terminal link was varied between 1 second and 60 seconds across conditions. In Experiment 2, the initial-link schedule preceding the longer of two terminal links was varied for each of three groups of pigeons. The terminal links of the concurrent chains for the three groups were variable-interval 10 seconds and 20 seconds, variable-interval 10 seconds and 30 seconds, and variable-interval 30 seconds and 50 seconds. In both experiments, preference for the shorter terminal link was either a bitonic function or an inverse function of the initial-link schedule preceding the longer terminal-link schedule. Consistent with the predictions of the delay-reduction hypothesis, the relative values of the terminal-link stimuli changed as a function of the overall frequency of primary reinforcement. Vaughan's (1985) melioration model, which was shown to be formally similar to Squires and Fantino's (1971) delay-reduction model, can be modified so as to predict these results without changing its underlying assumptions.     

Stimulus generalization and the response-reinforcement contingency

Generalization gradients along a line-tilt continuum were obtained from groups of pigeons that had been trained to peck a key on different schedules of reinforcement. In Exp. I, gradients following training on a differential-reinforcement-of-low-rate (DRL) schedule proved to be much flatter than gradients following the usual 1-min variable interval (VI) training. In Exp. II, the value of the VI schedule itself was parametrically studied; Ss trained on long VI schedules (e.g., 4-min) produced much flatter gradients than Ss trained on short VI schedules (30-sec; 1-min). The results were interpreted mainly in terms of the relative control exerted by internal, proprioceptive cues on the different reinforcement schedules. Several implications of the results for other problems in the field of stimulus generalization are discussed.     

The role of contingencies and "principles of behavioral variation" in pigeons' pecking

Staddon and Simmelhag's proposal that behavior is produced by “principles of behavioral variation” instead of contingencies of reinforcement was tested in two experiments. In the first experiment pigeons were exposed to either a fixed-interval schedule of response-contingent reinforcement, an autoshaping schedule of stimulus-contingent reinforcement, or a fixed-time schedule of noncontingent reinforcement. Pigeons exposed to contingent reinforcement came to peck more rapidly than those exposed to noncontingent reinforcement. Staddon and Simmelhag's “principles of behavioral variation” included the proposal that patterns (interim and terminal) were a function of momentary probability of reinforcement. In the second experiment pigeons were exposed to either a fixed-time or a random-time schedule of noncontingent reinforcement. Pecking showed a constant frequency of occurrence over postfood time on the random-time schedule. Most behavior showed patterns on the fixed-time schedule that differed in overall shape (i.e., interim versus terminal) from those shown on the random-time schedule. It was concluded that both the momentary probability of reinforcement and postfood time can affect patterning.     

Species differences in temporal control of behavior

Temporal control of rats' and pigeons' responding was analyzed and compared in detail on fixed-interval and fixed-time schedules with parameters of 30, 60, and 120 seconds. On fixed-time schedules, rats' responding decreased greatly or ceased, whereas pigeons continued to respond, especially on low schedule values. The running rate of responses (calculated by excluding the postreinforcement pause) was related to the duration of the preceding postreinforcement pause for rats but not for pigeons. Changes in response rate in successive segments of the interval were best described by normal curves. The relationship between midpoints of the normal curves and schedule value was a power function, with an exponent of less than one for pigeons but greater than one for rats. These differences could be explained in terms of a basic difference between the key-peck and lever-press responses, the two being differently affected by the response-eliciting properties of food.     

The effect of acceleration on food-reinforced DRL and FR

Performance on DRL 10 sec and FR 5 was studied after exposure to acceleration. After four rats, two on each of the above schedules, had stabilized they were exposed to 5 hr of acceleration at 5 G immediately before daily experimental sessions. Food intake was also studied in rats given access to food daily in their home cages and exposed to acceleration immediately before the free-feeding session. Weight gain of free-feeding animals and reinforcement intake of experimental animals dropped after acceleration. Over-all response rate on the FR was depressed markedly by acceleration but local response rates did not appear to be affected. IRT distributions of DRL sessions after acceleration were markedly shifted toward the long intervals. A sequential plot of IRTs on acceleration days showed an altered, but relatively stable, temporal patterning of responses followed by an abrupt return to the normal baseline toward the end of the session.     

Control of pigeons' matching-to-sample performance by differential sample response requirements

Pigeons were trained on a matching-to-sample task in which sample hue and required sample-specific observing behavior provided redundant, relevant cues for correct choices. On trials that involved red and yellow hues as comparison stimuli, a fixed-ratio 16 schedule (FR 16) was required to illuminate the comparisons when the sample was red, and a differential-reinforcement-of-low-rates 3-sec schedule (DRL 3-sec) was required when the sample was yellow. On trials involving blue and green hues as comparison stimuli, an FR 16 schedule was required when the sample was blue and a DRL 3-sec schedule was required when the sample was green. For some pigeons, a 0-sec delay intervened between sample offset and comparison onset, whereas other pigeons experienced a random mixture of 0-sec and 2-sec delay trials. Test trial performance at 0-sec delay indicated that sample-specific behavior controlled choice performance considerably more than sample hue did. Test performance was independent of whether original training involved all 0-sec delay trials or a mixture of 0-sec and 2-sec delays. Sample-specific observing response requirements appear to facilitate pigeons' matching-to-sample performance by strengthening associations between the observing response and correct choice.     

Conditioned suppression of drl responding: Effect of ucs intensity, schedule parameter, and schedule context

In Experiment I, groups of rats were trained to press a lever for food reinforcement on differential reinforcement of low rate (DRL) schedules which differed in parameter value. A stimulus which terminated with either a 0.5-mA or 2.0-mA electric shock was then superimposed upon each DRL baseline. In general, the magnitude of conditioned suppression was an inverse function of DRL schedule parameter and a direct function of shock intensity. Experiment II demonstrated that the rate of responding maintained by the DRL component of a multiple DRL-extinction schedule decreased during a stimulus preceding a 0.5-mA shock, whereas the rate of responding maintained by the DRL component of a multiple DRL-variable interval schedule showed little change or increased slightly during a stimulus preceding a 0.5-mA shock.     

Behavioral variability and frequency-dependent selection.

In Experiment 1, two conditions were compared: (a) a variability schedule in which food reinforcement was delivered for the fourth peck in a sequence that differed from the preceding N four-peck sequences, with the value of N continuously adjusted to maintain reinforcement probability approximately constant; and (b) a control condition in which the variability constraint was dropped but reinforcement probability remained constant. Pigeons responded approximately randomly under the variability schedule but showed strong stereotyped behavior under the control condition. Experiments 2 and 3 tested the idea that variability is the outcome of a type of frequency-dependent selection, namely differential reinforcement of infrequent behavior patterns. The results showed that pigeons alternate when frequency-dependent selection is applied to single pecks because alternation is an easy-to-learn stable pattern that satisfies the frequency-dependent condition. Nevertheless, 2 of 4 pigeons showed random behavior when frequency-dependent selection was applied to two pecks, even though double alternation is a permissible and stable stereotype under these conditions. It appears that random behavior results when pigeons are unable to acquire the stable stereotyped behavior under a given frequency-dependent schedule.     

On the primacy of molecular processes in determining response rates under variable-ratio and variable-interval schedules

This study focused on variables that may account for response-rate differences under variable-ratio (VR) and variable-interval (VI) schedules of reinforcement. Four rats were exposed to VR, VI, tandem VI differential-reinforcement-of-high-rate, regulated-probability-interval, and negative-feedback schedules of reinforcement that provided the same rate of reinforcement. Response rates were higher under the VR schedule than the VI schedule, and the rates on all other schedules approximated those under the VR schedule. The median reinforced interresponse time (IRT) under the VI schedule was longer than for the other schedules. Thus, differences in reinforced IRTs correlated with differences in response rate, an outcome suggestive of the molecular control of response rate. This conclusion was complemented by the additional finding that the differences in molar reinforcement-feedback functions had little discernible impact on responding.