Effects of reinforcer magnitude on responding under differential-reinforcement-of-low-rate schedules of rats and pigeons

Experiment I investigated the effects of reinforcer magnitude on differential-reinforcement-of-low-rate (DRL) schedule performance in three phases. In Phase 1, two groups of rats (n = 6 and 5) responded under a DRI. 72-s schedule with reinforcer magnitudes of either 30 or 300 microl of water. After acquisition, the water amounts were reversed for each rat. In Phase 2, the effects of the same reinforcer magnitudes on DRL 18-s schedule performance were examined across conditions. In Phase 3, each rat responded unider a DR1. 18-s schedule in which the water amotnts alternated between 30 and 300 microl daily. Throughout each phase of Experiment 1, the larger reinforcer magnitude resulted in higher response rates and lower reinforcement rates. The peak of the interresponse-time distributions was at a lower value tinder the larger reinforcer magnitude. In Experiment 2, 3 pigeons responded under a DRL 20-s schedule in which reinforcer magnitude (1-s or 6-s access to grain) varied iron session to session. Higher response rates and lower reinforcement rates occurred tinder the longer hopper duration. These results demonstrate that larger reinforcer magnitudes engender less efficient DRL schedule performance in both rats and pigeons, and when reinforcer magnitude was held constant between sessions or was varied daily. The present results are consistent with previous research demonstrating a decrease in efficiency as a function of increased reinforcer magnituide tinder procedures that require a period of time without a specified response. These findings also support the claim that DRI. schedule performance is not governed solely by a timing process.     

Interresponse-time analysis of stimulus control in multiple schedules

Interresponse-time distributions were recorded in two components of multiple variable-interval schedules that were varied over several conditions. Values of the exponent for power functions relating ratios of interresponse times emitted per opportunity to ratios of reinforcers obtained in the two components varied with interresponse-time class interval. The exponent (sensitivity to reinforcement) afforded a measure of stimulus control exerted by the discriminative stimuli. Exponents were near zero for short interresponse times, consistent with previous conclusions that responses following short interresponse times are controlled by response-produced or proprioceptive stimuli. Values of exponents increased with longer interresponse times, indicating strong control by exteroceptive stimuli over responses following interresponse times of approximately one second or longer.     

A procedure for generating differential "sample" responding without different exteroceptive stimuli

Sidman (1994, 2000) suggested that responses as well as stimuli can join equivalence classes, a hypothesis difficult to test because differential responding typically requires different stimuli. The present experiments describe a procedure with pigeons that avoids this potential confounding effect. In Experiment 1, spacing two responses 3 s apart (a differential-reinforcement-of-low-rate [DRL] schedule) to a white stimulus on some trials produced food or the comparison stimuli in a matching task, whereas pecking 10 or more times with no temporal restrictions (a fixed-ratio [FR] schedule) produced the same effect on other trials. Completing the alternative (unscheduled) requirement terminated the white stimulus and repeated the trial. Following such errors, pigeons learned to switch to the alternative response pattern on the repeat trials. In addition, the correct response pattern functioned as a conditional cue for comparison choice. In Experiment 2, mixed DRL-FR training was preceded by two-sample/two-alternative matching-to-sample with DRL and FR sample-response requirements. In a subsequent transfer test in which the correct response pattern to white served as the sample, pigeons preferentially chose the comparison previously reinforced following that pattern in the baseline task. This "unsignaled response" procedure may be useful for assessing whether differential responses can be members of acquired equivalence classes.     

Stimulus control of Pavlovian facilitation

Two experiments were conducted using an autoshaping procedure with pigeons to examine whether dimensional stimulus control by a Pavlovian facilitator parallels the control established following operant discrimination training. Facilitation training consisted of the presentation of a black vertical line on a white background as the B stimulus in a feature-positive discrimination in which the A stimulus (white keylight) was followed by grain presentation only if preceded by B. In this way, B facilitates or sets the occasion for pecking at A. Subsequent testing for generalization along the line-orientation dimension produced decremental gradients when the facilitation paradigm incorporated an explicit feature-negative stimulus (B−). These results parallel the decremental control obtained following operant discrimination training and suggest that Pavlovian facilitators and instrumental discriminative stimuli are functionally equivalent.     

Within-session changes in responding during several simple schedules

Pigeons' key pecking was reinforced by food delivered by several fixed-interval, variable-ratio, and differential-reinforcement-of-low-rate schedules. Rate of responding, number of responses per reinforcer, length of postreinforcement pause, running response rate, and the time required to collect an available reinforcer changed systematically within sessions when the schedules provided high rates of reinforcement, but usually not when they provided low rates. These results suggest that the factors that produce within-session changes in responding are generally similar for different schedules of reinforcement. However, a separate factor may also contribute during variable-ratio schedules. The results question explanations for within-session changes that are related solely to the passage of time, to responding, and to one interpretation of attention. They support the idea that one or more factors related to reinforcement play a role.     

Stimulus control of behavioral history.

Pigeons were exposed to two different reinforcement schedules under different stimulus conditions in each of two daily sessions separated by 6 hr (Experiments 1 and 2) or in a single session (Experiment 3). Following this, either a fixed-interval (Experiment 1) or a variable-interval schedule (Experiments 2 and 3) was effected in both stimulus conditions. In the first two experiments, exposure to fixed-ratio or differential-reinforcement-of-low-rate schedules led to response-rate, but not pattern, differences in subsequent performance on fixed- or variable-interval schedules that persisted for up to 60 sessions. The effects of reinforcement-schedule history on fixed-interval schedule performance generally were more persistent. In Experiment 3, a history of high and low response rates in different components of a multiple schedule resulted in subsequent response-rate differences under identical variable-interval schedules. Higher response rates initially occurred in the component previously correlated with high response rates. For 3 of 4 subjects, the differences persisted for 20 or more sessions. Previous demonstrations of behavioral history effects have been confined largely to between-subject comparisons. By contrast, the present results demonstrate strong behavioral effects of schedule histories under stimulus control within individual subjects.     

Stimulus control and response bias in an analogue prey-detection procedure

The present study compared the performance of 6 pigeons trained to detect luminance differences in two different signal-detection procedures. Exposed to a three-key array, the pigeons were trained to peck the left key when the brighter of two light intensities had been presented on the center key and to peck the right key when the dimmer of two light intensities had been presented on the center key. Procedure A was a standard signal-detection procedure in which left/bright and right/dim responses produced food reinforcement and left/dim and right/bright responses produced periods of timeout. Procedure B was designed to simulate some of the contingencies operating in a prey-detection situation. Left-key responses produced reinforcement following the brighter center-key stimulus and a period of timeout following the dimmer center-key stimulus. Right-key responses always produced a short period of timeout irrespective of the stimulus. Within each procedure, the duration of timeout arranged for false alarms (left/dim responses) was varied between 3 s and 120 s. Measures of accuracy and response bias were compared between the two procedures. The timeout manipulation produced systematic, but relatively small, changes in these measures when right/dim responses (i.e., correct rejections) produced reinforcement (Procedure A). Arranging timeout for right/dim responses in Procedure B produced greater variability in accuracy and response bias than did arranging reinforcement, but this variability was not related to timeout duration. Overall, discrimination accuracy was considerably higher when right/dim responses produced timeout than when they resulted in reinforcement, and accuracy was accompanied by a large bias toward the response associated with reinforcement. These results are consistent with a recently proposed model of signal detection.     

Stimulus control in the use of landmarks by pigeons in a touch-screen task

Pigeons were tested in a search task on the surface of a monitor on which their responses were registered by a touch-sensitive device. A graphic landmark array was presented consisting of a square outline (the frame) and a colored “landmark.” The unmarked goal, pecks at which produced reward, was located near the center of one edge of the frame, and the landmark was near it. The entire array was displaced without rotation on the monitor from trial to trial. On occasional no-reward tests, the following manipulations were made to the landmark array: (a) either the frame or the landmark was removed; (2) either one edge of the frame or the landmark was shifted; and (3) two landmarks were presented with or without the frame present. On these two-landmark tests, the frame, when present, defined which was the “correct” landmark. When the frame was absent, the “correct” landmark was arbitrarily determined. Results showed that pecks of 2 pigeons were controlled almost solely by the landmark, pecks of 3 were controlled primarily by the landmark but the frame could distinguish the correct landmark, and 1 bird's behavior was controlled primarily by the frame. Stimulus control in this search task is thus selective and differs across individuals. Comparisons to other search tasks and to other stimulus control experiments are made.     

Stimulus effects on concurrent performance in transition

Six experimentally naive pigeons were exposed to concurrent variable-interval variable-interval schedules in a three-key procedure in which food reinforcement followed pecks on the side keys and pecks on the center key served as changeover responses. In Phase 1, 3 birds were exposed to 20 combinations of five variable-interval values, with each variable-interval value consistently associated with a different color on the side keys. Another 3 pigeons were exposed to the same 20 conditions, but with a more standard procedure that used a nondifferential discriminative stimulus on the two side keys throughout all conditions. In Phase 2, the differential and nondifferential stimulus conditions were reversed for each pigeon. Each condition lasted for one 5-hr session and one subsequent 1-hr session. In the last 14 conditions of each phase, the presence of differential discriminative stimuli decreased the time necessary for differential responding to develop and increased the sensitivity of behavior to reinforcement distribution in the 1st hr of training; during the last hours of training in each condition, however, the effects of the differential discriminative stimuli could not be distinguished from the effects of reinforcement distribution per se. These results show the importance of studying transitions in behavior as well as final performance. They may also be relevant to discrepancies in the results of previous experiments that have used nonhuman and human subjects.     

Temporal patterns of responding in small fixed-ratio schedules

Pigeons were exposed to an ascending series of small fixed-ratio schedules from fixed-ratio 1 to 7. Two of those pigeons were later placed on a fixed-ratio 30 schedule. The two primary dependent variables were the postreinforcement pause and the interresponse time. Changes in these variables under small fixed ratios were sometimes opposite to changes reported with large fixed ratios. For example, postreinforcement pauses decreased in length as the fixed-ratio requirement increased from fixed-ratio 1 to fixed-ratio 3. Also, the interresponse times early in the small fixed-ratio schedule were shorter than those immediately preceding reinforcement. These findings question the role of interresponse-time reinforcement in determining temporal patterns of responding under small fixed-ratio schedules. They also suggest that there may be a limited region in which the independent variable, fixed-ratio size, does not operate as previously described.     

Free-operant forgetting: Delayed stimulus control of multiple-schedule performance

Pigeons' responses were reinforced in two components of several multiple variable-interval variable-interval schedules of food reinforcement. In one component, the key was illuminated green for 15 seconds and white for 45 seconds. In the other component, the key was illuminated red for 15 seconds and white for 45 seconds. Values for the exponent of the power functions relating response ratios to reinforcement ratios were higher in the presence of the discriminative stimuli (green or red) than in the presence of white. Sensitivity of response ratios to changes in reinforcement ratios provided an index of the extent to which responding was under delayed stimulus control by prior discriminative stimuli.     

Disruption of responding maintained by conditioned reinforcement: alterations in response-conditioned-reinforcer relations

An observing procedure was used to investigate the effects of alterations in response-conditioned-reinforcer relations on observing. Pigeons responded to produce schedule-correlated stimuli paired with the availability of food or extinction. The contingency between observing responses and conditioned reinforcement was altered in three experiments. In Experiment 1, after a contingency was established in baseline between the observing response and conditioned reinforcement, it was removed and the schedule-correlated stimuli were presented independently of responding according to a variable-time schedule. The variable-time schedule was constructed such that the rate of stimulus presentations was yoked from baseline. The removal of the observing contingency reliably reduced rates of observing. In Experiment 2, resetting delays to conditioned reinforcement were imposed between observing responses and the schedule-correlated stimuli they produced. Delay values of 0, 0.5, 1, 5, and 10 s were examined. Rates of observing varied inversely as a function of delay value. In Experiment 3, signaled and unsignaled resetting delays between observing responses and schedule-correlated stimuli were compared. Baseline rates of observing were decreased less by signaled delays than by unsignaled delays. Disruptions in response-conditioned-reinforcer relations produce similar behavioral effects to those found with primary reinforcement.     

Stimulus control topographies and tests of symmetry in pigeons

Pigeons were tested for symmetry after A-B training under conditions designed to avoid problems that may prevent its emergence, namely the change of stimulus location in testing relative to training and the lack of requisite discrimination training. In Experiment 1, samples appeared in two locations during baseline training to minimize the impact of stimulus location. Experiments 2 and 3 included multiple-location training along with additional identity and symbolic matching training, respectively, to explicitly train all of the simultaneous and successive stimulus discriminations required for testing. Experiment 4 provided reinforcement for symmetrical matching relations with some stimulus sets (with multiple-location training) prior to symmetry testing with different sets. In all experiments, pigeons showed no evidence of symmetry despite the fact that baseline (A-B) matching transferred to novel locations. Additional tests for reflexivity (Experiment 2) yielded similar outcomes. These results indicate that the change in stimulus location is not the sole reason that pigeons do not show symmetry and increase the plausibility of arguments that symmetry and other indexes of stimulus equivalence may be beyond the capabilities of the pigeon.     

THE EFFECT OF CHANGES IN CRITERION VALUE ON DIFFERENTIAL REINFORCEMENT OF LOW RATE SCHEDULE PERFORMANCE

The differential reinforcement of low rate (DRL) schedule is commonly used to assess impulsivity, hyperactivity, and the cognitive effects of pharmacological treatments on performance. A DRL schedule requires subjects to wait a certain minimum amount of time between successive responses to receive reinforcement. The DRL criterion value, which specifies the minimum wait time between responses, is often shifted towards increasingly longer values over the course of training. However, the process invoked by shifting DRL values is poorly understood. Experiment 1 compared performance on a DRL 30‐s schedule versus a DRL 15‐s schedule that was later shifted to a DRL 30‐s schedule. Dependent measures assessing interresponse time (IRT) production and reward‐earning efficiency showed significant detrimental effects following a DRL schedule transition in comparison with the performance on a maintained DRL 30‐s schedule. Experiments 2a and 2b assessed the effects of small incremental changes vs. a sudden large shift in the DRL criterion on performance. The incremental changes produced little to no disruption in performance compared to a sudden large shift. The results indicate that the common practice of incrementing the DRL criterion over sessions may be an inefficient means of training stable DRL performance.     

Delayed and current stimulus control in successive discriminations

In a successive discrimination in which successively alternating red and green hues signaled component variable-interval schedules, sensitivity of the ratio of responses in the two components to variation in the component reinforcer ratio decreased systematically during the course of the component. This decrease in stimulus control or discrimination over the course of the component was shown to be the result of delayed control of responding during the component by the stimulus transition between components. When the red–green stimulus transition was altered by interpolating a white stimulus at the end of each 60-s component, discrimination at the beginning of the component (measured by the power-function exponent for sensitivity to reinforcement) was reduced. Conditions with the white stimulus inserted in other quarters of the component indicated that the current discriminative stimulus exerts control over responding throughout the component, whereas during about the first half of the component, response differentials are influenced by the transition between discriminative stimuli.     

Stimulus and reinforcer relativity in multiple schedules: Local and dimensional effects on sensitivity to reinforcement

Pigeons' responses in two successive components of multiple schedules were reinforced according to variable-interval schedules of reinforcement that varied over five different conditions. Within each session of all conditions, line orientations of 0°, 30°, or 45° in Component 1 alternated with orientations of 45°, 60°, or 90° in Component 2. Response rates were recorded in three successive subintervals of each component. Ratios were taken between the response rate in each Component 1 line orientation and the response rate in each Component 2 orientation. These ratios were found to be power functions of the corresponding ratios of obtained reinforcement rates. Sensitivity of response ratios to changes in reinforcer ratios, given by the value of the exponent of the power function, increased systematically with increasing disparity between the dimensional values of orientation stimuli. In addition, sensitivity decreased systematically over successive subintervals of components, that is, with increasing time since component alternation. Dimensional and local (subinterval) effects interacted in that sensitivity increased with stimulus disparity to a far greater extent in the first subinterval than later in components. The data could be described by a combination of rectangular hyperbolae which attributed the interaction between local and dimensional effects to limits set by local effects on the extent that stimulus differences could affect sensitivity.     

Intractable properties of responding under a fixed-interval schedule

The behavior engendered by the fixed-interval schedule is characterized by its variability within and across intervals. The present experiment was designed to assess further the magnitude of interval-to-interval dynamics and to explore conditions which might enhance control by response number for subsequent output. Pigeons were exposed to three experimental manipulations after responding had stabilized under a fixed-interval five-minute schedule. First, a discrete five-stimulus counter was added so that the key color changed after a fixed number of responses. Then additional grain presentations were made at the end of the interval so that high response output was differentially reinforced in the presence of the counter stimuli. Finally, the counter stimuli were presented as an irregular clock (i.e., independently of responding), but the durations were yoked to performance under the counter condition. The data show that response number can exert influence from one interval to the next, but this source of control is weak and not influenced by the experimental manipulations. Results from the clock arrangement indicate that behavior is controlled largely by the stimulus conditions prevailing at the time of interval onset.     

Stimulus generalization from feeder to response key in the acquisition of autoshaped pecking

During autoshaping, a 6-second presentation of one stimulus and a variable time 30-second presentation of a second stimulus alternated in appearance on a pigeon key. Grain always was delivered for 3 seconds at the end of the first stimulus interval. In the first experiment, autoshaped pecking of the stimulus preceding grain delivery began much sooner when that stimulus was a black vertical line on a white background and the other stimulus was green than when the opposite stimulus arrangement was used. Because these two stimuli differed in form, hue, brightness, and similarity in hue and brightness to the illumination of the raised feeder, three subsequent experiments examined whether the differential speed of autoshaping in the two groups was due to a feature-positive, feature-negative effect, a preference for brighter over darker stimuli, a simple preference for white over green, or stimulus generalization from the brightness or hue of the illuminated, raised feeder to the stimulus on the key preceding grain delivery. The data from these experiments showed that the first autoshaped key peck was most likely to be made to the stimulus of the same hue as that illuminating the feeder, regardless of whether that stimulus was positively or negatively associated with grain delivery. At least under some conditions, therefore, stimulus-generalization mediated response transfer of pecking grain in the presence of the hue illuminating the feeder to pecking the key illuminated by a similar hue appears to account for the occurrence of autoshaped key pecking.     

An interresponse time analysis of variable-ratio punishment

An interresponse time analysis was used to study the effects of variable-ratio punishment schedules on the temporal pattern of reinforced responding. Twelve pigeons responded on a baseline variable-interval schedule of food reinforcement. A variable-ratio ten schedule of electric shock punishment was then introduced. The shock intensity was systematically increased to the highest intensity at which responding could be maintained. At this intensity, the mean variable-ratio value was increased and then decreased. Variable-ratio punishment resulted in an increased relative frequency of very short unreinforced interresponse times (response bursting). Increased response bursting accounted for instances of response rate facilitation. In addition, shock was followed by interresponse times of decreasing mean length over the first several responses after shock.     

Inhibitory stimulus control following errorless discrimination learning

Three generalization procedures were used to investigate inhibitory stimulus control following discrimination learning with few errors. Three groups of pigeons acquired a discrimination between a green stimulus (the positive stimulus) and a vertical or horizontal line (the negative stimulus) through differential autoshaping followed by multiple schedule presentation of the two stimuli with gradually increasing stimulus durations. Genereralization testing was along a line-tilt continuum. For one group, the test involved a resistance-to-reinforcement procedure in which responses to all line tilts were reinforced on a variable-interval schedule. For a second group, also tested with the resistance-to-reinforcement procedure, the lines were superimposed on the green field that formerly served as the positive stimulus. A third group was tested in extinction with the combined stimuli. Control groups had no discrimination training but responding to green was nondifferentially reinforced. The control subjects responded more to all line tilts during testing than did the comparable experimental subjects, indicating that the negative stimulus had become an inhibitory stimulus. Both resistance-to-reinforcement groups revealed inhibitory gradients around the negative stimulus, but the gradient for the extinction group was relatively flat. These data are consistent with others that modify Terrace's early conclusion concerning the failure of inhibition to develop during errorless training.