Disruption of responding maintained by conditioned reinforcement: alterations in response-conditioned-reinforcer relations

An observing procedure was used to investigate the effects of alterations in response-conditioned-reinforcer relations on observing. Pigeons responded to produce schedule-correlated stimuli paired with the availability of food or extinction. The contingency between observing responses and conditioned reinforcement was altered in three experiments. In Experiment 1, after a contingency was established in baseline between the observing response and conditioned reinforcement, it was removed and the schedule-correlated stimuli were presented independently of responding according to a variable-time schedule. The variable-time schedule was constructed such that the rate of stimulus presentations was yoked from baseline. The removal of the observing contingency reliably reduced rates of observing. In Experiment 2, resetting delays to conditioned reinforcement were imposed between observing responses and the schedule-correlated stimuli they produced. Delay values of 0, 0.5, 1, 5, and 10 s were examined. Rates of observing varied inversely as a function of delay value. In Experiment 3, signaled and unsignaled resetting delays between observing responses and schedule-correlated stimuli were compared. Baseline rates of observing were decreased less by signaled delays than by unsignaled delays. Disruptions in response-conditioned-reinforcer relations produce similar behavioral effects to those found with primary reinforcement.     

Choice with probabilistic reinforcement: effects of delay and conditioned reinforcers.

Two experiments measured pigeons' choices between probabilistic reinforcers and certain but delayed reinforcers. In Experiment 1, a peck on a red key led to a 5-s delay and then a possible reinforcer (with a probability of .2). A peck on a green key led to a certain reinforcer after an adjusting delay. This delay was adjusted over trials so as to estimate an indifference point, or a duration at which the two alternatives were chosen about equally often. In all conditions, red houselights were present during the 5-s delay on reinforced trials with the probabilistic alternative, but the houselight colors on nonreinforced trials differed across conditions. Subjects showed a stronger preference for the probabilistic alternative when the houselights were a different color (white or blue) during the delay on nonreinforced trials than when they were red on both reinforced and nonreinforced trials. These results supported the hypothesis that the value or effectiveness of a probabilistic reinforcer is inversely related to the cumulative time per reinforcer spent in the presence of stimuli associated with the probabilistic alternative. Experiment 2 tested some quantitative versions of this hypothesis by varying the delay for the probabilistic alternative (either 0 s or 2 s) and the probability of reinforcement (from .1 to 1.0). The results were best described by an equation that took into account both the cumulative durations of stimuli associated with the probabilistic reinforcer and the variability in these durations from one reinforcer to the next.     

Cooperative responding in rats: II. Performance on fixed-ratio schedules of mutual reinforcement

Coordinated responses of 5 dyads of rats were investigated under fixed-ratio (FR) schedules of mutual water reinforcement. Coordinated responding was defined as 2 consecutive lever-presses, 1 from each of 2 rats, occurring <.5 s apart. In the FR schedules, each coordinated episode was defined as 1 response in the FR sequence. The size of FR schedules was parametrically manipulated assuming the values of FR 1, 6, 12, 18, 24, 30, 50, and 9, in this order. Each FR remained in effect until responding reached stability. Under all conditions, pairs of rats received access to water simultaneously (mutual reinforcement). Rates and proportions of coordinated responding showed a bitonic inverted U-shaped function of ratio size. Postreinforcement pauses increased systematically as the interreinforcement interval increased. Local rates and proportions increased as a function of response location within ratios. Results of a control condition with relaxed temporal constraints for mutual reinforcement showed decreases in rates and proportion of coordinated responses, suggesting that the coordinated responses were controlled by the mutual reinforcement contingencies. The present experiment showed that coordinated responding is quantitatively affected by 3 properties of FR schedules: response requirement, reinforcement rates, and proximity to reinforcement.     

The effects of delayed reinforcement on free-operant responding

In previous studies of delayed reinforcement, response rate has been found to vary inversely with the response-reinforcer interval. However, in all of these studies the independent variable, response-reinforcer time, was confounded with the number of reinforcers presented in a fixed period of time (reinforcer frequency). In the present study, the frequency of available reinforcers was held constant, while temporal separation between response and reinforcer was independently manipulated. A repeating time cycle, T, was divided into two alternating time periods, t^D and t^Δ. The first response in t^D was reinforced at the end of the prevailing T cycle and extinction prevailed in t^Δ. Two placements for t^D were defined, an early t^D placement in which t^D precedes t^Δ and a late t^D placement in which t^D follows t^Δ. The duration of the early and late t^D was systematically decreased from 30 seconds (i.e., t^D = T) to 0.1 second. Manipulation of t^D placement and duration controlled the temporal separation between response and reinforcement, but it did not affect the frequency of programmed reinforcers, which was 1/T. The results show that early and late t^D placements of equal duration have similar overall effects upon response rate, reinforcer frequency, responses per reinforcer, and obtained response-reinforcer temporal separation. A stepwise regression analysis using log response rate as the dependent variable showed that the obtained delay was a significant first-step variable for six of eight subjects, with obtained reinforcer frequency significant for the remaining two subjects.     

Differential responding without differential reinforcement: Intensity difference, continuum position, and reinforcement density effects

The response rates of five groups of rats were observed during exposure to different intensities of a four kilohertz tone within a two-component multiple schedule of nondifferential reinforcement. Response rates were found to be higher during the multiple schedule component which contained the higher intensity tone. Larger differences in response rates between the two multiple schedule components occurred with greater intensity separations (30 versus 20 decibels). At the 30 decibel separation a low absolute magnitude produced larger response rate differences than a high absolute magnitude, while at the 20 decibel separation a high absolute magnitude produced larger response rate differences. Increases in reinforcement density were accompanied by decreases in response rate differences between high and low intensity components only when over-all response rates also increased.     

The effects of magnitude and quality of reinforcement on choice responding during play activities

Three boys with autism participated in a study of the effects of magnitude and quality of reinforcement on choice responding. Two concurrent response alternatives were arranged: (a) to play in an area where a peer or sibling was located, or (b) to play in an area where there was no peer or sibling. During one condition, the magnitude (i.e., duration of access to toys) or quality (level of preference) of reinforcement provided for both responses was equal. During the other condition, the magnitude or quality of reinforcement was relatively greater for choosing the play area where the peer or sibling was located than the area where the peer or sibling was not located. Results showed that after repeated exposure to the unequal magnitude or quality condition, the participant increasingly allocated his responses to the play area where the peer or sibling was located. For 2 participants, this pattern of responding was maintained in the subsequent equal magnitude or quality condition. Overall, the analysis suggests that the dimensions of magnitude and quality of reinforcement can be arranged to influence choice responding in favor of playing near a peer or sibling rather than playing alone.     

Response acquisition by zebrafish (Danio rerio) with delayed reinforcement

Zebrafish (Danio rerio) is a common vertebrate animal model in biomedical research and is a promising species for studying how genes interact with environmental factors in determining behavior. The present study investigated how reinforcement parameters affect zebrafish behavior by assessing response acquisition with delayed reinforcement, which has been studied with other species (e.g., rats, pigeons, humans, etc.) but not with zebrafish. Twenty‐four experimentally naïve subjects were exposed to a tandem fixed‐ratio 1 differential‐reinforcement‐of‐other‐behavior x‐s schedule of reinforcement, where x varied across subjects. There were six different delay‐to‐reinforcement durations and sets of four fish were assigned to each delay duration. All of the fish assigned to a 0‐, 0.5‐, or 1‐s delay acquired responding. Two fish acquired responding with a 3‐s delay and one fish appeared to have acquired it with a 6‐s delay although the latter result was less clear. None acquired responding with a 12‐s delay. These results suggest that zebrafish behavior is sensitive to delays to reinforcement and the time frame over which reinforcement is effective may be limited approximately to 6 s. This time frame is shorter than that found with other species. Practical and theoretical implications of the present finding are discussed.     

The effects of concurrent responding and reinforcement on behavioral output

Four birds key pecked on concurrent variable-interval one-minute variable-interval four-minute schedules with a two-second changeover delay. Response rates to the variable-interval one-minute key were then reduced by signaling its reinforcer availability and later by providing its reinforcers independently of responding. Each manipulation increased response rates to the variable-interval four-minute key even though relative reinforcement rates were unchanged. In a final phase, eliminating the variable-interval one-minute key and its schedule produced the highest rates of all to the variable-interval four-minute key. These results show that both reinforcement and response rates to one schedule influence response rates to another schedule. These results join those of Guilkey, Shull, & Brownstein (1975) in failing to replicate Catania (1963). Moreover, they violate the predictions of the equation for simple action (de Villiers & Herrnstein, 1976). In terms of a median-rate measure (reciprocal of the median interresponse time), rates to the variable-interval four-minute key were high when responding was not reduced to the variable-interval one-minute key and were low when it was reduced. This rate difference suggests a process difference between concurrent-schedule procedures that maintain high concurrent response rates versus those that do not. This process difference jeopardizes attempts to integrate single- and concurrent-operant performances within a single formulation.     

Elicited responding to signals for reinforcement: the effects of overall versus local changes in reinforcement probability

Pigeons were studied on a three-component multiple schedule where all reinforcement was independent of responding. Two components were cued by different keylights and were associated with different rates of reinforcement. The third was always a no-key period associated with extinction. After a few sessions, pecking was elicited by the keylights signalling the reinforcement and continued to be maintained indefinitely. The duration and sequence of the three components were varied to determine if the primary controlling variable was differences in the overall probability of reinforcement, or if it was the immediate change in reinforcement signalled by the onset and/or offset of the stimulus. Both variables were found to control behavior. When 30-sec components were used, the primary controlling variable was the overall probability of reinforcement, but when 3-min components were used, overall probability had little effect. Control by local changes in reinforcement also occurred, although the type of local control varied both across subjects and experimental conditions. Some behaviors were controlled more by the change in reinforcement signalled by the onset of the stimulus, while others were controlled more by the change signalled by the offset of the stimulus.     

Human self-control and the density of reinforcement

Choice responding in adult humans on a discrete-trial button-pressing task was examined as a function of amount, delay, and overall density (points per unit time) of reinforcement. Reinforcement consisted of points that were exchangeable for money. In T 0 conditions, an impulsive response produced 4 points immediately and a self-control response produced 10 points after a delay of 15 s. In T 15 conditions, a constant delay of 15 s was added to both prereinforcer delays. Postreinforcer delays, which consisted of 15 s added to the end of each impulsive trial, equated trial durations regardless of choice, and was manipulated in both T 0 and T 15 conditions. In all conditions, choice was predicted directly from the relative reinforcement densities of the alternatives. Self-control was observed in all conditions except T 0 without postreinforcer delays, where the impulsive choices produced the higher reinforcement density. These results support previous studies showing that choice is a direct function of the relative reinforcement densities when conditioned (point) reinforcers are used. In contrast, where responding produces intrinsic (immediately consumable) reinforcers, immediacy of reinforcement appears to account for preference when density does not.     

Contingency tracking during unsignaled delayed reinforcement: Effects of delay duration and d‐amphetamine

In two experiments, the role of the response–reinforcer relation in maintaining low‐rate responding under unsignaled delay conditions was investigated. In both experiments pecking by pigeons on one response key, denoted the relevant key, was reinforced under an unsignaled delay‐of‐reinforcement procedure (defined as tandem variable‐interval (VI) differential‐reinforcement‐of‐other behavior [DRO] schedule). Responding on a second key, denoted the irrelevant key, had no programmed consequences. Between sessions, the location of the relevant key varied (after one, two, or three sessions) pseudorandomly. In Experiment 1, the delay (DRO) duration was manipulated parametrically. Overall, proportional relevant‐key response rates (relevant‐key response rates / [relevant‐key response rates + irrelevant key response rates]) increased across 3‐session sequences in which the relevant key remained in the same location and decreased as the DRO duration was changed systematically (2, 5, and 10 s). In Experiment 2, acute administration of d‐amphetamine increased proportional relevant‐key response rates during 1‐day sequences for only the DRO 5‐s duration, and results over 3‐day sequences, once a discrimination had already been established, were inconsistent. Results support that the response–reinforcer relation is the primary determinant of responding, and such discriminations are relatively resistant to disruption or potentiation by behaviorally active doses of d‐amphetamine.     

Effects of alternative reinforcement on human behavior: the source does matter

Competing theories regarding the effects of delivering periodic response-independent reinforcement (more accurately, response-independent points exchanged for money) on a baseline rate of behavior were evaluated in human subjects. Contiguity theory holds that these events decrease target responding because incompatible behavior is adventitiously strengthened when the point deliveries follow target behavior closely in time. Matching theory holds that response-independent points, like any other alternative reinforcer, should reduce target responding. On this view, temporal contiguity between target responding and response-independent point delivery is unimportant. In our experiment, four different responses (moving a joystick in four different directions) were reinforced with points exchangeable for money according to four independent variable-interval schedules. Different schedules of point delivery were then superimposed on these baselines. When all superimposed point deliveries occurred immediately after one of the four responses (the target response), time allocated to target responding increased. When the superimposed point deliveries could be delivered at any time, time allocated to target responding declined and other behavior increased. When superimposed points could never immediately follow target responses, time allocated to target responding decreased further and other behavior or pausing predominated. The findings underscore the contribution of temporal contiguity in the effects of response-independent deliveries of food, money, points, etc.     

The effect of signaled reinforcement on rats' fixed-interval responding

Four experiments examined the effect on rats' response rate of presenting a brief (500 ms) stimulus simultaneously with the delivery of food on fixed-interval (FI) schedules. In Experiment 1, reinforcement signals that were spatially diffuse (both tones and lights) elevated rates of responding, but responding was attenuated by localized visual stimuli. The remaining experiments examined the signal-induced potentiation of responding. In Experiment 2, a tone reinforcement signal potentiated response rates on an FI schedule, but attenuated response rates on a variable-interval (VI) schedule. This difference was obtained even though the overall rate of responding was equated on the two schedules before the introduction of the signal. Signal-induced potentiation of responding occurred over a range of FI values employed in Experiment 3. In Experiment 4, presenting a reinforcement signal when high local rates of response had occurred immediately before reinforcement resulted in potentiated rates of responding on an FI schedule. The opposite effect on response rate occurred when the reinforcement signal followed only low local rates of response. These results indicate that a variety of factors influence the effects of a reinforcement signal. They imply, however, that the local rate of response at the time of reinforcement is a key factor in establishing the nature of the signaling effect.     

Acquisition of lever-press responding in rats with delayed reinforcement: A comparison of three procedures

The present study examined the acquisition of lever pressing in rats under three procedures in which food delivery was delayed by 4, 8, and 16 seconds relative to the response. Under the nonresetting delay procedure, food followed the response selected for reinforcement after a specified interval elapsed; responses during this interval had no programmed effect. Under the resetting procedure, the response selected for reinforcement initiated an interval to food delivery that was reset by each subsequent response. Under the stacked delay procedure, every response programmed delivery of food t seconds after its occurrence. Two control groups were studied, one that received food immediately after each lever press and another that never received food. With the exception of the group that did not receive food, responding was established with every procedure at every delay value without autoshaping or shaping. Although responding was established under the resetting delay procedure, response rates were generally not as high as under the other two procedures. These findings support the results of other recent investigations in demonstrating that a response not previously reinforced can be brought to strength by delayed reinforcement in the absence of explicit training.     

Short-component multiple schedules: effects of relative reinforcement duration

Pigeons were exposed to multiple variable-interval 2-min variable-interval 2-min schedules of food presentation in which relative duration of food presentation was manipulated. When components alternated every 5 sec and were scheduled on separate response keys, relative response rates closely matched relative reinforcement duration in three of four pigeons. On the other hand, relative response rates were insensitive to relative reinforcement duration when components scheduled on a single response key alternated every 5 sec, and when components scheduled on separate response keys alternated every 2 min. Thus, both rapid alternation and spatial separation of components were necessary to produce approximate matching of relative responding to relative reinforcement duration. This finding contrasts with previous findings that only rapid component alternation is necessary for matching when relative rate of reinforcement is manipulated.     

Effects of reinforcer magnitude on responding under differential-reinforcement-of-low-rate schedules of rats and pigeons

Experiment I investigated the effects of reinforcer magnitude on differential-reinforcement-of-low-rate (DRL) schedule performance in three phases. In Phase 1, two groups of rats (n = 6 and 5) responded under a DRI. 72-s schedule with reinforcer magnitudes of either 30 or 300 microl of water. After acquisition, the water amounts were reversed for each rat. In Phase 2, the effects of the same reinforcer magnitudes on DRL 18-s schedule performance were examined across conditions. In Phase 3, each rat responded unider a DR1. 18-s schedule in which the water amotnts alternated between 30 and 300 microl daily. Throughout each phase of Experiment 1, the larger reinforcer magnitude resulted in higher response rates and lower reinforcement rates. The peak of the interresponse-time distributions was at a lower value tinder the larger reinforcer magnitude. In Experiment 2, 3 pigeons responded under a DRL 20-s schedule in which reinforcer magnitude (1-s or 6-s access to grain) varied iron session to session. Higher response rates and lower reinforcement rates occurred tinder the longer hopper duration. These results demonstrate that larger reinforcer magnitudes engender less efficient DRL schedule performance in both rats and pigeons, and when reinforcer magnitude was held constant between sessions or was varied daily. The present results are consistent with previous research demonstrating a decrease in efficiency as a function of increased reinforcer magnituide tinder procedures that require a period of time without a specified response. These findings also support the claim that DRI. schedule performance is not governed solely by a timing process.     

Choice with uncertain outcomes: conditioned reinforcement effects.

Pigeons responded on concurrent chains with equal initial- and terminal-link durations. In all conditions, the terminal links of one chain ended reliably in reinforcement; the terminal links on the alternative chain ended in either food or blackout. In Experiment 1, the terminal-link stimuli were correlated with (signaled) the outcome, and the durations of the initial and terminal links were varied across conditions. Preference did not vary systematically across conditions. In Experiment 2, terminal-link durations were varied under different stimulus conditions. The initial links were variable-interval 80-s schedules. Preference for the reliable alternative was generally higher in unsignaled than in signaled conditions. Preference increased with terminal-link durations only in the unsignaled conditions. There were no consistent differences between conditions with and without a common signal for reinforcement on the two chains. In the first series of conditions in Experiment 3, a single response was required in the initial links, and the stimulus conditions during 50-s terminal links were varied. Preference for the reliable outcome approached 1.0 in unsignaled conditions and was considerably lower (below .50 for 3 of 5 subjects) in signaled conditions. In a final series of signaled conditions with relatively long terminal links, preference varied with duration of the initial links. The results extend previous findings and are discussed in terms of the delay reduction signaled by terminal-link stimuli.