Body weight and response acquisition with delayed reinforcement.

The relation between body weight and responding established with unsignaled delayed reinforcement was investigated. In three experiments, naive rats were deprived to either 70%, 80%, or 90% of ad libitum weight and were then exposed to tandem variable-interval 15-s differential-reinforcement-of-other-behavior 30-s schedules. The tandem schedule defined a resetting unsignaled delay-of-reinforcement procedure. In the first experiment, speed of magazine training, acquisition of lever pressing, and final rate of lever pressing were related to body weight. In the next experiment, lever pressing was established and maintained in rats that were magazine trained at 70% of ad libitum weight but that were then exposed to the delay procedure at 90% of ad libitum weight. Responding did not change consistently either across or within subjects in subsequent conditions in which body weight was manipulated. In the final experiment, lever pressing was established and maintained with delayed reinforcement in the absence of magazine training for each of 2 rats at 70% and for 1 of 2 rats at 90% of ad libitum weight. The results further illuminate the conditions under which responding can be established in the absence of training and when such responses are reinforced only following an unsignaled delay period.     

Social reinforcement of operant behavior in rats: a methodological note.

An apparatus was developed to study social reinforcement in the rat. Four Long-Evans female rats were trained to press a lever via shaping, with the reinforcer being access to a castrated male rat. Responding under a fixed-ratio schedule and in extinction was also observed. Social access was found to be an effective reinforcer. When social reinforcement was compared with food reinforcement under similar conditions of deprivation and reinforcer duration, no significant differences were observed.     

Acquisition of lever-press responding in rats with delayed reinforcement: A comparison of three procedures

The present study examined the acquisition of lever pressing in rats under three procedures in which food delivery was delayed by 4, 8, and 16 seconds relative to the response. Under the nonresetting delay procedure, food followed the response selected for reinforcement after a specified interval elapsed; responses during this interval had no programmed effect. Under the resetting procedure, the response selected for reinforcement initiated an interval to food delivery that was reset by each subsequent response. Under the stacked delay procedure, every response programmed delivery of food t seconds after its occurrence. Two control groups were studied, one that received food immediately after each lever press and another that never received food. With the exception of the group that did not receive food, responding was established with every procedure at every delay value without autoshaping or shaping. Although responding was established under the resetting delay procedure, response rates were generally not as high as under the other two procedures. These findings support the results of other recent investigations in demonstrating that a response not previously reinforced can be brought to strength by delayed reinforcement in the absence of explicit training.     

Response acquisition with delayed reinforcement: a comparison of two-lever procedures.

Groups of 8 experimentally naive rats were exposed during 8-hr sessions to resetting delay procedures in which responses on one lever (the reinforcement lever) produced water after a delay of 8, 16, 32, or 64 s. For rats in one condition, responses on a second (no-consequences) lever had no programmed consequences. For rats in another condition, responses on a second (cancellation) lever during a delay initiated by a response on the reinforcement lever prevented delivery of the scheduled reinforcer; responses on the cancellation lever at other times had no programmed consequences. Under both conditions and at all delays, most subjects emitted more responses on the reinforcement lever than did control rats that never received water emitted on either lever. At 8-s delays, both conditions engendered substantially more responding on the reinforcement lever than on the other lever, and performance closely resembled that of immediate-reinforcement controls. At delays of 16 and 32 s, however, there was clear differential responding on the two levers under the cancellation condition but not under the other condition. When the delay was 64 s, differential responding on the two levers did not occur consistently under either condition. These findings provide strong evidence that the behavior of rats is sensitive to consequences delayed by 8, 16, and 32 s, but only equivocal evidence of such sensitivity to consequences delayed 64 s. They also indicate that acquisition depends, in part, on the measure of performance used to index it.     

The effects of behavioral history on response acquisition with immediate and delayed reinforcement

Effects of prior exposure to the experimental chamber with levers present or absent and variable-time (VT) 60-s water deliveries arranged during one, five, or no 1-hr sessions were examined in rats during a 6-hr response-acquisition session in which presses on one lever produced water delivery immediately or after a 15-s resetting delay, and presses on the other lever canceled scheduled water deliveries. Response acquisition was (a) slower to occur when water deliveries were delayed, (b) most consistent in groups that had received five VT sessions, and (c) impaired by the presence of levers only when there had been five VT sessions and water deliveries were delayed during the acquisition session.     

Response induction during the acquisition and maintenance of lever pressing with delayed reinforcement

The acquisition of lever pressing by rats and the occurrence of unreinforced presses at a location different from that of the reinforced response were studied using different delays of reinforcement. An experimental chamber containing seven identical adjoining levers was used. Only presses on the central (operative) lever produced food pellets. Groups of 3 rats were exposed to one of seven different tandem random-interval (RI) fixed-time (FT) schedules. The average RI duration was the complement of the FT duration such that their sum yielded a nominal 32-s interreinforcement interval on average. Response rate on the operative lever decreased as the FT value was lengthened. The spatial distribution of responses on the seven levers converged on the operative lever when the FT was 0 or 2 s and spread across the seven levers as the FT value was lengthened to 16 or 32 s. Presses on the seven levers were infrequent during the FT schedule. Both operative- and inoperative-lever pressing intertwined in repetitive patterns that were consistent within subjects but differed between subjects. These findings suggest that reinforcer delay determined the response-induction gradient.     

Effects of pre-operant running and sucrose concentration on operant wheel running on a fixed interval schedule of reinforcement

Prior research proposed that temporal control over the pattern of operant wheel running on a fixed interval (FI) schedule of sucrose reinforcement is a function of automatic reinforcement generated by wheel running and the experimentally arranged sucrose reinforcement. Two experiments were conducted to assess this prediction. In the first experiment, rats ran for different durations (0, 30, 60, and 180 min) prior to a session of operant wheel running on a FI 120-s schedule. In the second experiment, the concentration of sucrose reinforcement on a FI 180-s schedule was varied across values of 0, 5, 15, and 25%. In Experiment 1, as the duration of pre-operant running increased, the postreinforcement pause before initiation of running lengthened while wheel revolutions in the latter part of the FI interval increased. In Experiment 2, wheel revolutions markedly increased then decreased to a plateau early in the FI interval. Neither manipulation increased temporal control of the pattern of wheel running. Instead, results indicate that operant wheel running is regulated by automatic reinforcement generated by wheel activity and an adjunctive pattern of running induced by the temporal presentation of sucrose. Furthermore, the findings question whether the sucrose contingency regulates wheel running as a reinforcing consequence.     

Effects of D-amphetamine on response acquisition with immediate and delayed reinforcement.

The present study examined in 8-hour sessions the effects of d-amphetamine (1.0, 5.6, and 10 mg/kg) on the acquisition of lever-press responding in rats that were exposed to procedures in which water delivery was delayed by 0, 8, or 16 seconds relative to the response that produced it. Both nonresetting- and resetting-delay conditions were studied. Although neither shaping nor autoshaping occurred, substantial levels of operative-lever responding developed under all conditions in which responses produced water. The lowest dose (1.0 mg/kg) of d-amphetamine either had no effect on or increased operative-lever pressing, whereas higher doses typically produced an initial reduction in lever pressing. Nonetheless, overall rates of operative-lever pressing at these doses were as high as, or higher than, those observed with vehicle. Thus, response acquisition was observed under all reinforcement procedures at all drug doses. In the absence of the drug, most responding occurred on the operative lever when reinforcement was immediate. Such differential responding also developed under both nonresetting- and resetting-delay procedures when the delay was 8 seconds, but not when it was 16 seconds. d-Amphetamine did not affect the development of differential responding under any procedure. Thus, consistent with d-amphetamine's effects under repeated acquisition procedures, the drug had no detrimental effect on learning until doses that produced general behavioral disruption were administered.     

Effects of reinforcement rate and delay on the acquisition of lever pressing by rats.

The acquisition of lever pressing by naive rats, in the absence of shaping, was studied as a function of different rates and unsignaled delays of reinforcement. Groups of 3 rats were each exposed to tandem schedules that differed in either the first or the second component. First-component schedules were either continuous reinforcement or random-interval 15, 30, 60 or 120 s; second-component schedules were fixed-time 0, 1, 3, 6, 12, or 24 s. Rate of responding was low under continuous immediate reinforcement and higher under random-interval 15 s. Random interval 30-s and 60-s schedules produced lower rates that were similar to each other. Random-interval 120 s controlled the lowest rate in the immediate-reinforcement condition. Adding a constant 12-s delay to each of the first-component schedule parameters controlled lower response rates that did not vary systematically with reinforcement rate. The continuous and random-interval 60-s schedules of immediate reinforcement controlled higher global and first-component response rates than did the same schedules combined with longer delays, and first-component rates showed some graded effects of delay duration. In addition, the same schedules controlled higher second-component response rates in combination with a 1-s delay than in combination with longer delays. These results were related to those from previous studies on acquisition with delayed reinforcement as well as to those from similar reinforcement procedures used during steady-state responding.     

Acquisition of cocaine self-administration with unsignaled delayed reinforcement in rhesus monkeys

Six experimentally naive rhesus monkeys produced 0.01 mg/kg/infusion cocaine by lever pressing under a tandem fixed-ratio 1 differential-reinforcement-of-other-behavior schedule. One lever press initiated an unsignaled 15- or 30-s delay culminating in cocaine delivery. Each press made during the delay reset the delay interval. With two exceptions, responding was acquired and maintained at higher rates than responding on a second (inoperative) lever. For the exceptions, a cancellation contingency was arranged in which each formerly inoperative-lever response reset the tandem schedule. This manipulation reduced presses on the inoperative lever. Subsequently, the consequences of responding on the two levers were reversed, and the monkeys again responded at higher rates on the operative lever. As a comparison, 3 additional experimentally naive monkeys received response-independent cocaine deliveries. Although lever pressing was observed, it extinguished and was subsequently reestablished under the tandem schedule. The results suggest that although response-reinforcer contiguity is not required for cocaine to acquire reinforcing functions, a response-reinforcer relation appears necessary.     

Response duration is sensitive to both immediate and delayed reinforcement

We investigated the duration of lever pressing by rats when the delivery of appetitive reinforcers was contingent upon response duration. In the first experiment, response durations increased when duration requirements were imposed, and they decreased when duration requirements were removed. This effect occurred whether reinforcers were immediate or delayed by 8 s. In order to maintain the integrity of the delay intervals, reinforcer delivery was dependent upon both lever depression and release. In a second experiment, lever depression only and a response duration of at least 4 s were required for reinforcer delivery. Compared to immediate reinforcement conditions, delayed reinforcers increased both variability and the length of the maximum response durations. In a third experiment, immediate reinforcers were delivered contingent upon lever depression and release under a variety of duration requirements. Median lever‐press durations tracked the contingencies rapidly. Across all three experiments, rats emitted numerous response durations that were too short to satisfy the reinforcer requirements, and bimodal distributions similar to those produced by differential reinforcement of low rate schedules were evident for most rats. In many aspects, response duration responds to reinforcement parameters in a fashion similar to rate of discrete responding, but an examination of this continuous dimension of behavior may provide additional information about environment–behavior relationships.     

A cellular analogue of operant conditioning.

Using the hippocampal-slice preparation, we attempted to model operant conditioning in vitro by reinforcing pyramidal cell bursting responses with local micropressure applications of transmitters and drugs. The same injections were administered independently of bursting to provide a "noncontingent" control for direct pharmacological stimulation or facilitation of firing. The results suggested that the bursting responses of individual CA1 pyramidal neurons may be reinforced in a dose-related manner by response-contingent (but not noncontingent) injections of dopamine and the selective dopamine D2 agonist, N-0923. N-0924, a stereoisomer of N-0923 that is largely devoid of D2-agonist activity, failed to reinforce CA1 bursting. Burst-contingent injections of the excitatory neurotransmitter glutamate also failed to reinforce CA1 bursting; indeed, the glutamate applications (whether contingent or random) reduced the likelihood of bursts while increasing the frequency of solitary spikes. Reinforcement delays exceeding 200 ms largely eliminated the reinforcing efficacy of the D2 agonist N-0437 in CA1 operant conditioning. The results are consistent with the suggestion that the behaviorally reinforcing effects of dopaminergic agents can be modeled in vitro in the hippocampal-slice preparation.     

Conditioned reinforcement and choice with delayed and uncertain primary reinforcers.

In an adjusting-delay choice procedure, pigeons could peck on either a red key or a green key. A peck on the red key always led to a delay associated with red houselights and then food. The delay was adjusted over trials to estimate an indifference point--a delay at which the two keys were chosen about equally often. In some conditions, a peck on the green key led to food on all trials after delays of either 10 s or 30 s, and green houselights were lit during the delays. In other conditions, food was presented on only half of the green-key trials. If the green houselights continued to occur on both reinforcement and nonreinforcement trials, preference for the green key always decreased. Preference for the green key also decreased if half of the trials had 30-s houselights followed by food and the other half had no green houselights and no food. However, preference for the green key actually increased if half of the trials had 10-s green houselights followed by food and the other half had no green houselights followed by no food. The latter condition therefore demonstrated a case in which preference for an alternative increased when food was removed from half of the trials. The results suggest that the red and green houselights served as conditioned reinforcers. A hyperbolic decay model (Mazur, 1989) provided good predictions for all conditions by assuming that the strength of a conditioned reinforcer is inversely related to the total time spent in its presence before food is delivered.     

Behavioral effects of delayed timeouts from reinforcement

Timeouts are sometimes used in applied settings to reduce target responses, and in some circumstances delays are unavoidably imposed between the onset of a timeout and the offset of the response that produces it. The present study examined the effects of signaled and unsignaled timeouts in rats exposed to concurrent fixed‐ratio 1 fixed‐ratio 1 schedules of food delivery, where each response on one lever, the location of which changed across conditions, produced both food and a delayed 10‐s timeout. Delays of 0 to 38 s were examined. Delayed timeouts often, but not always, substantially reduced the number of responses emitted on the lever that produced timeouts relative to the number emitted on the lever that did not produce timeouts. In general, greater sensitivity was observed to delayed timeouts when they were signaled. These results demonstrate that delayed timeouts, like other delayed consequences, can affect behavior, albeit less strongly than immediate consequences.     

Further evaluation of a nonsequential approach to studying operant renewal

Basic‐laboratory assessments of renewal may inform clinical efforts to maintain reduction of severe destructive behavior when clients transition between contexts. The contextual changes arranged during standard renewal procedures, however, do not necessarily align with those that clients experience during outpatient therapy. More specifically, clients transition between clinical (associated with extinction for target behavior) and home/community (associated with reinforcement for target behavior) contexts during outpatient treatment. Standard renewal assessments do not incorporate these contextual alternations during treatment. The present experiment aimed to directly compare renewal of rats' lever pressing following a standard (“sequential”) ABA renewal procedure (i.e., baseline in Context A, extinction in Context B, renewal test in Context A) and a “nonsequential” renewal assessment wherein treatment consisted of daily alternation between Context A (associated with reinforcement for lever pressing) and Context B (associated with extinction). Lever pressing renewed to a greater extent for rats in the Nonsequential group than for rats in the Sequential group, suggesting the contextual changes that clients experience during outpatient treatment for severe destructive behavior may be a variable that is important to consider in translational research on renewal. Potential implications of these findings for basic and clinical research on renewal are discussed.     

Tactical contingencies in the experimental analysis of reinforcement and operant classes

In his “Tactics of Scientific Research” (1960), his work on avoidance, his discovery of equivalence classes and his cautions on applications of coercion, Murray Sidman created high standards for behavior analytic research. I illustrate his influence in the context of three examples he might have characterized as pilot studies. Each examined trial N+1 response probabilities depending on whether trial N responding had produced a reinforcer. Differentially reinforced interresponse times, keys pecked in arbitrary matching, and two-key response sequences provided no robust evidence that reinforcing some response property on trial N raises the probability of responding with that property on trial N+1. These negative findings shed light on the nature of operant classes and on the relation of reinforcers to the responses that produced them. Through selection, reinforcers create operant classes and engender variations of the responses within those classes; operant classes are held together by common contingencies. Sidman extended our understanding of operant classes by expanding them to include equivalence relations.     

Contingency tracking during unsignaled delayed reinforcement

Three experiments were conducted with rats in which responses on one lever (labeled the functional lever) produced reinforcers after an unsignaled delay period that reset with each response during the delay. Responses on a second, nonfunctional, lever did not initiate delays, but, in the first and third experiments, such responses during the last 10 s of a delay did postpone food delivery another 10 s. In the first experiment, the location of the two levers was reversed several times. Responding generally was higher on the functional lever, though the magnitude of the difference diminished with successive reversals. In the second experiment, once a delay was initiated by a response on the functional lever, in different conditions responses on the nonfunctional lever either had no effect or postponed food delivery by 30 s. The latter contingency typically lowered response rates on the nonfunctional lever. In the first two experiments, both the functional and nonfunctional levers were identical except for their location; in the third experiment, initially, a vertically mounted, pole-push lever defined the functional response and a horizontally mounted lever defined the nonfunctional response. Higher response rates occurred on the functional lever. These results taken together suggest that responding generally tracked the response-reinforcer contingency. The results further show how nonfunctional-operanda responses are controlled by a prior history of direct reinforcement of such responses, by the temporal delay between such responses and food delivery, and as simple generalization between the two operanda.     

Effects of delayed conditioned reinforcement in chain schedules.

The contingency between responding and stimulus change on a chain variable-interval 33-s, variable-interval 33-s, variable-interval 33-s schedule was weakened by interposing 3-s delays between either the first and second or the second and third links. No stimulus change signaled the delay interval and responses could occur during it, so the obtained delays were often shorter than the scheduled delay. When the delay occurred after the initial link, initial-link response rates decreased by an average of 77% with no systematic change in response rates in the second or third links. Response rates in the second link decreased an average of 59% when the delay followed that link, again with little effect on response rates in the first or third links. Because the effect of delaying stimulus change was comparable to the effect of delaying primary reinforcement in a simple variable-interval schedule, and the effect of the unsignaled delay was specific to the link in which the delay occurred, the results provide strong evidence for the concept of conditioned reinforcement.     

Effects of delayed reinforcement on infant vocalization rate.

Three previous studies have failed to demonstrate conditioning in infants using a 3-s delay of reinforcement. The effects of a delayed reinforcement schedule on vocalization rates therefore were explored in a single-subject repeated-reversal experimental design for 3 4- to 6-month-old normally developing infants. Each infant received delayed social reinforcement from his or her parent for vocalizing. The comparison condition was a schedule of differential reinforcement of behavior other than vocalizations to control for elicitation by social stimulation. An operant level of infant vocalizations was the initial condition, after which the differential reinforcement schedule was implemented in an across-subjects multiple baseline design. Infants' vocalization rates increased above levels measured during differential reinforcement following onset of the delayed reinforcement condition. Also, vocalization rates decreased during differential reinforcement compared to operant levels. The successful use of delayed reinforcement schedules with infants in this study, as opposed to others, is discussed in terms of procedural differences among them.     

Evaluating different values of effort and reinforcement parameters under concurrent‐ and single‐operant arrangements

If particular parameters (e.g., reinforcer quality) are found to be preferred in a concurrent‐operant arrangement, then these should be incorporated into interventions for maintaining skills in individuals with intellectual disabilities. However, because results from a concurrent‐operant arrangement may not predict those of a single‐operant assessment, interventions with less preferred parameters may also be effective. The purpose of this study was to determine if preference for a particular parameter, determined via concurrent‐operant arrangements, predicted responding when two values of highly and less preferred parameters were manipulated in a single‐operant arrangement. Participants allocated responding to one parameter over another during concurrent‐operant assessments, indicating that certain parameters were highly preferred. However, contingent presentation of two values of less preferred parameters increased compliance to similar levels achieved with two highly preferred parameters in a single‐operant arrangement. These findings suggest that clinicians may have multiple options when selecting parameters for use during intervention.