Signaled Delay of Reinforcement: Effects of Postconditioning Manipulation of Context Associative Strength on Instrumental Performance

Two experiments examined the influence of postconditioning treatments of contextual cues on instrumental responding acquired with a signaled delay of reinforcement schedule. In Experiment 1, mere exposure to the conditioning context after instrumental training resulted in an attenuated response rate during an extinction test. In the second experiment, responding was decreased by exposure to the contextual cues or sessions in which signaled noncontingent reinforcements occurred. The greatest response decrement, however, occurred following unsignaled noncontingent food presentations. The results are discussed with respect to the different roles of contextual cues on operant responding.     

The effect of reinforcement differences on choice and response distribution during stimulus compounding

In Experiments I and II, rats were trained to respond on one lever during light and another during tone. The absence of tone and light controlled response cessation. In the multiple schedule of Experiment I, all reinforcements were received for responding in tone or light; in the chain schedule of Experiment II, all reinforcements were received in no tone + no light for not responding. Experiment I subjects, for which tone and light were associated with response and reinforcement increase, responded significantly more to tone-plus-light than to tone or light alone (additive summation). Experiment II subjects, for which tone and light were associated with response increase and reinforcement decrease, responded comparably to tone, light, and tone + light. Thus, additive summation was observed when stimulus-response and stimulus-reinforcer associations in tone and light were both positive, but not when they were conflicting. All subjects in both experiments responded predominantly on the light-correlated lever during tone + light, even when light intensity was reduced in testing. Furthermore, when a light was presented to a subject engaged in tone-associated responding, all subjects immediately switched the locus of responding to the light-correlated lever. No change in locus occurred when a tone was presented to a subject engaged in light-associated responding, irrespective of the stimulus-reinforcer association conditioned to tone. The light-lever preference in tone + light indicates that the heightened responding observed in Experiment I was not the summation of tone-associated behavior with light-associated behavior. Rather, it appears to be the result of a facilitation of one operant (light-associated responding) by the reinforcement-associated cue for the other.     

Positive contrast, negative induction, and inhibitory stimulus control in the rat

In Experiment I, 24 rats were trained on a multiple variable-interval variable-interval schedule with a doorlight and white noise serving as component cues. Two groups were then shifted to a multiple extinction variable-interval schedule, and a third group was maintained on the multiple variable-interval variable-interval schedule. The multiple extinction variable-interval condition produced positive contrast when either the light or noise signalled extinction, and both of these cues acquired inhibitory stimulus control as measured by a combined cue test. In Experiment II, the multiple variable-interval variable-interval condition was shifted to multiple extinction variable-interval for one group, to multiple variable-time variable-interval for a second group, and was unchanged for the third group. The two experimental conditions produced identical patterns of response-rate reduction in the altered component, but the multiple extinction variable-interval condition produced positive contrast, whereas the multiple variable-time variable-interval condition did not. Subsequent combined cue and resistance to reinforcement tests revealed that the cue signalling extinction acquired stronger inhibitory stimulus control than the cue signalling variable time.     

Disruption of responding maintained by conditioned reinforcement: alterations in response-conditioned-reinforcer relations

An observing procedure was used to investigate the effects of alterations in response-conditioned-reinforcer relations on observing. Pigeons responded to produce schedule-correlated stimuli paired with the availability of food or extinction. The contingency between observing responses and conditioned reinforcement was altered in three experiments. In Experiment 1, after a contingency was established in baseline between the observing response and conditioned reinforcement, it was removed and the schedule-correlated stimuli were presented independently of responding according to a variable-time schedule. The variable-time schedule was constructed such that the rate of stimulus presentations was yoked from baseline. The removal of the observing contingency reliably reduced rates of observing. In Experiment 2, resetting delays to conditioned reinforcement were imposed between observing responses and the schedule-correlated stimuli they produced. Delay values of 0, 0.5, 1, 5, and 10 s were examined. Rates of observing varied inversely as a function of delay value. In Experiment 3, signaled and unsignaled resetting delays between observing responses and schedule-correlated stimuli were compared. Baseline rates of observing were decreased less by signaled delays than by unsignaled delays. Disruptions in response-conditioned-reinforcer relations produce similar behavioral effects to those found with primary reinforcement.     

Positive behavioral contrast in 3-month-old infants on multiple conjugate reinforcement schedules.

Positive behavioral contrast was assessed in two experiments with young infants using multiple conjugate reinforcement schedules. Reinforcement was produced by footkicks which activated the objects of an overhead crib mobile in a manner proportional to the vigor and rate of responding. Distinctive color/pattern cues on the sides of the objects served as discriminative stimuli for components of the multiple schedule. In Experiment 1, infants were trained with one cue (S+) only before insertion of S+ into a multiple schedule with an extinction component. A control group received S+ throughout all sessions. In Experiment 2, a multiple schedule was introduced at the outset, and responses in both components were reinforced before the introduction of extinction in the second component. In a final phase, reinforcement was reintroduced into the second component. Positive behavioral contrast occurred in both experiments. Response reduction in the extinction component was seen only in individual relative response curves. In both experiments, negative emotional behaviors accompanied the extinction component, and in Experiment 1, cooing accompanied presentations of S+.     

Responding of pigeons under variable-interval schedules of unsignaled, briefly signaled, and completely signaled delays to reinforcement

In Experiment 1, three pigeons' key pecking was maintained under a variable-interval 60-s schedule of food reinforcement. A 1-s unsignaled nonresetting delay to reinforcement was then added. Rates decreased and stabilized at values below those observed under immediate-reinforcement conditions. A brief stimulus change (key lit red for 0.5 s) was then arranged to follow immediately the peck that began the delay. Response rates quickly returned to baseline levels. Subsequently, rates near baseline levels were maintained with briefly signaled delays of 3 and 9 s. When a 27-s briefly signaled delay was instituted, response rates decreased to low levels. In Experiment 2, four pigeons' responding was first maintained under a multiple variable-interval 60-s (green key) variable-interval 60-s (red key) schedule. Response rates in both components fell to low levels when a 3-s unsignaled delay was added. In the first component delays were then briefly signaled in the same manner as Experiment 1, and in the second component they were signaled with a change in key color that remained until food was delivered. Response rates increased to near baseline levels in both components, and remained near baseline when the delays in both components were lengthened to 9 s. When delays were lengthened to 27 s, response rates fell to low levels in the briefly signaled delay component for three of four pigeons while remaining at or near baseline in the completely signaled delay component. In Experiment 3, low response rates under a 9-s unsignaled delay to reinforcement (tandem variable-interval 60 s fixed-time 9 s) increased when the delay was briefly signaled. The role of the brief stimulus as conditioned reinforcement may be a function of its temporal relation to food, and thus may be related to the eliciting function of the stimulus.     

Behavioral and dimensional contrast in rats.

Rats pressed a nose key for brain stimulation reinforcement presented on a fixed-interval schedule. Stimuli were drawn at random from a continuum of 12 white noise intensities in the range 62-95 decibels, spaced in 3 decibel steps. Experiment 1 varied the number of stimuli and the reinforcement contingencies associated with them. In Condition I (baseline) all stimuli signaled reinforcement; in Conditions II and III stimuli from one half of the continuum signaled reinforcement and those from the other half, extinction. However, in Condition II the 6 stimuli from the middle of the continuum were omitted. Experiment 2 held constant the number of stimuli and varied their spacing. In Condition I, each of 6 sounds signaled reinforcement. In Conditions II and II, three stimuli from one half of the continuum signaled reinforcement and three from the other half, extinction. However, in Condition II the stimuli were near the extremes of the continuum (Stimuli 1, 3, 4, 9, 10, 12). Condition III replaced Stimulus 3 with Stimulus 6 and Stimulus 10 with Stimulus 7. Behavioral contrast was seen in an increase over baseline in response rate to the stimuli associated with the constant schedule component when the variable component was changed to extinction. Dimensional contrast was seen in a further elevation of rate to intermediate positive stimulus values when stimuli were added to the border region between positive and negative values.     

Effect of signaled reinforcement on response variability

Three experiments examined the effect of signaling reinforcement on rats' lever pressing on contingencies that reinforced variable responding to extend the exploration of signaled reinforcement to a schedule that has previously not been examined in this respect. In Experiment 1, rats responding on a lag-8 variability schedule with signaled reinforcement displayed greater levels of variability (U values) than rats on the same schedule lacking a reinforcement signal. In Experiment 2, rats responding on a differential reinforcement of least frequent responses schedule also displayed greater operant variability with a signal for reinforcement compared with rats without a reinforcement signal. In Experiment 3, a reinforcement signal decreased the variability of a response sequence when there was no variability requirement. These results offer empirical corroboration that operant variability responds to manipulations in the same manner as do other forms of operant response and that a reinforcement signal facilitates the emission of the required operant.     

Signaled alternative reinforcement and the persistence of operant behavior

Differential reinforcement of alternative behavior (DRA) is a treatment designed to eliminate problem behavior by reinforcing an alternative behavior at a higher rate. Availability of alternative reinforcement may be signaled, as with Functional Communication Training, or unsignaled. Whether or not alternative reinforcement is signaled could influence both the rate and persistence of problem behavior. The present study investigated whether signaling the availability of alternative reinforcement affects the rate and persistence of a concurrently available target response with pigeons. Three components of a multiple concurrent schedule arranged equal reinforcement rates for target responding. Two of the components also arranged equal reinforcement rates for an alternative response. In one DRA component, a discrete stimulus signaled the availability of response‐contingent alternative reinforcement by changing the keylight color upon reinforcement availability. In the other DRA component, availability of alternative reinforcement was not signaled. Target responding was most persistent in the unsignaled DRA component when disrupted by satiation, free food presented between components, and extinction, relative to the signaled DRA and control components. These findings suggest the discrete stimulus functionally separated the availability of alternative reinforcement from the discriminative stimuli governing target responding. These findings provide a novel avenue to explore in translational research assessing whether signaling the availability of alternative reinforcement with DRA treatments reduces the persistence of problem behavior.     

Occasional reinforced trials during extinction can slow the rate of rapid reacquisition

Two appetitive conditioning experiments with rats examined reacquisition after conditioned responding was eliminated by either extinction or by a partial reinforcement procedure in which reinforced trials were occasionally presented among many nonreinforced trials. In Experiment 1, reacquisition to a conditional stimulus (CS) that had been conditioned and extinguished was more rapid than acquisition in a group that had received no prior conditioning. However, the addition of occasional reinforced trials to extinction slowed this rapid reacquisition effect. Experiment 2 replicated the result and showed that a procedure in which the CS and the unconditional stimulus (US) were unpaired in extinction interfered even further with reacquisition. The results suggest that rapid reacquisition is ordinarily produced when reinforced trials provide a contextual cue that can renew responding by signaling other acquisition trials (Ricker & Bouton, 1996). The effects of partial reinforcement in extinction are surprising from several theoretical perspectives and have useful clinical implications.     

Acquisition and extinction of signaled avoidance as a function of intermittent reinforcement

Signaled, shuttle-box avoidance responding in female rats of the Fischer₃₄₄ strain was examined as a function of four separate contingencies of intermittent reinforcement. In Experiment 1, when avoidance responses during acquisition were reinforced 25% of the time with prompt CS termination, animals responded equally often during acquisition and significantly more often during extinction than animals who received such reinforcement on a 100% schedule. Similar results were found under a trace procedure in Experiment 2 when avoidance responses were reinforced 25% of the time with informational feedback stimuli. In contrast, during Experiment 3, when animals were shocked on only 25% of the trials on which they failed to respond, the level of avoidance responding during both acquisition and extinction was significantly less than it was when animals were shocked on a 100% schedule. Comparable results were found in Experiment 4 when avoidance responses during acquisition averted shock on only 25% of the trials. Thus, intermittent reinforcement contingencies involving response-contingent feedback stimuli and shock have differential effects on avoidance responding during both acquisition and extinction trials under the signaled avoidance procedure.     

Behavioral contrast produced by a signaled decrease in local rate of reinforcement

Pigeons' key pecking in the presence of one stimulus (S1) was reinforced according to a response-dependent variable-interval schedule. Pecking rate during S1 increased (behavioral contrast) when a second stimulus (S2) [associated with either a response-dependent fixed-interval schedule (Experiment I) or a response-independent reinforcement schedule in which reinforcement availability was signaled by visual (Experiment II) or temporal (Experiment III) stimuli] alternated with S1. These experiments suggest that a discriminable, signaled decrease in local reinforcement rate during S2 is an antecedent of the behavioral contrast response rate increases during S1.     

Food-paired stimuli as conditioned reinforcers: effects of d-amphetamine.

Seven pigeons were studied in two experiments in which key pecks were reinforced under a second-order schedule wherein satisfaction of variable-interval schedule requirements produced food or a brief stimulus. In the second part of each session, responses produced only the brief stimulus according to a variable-interval schedule (food extinction). For the 4 pigeons in Experiment 1, the response key was red throughout the session. In separate phases, the brief stimulus was either paired with food, not paired with food, or not presented during extinction. d-Amphetamine (0.3 to 10.0 mg/kg) dose-dependently reduced food-maintained responding during the first part of the session and, at intermediate dosages, increased responding during the extinction portion of the session. The magnitude of these increases, however, did not consistently depend on whether the brief stimulus was paired, not paired, or not presented. It was also true that under nondrug conditions, response rates during extinction did not differ reliably depending on pairing operations for the brief stimulus. In Experiment 2, 3 different pigeons responded under a procedure wherein the key was red in the component with food presentations and blue in the extinction component (i.e., multiple schedule). Again, d-amphetamine produced dose-related decreases in responding during the first part of a session and increases in responding in the second part of the session. These increases, however, were related to the pairing operations; larger increases were observed when the brief stimulus was paired with food than when it was not or when it was not presented at all. Under nondrug conditions, the paired brief stimulus controlled higher response rates during extinction than did a nonpaired stimulus or no stimulus. These findings suggest that d-amphetamine can enhance the efficacy of conditioned reinforcers, and that this effect may be more robust if conditioned reinforcers occur in the context of a signaled period of extinction.     

Effects of diazepam on schedule-controlled and schedule-induced behavior under signaled and unsignaled shock.

Schedule-controlled lever pressing and schedule-induced licking were studied in rats under a multiple fixed-interval fixed-interval schedule of food reinforcement upon which was superimposed a multiple variable-time variable-time schedule of electric-shock delivery. Shocks were signaled in one component of the multiple schedule and unsignaled in the other. The effects of diazepam upon the suppression of behavior during the signal (conditioned suppression) and during signaled and unsignaled shock (differential suppression) were studied under several shock intensities (Experiment 1) and at increased body weight (Experiment 2). In each study, diazepam led to dose-dependent increases in the rate of pressing and licking during signaled and unsignaled shock, but had little effect on conditioned suppression. the rate-enhancing effects of diazepam depended upon the intensity of shock, nature of the response, and whether or not shocks were signaled. The data was discussed in terms of (1) implications for understanding the effects of signaled and unsignaled shock on behavior, (2) the effects of diazepam on behavior suppressed by response-independent shock, and (3) comparison between operant and schedule-induced behavior.     

Pavlovian contingencies and resistance to change in a multiple schedule

According to theoretical accounts of behavioral momentum, the Pavlovian stimulus—reinforcer contingency determines resistance to change. To assess this prediction, 8 pigeons were exposed to an unsignaled delay-of-reinforcement schedule (a tandem variable-interval fixed-time schedule), a signaled delay-of-reinforcement schedule (a chain variable-interval fixed-time schedule), and an immediate, zero-delay schedule of reinforcement in a three-component multiple schedule. The unsignaled delay and signaled delay schedules employed equal fixed-time delays, with the only difference being a stimulus change in the signaled delay schedule. Overall rates of reinforcement were equated for the three schedules. The Pavlovian contingency was identical for the unsignaled and immediate schedules, and response—reinforcer contiguity was degraded for the unsignaled schedule. Results from two disruption procedures (prefeeding subjects prior to experimental sessions and adding a variable-time schedule to timeout periods separating baseline components) demonstrated that response—reinforcer contiguity does play a role in determining resistance to change. The results from the extinction manipulation were not as clear. Responding in the unsignaled delay component was consistently less resistant to change than was responding in both the immediate and presignaled segments of the signaled delay components, contrary to the view that Pavlovian contingencies determine resistance to change. Probe tests further supported the resistance-to-change results, indicating consistency between resistance to change and preference, both of which are putative measures of response strength.     

An investigation of peak shift and behavioral contrast for autoshaped and operant behavior.

Instrumental treadle press and nonreinforced key peck responses were monitored during discrimination training and generalization testing in pigeons on positive and negative reinforcement schedules. In Experiment 1, six pigeons pressed a treadle for food on a multiple variable-interval extinction schedule. In Experiment 2, three pigeons pressed a treadle to avoid shock on a multiple free-operant avoidance extinction schedule. Different color keylights signaled S+ and S- components. Some positive behavioral contrast occurred during discrimination training, but the effect was small. Pecking occurred to the S+ keylight in Experiment 1 but not in Experiment 2. On stimulus generalization tests, all subjects displayed a positive peak shift when pressing the treadle for food or to avoid shock. However, peak shift was not found for nonreinforced "autopecks" on the stimulus key, although an area shift was observed in Experiment 1. This is the first demonstration of peak shift for pigeons pressing treadles and the only reliable demonstration of peak shift when negative reinforcement maintained responding. These results, in combination with previous demonstrations of peak shift for rats pressing levers and pigeons pecking keys, indicate that peak shift is a general by-product of operant discrimination learning, since it occurs across a variety of the organisms, responses, and reinforcers.     

Appetitive pavlovian-instrumental interactions: Effects of inter-stimulus interval and baseline reinforcement conditions

Experiment I manipulated two variables which appear to influence whether a signal for food enhances or suppresses food-rewarded instrumental performance: interstimulus interval (ISI) during classical conditioning and instrumental reinforcement schedule during testing. In two groups a 10-s conditioned stimulus (CS) and food were paired (10-s and 20- to 100-s ISI), while in a third group they were unpaired. During signalled reinforcement of lever-pressing (S+), the CS suppressed responding in both paired groups. During signalled extinction (S-), responding in the 10-s ISI group was suppressed during the CS and enhanced for 60 s after CS offset; responding in the 20- to 100-s ISI group was enhanced both during the CS and for 120 s after CS offset. Experiment II examined whether the long ISI enhancement effect would occur when the baseline response rate was lowered by satiation rather than extinction. A 20- to 100-s CS and food were paired in one group and unpaired in another. After near-satiation on a CRF schedule, CS presentations caused a reduction in responding in both groups, with no significant difference between the two groups. The results of the two experiments were interpreted in terms of an interaction between the expectancy of food generated by stimuli conditioned at short and long ISIs and the expectancy of food availability controlled by the instrumental schedule.     

Within-subject testing of the signaled-reinforcement effect on operant responding as measured by response rate and resistance to change

Response rates under random-interval schedules are lower when a brief (500 ms) signal accompanies reinforcement than when there is no signal. The present study examined this signaled-reinforcement effect and its relation to resistance to change. In Experiment 1, rats responded on a multiple random-interval 60-s random-interval 60-s schedule, with signaled reinforcement in only one component. Response resistance to alternative reinforcement, prefeeding, and extinction was compared between these components. Lower response rates, and greater resistance to change, occurred in the component with the reinforcement signal. In Experiment 2, response rates and resistance to change were compared after training on a multiple random-interval 60-s random-interval 60-s schedule in which reinforcer delivery was unsignaled in one component and a response-produced uncorrelated stimulus was presented in the other component. Higher response rates and greater resistance to change occurred with the uncorrelated stimulus. These results highlight the significance of considering the effects of an uncorrelated signal when used as a control condition, and challenge accounts of resistance to change that depend solely on reinforcer rate.     

Choice and conditioned reinforcement.

A potential weakness of one formulation of delay-reduction theory is its failure to include a term for rate of conditioned reinforcement, that is, the rate at which the terminal-link stimuli occur in concurrent-chains schedules. The present studies assessed whether or not rate of conditioned reinforcement has an independent effect upon choice. Pigeons responded on either modified concurrent-chains schedules or on comparable concurrent-tandem schedules. The initial link was shortened on only one of two concurrent-chains schedules and on only one of two corresponding concurrent-tandem schedules. This manipulation increased rate of conditioned reinforcement sharply in the chain but not in the tandem schedule. According to a formulation of delay-reduction theory, when the outcomes chosen (the terminal links) are equal, as in Experiment 1, choice should depend only on rate of primary reinforcement; thus, choice should be equivalent for the tandem and chain schedules despite a large difference in rate of conditioned reinforcement. When the outcomes chosen are unequal, however, as in Experiment 2, choice should depend upon both rate of primary reinforcement and relative signaled delay reduction; thus, larger preferences should occur in the chain than in the tandem schedules. These predictions were confirmed, suggesting that increasing the rate of conditioned reinforcement on concurrent-chains schedules may have no independent effect on choice.     

Control of responding during stimuli that precede transitions in reinforcement frequency

Pigeons' responses were reinforced on a variant of a mixed variable-interval extinction schedule of reinforcement in which the transition to the higher reinforcement rate was signaled by a trace stimulus projected on the response key prior to the onset of the component correlated with food delivery. In the first of two experiments, the duration of the trace stimulus preceding the component correlated with food delivery was varied from 1.5 to 50.0 s and in the second experiment, the reinforcement frequency in the same component was varied from 10 to 60 reinforcers per hour. Pigeons pecked at the trace stimulus preceding the onset of the component correlated with food delivery even though responding was not reinforced in its presence and only one of the changes in reinforcement rate (i.e., from extinction to reinforcement) was signaled. The rate of pecking during the trace stimulus was a function of its duration but not of the reinforcement frequency in the following component. Higher rates generally occurred at the shorter trace-stimulus durations. Component responding following the offset of the trace stimulus was under discriminative control of the trace stimulus whether or not responding occurred in the presence of the trace stimulus.