Stimulus control of schedule-induced activity in pigeons during multiple schedules

Stimulus control of schedule-induced general activity was demonstrated with pigeons using multiple schedules of response-independent food delivery. In Experiment 1, the introduction of food during a multiple variable-time 30-second variable-time 30-second schedule produced a tenfold increase in activity above the no-food baseline. Each pigeon developed stable differential activity rates during the components (correlated with red and green lights) of a multiple variable-time 30-second extinction schedule. Lengthening the extinction component from 1 to 7 minutes increased the rate differences and produced a reliable pattern of responding during S− (the stimulus correlated with extinction): Activity rate was high immediately following the change from S+ (the stimulus correlated with variable-time 30-second) to S−, then decreased abruptly and remained low throughout the middle of the interval, and subsequently showed a positively accelerated increase until the stimulus changed to S+. In Experiment 2, three pigeons were exposed to a mixed variable-time extinction schedule prior to a multiple variable-time extinction schedule. Auditory rather than visual stimuli were used to determine the generality of Experiment 1 results. The multiple- versus mixed-schedule results indicated that stimulus control of activity occurred for two of the birds, but rate differences between S+ and S− were much less than those demonstrated with visual stimuli. A direct comparison of visual and auditory stimulus control in Experiment 3 supported this conclusion. These parallels between the stimulus control of reinforced responding and that of schedule-induced activity suggest that the stimulus control of induced activity may be a factor in operant stimulus control.     

Control of responding during stimuli that precede transitions in reinforcement frequency

Pigeons' responses were reinforced on a variant of a mixed variable-interval extinction schedule of reinforcement in which the transition to the higher reinforcement rate was signaled by a trace stimulus projected on the response key prior to the onset of the component correlated with food delivery. In the first of two experiments, the duration of the trace stimulus preceding the component correlated with food delivery was varied from 1.5 to 50.0 s and in the second experiment, the reinforcement frequency in the same component was varied from 10 to 60 reinforcers per hour. Pigeons pecked at the trace stimulus preceding the onset of the component correlated with food delivery even though responding was not reinforced in its presence and only one of the changes in reinforcement rate (i.e., from extinction to reinforcement) was signaled. The rate of pecking during the trace stimulus was a function of its duration but not of the reinforcement frequency in the following component. Higher rates generally occurred at the shorter trace-stimulus durations. Component responding following the offset of the trace stimulus was under discriminative control of the trace stimulus whether or not responding occurred in the presence of the trace stimulus.     

Factors influencing inhibitory stimulus control: discrimination training and prior non-differential reinforcement

In Exp. I, shallow U-shaped gradients of inhibition in the line-orientation dimension were obtained from birds that had a vertical (0°) line on a green surround correlated with extinction and a blank green surround correlated with reinforcement. Birds that had massed extinction in the presence of the 0° line showed flat gradients. Thus, discrimination training, but not massed extinction, appears to generate inhibitory control. In Exp. II, as in studies of control by a stimulus correlated with punishment, non-differential training across the line-orientation dimension preceded further sessions. Steep inverted gradients about the 0° line were obtained after discrimination training with the 0° line correlated with extinction. Gradients obtained after massed extinction tended to be flat. Again, discrimination training was critical in obtaining negative gradients of stimulus control.     

Trial spacing is a determinant of cue interaction

Four conditioned lick suppression experiments with rats examined the effect of trial spacing on cue interaction. Experiments 1 and 2 found overshadowing to be eliminated with massed compound stimulus-outcome pairings and the usual trial spacing effect to be reversed with compound acquisition trials. Experiment 3 found that whether acquisition compound-outcome pairings were massed or spaced determined the effect of posttraining extinction treatment. Extinction of the overshadowing cue reduced responding following massed training and increased responding following spaced training. Extinction of the context decreased responding following massed training. Experiment 4 found the conditioning and devaluation results to be associative and stimulus specific. These results are in accord with the extended comparator hypothesis (J. C. Denniston, H. I. Savastano, & R. R. Miller, 2001).     

Inhibitory stimulus control following errorless discrimination learning

Three generalization procedures were used to investigate inhibitory stimulus control following discrimination learning with few errors. Three groups of pigeons acquired a discrimination between a green stimulus (the positive stimulus) and a vertical or horizontal line (the negative stimulus) through differential autoshaping followed by multiple schedule presentation of the two stimuli with gradually increasing stimulus durations. Genereralization testing was along a line-tilt continuum. For one group, the test involved a resistance-to-reinforcement procedure in which responses to all line tilts were reinforced on a variable-interval schedule. For a second group, also tested with the resistance-to-reinforcement procedure, the lines were superimposed on the green field that formerly served as the positive stimulus. A third group was tested in extinction with the combined stimuli. Control groups had no discrimination training but responding to green was nondifferentially reinforced. The control subjects responded more to all line tilts during testing than did the comparable experimental subjects, indicating that the negative stimulus had become an inhibitory stimulus. Both resistance-to-reinforcement groups revealed inhibitory gradients around the negative stimulus, but the gradient for the extinction group was relatively flat. These data are consistent with others that modify Terrace's early conclusion concerning the failure of inhibition to develop during errorless training.     

Discriminative effects of massed extincttion

Prior studies have reported that generalization gradients are not steepened if periods of non-reinforcement in S− follow and are not interspersed with periods of reinforcement in S+. Sharper gradients are produced by this massed-extinction procedure if it is preceded by prior discriminative training on a dimension orthogonal to the S+, S− dimension. The present study, using pigeons, found that generalization gradients along the wavelength dimension were steepened by massed-extinction sessions in 570 nm that had been preceded by: (1) discriminative training in which the S+ was a 550-nm light and the S− was a black vertical line superimposed on the 550-nm light; (2) non-differential reinforcement training with a 550-nm light and a black vertical line superimposed on the 550-nm light; (3) reinforcement training with only the 550-nm light. Massed-extinction sessions were administered until the response rate in the presence of the 570-nm stimulus was one-tenth of the mean response rate in the presence of the 550-nm stimulus during prior reinforcement training. Prior studies have used a time-dependent criterion, rather than a response-rate criterion of extinction, and this difference may be responsible for the differences in the effects of massed extinction on stimulus control.     

Stimulus-specific contrast effects during operant discrimination learning

In two experiments, pigeons' responding was equally reinforced in the presence of four line-orientation stimuli. Responding was then reinforced when only two of the four orientation stimuli were present; the remaining two orientations appeared during extinction. Response rates were often highest in the stimulus adjacent to the orientations presented during extinction and often lowest in that orientation adjacent to the orientations presented with reinforcement. These effects were stronger and more persistent when the stimuli were separated by a smaller angle, rendering the discrimination more difficult. These and other data suggest that discrimination training may not be accurately explained in terms of the simple effects of reinforcement and nonreinforcement associated with isolated stimuli, nor by accounts that depend upon stimulus generalization. Recent accounts of contrast that depend upon “emotionality” produced by nonreinforced responding or upon reinforcement-elicited responses are also difficult to apply to these data.     

Food-paired stimuli as conditioned reinforcers: effects of d-amphetamine.

Seven pigeons were studied in two experiments in which key pecks were reinforced under a second-order schedule wherein satisfaction of variable-interval schedule requirements produced food or a brief stimulus. In the second part of each session, responses produced only the brief stimulus according to a variable-interval schedule (food extinction). For the 4 pigeons in Experiment 1, the response key was red throughout the session. In separate phases, the brief stimulus was either paired with food, not paired with food, or not presented during extinction. d-Amphetamine (0.3 to 10.0 mg/kg) dose-dependently reduced food-maintained responding during the first part of the session and, at intermediate dosages, increased responding during the extinction portion of the session. The magnitude of these increases, however, did not consistently depend on whether the brief stimulus was paired, not paired, or not presented. It was also true that under nondrug conditions, response rates during extinction did not differ reliably depending on pairing operations for the brief stimulus. In Experiment 2, 3 different pigeons responded under a procedure wherein the key was red in the component with food presentations and blue in the extinction component (i.e., multiple schedule). Again, d-amphetamine produced dose-related decreases in responding during the first part of a session and increases in responding in the second part of the session. These increases, however, were related to the pairing operations; larger increases were observed when the brief stimulus was paired with food than when it was not or when it was not presented at all. Under nondrug conditions, the paired brief stimulus controlled higher response rates during extinction than did a nonpaired stimulus or no stimulus. These findings suggest that d-amphetamine can enhance the efficacy of conditioned reinforcers, and that this effect may be more robust if conditioned reinforcers occur in the context of a signaled period of extinction.     

Stimulus control of responding in the early trials of differential conditioning

In Experiment 1 two groups of rats were given 12 differential conditioning trials, seven to the rewarded alley (S+) and five to the nonrewarded alley (S?), prior to being extinguished in both alleys. Group S?S+ received S+ trials, following S? trials in acquisition, while Group S+S? did not receive S+ trials following S? trials in acquisition. In extinction S+ and S? trials were presented according to a quasi-random sequence for both groups. Running on the last 3 trials of acquisition was found to be faster following S+ than following S? trials. Group S?S+ showed greater resistance to extinction and less discriminative responding in extinction than Group S+S?. These results suggest that responding in differential conditioning is controlled not merely by S+ and S? but by the memories of reward (S^R) and of nonreward (S^N) as well. When the joint effects of both classes of cues were considered, e.g., S^R+S+, responding in the early trials of differential conditioning was shown to be highly orderly. Experiment 2 was highly similar to Experiment 1 except that Groups S?S+ and S+S? were equated along dimensions not equated in Experiment 1. The results obtained in Experiment 2 were highly similar to those obtained in Experiment 1.     

Responding to a positive stimulus by “satiated” pigeons

Skinner (1938) found that rats given discrimination training (Phase I) and then reinforced to “satiation” for responses in the presence of the negative stimulus (S?) (Phase II), began to respond again when the positive stimulus (S+) was reintroduced (Phase III). Experiment I replicated Skinner's finding with pigeons, alternating S+ and S? presentations during Phase III. In Experiment II, Phase II was extended, and Phase III results were similar to those of Experiment I, demonstrating that the recovery of S+ responding could not be attributed to a lax Phase II satiation criterion. In Experiment III, a uniform schedule of reinforcement was maintained throughout the three phases, and results similar to those of Experiment I were found, indicating that renewed S+ responding was not due to the shift in schedule between phases. In Experiment IV, Phase I consisted of discrimination training with two positive stimuli (S+_s and S+_n), and Phase II consisted of reinforcement for responses in the presence of S? and S+_s. During Phase III, significantly more responding was found to S+_s and S+_n than to S?, but no difference in responding was found between the two positive stimuli. In Experiment V, Phase I consisted of simple discrimination training, and during Phase II, responses in the presence of both S? and a novel stimulus (So) were reinforced. During Phase III, significantly more responding was found to S+ than to either S? or So, with no difference found between S? and So responding. Renewed responding to S+ during Phase III in the present experiments is best explained by behavioral contrast developed during Phase I.     

Occasional reinforced trials during extinction can slow the rate of rapid reacquisition

Two appetitive conditioning experiments with rats examined reacquisition after conditioned responding was eliminated by either extinction or by a partial reinforcement procedure in which reinforced trials were occasionally presented among many nonreinforced trials. In Experiment 1, reacquisition to a conditional stimulus (CS) that had been conditioned and extinguished was more rapid than acquisition in a group that had received no prior conditioning. However, the addition of occasional reinforced trials to extinction slowed this rapid reacquisition effect. Experiment 2 replicated the result and showed that a procedure in which the CS and the unconditional stimulus (US) were unpaired in extinction interfered even further with reacquisition. The results suggest that rapid reacquisition is ordinarily produced when reinforced trials provide a contextual cue that can renew responding by signaling other acquisition trials (Ricker & Bouton, 1996). The effects of partial reinforcement in extinction are surprising from several theoretical perspectives and have useful clinical implications.     

A stimulus-compounding assay of conditioned inhibition and excitation

Rats were trained in an operant discrimination involving two panel lights, A and B. Group CI (Conditioned Inhibition) was trained to lever press in the presence of A (S+), and not in the presence of the compound AB (S?). The opposite contingencies held in Group CE (Conditioned Excitation). Response rates were higher in Group CE than in Group CI, especially to the positive stimulus. However, the groups acquired the discrimination at comparable rates. In a separate phase of the experiment, a tone (C) was separately trained in the absence of the lights to control responding. All seven possible combinations of A, B, and C were then presented randomly during extinction. The inhibitory effect of B on C in Group CI was comparable in magnitude to the excitatory effect of B on C in Group CE. B's effect on C was not altered by the presence or absence of A, regardless of whether A was inhibitory or excitatory. The symmetry between CI and CE revealed by the stimulus-compounding test may be partly artifactual.     

Stress-enhanced fear learning in rats is resistant to the effects of immediate massed extinction

Enhanced fear learning occurs subsequent to traumatic or stressful events and is a persistent challenge to the treatment of post-traumatic stress disorder (PTSD). Facilitation of learning produced by prior stress can elicit an exaggerated fear response to a minimally aversive event or stimulus. Stress-enhanced fear learning (SEFL) is a rat model of PTSD; rats previously exposed to the SEFL 15 electrical shocks procedure exhibit several behavioral responses similar to those seen in patients with PTSD. However, past reports found that SEFL is not mitigated by extinction (a model of exposure therapy) when the spaced extinction began 24?h after stress. Recent studies found that extinction from 10?min to 1?h subsequent to fear conditioning "erased" learning, whereas later extinction, occurring from 24 to 72?h after conditioning did not. Other studies indicate that massed extinction is more effective than spaced procedures. Therefore, we examined the time-dependent nature of extinction on the stress-induced enhancement of fear learning using a massed trial's procedure. Experimental rats received 15 foot shocks and were given either no extinction or massed extinction 10?min or 72?h later. Our present data indicate that SEFL, following traumatic stress, is resistant to immediate massed extinction. Experimental rats showed exaggerated new fear learning regardless of when extinction training occurred. Thus, post-traumatic reactivity such as SEFL does not seem responsive to extinction treatments.     

Effects of stimulus control and deprivation upon discriminative responding

Discriminative responding in pigeons was studied under multiple variable-interval extinction schedules in which extinction was correlated with either a tone or a white keylight. The two procedures resulted in weak and strong stimulus control, respectively. In the first experiment, there was no interaction between schedule components when stimulus control was strong and reinforcement was omitted under the previously reinforced component. However, there was marked induction between components when stimulus control was weak and responding was extinguished under the previously reinforced component. In the second experiment, hours of food deprivation was varied under two levels of stimulus control. Deprivation mainly influenced response rates under the extinction stimulus, with greater absolute rate increases occurring the lower the existing level of stimulus control. Increases in responding during the extinction stimulus were four times as great from 24 to 72 hours of deprivation as from 24 to 48 hours under conditions of both high and low stimulus control.     

Non-differential return of fear in humans after a reinstatement procedure

The present experiment investigated reinstatement of fear in humans using a differential fear conditioning preparation. In this experiment, one neutral stimulus (conditioned stimulus; CS+) was paired with an aversive stimulus (unconditioned stimulus; US) during the acquisition phase, while another neutral stimulus was not (CS−). This procedure led to a difference in responding between the CS+ and the CS− (i.e., differential conditioning). After this acquisition phase, an extinction phase followed, during which both CSs were presented without the US, resulting in a decrease in differential conditioned responding. Reinstatement refers to the return of extinguished conditioned responses due to the experience of US-only trials after the extinction phase. This phenomenon was investigated by presenting half of the participants (reinstatement group) with unpredictable USs after the extinction phase. The control group did not receive these USs after the extinction procedure. The results show that return of fear was clearly apparent after the reinstating USs. This return of fear was, however, not limited to the CS+. An increase in ‘conditioned’ responding was also observed for the control stimulus. This interesting observation will be discussed against the background of a number of recent theoretical conceptualizations of reinstatement.     

Testing for inhibitory stimulus control with S- superimposed on S+

Pigeons learned a successive discrimination between a positive stimulus (red) correlated with a variable-interval 1-min reinforcement schedule and a negative stimulus (vertical line) correlated with either a variable-interval 5-min schedule or extinction. Transfer tests measured the rate of responding to the positive stimulus alone, to various orientations of the negative stimulus, and to the same line orientations superimposed on the positive stimulus. Although there were no gradients with minima at the training value for the negative stimulus dimension, the addition of the negative stimulus dimension to the positive stimulus always resulted in a lower response rate than that for the positive stimulus alone. The results demonstrate that an operational definition of inhibitory stimulus control that requires increased responding to stimuli more distant from a negative stimulus (along some dimension) is not always consistent with a definition that requires the suppression of responding in the presence of one stimulus, the positive stimulus, by the simultaneous presentation of another, the negative stimulus.     

Resistance to change and relapse of observing

Four experiments examined relapse of extinguished observing behavior of pigeons using a two-component multiple schedule of observing-response procedures. In both components, unsignaled periods of variable-interval (VI) food reinforcement alternated with extinction and observing responses produced stimuli associated with the availability of the VI schedule (i.e., S+). The components differed in the rate of food arranged (Rich = VI 30 s; Lean = VI 120 s). In Experiment 1, following baseline training, extinction of observing involved removal of both food and S+ deliveries, and reinstatement was examined by presenting either response-independent food or S+ deliveries. In Experiment 2, extinction involved removal of only food deliveries while observing responses continued to produce S+. Reinstatement was examined by delivering food contingent upon the first two food-key responses occurring in the presence of the S+. Experiment 3 assessed ABA renewal of observing by extinguishing food-key and observing responses in the presence of one contextual stimulus (i.e., B) and then returning to the original training context (i.e., A) during continued extinction. Experiment 4 examined resurgence by introducing food reinforcement for an alternative response during extinction, and subsequently removing that alternative source of food. Across experiments, relative resistance to extinction and relapse of observing tended to be greater in the component previously associated with the higher rate of primary reinforcement. Relapse of observing or attending to stimuli associated with primary reinforcement appears to be impacted by frequency of primary reinforcement in a manner similar to responding maintained directly by primary reinforcement.     

Temporally massed CS presentations generate more fear extinction than spaced presentations

Rodent fear conditioning models both excitatory learning and the pathogenesis of human anxiety, whereas extinction of conditional fear is a paradigm of inhibitory learning and the explicit model for behavior therapy. Many studies support a general learning rule for acquisition: Temporally spaced training is more effective than massed training. The authors asked whether this rule applies to extinction of conditional fear in mice. The authors find that both short- and long-term fear extinction are greater with temporally massed presentations of the conditional stimulus (CS). The data also indicate that once CS presentations are sufficiently massed to initiate, or "induce," extinction learning, further CS presentations are more effective when spaced.     

Generalization of the disruptive effects of alternative stimuli when combined with target stimuli in extinction

Differential‐reinforcement treatments reduce target problem behavior in the short term but at the expense of making it more persistent long term. Basic and translational research based on behavioral momentum theory suggests that combining features of stimuli governing an alternative response with the stimuli governing target responding could make target responding less persistent. However, changes to the alternative stimulus context when combining alternative and target stimuli could diminish the effectiveness of the alternative stimulus in reducing target responding. In an animal model with pigeons, the present study reinforced responding in the presence of target and alternative stimuli. When combining the alternative and target stimuli during extinction, we altered the alternative stimulus through changes in line orientation. We found that (1) combining alternative and target stimuli in extinction more effectively decreased target responding than presenting the target stimulus on its own; (2) combining these stimuli was more effective in decreasing target responding trained with lower reinforcement rates; and (3) changing the alternative stimulus reduced its effectiveness when it was combined with the target stimulus. Therefore, changing alternative stimuli (e.g., therapist, clinical setting) during behavioral treatments that combine alternative and target stimuli could reduce the effectiveness of those treatments in disrupting problem behavior.     

Stimulus selection with duration as a relevant cue

To date, research has not explicitly examined how duration competes with other stimulus dimensions for control over responding. The present study investigated some familiar selective mechanisms of stimulus control over key pecking in pigeons, with duration and line tilt as discriminative stimuli in successive discrimination procedures. Specifically, pigeon's key pecking was reinforced with food or extinguished following compound stimuli comprising one of two line orientation stimuli presented for one of two different durations. Traditional experimental designs explored stimulus additivity, overshadowing, blocking, and learned irrelevance. Although stimulus additivity was observed, control by duration was masked by line tilt in extinction testing that followed facilitated acquisition with redundant, relevant cues. In addition, although prior training with duration relevant partially blocked subsequent acquisition of control by line tilt when both stimulus dimensions were relevant, there was a tendency for control by duration to decrease with continued compound training. It was suggested that the greater time required to distinguish duration on a trial—compared to more commonly studied stimulus dimensions, which can be distinguished almost immediately—puts duration at a competitive disadvantage in situations where other relevant dimensions are also available.