Occasional reinforced trials during extinction can slow the rate of rapid reacquisition

Two appetitive conditioning experiments with rats examined reacquisition after conditioned responding was eliminated by either extinction or by a partial reinforcement procedure in which reinforced trials were occasionally presented among many nonreinforced trials. In Experiment 1, reacquisition to a conditional stimulus (CS) that had been conditioned and extinguished was more rapid than acquisition in a group that had received no prior conditioning. However, the addition of occasional reinforced trials to extinction slowed this rapid reacquisition effect. Experiment 2 replicated the result and showed that a procedure in which the CS and the unconditional stimulus (US) were unpaired in extinction interfered even further with reacquisition. The results suggest that rapid reacquisition is ordinarily produced when reinforced trials provide a contextual cue that can renew responding by signaling other acquisition trials (Ricker & Bouton, 1996). The effects of partial reinforcement in extinction are surprising from several theoretical perspectives and have useful clinical implications.     

Effect of schedule and magnitude of reinforcement on instrumental learning in the toad, Bufo arenarum

Extinction after training with continuous (CR) or 50% partial (PR) reinforcement, and with different magnitudes of reward, was studied in the amphibian Bufo arenarum, in a runway situation. In Experiment 1, a group of toads received massed-trial, CR training with access to water as the reward. Performance improved during acquisition, including an improvement on the first trial of each session. Extinction was rapid and there was evidence for spontaneous recovery of the running response. In Experiment 2, groups of toads received PR or CR training at a rate of one trial per day. PR impaired acquisition and resulted in poor responding during extinction, compared to CR. Experiment 3 factorially studied the effects of schedule (PR vs CR) and distribution of practice (15 s vs 300 s intertrial interval). Acquisition was impaired by PR training but had little effect on extinction performance. Different magnitudes of water reinforcement were used in Experiment 4 in a one-trial-per-day situation. Terminal acquisition performance was a monotonic function of reward magnitude, but there were no differences in extinction performance across groups. The results are discussed in relation to comparative and developmental data on the paradoxical effects of reward.     

The effects of amount of training per reversal on successive reversals of a color discrimination

Three groups of pigeons were trained on a red-green discrimination in which the stimuli were alternately presented in a multiple schedule of reinforcement. The discrimination was reversed 24 times. Groups were given 1, 2, or 4 hr of training on each discrimination. Increasing the length of training had two principal effects on reversal performance: it increased the rate of extinction of responding to one of the stimuli and increased the rate of reacquisition of responding to the other. The latter effect involved both an increase in reacquisition of responding to a positive stimulus within reversals and an increase in recovery of responding to the previous negative stimulus between reversals. Improvements in performance of each group over the series of reversals were qualitatively similar to the two effects of length of training on each discrimination, and were analogous to effects obtained in other studies involving overtraining and successive reversals of simultaneous discriminations.     

An investigation of peak shift and behavioral contrast for autoshaped and operant behavior.

Instrumental treadle press and nonreinforced key peck responses were monitored during discrimination training and generalization testing in pigeons on positive and negative reinforcement schedules. In Experiment 1, six pigeons pressed a treadle for food on a multiple variable-interval extinction schedule. In Experiment 2, three pigeons pressed a treadle to avoid shock on a multiple free-operant avoidance extinction schedule. Different color keylights signaled S+ and S- components. Some positive behavioral contrast occurred during discrimination training, but the effect was small. Pecking occurred to the S+ keylight in Experiment 1 but not in Experiment 2. On stimulus generalization tests, all subjects displayed a positive peak shift when pressing the treadle for food or to avoid shock. However, peak shift was not found for nonreinforced "autopecks" on the stimulus key, although an area shift was observed in Experiment 1. This is the first demonstration of peak shift for pigeons pressing treadles and the only reliable demonstration of peak shift when negative reinforcement maintained responding. These results, in combination with previous demonstrations of peak shift for rats pressing levers and pigeons pecking keys, indicate that peak shift is a general by-product of operant discrimination learning, since it occurs across a variety of the organisms, responses, and reinforcers.     

Different mechanisms of fear extinction dependent on length of time since fear acquisition

Fear extinction is defined as a decline in conditioned fear responses (CRs) following nonreinforced exposure to a feared conditioned stimulus (CS). Behavioral evidence indicates that extinction is a form of inhibitory learning: Extinguished fear responses reappear with the passage of time (spontaneous recovery), a shift of context (renewal), and unsignaled presentations of the unconditioned stimulus (reinstatement). However, there also is evidence to suggest that extinction is an "unlearning" process corresponding to depotentiation of potentiated synapses within the amygdala. Because depotentiation is induced more readily at short intervals following LTP induction and is not inducible at all at a sufficient delay, it may be that extinction initiated shortly following fear acquisition preferentially engages depotentiation/"unlearning," whereas extinction initiated at longer delays recruits a different mechanism. We investigated this possibility through a series of behavioral experiments examining the recoverability of conditioned fear following extinction. Consistent with an inhibitory learning mechanism of extinction, rats extinguished 24-72 h following acquisition exhibited moderate to strong reinstatement, renewal, and spontaneous recovery. In contrast, and consistent with an erasure mechanism, rats extinguished 10 min to 1 h after acquisition exhibited little or no reinstatement, renewal, or spontaneous recovery. These data support a model in which different neural mechanisms are recruited depending on the temporal delay of fear extinction.     

Infantile stimulation and its effects on frustration- and fear-motivated behavior in rats

Infant rats that were either removed from the nest each day (handled) or left undisturbed (nonhandled) were, in adulthood, given 72 food-reinforced runway acquisition trials followed by 24 trials of extinction training with or without shock. Handled and nonhandled control animals were given runway training without food reinforcement. Reinforced rats ran faster than nonreinforced rats, and handled rats ran faster than nonhandled rats during the initial trials of runway acquisition irrespective of the reinforcement condition. Nonhadled rats stopped running sooner than handled rats when shock was introduced in the goalbox, but differences between handled and nonhandled rats given extinction training without shock were small. Results of a second experiment showed no differences between handled and non-handled rats in the magnitude of the depression effect after an incentive shift. It was concluded that infantile handling had little effect on frustration-motivated behavior, but did affect fear-motivated behavior.     

Changes Accompanying Practice upon Successive Samples of Verbal Material

Three groups of rats received either continuous, partial, or zero reinforcement in a first acquisition phase which was followed by an extended extinction phase. Then all Ss were given a reacquisition phase under continuous reinforcement conditions followed by a second extinction phase. While the usual partial reinforcement extinction effect (PREE) was found during the major part of the first extinction phase, it disappeared during the last few trials of that phase. No PREE was obtained during the second extinction phase in any of the three sections of the runway. The abolition of the PREE was attributed to the extinction of the r_F-s_F mechanism in the partial reinforcement group.     

An analysis of sequential variables in Pavlovian conditioning employing extended and limited acquisition training

Sequential theory’s memory model of learning has been successfully applied in response contingent instrumental conditioning experiments (Capaldi, 1966, Capaldi, 1967, Capaldi, 1994 and Capaldi and Miller, 2003). However, it has not been systematically tested in nonresponse contingent Pavlovian conditioning experiments. The present experiments attempted to determine if several sequential variables affect responding in Pavlovian situations as they do in instrumental ones. Of primary concern here were the effects on extinction of number of NR transitions (the number of times a nonreinforced trial is followed by a reinforced trial), N-length (the number of successive nonreinforced trials that precede a reinforced trial), and percentage of reinforcement (50 versus 100%) following either extended acquisition training (Experiment 1, 720 trials) or limited acquisition training (Experiment 3, 24 trials). In agreement with a sequential analysis, N-length increased resistance to extinction more than number of NR transitions following extensive training with the opposite occurring following limited training. In Experiment 1, greater resistance to extinction was associated with 50% than with 100% reinforcement, a partial reinforcement extinction effect (PREE). Experiment 2 examined an anomalous finding obtained in Experiment 1. A major theoretical difference between instrumental and Pavlovian conditioning has been held to be the greater ease of producing a PREE in instrumental than in Pavlovian conditioning (Kimble, 1961 and Mackintosh, 1974). However, the findings obtained here suggest that the probability of obtaining a PREE and other Pavlovian extinction effects, as in instrumental conditioning, increases along with the effectiveness of the sequential variables employed.     

Partial reinforcement effects in instrumental escape conditioning

In Experiment I acquisition and extinction of instrumental escape conditioning with rats (N = 64) were studied as a function of reinforcement magnitude under conditions of partial and continuous reinforcement. In Experiment II the effects of partial and continuous reinforcement were studied in rats (N = 96) during acquisition followed by small, medium, and large reductions in reinforcement magnitude. A water-tank escape apparatus was used with temperature as the relevant variable. It was found that (1) with large reinforcement magnitude a continuously reinforced group was superior in acquisition to one that was partially reinforced; there were no differences with small reinforcement; (2) disruptive effects of a nonreinforced trial (a) appear early in learning, (b) are quite strong after each nonreinforced trial, and (c) persist through several succeeding reinforced trials; (3) major competing behaviors persist throughout acquisition for small reinforcement magnitude regardless of schedule, decline with large reinforcement (more so with continuous than with partial), and return to a high level in extinction for all conditions; (4) the partial reinforcement extinction effect occurs after large reinforcement but not after small, and it appears only with large reductions in reinforcement magnitude which approach extinction conditions. Only the first part of the last finding appears to be consistent with the appetitive conditioning literature.     

Deepened extinction from compound stimulus presentation

Three experiments with rats and 2 with pigeons explored the effect of presenting 2 extinguished excitatory stimuli in compound. Four learning situations were used: Pavlovian magazine approach, Pavlovian fear conditioning, and instrumental discriminative instrumental learning in rats, as well as Pavlovian sign tracking in pigeons. All 5 experiments confirmed D. Reberg's (1972) observation that even after extinction of the individual stimuli, presenting them in compound evoked substantial responding. Moreover, nonreinforcement of that compound deepened extinction of an element more substantially than did additional presentation of that element alone. Such compound exposure reduced spontaneous recovery, reduced reinstatement, and slowed subsequent reconditioning. The primary determinant seemed to be the enhanced associative strength rather than the enhanced conditioned responding that occurred during the nonreinforced compound.     

Sequential effects of non-reward in a skinner box

The partial reinforcement extinction effect was examined within subjects in a simultaneous discrimination in a two bar Skinner box. Discrete trials were used, rats being required to press the bar under the illuminated cue light; one bar was correlated with 100% the other with 50% reinforcement. The three groups differed in the probability of a change in the cue light between trials during acquisition. When this probability was low, the 50% bar was preferred in extinction, while when it was higher (0.433 or 0.875) the 100% bar was preferred. These results confirm Capaldi's (1966) hypothesis of the partial reinforcement extinction effect, and support a suggested explanation of some conflicting results on partial reinforcement effects in a Skinner box.     

‘Preparedness’ and ‘arousability’ as determinants of electrodermal conditioning

‘Arousability’, as defined through spontaneous electrodermal responses, has been empirically linked to anxiety, phobic symptoms and outcome of systematic desensitization. Previous data from our laboratory indicate that ‘preparedness’, as defined through potentially phobic vs. fear-irrelevant or ‘neutral’ conditioned stimuh, is an important determinant of electrodermal conditioning. The present experiment compared groups selected to be high or low in spontaneous responding during differential conditioning to potentially phobic or neutral stimuh. It was found that the effects of these two factors were essentially additive, i.e. conditioning and resistance to extinction were better for phobic stimuli and for high-arousal groups. The high-aroused group with phobic stimuh showed diffuse responding during acquisition, not differentiating between reinforced and unreinforced cues. However, it was the only group that failed to extinguish during 20 trials, which indicates that high arousal gives superior resistance to extinction particularly for phobic stimuli.     

Differential runway performance using goal box placement and running trials as discriminative stimuli

Two experiments compared the effects of reinforced, partially reinforced and nonreinforced intertrial goal box placements (ITR, ITP, and ITN respectively) on runway performance. In Experiment I differential responding (animals running slower on nonreinforced (N) trials than on reinforced (R) trials) was observed during acquisition for subjects receiving ITP preceding N trials or subjects receiving ITP before R trials, and also for subjects receiving ITR preceding N trials. No differential responding was observed in subjects receiving ITP prior to both N and R trials or ITN prior to N trials. In extinction, the subjects which had responded differentially during acquisition demonstrated reduced resistance to extinction in comparison to the subjects which had not. In Experiment II, a 2 × 2 factorial design was utilized with placement schedule (ITN and ITP) and type of trial predicted by placement (N and R) serving as the factors. Differential responding was observed in all four groups. The apparent contradiction between the results and the discrimination hypothesis proposed by Capaldi and Olivier (1967) to explain the attenuation of ITR and ITN effects on resistance to extinction is discussed.     

Does delay of reinforcement produce durable persistence?

In a foru-phase experiment, phase I was runway training under four different reinforcement conditions: partial reinforcement (PRF), partial delayed reinforcement (PDR), constant delayed reinforcement (CDR), and consistent reinforcement (CRF). During phase 2 extinction, PRF and PDR groups did not differ; both groups were more persistent than group CDR, which was in turn superior to the CRF control. Phase 3 was CRF reacquisition for all groups. During phase 4 extinction, PRF group was more presistent than the other three groups which did not differ. A Pavlovian counter-conditioning hypothesis was proposed to account for the absence of durable persistence following PDR training.     

An assessment of the role of handling cues in "spontaneous recovery" after extinction

Three experiments examined the assertion that presession handling cues that accompany training with reinforcement might account for spontaneous recovery when they reoccur following extinction. In Experiment 1, after extensive training on a variable-interval schedule, key pecking in pigeons was extinguished following either normal or distinctively different handling and transportational cues. Those cues resulted in enhanced spontaneous recovery 24 hr later when normal cues were reinstated. In Experiment 2, however, subjects tested following the normal handling cues showed no more spontaneous recovery than did subjects that spent the entire extinction-test interval in the experimental chambers and thus were tested without handling cues altogether. In Experiment 3, a group whose test for recovery began 10 min after being placed in the chambers yielded as much spontaneous recovery as did a group tested normally. Furthermore, a group for which extinction began at mid-session and for which handling therefore could not be a discriminative cue for extinction showed no more spontaneous recovery than did the other two groups. Handling cues thus contributed to spontaneous recovery only after explicit discrimination training, as provided in Experiment 1.     

Response elimination: A comparison of four procedures

Four procedures for eliminating an operant response were compared within a multiple schedule. Response reduction was most rapid when reinforcement was provided for a specific alternative response. The decline in responding produced by extinction and differential reinforcement of other behavior was similar. Responding initially increased and then gradually decreased when reinforcers were delivered independently of responding at fixed times. Removal of reinforcement from the alternative-response and DRO procedures did not result in recovery of the target response. However, the shift from response-independent dilivery of reinforcers to extinction caused an initial increase in responding.     

Food-avoidance in hungry pigeons, and other perplexities

Twenty-three pigeons were subjected to a series of procedures in which the key-peck's effects ranged from immediate, differential food reinforcement, through delayed reinforcement, the production of stimulus changes with and without probable secondary reinforcement, the prevention of food presentation ("food-avoidance"), to extinction. Neither primary nor secondary food reinforcement appeared to be essential for the maintenance or acquisition of key pecking. The food-avoidance contingency failed to suppress responding in any subject. Only complete extinction, when pecking produced neither food nor stimulus changes, eliminated all pecking for most subjects. A combination of stimulus-change reinforcement and food reinforcement appeared to account for the results, but only if it could be assumed that the presence of food in a procedure enhanced the reinforcing power of stimulus change, whether or not the food was also dependent upon responding. Such an interaction between reinforcers may be involved in the phenomenon of autoshaping.     

Priming and trial spacing in extinction: Effects on extinction performance,spontaneous recovery,and reinstatement in appetitive conditioning

Previous research in this laboratory suggests that priming of the conditional stimulus (CS) in short-term memory may play a role in the trial-spacing effects in appetitive conditioning. For example, a nonreinforced presentation of a CS 60 s before a reinforced trial with the same CS produced slower acquisition than a CS presentation that occurred 240 s before the reinforced trial. The results were consistent with the self-generated priming mechanism proposed by Wagner (e.g., Wagner 1978, 1981). The present experiments extended the earlier work by examining the effects of trial spacing in extinction rather than acquisition. After conditioning with a mixture of intertrial intervals (ITIs), rats received extinction with ITIs of 60 or 240 s, longer or shorter values, or different ways of “chunking” extinction trials in time. Although trial spacing produced effects on extinction performance that were consistent with our previous research on acquisition, there were few long-term differences in spontaneous recovery or in reinstatement. Short ITIs in extinction appear to affect extinction performance more than they affect extinction learning. Mechanisms of trial spacing in conditioning and extinction are discussed.     

Rapid reaquisition in conditioning of the rabbit's nictitating membrane response.

Reacquisition after extinction often appears faster than original acquisition. However, data from conditioned suppression studies indicate that this effect may arise from spontaneous recovery and reinstatement of unextinguished contextual stimuli related to the unconditioned stimulus (US). In the present experiments using the rabbit nictitating membrane preparation, spontaneous recovery was eradicated before reaquisition training. US contextual stimuli were controlled by retaining the US during extinction through explicit unpairings of the conditioned stimulus (CS) and US. Attempts were also made to drive the associative strength of the CS into the inhibitory region by differential conditioning and conditioned inhibition procedures. In all cases, reacquisition was very rapid in comparison with a rest control. The results are discussed with respect to their implications for CS and US processing models of conditioning.     

Return of fear in a human differential conditioning paradigm caused by a stimulus change after extinction

In a human fear conditioning experiment, 32 participants were trained in a differential conditioning procedure with geometrical shapes as CS+ and CS- (four presentations each), and an electric shock as US. Measures of conditioned responding were skin conductance response (SCR) and retrospective US-expectancy ratings. For half of the participants (Generalization Group, GG), the subsequent extinction phase consisted of four nonreinforced presentations of generalization stimuli (GS+ and GS-). Participants from the Extinction control Group received an equal amount of nonreinforced presentations of the CSs. Finally, all participants were tested with the original CSs. The results from both measures clearly show an increase in the size of the discrimination upon the stimulus change after extinction in the GG. Because this pattern is not observed in the Extinction control Group, extinction performance appears to be somehow restricted to the perceptual characteristics of the extinction stimulus. Interestingly, the size of the conditioned SCR discrimination in the GG is not influenced by the stimulus change after acquisition. This observation points to a differential impact of stimulus change after acquisition vs. extinction treatment. The findings are discussed from the theoretical perspective of renewal and the clinical perspective of Return of Fear.