Occasional reinforced trials during extinction can slow the rate of rapid reacquisition

Two appetitive conditioning experiments with rats examined reacquisition after conditioned responding was eliminated by either extinction or by a partial reinforcement procedure in which reinforced trials were occasionally presented among many nonreinforced trials. In Experiment 1, reacquisition to a conditional stimulus (CS) that had been conditioned and extinguished was more rapid than acquisition in a group that had received no prior conditioning. However, the addition of occasional reinforced trials to extinction slowed this rapid reacquisition effect. Experiment 2 replicated the result and showed that a procedure in which the CS and the unconditional stimulus (US) were unpaired in extinction interfered even further with reacquisition. The results suggest that rapid reacquisition is ordinarily produced when reinforced trials provide a contextual cue that can renew responding by signaling other acquisition trials (Ricker & Bouton, 1996). The effects of partial reinforcement in extinction are surprising from several theoretical perspectives and have useful clinical implications.     

Partial Reinforcement in Appetitive Pavlovian Conditioning with Rats

Four experiments examined the effects of a partial reinforcement schedule on extinction using appetitive Pavlovian conditioning. Extinction was slower after partial than after continuous reinforcement when the schedules were administered to different groups (Experiment 1). The opposite result was found in Experiments 2 and 3 when both schedules were presented to the same group in the same context. When the schedules were presented to the same group in different contexts, then extinction was again slower after partial reinforcement (Experiment 3). Experiment 4 demonstrated that a change of context facilitates extinction to a greater extent after conditioning with a partial reinforcement schedule than with a continuous one. The results are explained by assuming that the nonreinforced trials of a partial reinforcement schedule create an internal state that serves as a contextual cue.     

Partial reinforcement effects in instrumental escape conditioning

In Experiment I acquisition and extinction of instrumental escape conditioning with rats (N = 64) were studied as a function of reinforcement magnitude under conditions of partial and continuous reinforcement. In Experiment II the effects of partial and continuous reinforcement were studied in rats (N = 96) during acquisition followed by small, medium, and large reductions in reinforcement magnitude. A water-tank escape apparatus was used with temperature as the relevant variable. It was found that (1) with large reinforcement magnitude a continuously reinforced group was superior in acquisition to one that was partially reinforced; there were no differences with small reinforcement; (2) disruptive effects of a nonreinforced trial (a) appear early in learning, (b) are quite strong after each nonreinforced trial, and (c) persist through several succeeding reinforced trials; (3) major competing behaviors persist throughout acquisition for small reinforcement magnitude regardless of schedule, decline with large reinforcement (more so with continuous than with partial), and return to a high level in extinction for all conditions; (4) the partial reinforcement extinction effect occurs after large reinforcement but not after small, and it appears only with large reductions in reinforcement magnitude which approach extinction conditions. Only the first part of the last finding appears to be consistent with the appetitive conditioning literature.     

A partial reinforcement extinction effect despite equal rates of reinforcement during Pavlovian conditioning

In 4 experiments rats received appetitive Pavlovian conditioning followed by extinction. Food accompanied every trial with the conditioned stimulus (CS) for the continuously reinforced groups and only half of the trials for the partially reinforced groups. In contrast to previous experiments that have compared the effects of partial and continuous reinforcement, the rate at which food was delivered during the CS was the same for both groups. The strength of the conditioned response during extinction weakened more rapidly in the continuously than in the partially reinforced groups. The results demonstrate that the partial reinforcement extinction effect is a consequence of the nonreinforced trials with the CS, rather than the rate at which the unconditioned stimulus is delivered during the CS.     

An analysis of sequential variables in Pavlovian conditioning employing extended and limited acquisition training

Sequential theory’s memory model of learning has been successfully applied in response contingent instrumental conditioning experiments (Capaldi, 1966, Capaldi, 1967, Capaldi, 1994 and Capaldi and Miller, 2003). However, it has not been systematically tested in nonresponse contingent Pavlovian conditioning experiments. The present experiments attempted to determine if several sequential variables affect responding in Pavlovian situations as they do in instrumental ones. Of primary concern here were the effects on extinction of number of NR transitions (the number of times a nonreinforced trial is followed by a reinforced trial), N-length (the number of successive nonreinforced trials that precede a reinforced trial), and percentage of reinforcement (50 versus 100%) following either extended acquisition training (Experiment 1, 720 trials) or limited acquisition training (Experiment 3, 24 trials). In agreement with a sequential analysis, N-length increased resistance to extinction more than number of NR transitions following extensive training with the opposite occurring following limited training. In Experiment 1, greater resistance to extinction was associated with 50% than with 100% reinforcement, a partial reinforcement extinction effect (PREE). Experiment 2 examined an anomalous finding obtained in Experiment 1. A major theoretical difference between instrumental and Pavlovian conditioning has been held to be the greater ease of producing a PREE in instrumental than in Pavlovian conditioning (Kimble, 1961 and Mackintosh, 1974). However, the findings obtained here suggest that the probability of obtaining a PREE and other Pavlovian extinction effects, as in instrumental conditioning, increases along with the effectiveness of the sequential variables employed.     

The role of local context in autoshaping

Three experiments used an autoshaping procedure with pigeons to examine the effects of nonreinforced, nontarget stimuli (ITI-fillers) during the intertrial interval on responding to a reinforced target CS. Experiment 1 replicated a previous demonstration that an ITI-filler that occupied a substantial portion of the ITI attenuated responding to the target CS relative to a group trained with a similar ITI but lacking the ITI-filler. Experiment 2 found that the superiority of responding typically found with a long ITI relative to a short ITI, that is, the trial-spacing effect, can be reversed by imposing a filler stimulus during the ITI in the former condition. Experiment 3, using a within-subject design, found that when one background stimulus condition occupied a majority of the interreinforcement interval, pairings of one target CS with reinforcement that were embedded within this background condition, that is, a long duration local context occurred for this CS, yielded better performance that a second, reinforced, target CS paired with reinforcement in the shorter duration background condition. These results confirmed predictions derived from a local context view of cycle time. Comparator theories of classical conditioning require incorporation of this notion into their conceptualization of effective cycle time in order to explain the present findings.     

Extinction of the context and latent inhibition

The hypothesis that latent inhibition could be reduced by extinguishing the experimental context, that is, exposing the rats to the context between exposure to the conditional stimulus (CS) and conditioning, was tested. All experiments used the conditioned emotional response procedure. In Experiment 1, extinction was not effective when the animals were exposed to the clicker 40 times off the baseline of responding for food and when the clicker CS was partially reinforced with shocks during the test phase. In Experiments 2 and 3, latent inhibition could be reduced by extinction if the animals were exposed to the CS 24 or 16 times on-baseline, and if continuous reinforcement was used during the test. In Experiments 4, 5, and 6, we attempted to determine which variable was responsible for the discrepant results. In Experiment 4, extinction was effective with 20 or 40 on-baseline exposures to the CS, using continuous reinforcement during the test. In Experiment 5, extinction was not effective with exposure on- or off-baseline, using 24 exposures and partial reinforcement. Finally, in Experiment 6, extinction reduced latent inhibition using continuous, but not partial, reinforcement with 40 exposures off-baseline. From these results, we concluded that Wagner's model of habituation was not sufficient to account for latent inhibition and that a hybrid model, using both associative and cognitive representational processes, was necessary.     

The number of trials of compound conditioning: its effect on latent inhibition in rats.

Two experiments interspersed nonreinforced tests for the strength of the preexposed and nonpreexposed elements of a compound CS throughout conditioning. The nonreinforced exposures and the conditioning trials were superimposed on an appetitive operant baseline, and conditioned suppression of the rats' bar pressing was the dependent variable. In both experiments, the preexposed element failed to suppress responding and failed to gain strength over trials. The nonpreexposed element at first suppressed responding completely but gradually lost control of responding, even while the compound continued to suppress responding asymptotically. The results are discussed in terms of 'loss of salience' and 'discrimination' hypotheses.     

Successive negative contrast after partial reinforcement in the consummatory behavior of rats

Rats given access to a 32% sucrose solution later reject a 4% solution significantly more than controls that have only received the 4% solution. In Experiment 1, this consummatory successive negative contrast (cSNC) effect was attenuated by previous exposure to 50% partial reinforcement. Furthermore, recovery from cSNC was also facilitated by partial reinforcement. In Experiment 2, the attenuating effects of partial reinforcement on cSNC were eliminated by administration of the benzodiazepine anxiolytic chlordiazepoxide (5 mg/kg) before nonreinforced trials. In Experiment 3, the attenuating effect of partial reinforcement was greater after a shift from 32 to 6% solution, than after a shift from 32 to 2% solution. The parallels between the effects of partial reinforcement on consummatory and instrumental behavior are discussed.     

Stimulus generalization of auto-shaped key-pecking following interdimensional and extradimensional training

In three experiments, groups of pigeons were tested for wavelength generalization of auto-shaped key-pecking following different types of training. In Experiment I, one group received auto-shape trials with a 555-nm CS. Another group received auto-shape trials with a 555-nm CS and a white CS. A third group received auto-shape trials with a 555-nm CS and a white CS, except the white CS was nonreinforced. The third group yielded the sharpest gradient, whereas the second group yielded the flattest gradient. In Experiment II, one group received auto-shape trials with a 555-nm CS. A second group received auto-shape trials with a 555-nm CS and a compound CS consisting of a vertical white line on a 555-nm background. A third group received auto-shape trials with a 555-nm CS and a compound CS, except that the compound was nonreinforced. The third group yielded the sharpest wavelength gradient, whereas the second group yielded the flattest. In Experiment III, exposure differences to 555 nm in Experiment II resulting from differential speeds of auto-shape acquisition were controlled by the use of an ITI manipulation. Amount of exposure to 555 nm was shown not to be responsible for the results of Experiment II.     

Effects of partial reinforcement on conditioning,conditional probabilities,asymptotic performance,and extinction of the rabbit’s nictitating membrane response

Classical conditioning and extinction of the rabbit nictitating membrane response was investigated under 100 per cent, and two 50 per cent reinforcement conditions (50% equated total trials, and 50% equated reinforced trials) at average intertriai intervals (ITIs) of 30 and 60 sec. It was found that: (1) partial reinforcement reduced the rate of conditioning, but all groups eventually attained asymptomatic performance levels near 100 per cent; (2) first-order and second-order conditional probabilities following reinforced and nonreinforced trials were virtually identical; and (3) no partial reinforcement extinction effect (PREE) was observed.     

The role of contextual associations in producing the partial reinforcement acquisition deficit

Three conditioned suppression experiments with rats as subjects assessed the contributions of the conditioned stimulus (CS)-context and context-unconditioned stimulus (US) associations to the degraded stimulus control by the CS that is observed following partial reinforcement relative to continuous reinforcement training. In Experiment 1, posttraining associative deflation (i.e., extinction) of the training context after partial reinforcement restored responding to a level comparable to the one produced by continuous reinforcement. In Experiment 2, posttraining associative inflation of the context (achieved by administering unsignaled outcome presentations in the context) enhanced the detrimental effect of partial reinforcement. Experiment 3 found that the training context must be an effective competitor to produce the partial reinforcement acquisition deficit. When the context was down-modulated, the target regained behavioral control thereby demonstrating higher-order retrospective revaluation. The results are discussed in terms of retrospective revaluation, and are used to contrast the predictions of a performance-focused model with those of an acquisition-focused model.     

The decrement in conditioned fear with increased trials of simultaneous conditioning is not specific to the simultaneous procedure

Three experiments using a conditioned punishment paradigm with rat subjects examined the possibility that the nonmonotonic acquisition function previously found to characterize simultaneous conditioning was due to the noninformative nature of the conditioned stimulus (CS). In Experiment 1 the suppressive effects of a CS previously presented with an unconditioned stimulus (US) in a simultaneous and forward (informative) manner were compared following 20 and an additional 60 conditioning trials. Excitatory conditioning similarly diminished with increased trials for both the simultaneous and forward procedures. Experiment 2 employed a between-groups design. Simultaneous, forward, and trace conditioning procedures were compared following 20 or 100 trials. Each of the three 100-trial groups showed less resistance to extinction than their 20-trial counterparts. Experiment 3 determined that the decrement in excitatory conditioning for the 100-trial groups was not due to the greater number of US presentations, per se, but rather to the number of CS-US pairings. The nonmonotonic acquisition function observed with all three conditioning procedures indicated that informational factors were not responsible for the decrement observed in simultaneous conditioning. The pattern of results suggested that subjects receiving extended conditioning trials were better able to discriminate between training and testing.     

Classical skin conductance response conditioning: effects of intermittent reinforcement and information about schedule contingencies.

Skin conductance responses were differentially conditioned using reinforcement schedules of 25%, 50%, 75%, and 100%, manipulated between subjects. Half of the subjects were informed about schedule contingencies, and half were uninformed. The interstimulus interval was 6 sec. Discrimination of first-interval responses (1.0-3.5 sec after conditioned stimulus [CS] onset) by informed subjects did not vary with the ratio variable, but that by uninformed subjects improved with increasing reinforcement ratio because of diminished response levels to the nonreinforced CS (CS-). Discrimination of second-interval responses (3.6-7.0 sec after CS onset) improved as a function of increasing reinforcement ratio because of elevated response levels to the reinforced CS (CS+), but the effect was not persistent across trials in informed subjects. Performance in the first and second intervals did not reflect sequential increments and decrements as a function of reinforced and nonreinforced trials. Third-interval responses (7.1-9.9 sec after CS on nonreinforced trials) were not affected by schedule manipulations, but unconditioned responses diminished with increasing reinforcement ratio. Information about schedule contingencies led to superior discrimination of first-, second-, and third-interval responses and to suppression of unconditioned responses.     

Within-compound associations in a variety of compound conditioning procedures

Three experiments used a flavor-aversion preparation to demonstrate the occurrence of within-compound associations in various conditioning procedures. In Experiment 1 rats first received two separate two-flavor compounds, each followed by a mild US. They then received one element from one compound paired with a strong US and one element from the other compound nonreinforced. A subsequent choice test of the other two flavors from the compounds revealed that the consumption of each test flavor reflected the conditioning history of the element with which it had been paired. Rats drank less of the flavor whose associate had been paired with the strong US than of the flavor whose associate had been nonreinforced. In Experiment 2 within-compound associations were observed using a similar design in a blocking procedure in which one element from each of the two compounds was conditioned previously. Experiment 3 identified within-compound associations in a procedure where reinforced single element presentations were intermixed with nonreinforced compound presentations, as in a conditioned inhibition paradigm. These results suggest that within-compound associations occur in several important conditioning procedures which use multiple CS presentations.     

Partial reinforcement and extinction in vasomotor conditioning

In order to investigate cognitive versus traditional accounts of responding in extinction and the discrimination hypothesis for the partial reinforcement effect, 40 human subjects were randomly divided into two groups and were treated according to thermal vasomotor conditioning procedures using either 25 trials of continuous reinforcement or 100 trials of 25% partial reinforcement. At the onset of extinction, half of each group was given traditional noninformed extinction, while the other (informed) half had the thermal stimulator removed. The usual greater resistance to extinction was obtained after partial reinforcement than after continuous reinforcement in the two noninformed groups; however, immediate extinction of responding was obtained from the first extinction trial in the two informed groups. These results are consistent both with the discrimination hypothesis for the partial reinforcement extinction effect and with cognitive explanations of responding in extinction. Consequences for the behavioral therapies are discussed.     

Dissociative effects of different types and amounts of nonreinforced CS exposure on avoidance extinction and the CER

Two experiments examined the effectiveness of two amounts of flooding or response-prevention on hastening avoidance response extinction and on reducing CS-produced suppression of bar-pressing for food. In Experiment 1, 20 and 30 flooding trials were both shown to be effective in hastening the extinction of a well-learned shuttlebox avoidance response. In Experiment 2, rats trained under comparable conditions to those in Experiment 1 were tested following flooding for the CER in a different apparatus. The results indicated that 30, but not 20, flooding trials were sufficient to reduce the CER. In each experiment the results of additional control groups equated with the flooded groups for nonreinforced CS exposure also revealed a dissociation between the effectiveness of this CS time control procedure in hastening avoidance response extinction and in reducing the CER. Further comparisons showed that although 30 flooding trials did reduce the CER, the same total duration of nonreinforced CS Exposure in the form of avoidance extinction trials did not. Thus the context in which CS exposure occurs may affect the dynamics of extinction of the CER. The experiments are discussed in the broader context of dissociation of various indices of fear in humans.     

Contextual control of inhibition with reinforcement: adaptation and timing mechanisms

Four experiments with rats studied the effects of switching the context after Pavlovian conditioning. In three conditioned suppression experiments, a large number of conditioning trials created "inhibition with reinforcement" (IWR), in which fear of the conditional stimulus (CS) reached a maximum and then declined despite continued CS-unconditional stimulus pairings. When IWR occurred, a context switch augmented fear of the CS; IWR and augmentation were highly correlated. Neither IWR nor augmentation resulted from inhibition of delay (IOD): In conditioned suppression, IWR and augmentation occurred without IOD (Experiment 3), and in appetitive conditioning (Experiment 4), IOD occurred without IWR or augmentation. IWR may occur in conditioned suppression because the animal adapts to fear of the CS in a context-specific manner. The authors discuss several implications.     

Blocking and unconditioned response diminution in human classical autonomic conditioning

Four experiments examined blocking and UR diminution in human SCR conditioning. In Experiment 1, CS_X was conditioned in compound with previously conditioned CS_A, in one group, or in a compound with another novel stimulus, CS_N, in an additional group. UR diminution occurred, but there was no post-compound difference in CRs in the two groups and no difference in responding to CS_X and CS_A following equal training on each, i.e., no blocking. The correlation between blocking and UR diminution was not significant. In Experiment 2, two groups received a blocking procedure, but one was tested on CS_X and one on CS_A. UR diminution occurred, but there were no post-compound CR or UR differences. There was again no within-subjects blocking effect. The correlation between blocking and UR diminution was not significant, but this correlation was significant when data from Experiments 1 and 2 were combined. In Experiment 3, a differential conditioning procedure was used, followed by reinforcement of two compounds, one with CS_A+ and one with CS_A-. One subgroup was tested on CS_A+ and CS_A- with reinforcement; another was tested without reinforcement. UR diminution occurred, and there was significant blocking, but only after two nonreinforced tests. Blocking and UR diminution were significantly correlated. In Experiment 4, four groups received compound conditioning after initial training with CS_A. The compound contained CS_X and CS_A for three groups and two novel CSs for the fourth. Ten nonreinforced test trials were run. The three blocking groups received two CS_As plus eight CS_Xs, four CS_As plus six CS_Xs, or all CS_Xs. The fourth group received ten nonblocked CS_Xs. CR magnitude to blocked CS_X was significantly smaller than to nonblocked CS_X during the last six test trials, but not initially. Blocking and UR diminution were not correlated, in disagreement with the first three studies. The results indicated that repeated nonreinforced test trials are needed for blocking to be observed in this preparation. The proposition that blocking and UR diminution are both due to reduced processing of the US received only partial support.     

Appetitive conditioning in Octopus cyanea

The performance of Octopus cyanea was studied in 3 appetitive conditioning situations. In Experiment 1, 2 groups were trained in a runway; a large reward produced faster acquisition when reinforcement was consistent and better subsequent performance on a partial schedule than did a small reward. In Experiment 2, activity in the vicinity of a feeder was measured, and in Experiment 3, latency and probability of response were measured in an automated version of a traditional conditioned attack situation (Boycott & Young, 1950). There was evidence of acquisition with continuous reinforcement in both experiments but in neither with partial reinforcement. All of the results can be understood in terms of growth and decline in the strength of stimulus-reinforcer associations with reinforcement and nonreinforcement.