Extinction in human learning and memory

In two experiments, participants were given extinction training in a human causal learning task. In both experiments, three critical experimental cues were paired with different outcomes in a first phase of training and were then extinguished in a second phase. Three control cues were given the same treatment in the first phase of training, but were not then presented in the second phase. Participants' ability to correctly identify the outcome with which each cue had been paired in the first phase was lower for extinguished than for control cues. Causal attributions to the extinguished cues were also lower than those to the control cues, a difference that correlated with outcome memory. These data are consistent with the idea that extinction in causal judgement is due, at least in part, to a failure to remember the cue-outcome relationship encoded in the first phase of training.     

Re-examination of the role of within-compound associations in the retrospective revaluation of causal judgements

We investigated blocking and retrospective revaluation of causal judgements using a scenario in which food cues acted as potential causes of an allergic reaction as the outcome. In the blocking contingency,the treatment cues were either paired or unpaired with the outcome prior to a second stage in which sequential compounds of treatment and target cues were paired with the outcome. The order of this compound and treatment training was reversed in retrospective revaluation contingencies. When the interstimulus interval between the treatment and target cues was unfilled on compound trials (Experiments 1 and 3), both blocking and retrospective revaluation were observed in that the target cue trained in compound with the paired treatment cue attracted lower causal ratings than the target cue trained in compound with the unpaired treatment cue. By contrast, performing a mental arithmetic task using numerals presented during the interstimulus interval had no effect on the magnitude of blocking but rendered retrospective revaluation unreliable (Experiments 2 and 3). These results provide further support for accounts of revaluation based upon within-compound associations.     

Partial reinforcement and breadth of learning

Two experiments on partial reinforcement were undertaken to test predictions made by a two process model of discrimination learning. In the first experiment rats were trained on a discrimination involving two relevant cues: one group (C) was trained on a 100: o schedule, the other (P) on a 50:0 schedule. Both groups were then given transfer tests with the two cues presented individually; finally all animals were extinguished on the original training stimuli and on the single cue stimuli. During extinction there was a negative correlation between the number of correct responses made by individual subjects of Group C to each single cue; whereas the correlation was positive for subjects of Group P. The second experiment employed basically the same design, but subjects were trained with seven relevant cues. The results of transfer tests showed that subjects of Group P learned to attach the correct response to many more cues than subjects of Group C. This suggests that the breadth of learning is greater under partial than under consistent reinforcement. The results were predicted by the model of discrimination learning under test.     

The power of causal beliefs and conflicting evidence on causal judgments and decision making

In two experiments, we investigated the relative impact of causal beliefs and empirical evidence on both decision making and causal judgments, and whether this relative impact could be altered by previous experience. Participants had to decide which of two alternatives would attain a higher outcome on the basis of four cues. After completing the decision task, they were asked to estimate to what extent each cue was a reliable cause of the outcome. Participants were provided with instructions that causally related two of the cues to the outcome, whereas they received neutral information about the other two cues. Two of the four cues—a causal and a neutral cue—had high validity and were both generative. The remaining two cues had low validity, and were generative in Experiment 1, but almost not related to the outcome in Experiment 2. Selected groups of participants in both experiments received pre-training with either causal or neutral cues, or no pre-training was provided. Results revealed that the impact of causal beliefs and empirical evidence depends on both the experienced pre-training and cue validity. When all cues were generative and participants received pre-training with causal cues, they mostly relied on their causal beliefs, whereas they relied on empirical evidence when they received pre-training with neutral cues. In contrast, when some of the cues were almost not related to the outcome, participants’ responses were primarily influenced by validity and—to a lesser extent—by causal beliefs. In either case, however, the influence of causal beliefs was higher in causal judgments than in decision making. While current theoretical approaches in causal learning focus either on the effect of causal beliefs or empirical evidence, the present research shows that both factors are required to explain the flexibility involved in human inferences.     

Contingency bias in probability judgement may arise from ambiguity regarding additional causes

We tested whether preventive and generative reasoning processes are symmetrical by keeping the training and testing of preventive (inhibitory) and generative (excitatory) causal cues as similar as possible. In Experiment 1, we extinguished excitors and inhibitors in a blocking design, in which each extinguished cause was presented in compound with a novel cause, with the same outcome occurring following the compound and following the novel cause alone. With this novel extinction procedure, the inhibitory cues seemed more likely to lose their properties than the excitatory cues. In Experiment 2, we investigated blocking of excitatory and inhibitory causes and found similar blocking effects. Taken together, these results suggest that acquisition of excitation and inhibition is similar, but that inhibition is more liable to extinguish with our extinction procedure. In addition, we used a variable outcome, and this enabled us to test the predictions of an inferential reasoning account about what happens when the outcome level is at its minimum or maximum (De Houwer, Beckers, & Glautier, 2002). We discuss the predictions of this inferential account, Rescorla and Wagner's (1972) model, and a connectionist model—the auto-associator.     

Backward conditioning: mediation by the context

The information acquired in backward conditioning (i.e., outcome-->cue) was assessed in 3 Pavlovian lick-suppression experiments with water-deprived rats as subjects. Experiment 1 confirmed previous research that few outcome-->cue pairings made the cue into a conditioned excitor and additionally showed that massive posttraining extinction of the training context attenuated a backward-trained cue's excitatory value. Experiment 2 found that many outcome-->cue pairings made the cue into a conditioned inhibitor and that the same context manipulation attenuated this inhibitory value. Experiment 3 confirmed the observations of Experiments 1 and 2 and demonstrated that these effects of context extinction were specific to backward-trained cues conditioned in the extinguished context. These results are interpreted in terms of cue-->context and context-->outcome associations.     

Encouraging children to think counterfactually enhances blocking in a causal learning task

According to a higher order reasoning account, inferential reasoning processes underpin the widely observed cue competition effect of blocking in causal learning. The inference required for blocking has been described as modus tollens (if p then q, not q therefore not p). Young children are known to have difficulties with this type of inference, but research with adults suggests that this inference is easier if participants think counterfactually. In this study, 100 children (51 five-year-olds and 49 six- to seven-year-olds) were assigned to two types of pretraining groups. The counterfactual group observed demonstrations of cues paired with outcomes and answered questions about what the outcome would have been if the causal status of cues had been different, whereas the factual group answered factual questions about the same demonstrations. Children then completed a causal learning task. Counterfactual pretraining enhanced levels of blocking as well as modus tollens reasoning but only for the younger children. These findings provide new evidence for an important role for inferential reasoning in causal learning.     

An assessment of the role of handling cues in "spontaneous recovery" after extinction

Three experiments examined the assertion that presession handling cues that accompany training with reinforcement might account for spontaneous recovery when they reoccur following extinction. In Experiment 1, after extensive training on a variable-interval schedule, key pecking in pigeons was extinguished following either normal or distinctively different handling and transportational cues. Those cues resulted in enhanced spontaneous recovery 24 hr later when normal cues were reinstated. In Experiment 2, however, subjects tested following the normal handling cues showed no more spontaneous recovery than did subjects that spent the entire extinction-test interval in the experimental chambers and thus were tested without handling cues altogether. In Experiment 3, a group whose test for recovery began 10 min after being placed in the chambers yielded as much spontaneous recovery as did a group tested normally. Furthermore, a group for which extinction began at mid-session and for which handling therefore could not be a discriminative cue for extinction showed no more spontaneous recovery than did the other two groups. Handling cues thus contributed to spontaneous recovery only after explicit discrimination training, as provided in Experiment 1.     

Recovery from blocking between outcomes

Contemporary associative learning research largely focuses on cue competition phenomena that occur when 2 cues are paired with a common outcome. Little research has been conducted to investigate similar phenomena occurring when a single cue is trained with 2 outcomes. Three conditioned lick suppression experiments with rats assessed whether treatments known to alleviate blocking between cues would also attenuate blocking between outcomes. In Experiment 1, conditioned responding recovered from blocking between outcomes when a long retention interval was interposed between training and testing. Experiment 2 obtained recovery from blocking between outcomes when the blocking outcome was extinguished after the blocking treatment. In Experiment 3, a recovery from blocking between outcomes occurred when a reminder stimulus was presented in a novel context prior to testing. Collectively, these studies demonstrate that blocking of outcomes, like blocking of cues, appears to be caused by a deficit in the expression of an acquired association.     

Transfer of the partial reinforcement extinction effect between escape (shock) and appetitive (food) conditioning

In the first experiment rats were given partial reinforcement or continuous reinforcement in either an escape or an appetitive paradigm. Subsequently, the rats received continuous reinforcement training under motivational conditions opposite those experienced earlier. Finally, responses were extinguished according to the motivational conditions experienced in the second phase. The results indicated that partial reinforcement in the initial phase operated to increase resistance to extinction in the last. In a second experiment this intermotivational partial reinforcement extinction effect was shown to survive interpolated experiences with extinction, a 1-week rest period, and continuous reinforcement reacquisition. A third experiment examined the influence of intramodal versus intermodal nonreinforcement-reinforcement sequences on the intermotivational partial reinforcement extinction effect. Interactive effects between similarity of aversive outcome (escape nonreinforcement, appetitive nonreinforcement) and reinforcement type (negative, positive) were found. The theoretical implications of the data from all three experiments are discussed.     

Forward blocking in human learning sometimes reflects the failure to encode a cue–outcome relationship

In two “allergist” causal judgement experiments, participants were trained with a blocking design (A?+?|AB+). The procedure allowed different food cues to be paired with different fictitious allergic reactions. On test, participants were asked to rate the causal efficacy of the target cues and to recall the particular allergic reaction (outcome) that had followed each cue during training. Forward blocking was observed on the causal judgement measure and on the outcome recall measure in both Experiment 1 and Experiment 2. A backward blocking contingency was also trained in Experiment 2 (AB?+?|A+). Backward blocking was not observed either on the causal judgement or on the outcome recall measure. The evidence from the recall measure suggests that forward blocking in this task results from a failure to encode the B–outcome relationship during training. Associative and nonassociative mechanisms of forward blocking are discussed.     

Forward blocking in human learning sometimes reflects the failure to encode a cue-outcome relationship

In two “allergist” causal judgement experiments, participants were trained with a blocking design (A + |AB+). The procedure allowed different food cues to be paired with different fictitious allergic reactions. On test, participants were asked to rate the causal efficacy of the target cues and to recall the particular allergic reaction (outcome) that had followed each cue during training. Forward blocking was observed on the causal judgement measure and on the outcome recall measure in both Experiment 1 and Experiment 2. A backward blocking contingency was also trained in Experiment 2 (AB + |A+). Backward blocking was not observed either on the causal judgement or on the outcome recall measure. The evidence from the recall measure suggests that forward blocking in this task results from a failure to encode the B-outcome relationship during training. Associative and nonassociative mechanisms of forward blocking are discussed.     

Interference between Elementally Trained Stimuli Can Take Place in One Trial

Recent research has shown that the acquisition of a second cue–outcome association can interfere with responding appropriate to a previously acquired association between another cue and the same outcome, even if the two cues had never received compound training (Matute & Pineño, 1998a). This is similar to other results in the paired-associate literature but it is problematic for associative theories of learning because all of them assume that compound training is necessary for cues to interfere with each other. However, given several assumptions, a recent revision of Wagner's (1981) SOP model proposed by Dickinson and Burke (1996) could account for most of the data available on interference between elementally trained cues. According to the modified SOP model, the target cue that is paired with the outcome during Phase 1 could acquire an inhibitory association with the outcome during the Phase 2 trials in which the interfering cue is trained and the target cue is absent. This inhibitory association could be responsible for the weak responding observed to the target cue during testing because it could interfere with the excitatory association acquired during Phase 1. If this is true, interference should be weaker as the number of Phase 2 interfering trials is reduced. However, the three experiments reported here show that interference can occur even when only one interfering trial is given during Phase 2. The results of these experiments, along with other results in the literature, add support to the idea that interference between elementally trained cues occurs during retrieval and that it is not due to the formation of inhibitory associations between an absent cue and the outcome.     

A Bayesian Theory of Sequential Causal Learning and Abstract Transfer

Two key research issues in the field of causal learning are how people acquire causal knowledge when observing data that are presented sequentially, and the level of abstraction at which learning takes place. Does sequential causal learning solely involve the acquisition of specific cause‐effect links, or do learners also acquire knowledge about abstract causal constraints? Recent empirical studies have revealed that experience with one set of causal cues can dramatically alter subsequent learning and performance with entirely different cues, suggesting that learning involves abstract transfer, and such transfer effects involve sequential presentation of distinct sets of causal cues. It has been demonstrated that pre‐training (or even post‐training) can modulate classic causal learning phenomena such as forward and backward blocking. To account for these effects, we propose a Bayesian theory of sequential causal learning. The theory assumes that humans are able to consider and use several alternative causal generative models, each instantiating a different causal integration rule. Model selection is used to decide which integration rule to use in a given learning environment in order to infer causal knowledge from sequential data. Detailed computer simulations demonstrate that humans rely on the abstract characteristics of outcome variables (e.g., binary vs. continuous) to select a causal integration rule, which in turn alters causal learning in a variety of blocking and overshadowing paradigms. When the nature of the outcome variable is ambiguous, humans select the model that yields the best fit with the recent environment, and then apply it to subsequent learning tasks. Based on sequential patterns of cue‐outcome co‐occurrence, the theory can account for a range of phenomena in sequential causal learning, including various blocking effects, primacy effects in some experimental conditions, and apparently abstract transfer of causal knowledge.     

Effects of contextual conditioning and unconditional stimulus presentation on performance in appetitive conditioning

Four experiments with rat subjects examined the effects of contextual conditioning on conditioned appetitive performance. Experiment 1 compared the effects of contextual conditioning on performance to conditioned stimuli (CSs) with different conditioning histories. Contextual conditioning enhanced performance to the CS if the CS had first been conditioned and then extinguished, but had no effect on performance when the CS had been merely paired or unpaired with food. Experiments 2 and 3 then asked whether the effect on the extinguished CS was due to contextual conditioning acting as a cue for conditioning. In Experiment 2, extinction procedures in which extra unconditioned stimuli (USs) were presented during the intertrial intervals were found to reduce the CS's sensitivity to enhancement by contextual conditioning, but had no effect on spontaneous recovery. In Experiment 3, USs added to conditioning or extinction acquired the ability to cue the corresponding performance. Under some conditions, USs added to conditioning could suppress performance (Experiment 4). The results suggest that contextual conditioning has complex effects that can be better understood by recognizing that contextual conditioning, as well as the USs that create it,Mayacquire discriminative control over conditioned responding.     

Effect of interpolated extinction after partial delay of reward training on subsequent reacquisition and extinction

Four groups of albino rats were run four trials a day in a straight runway for 44 days. On the first 15 days, two groups were given continuous immediate reward (IR) and two groups a 50 per cent, schedule of 30-sec. partial delay of reward (PDR). On the next 15 days, one IR group and one PDR group were extinguished, while the other IR and PDR groups remained on their original schedules. In the third phase, all groups received 8 days of training on IR. Finally, all groups were given 6 days of extinction training. In the first extinction, PDR produced greater resistance to extinction than IR. In the second extinction period, the PDR group which had previously been given extinction and the two IR groups extinguished relatively rapidly and at approximately the same rates, while the PDR group which had not been extinguished was significantly more resistant to extinction than the other three groups.     

Recovery of an overshadowed association achieved by extinction of the overshadowing stimulus

Three experiments are reported in which conditioned lick suppression by water-deprived rats was used as an index of associative strength. In Experiment 1, overshadowing of a light by a tone was observed when the light-tone compound stimulus was paired with foot shock. After initial compound pairings, the tone-shock association was extinguished in one group of subjects. Subsequently, these animals demonstrated significantly higher levels of suppression to the light relative to a control group in which the tone had not been extinguished. Experiment 2 replicated this effect while failing to find evidence to support the possibilities that extinction presentations of the overshadowing tone act as retrieval cues for the light-shock association, or that, via second-order conditioning, the light-shock association is actually formed during extinction of the tone. Experiment 3 determined that the recovery from overshadowing observed in Experiments 1 and 2 was specific to the extinction of the overshadowing stimulus rather than the extinction of any excitatory cue. Collectively, these results suggest that the debilitated response to an overshadowed stimulus does not represent an acquisition failure, but rather the failure of an acquired association to be manifest in behavior.     

Evidence that a transient but cognitively demanding process underlies forward blocking

Blocking is a learning phenomenon in which prior experience inhibits learning about novel cues. Though the phenomenon itself has been well documented, the details of blocking-related processes still remain contentious. Two experiments investigated whether participants were engaged in demanding cognitive processing during different portions of a standard blocking paradigm. Participants in Experiment 1 engaged in a simple secondary task while completing a standard blocking procedure. Results showed that performance on the secondary task was briefly diminished early in the second phase of the blocking paradigm, when the novel cue is first paired with the pretrained cue. Participants in Experiment 2 performed a difficult cognitive load task during either the early or the late portions of the second phase of blocking. The blocking effect was eliminated when learners were under load early in the second phase, but not when learners were under load late in the second phase. These results suggest that blocking relies on a cognitively demanding process with a distinct time course. Computational simulations illustrate how a model that includes top-down (i.e., cognitively demanding) attentional modulation can reproduce the observed behaviour. This suggests that purely associative processes are not sufficient to explain the observed behaviour. Implications for current accounts of blocking are discussed.     

Investigations of the reinstatement of extinguished fear

In two experiments, fear was conditioned to the situational cues in one compartment of a hurdle-jumping apparatus and was then extinguished. Subsequently, either one shock (Experiment 1) or three or nine shocks (Experiment 2) were given in a situation distinctively different from that in which conditioning and extinction had taken place. Although some associative strength between the situational cues and fear was shown to have remained after extinction, in neither experiment did the postextinction-shock treatment increase the fear elicited by these cues: Escape-from-fear performance was no better in the shocked groups than in control groups given no additional shock. Thus, the nonassociative hypothesis which postulates that inflating the value of the representation of the UCS with shock-alone presentations can reinstate the extinguished fear of a stimulus was not supported. Rather, the results showed that, after extinction, an increase in fear of a simulus depended on further conditioning to that stimulus. The data also indicated that the nonvisual components of the situational cues predominated over the visual component.     

The hide-and-seek of retrospective revaluation: recovery from blocking is context dependent in human causal learning

In a typical blocking procedure, pairings of a compound consisting of 2 stimuli, A and X, with the outcome are preceded by pairings of only A with the outcome (i.e., A+ then AX+). This procedure is known to diminish responding to the target cue (X) relative to a control group that does not receive the preceding training with blocking cue A. We report 2 experiments that investigated the effect of extinguishing a blocking cue on responding to the target cue in a human causal learning paradigm (i.e., A+ and AX+ training followed by A- training). The results indicate that extinguishing a blocking cue increases conditioned responding to the target cue. Moreover, this increase appears to be context dependent, such that increased responding to the target is limited to the context in which extinction of the blocking cue took place. We discuss these findings in the light of associative and propositional learning theories.