The effect of two ways of devaluing the unconditioned stimulus after first- and second-order appetitive conditioning.

Rats received first- and second-order conditioning based upon a food unconditioned stimulus (UCS). They then received one of two manipulations designed to reduce the value of that food, satiation, or pairing of food with high-speed rotation. The effects of these manipulations were assessed during extinction tests of the conditioned stimuli (CSs). Compared with controls, both manipulations reduced the activity produced by the first-order CS but did not affect that produced by a second-order CS. The results are interpreted as consistent with those from aversive UCSs in implying the involvement of a UCS representation in first- but not in second-order conditioning. They also suggest that a major effect of satiation is to reduce the value of the UCS.     

The amygdala is not necessary for unconditioned stimulus inflation after Pavlovian fear conditioning in rats

The basolateral complex (BLA) and central nucleus (CEA) of the amygdala play critical roles in associative learning, including Pavlovian conditioning. However, the precise role for these structures in Pavlovian conditioning is not clear. Recent work in appetitive conditioning paradigms suggests that the amygdala, particularly the BLA, has an important role in representing the value of the unconditioned stimulus (US). It is not known whether the amygdala performs such a function in aversive paradigms, such as Pavlovian fear conditioning in rats. To address this issue, Experiments 1 and 2 used temporary pharmacological inactivation of the amygdala prior to a US inflation procedure to assess its role in revaluing shock USs after either overtraining (Experiment 1) or limited training (Experiment 2), respectively. Inactivation of the BLA or CEA during the inflation session did not affect subsequent increases in conditioned freezing observed to either the tone conditioned stimulus (CS) or the conditioning context in either experiment. In Experiment 3, NBQX infusions into the BLA impaired the acquisition of auditory fear conditioning with an inflation-magnitude US, indicating that the amygdala is required for associative learning with intense USs. Together, these results suggest that the amygdala is not required for revaluing an aversive US despite being required for the acquisition of fear to that US.Pavlovian fear conditioning in rats is a behavioral model used to investigate the neurobiology underlying the development and maintenance of fear learning and memory (Grillon et al. 1996; LeDoux 1998, 2000; Bouton et al. 2001; Maren 2001b, 2005; Kim and Jung 2006). In this model, an innocuous conditioned stimulus (CS), such as a tone, is paired with an aversive unconditioned stimulus (US), such as a footshock. After one or more pairings, the rat learns that the CS predicts the US. As a consequence, CS presentations alone elicit a conditioned fear response (CR), which includes increases in heart rate, arterial blood pressure, hypoalgesia, potentiated acoustic startle, stress hormone release, and freezing (somatomotor immobility).The amygdala has been identified as one of the major regions in which fear memories are encoded and stored. Within the amygdala, the basolateral complex of the amygdala (BLA; consisting of the lateral, basolateral, and basomedial nuclei) and the central nucleus of the amygdala (CEA) receive convergent CS and US information and are involved in the acquisition of fear memories (LeDoux 1998, 2000; Fendt and Fanselow 1999; Davis and Whalen 2001; Maren 2001b; Schafe et al. 2001; Fanselow and Gale 2003; Wilensky et al. 2006; Zimmerman et al. 2007). In addition, the CEA has an important role in the expression of fear CRs (Fendt and Fanselow 1999; LeDoux 2000; Davis and Whalen 2001; Maren 2001b; Fanselow and Gale 2003). In support of this, many studies have shown that either permanent or temporary lesions of the BLA or CEA prevent the acquisition and/or expression of fear memories (Helmstetter 1992; Helmstetter and Bellgowan 1994; Campeau and Davis 1995; Maren et al. 1996a,b; Killcross et al. 1997; Muller et al. 1997; Walker and Davis 1997; Cousens and Otto 1998; Maren 1998, 1999, 2001a,b; Wilensky et al. 1999, 2000, 2006; Goosens and Maren 2001, 2003; Nader et al. 2001; Fanselow and Gale 2003; Gale et al. 2004; Koo et al. 2004; Zimmerman et al. 2007).In addition to its role in encoding CS–US associations during conditioning, recent work suggests that the amygdala is also involved in representing properties of the US itself. For example, temporary or permanent lesions of the BLA reduce both decrements in conditioned responding after devaluation of a food US (Hatfield et al. 1996; Killcross et al. 1997; Blundell et al. 2001; Balleine et al. 2003; Everitt et al. 2003; Pickens et al. 2003; Holland 2004) and increments in conditional responding after inflation of a shock US (Fanselow and Gale 2003). Moreover, recent electrophysiological studies in primates indicate that amygdala neurons represent the value of both aversive and appetitive outcomes (Paton et al. 2006; Belova et al. 2007, 2008; Salzman et al. 2007). These studies suggest that one function of the BLA is to represent specific properties of biologically significant events, such as the food or shock USs that are typically used in Pavlovian conditioning paradigms. By this view, the BLA may represent specific sensory properties of USs that shape the nature of learned behavioral responses to the US (Balleine and Killcross 2006) and allow CSs to gain access to the incentive value of the US (Everitt et al. 2003).In contrast to this view, we recently reported that rats with neurotoxic BLA lesions exhibit normal US revaluation after Pavlovian fear conditioning (Rabinak and Maren 2008). In this study, auditory fear conditioning (75 CS–US trials) with a moderate footshock (1 mA) was followed by several exposures (five US-alone trials) to an intense footshock (3 mA) during an inflation session. Both intact rats and rats with BLA lesions exhibit a robust increase in conditional freezing to the auditory CS during a subsequent retention test (Rabinak and Maren 2008). Control experiments suggested that this was due to a revaluation of the US with which the CS was associated, rather than nonassociative sensitization of fear engendered by exposure to intense shock. These data reveal that the BLA may not be necessary for representing properties of shock USs during Pavlovian fear conditioning. To address these issues further, we have examined the consequence of reversible pharmacological manipulations of the amygdala during US inflation on conditional fear responses established with either extensive or limited training.     

Effect of changing the unconditioned stimulus on appetitive blocking

In four experiments we examined the effects of changing the unconditioned stimulus (US) on appetitive blocking. In Experiments 1A, 1B, and 2 we established that substituting food for water, or water for food, in the compound stage did not attenuate blocking relative to groups that received the same US throughout conditioning. Experiment 3 showed that satiation with the US used prior to compound conditioning with a different US does not affect blocking. Experiment 4 revealed that changing the location of US delivery, as well as the quality of the US, also leaves blocking unaffected. It is suggested that these results demonstrate that blocking occurs, provided that there is no change in the affective properties of the US.     

Analysis of the learned irrelevance effect in appetitive Pavlovian conditioning

In Experiments 1 and 2 rats received uncorrelated presentations of a light conditioned stimulus (CS) and a food unconditioned stimulus (US) on each day of a preexposure phase. Control subjects received the same number of USs during the first half of preexposure and the same number of CSs during the second. Uncorrelated preexposure retarded inhibitory conditioning. Experiment 3 showed, however, that the different patterns of US preexposure experienced by the two groups could in itself influence the course of subsequent inhibitory conditioning. When this factor was equated by restricting the uncorrelated treatment to the first half of the pre-exposure phase (Experiment 2) or by extending the control treatment throughout the phase (Experiment 4) it was found that uncorrelated preexposure retarded excitatory conditioning, but facilitated inhibitory conditioning. This outcome challenges an interpretation in terms of the concept of learned irrelevance, which predicts that uncorrelated preexposure should retard both forms of conditioning.     

Facilitation of instrumental behavior by a Pavlovian appetitive conditioned stimulus

Three experiments examined appetitive Pavlovian-instrumental interactions by presenting separately trained conditioned stimuli (CSs) during reinforced instrumental responding in rabbits. Intra-oral reinforcement was used to minimize interference from peripheral responses such as magazine approach. In experiment 1, the rabbits were first trained to perform an instrumental head-raising response for sucrose reward. A conditioned jaw movement response was then established to a 2-sec CS by pairing it with sucrose; a control stimulus was unpaired with sucrose. Instrumental responding maintained by a variable-interval 40-sec schedule was enhanced during 10-sec presentations of the paired, but not the unpaired, CS. Responding on a variable-ratio 15 schedule was unaffected except on trials on which the pre-CS baseline response rate was low; in such cases the paired CS caused a long-lasting acceleration of responding. Noncontingent presentation of the sucrose reinforcer itself briefly suppressed responding but had no long-term effect. In Experiment 2, a CS that had been conditioned at a 10-sec duration produced the same pattern of effects as in the first study, indicating that facilitation resulted from CS presentation rather than from the frustrative effects of non-reinforcement of the CS. In Experiment 3 an inhibitory CS blocked facilitation by the excitatory CS but did not itself affect instrumental responding. These results support the view that Pavlovian processes play a positive role in instrumental performance and suggest that previous findings of suppression by a short-duration CS reflect peripheral interference. The dependence of facilitation on the baseline level of responding is discussed in terms of associative and motivational theories of Pavlovian mediation.     

Avoidance conditioning as a function of appetitive stimulus pretraining: Response and stimulus transfer effects

Pretrained appetitive discriminative stimuli were used as warning signals in subsequent avoidance learning. In Expt 1 identical responses were required in pretraining and in avoidance learning. An appetitive S+ facilitated avoidance learning in rats in comparison to S? or a stimulus previously uncorrelated with food. In Expt 2, the type of response in pretraining and in avoidance learning was varied. Groups with homogeneous responses in the two situations replicated Expt 1 results, whereas groups with different responses in pretraining and avoidance learning failed to show an advantage when S+ served as warning; in the heterogeneous response groups, S? was as effective as S+. Inhibitory factors in the heterogeneous groups were discussed as an explanation for these results.     

Modulation of unconditioned defense reflexes by a putative emotive Pavlovian conditioned stimulus

Four experiments showed differential modulation of defensive unconditioned responses (URs) in rabbits by contextual stimuli that Brandon and Wagner (1991) have shown similarly to modulate conditioned eyeblink responses. Two 30-s auditory cues, A and B, were differentially paired with shock. Tests were presentations of a response-eliciting probe stimulus within A, B, or a comparable blank interval, Experiments 1 and 2 demonstrated that A and B differentially facilitated eyeblink URs, and Experiments 3 and 4 showed that A and B similarly differentially facilitated startle responses elicited by airpuffs to the ear. These data are consistent with a characterization of Pavlovian conditioning that distinguishes between emotive and sensory conditioning and assumes that conditioned emotional responses similarly modulate specific conditioned and unconditioned defensive reflexes (Konorski, 1967; Wagner & Brandon, 1989).     

Differential effects of omission contingencies on various components of Pavlovian appetitive conditioned responding in rats

Five experiments with rat subjects investigated the effects of omission and partial reinforcement contingencies on five individual behaviors evoked by visual and auditory conditioned stimuli paired with a food unconditioned stimulus. The effects of omission depended on the behavior on which that contingency was placed: One behavior was eliminated, one was unaffected, and three were reduced relative to the performance of yoked controls. Partial reinforcement resulted in lower frequencies of three behaviors and higher frequencies of two behaviors, compared with performance under consistent reinforcement. A partial reinforcement extinction effect was noted with one behavior but not with the others. These results are related to the possible role of instrumental conditioning contingencies in generating conditioned behavior in this appetitive conditioning preparation and to the independence of individual components of a complex conditioned response.     

Effects of contextual conditioning and unconditional stimulus presentation on performance in appetitive conditioning

Four experiments with rat subjects examined the effects of contextual conditioning on conditioned appetitive performance. Experiment 1 compared the effects of contextual conditioning on performance to conditioned stimuli (CSs) with different conditioning histories. Contextual conditioning enhanced performance to the CS if the CS had first been conditioned and then extinguished, but had no effect on performance when the CS had been merely paired or unpaired with food. Experiments 2 and 3 then asked whether the effect on the extinguished CS was due to contextual conditioning acting as a cue for conditioning. In Experiment 2, extinction procedures in which extra unconditioned stimuli (USs) were presented during the intertrial intervals were found to reduce the CS's sensitivity to enhancement by contextual conditioning, but had no effect on spontaneous recovery. In Experiment 3, USs added to conditioning or extinction acquired the ability to cue the corresponding performance. Under some conditions, USs added to conditioning could suppress performance (Experiment 4). The results suggest that contextual conditioning has complex effects that can be better understood by recognizing that contextual conditioning, as well as the USs that create it,Mayacquire discriminative control over conditioned responding.     

Salivary conditioning with antennal gustatory unconditioned stimulus in an insect

Classical conditioning of olfactory conditioning stimulus (CS) with gustatory unconditioned stimulus (US) in insects has been used as a pertinent model for elucidation of neural mechanisms underlying learning and memory. However, a conditioning system in which stable intracellular recordings from brain neurons are feasibly obtained while monitoring the conditioning effect has remained to be established. Recently, we found classical conditioning of salivation in cockroaches Periplaneta americana, in which an odor was associated with sucrose solution applied to the mouth, and this conditioning could be monitored by activities of salivary neurons. Application of gustatory US to the mouth, however, leads to feeding movement accompanying a movement of the brain that prevents stable recordings from brain neurons. Here we investigated whether a gustatory stimulus presented to an antenna could serve as an effective US for producing salivary conditioning. Presentation of sucrose or sodium chloride solution to an antenna induced salivation and also increased activities of salivary neurons. A single pairing trial of an odor with antennal presentation of sucrose or sodium chloride solution produced conditioning of salivation or of activities of salivary neurons. Five pairing trials led to a conditioning effect that lasted for one day. Water or tactile stimulus presented to an antenna was not effective for producing conditioning. The results demonstrate that gustatory US presented to an antenna is as effective as that presented to the mouth for producing salivary conditioning. This conditioning system provides a useful model for studying the neural basis of learning at the level of singly identifiable neurons.     

Conditioned response-contingent delays of the unconditioned stimulus in human aversive conditioning.

Four groups of subjects were given either 0. 100, 500, or 1,000 msec delays of the unconditioned stimulus (UCS) contingent upon the occurrence of a conditioned response (CR) and were given a UCS 515 msec after conditioned stimulus (CS) onset when a CR did not occur. A fifth group received standard classical conditioning trials with an interstimulus interval of 515 msec. Overall performance decreased as CR-contingent UCS delay increased, with the classical conditioning group approximating the performance of the group receiving the 100-msec delay. The data were analyzed with the two-phase model of conditioning and the following results were obtained: The duration of Phase 1 of the model increased with contingent delay; operator limits associated with CR trials or with combined CR-CR (CR absent) trials decreased as a function of delay; and operator limits associated exclusively with CR trials were unaffected by the delay. Subjects receiving a contingent delay of 0 msec gave the shortest latency responses and exhibited reliable latency decreases across trials, suggesting an attempt to "beat" the UCS. The results were interpreted as contrary to what would be expected from low-of-effect theories which postulate that reinforcement results from a CR-UCS interaction, although they could be subsumed under a drive or an associative strength theory in which the aversive, or CR-supportive, strength of the UCS is assumed to be negatively correlated with contingent UCS delay.     

Analysis of us-preexposure effects in appetitive conditioning

In two experiments, rats received preexposure to one type of food followed by autoshaping in which presentation of one lever was associated with the preexposed food, and presentation of another lever with a novel type of food pellet. In both it was found that acquisition of the leverpress response occurred more readily on the lever associated with the novel food. This example of the US (unconditioned stimulus) preexposure effect is not to be explained in terms of the development of competing responses during preexposure. Explanations in terms of blocking by contextual cues and of habituation to the US are considered.     

Past experience influences the processing of stimulus compounds in human Pavlovian conditioning

In two human skin conductance conditioning experiments we investigated whether processing of stimulus compounds can be influenced by past experience. Participants were either pre-trained with a discrimination problem that could be solved elementally (A+, B−, AB+, C− in Experiment 1 and A+, AB+, C−, CB− in Experiment 2) or one that required a configural solution (AB+, BC−, CD+, DA− in Experiment 1 and A−, AB+, C+, CB− in Experiment 2). After pre-training, participants were shown an EX+, FX− discrimination. Subsequently, responding to individual components (E and F) was tested. After elemental pre-training, participants showed larger responses to the component from the previously reinforced compound (E) than to the component from the nonreinforced compound (F) whereas no such difference was found after configural pre-training. This means that the kind of pre-training influenced whether the later discrimination problem was processed elementally or configurally. The results indicate that organisms can flexibly process stimulus compounds in different ways.     

Asymmetrical effects of Pavlovian excitatory and inhibitory aversive transfer on Pavlovian appetitive responding and acquisition

Predictions of a theory of Pavlovian motivational transfer, which incorporates principles of both the theory of reciprocal inhibition and the Rescorla-Wagner model, were tested in several Pavlovian aversive to Pavlovian appetitive transfer tasks. As predicted, the presence of a signal for an aversive event, conditioned stimulus (AV CS+), reliably suppressed performance of appetitive conditioned responses (CRs) whether imposed during acquisition or on independently established responding. Acquisition of appetitive responding to a novel CS reinforced in compound with an AV CS+, however, was enhanced (“superconditioning”). This observation suggests that the effects of a discrepancy between expectation and actual outcome on a conditioning trial are influenced by the affective value of both the expectation and the reinforcer. These transfer effects were not symmetrical for an inhibitory aversive stimulus (AV CS?). An AV CS? did not enhance appetitive responding compared to a random control condition, nor did the AV CS? reduce (i.e., block) appetitive conditioning to a novel CS when appetitive reinforcement occurred in the presence of the AV CS?. Comparison of the two shock-exposed conditions with a naive control condition suggests that previous results that were apparently consistent with inhibitory aversive enhancement and blocking of appetitive conditioning may have been due to aversive context conditioning.