Facilitation of instrumental behavior by a Pavlovian appetitive conditioned stimulus

Three experiments examined appetitive Pavlovian-instrumental interactions by presenting separately trained conditioned stimuli (CSs) during reinforced instrumental responding in rabbits. Intra-oral reinforcement was used to minimize interference from peripheral responses such as magazine approach. In experiment 1, the rabbits were first trained to perform an instrumental head-raising response for sucrose reward. A conditioned jaw movement response was then established to a 2-sec CS by pairing it with sucrose; a control stimulus was unpaired with sucrose. Instrumental responding maintained by a variable-interval 40-sec schedule was enhanced during 10-sec presentations of the paired, but not the unpaired, CS. Responding on a variable-ratio 15 schedule was unaffected except on trials on which the pre-CS baseline response rate was low; in such cases the paired CS caused a long-lasting acceleration of responding. Noncontingent presentation of the sucrose reinforcer itself briefly suppressed responding but had no long-term effect. In Experiment 2, a CS that had been conditioned at a 10-sec duration produced the same pattern of effects as in the first study, indicating that facilitation resulted from CS presentation rather than from the frustrative effects of non-reinforcement of the CS. In Experiment 3 an inhibitory CS blocked facilitation by the excitatory CS but did not itself affect instrumental responding. These results support the view that Pavlovian processes play a positive role in instrumental performance and suggest that previous findings of suppression by a short-duration CS reflect peripheral interference. The dependence of facilitation on the baseline level of responding is discussed in terms of associative and motivational theories of Pavlovian mediation.     

Inhibitory sensory preconditioning detected with a sodium depletion procedure

In each of two experiments, two groups of rats were exposed to three flavoured solutions: A (citric acid), B (salt), and AX (a compound of citric acid and saccharin). Both experiments used a between-subjects design in which a paired group received presentations of A followed by B, alternating with presentations of AX (i.e., A → B/AX), and an unpaired group received alternating presentations of A, B, and AX (i.e., A/B/AX). This arrangement was expected to establish X as an inhibitor of B in group paired but not in group unpaired. In Experiment 1, after preexposure all subjects received a single presentation of an XB compound, then experienced sodium depletion, and were tested for their consumption of X, which was greater in group unpaired than in group paired. In Experiment 2, after preexposure, all subjects received four presentations of a new flavour, C, in compound with B and subsequently, under sodium depletion, were tested for consumption of XC. Intake of the XC compound was less in group paired than in group unpaired. These results suggest that, in group paired, X acquired an inhibitory relationship with B both retarding the acquisition of an excitatory association with B (retardation test, Experiment 1) and reducing the response to a new stimulus, C, strongly associated to B (summation test, Experiment 2). These results provide direct evidence of inhibition between two neutral stimuli and, therefore, of inhibitory sensory preconditioning.     

Analyzing the random control procedure: Effects of paired and unpaired CSs and USs on autoshaping the chick's key peck with heat reinforcement

A series of five experiments using a total of 264 subjects investigated the effects of paired and unpaired key light (CS) and heat (US) stimuli on autoshaping the chick's key peck. Experiment 1 established that paired presentations of CS and US promoted a more rapid rise in key pecking than did randomly presented CSs and USs and that the specific sequence of stimuli under the random control procedure affected key pecking performance. Experiment 2 used a trace conditioning procedure to determine the role of the CS-US interval on autoshaping and to define empirically unpaired CSs and USs. Key pecking declined as the trace delay interval was increased from 0 to 25 sec; at 25 sec, no conditioning of key pecking occurred. Experiments 3–5 assessed the effects on autoshaped key pecking of (a) number of daily CS-US pairings, (b) added unpaired CS presentations, and (c) added unpaired US presentations, since paired and random control schedules differed in all of these respects. Reduction in the number of CS-US pairings slowed the acquisition of key pecking as did the concurrent addition of nonreinforced CSs and unsignaled USs. These results support theories of association formation that stress the effects of both paired and unpaired CSs and USs.     

Inhibitory sensory preconditioning detected with a sodium depletion procedure

In each of two experiments, two groups of rats were exposed to three flavoured solutions: A (citric acid), B (salt), and AX (a compound of citric acid and saccharin). Both experiments used a between-subjects design in which a paired group received presentations of A followed by B, alternating with presentations of AX (i.e., A → B/AX), and an unpaired group received alternating presentations of A, B, and AX (i.e., A/B/AX). This arrangement was expected to establish X as an inhibitor of B in group paired but not in group unpaired. In Experiment 1, after preexposure all subjects received a single presentation of an XB compound, then experienced sodium depletion, and were tested for their consumption of X, which was greater in group unpaired than in group paired. In Experiment 2, after preexposure, all subjects received four presentations of a new flavour, C, in compound with B and subsequently, under sodium depletion, were tested for consumption of XC. Intake of the XC compound was less in group paired than in group unpaired. These results suggest that, in group paired, X acquired an inhibitory relationship with B both retarding the acquisition of an excitatory association with B (retardation test, Experiment 1) and reducing the response to a new stimulus, C, strongly associated to B (summation test, Experiment 2). These results provide direct evidence of inhibition between two neutral stimuli and, therefore, of inhibitory sensory preconditioning.     

Representation-mediated extinction of conditioned flavor aversions

Conditioned flavor aversions were extinguished by presenting without consequence auditory stimuli that had been previously paired with the aversive flavor. In Experiment 1, rats that received tone-sucrose pairings, then sucrose-lithium chloride (LiCl) pairings, and finally repeated tone-alone presentations showed greater sucrose consumption in subsequent testing than rats that received similar sucrose-LiCl pairings and tone-alone presentations but no initial tone-sucrose pairings. Experiment 2 demonstrated the stimulus specificity of the mediated extinction observed in Experiment 1. In Experiment 3, rats that received first-order light-food and second-order tone-light pairings prior to sucrose-LiCl pairings did not show greater subsequent sucrose consumption when extinction of the second-order tone intervened. These results suggest that conditioned stimulus (CS)-evoked representations of events can substitute for those events themselves in the extinction of previously established associations.     

Rapid, Labile, and Protein Synthesis– Independent Short-Term Memory in Conditioned Taste Aversion

Short-term memory is a rapid, labile, and protein-synthesis-independent phase of memory. The existence of short-term memory in conditioned taste aversion (CTA) learning has not been demonstrated formally. To determine the earliest time at which a CTA is expressed, we measured intraoral intake of sucrose at 15 min, 1 hr, 6 hr, or 48 h after contingent pairing of an intraoral infusion of 5% sucrose (6.6 ml over 6 min) and toxic lithium chloride injection (76 mg/kg). Rats were implanted with intraoral catheters to allow presentation of taste solutions at arbitrary times. Intraoral intake was measured under conditions of long-delay, single-trial learning typical of CTA. Rats decreased intraoral intake of sucrose at 15 min after contingent pairing of sucrose and LiCl, but not after noncontingent LiCl or sucrose. Thus CTA learning can be expressed rapidly. To determine if short-term CTA memory is labile and decays in the absence of long-term memory, we measured intraoral intake of sucrose after pairing sucrose with low doses of LiCl. Rats received an intraoral infusion of 5% sucrose (6 ml/6 min); 30 min later LiCl was injected at three different doses (19, 38, or 76 mg/kg). A second intraoral infusion of sucrose was administered 15 min, 1 hr, 3 hr, 4.5 hr, 6 hr, or 48 hr later. The formation of long-term CTA memory was dependent on the dose of LiCl paired with sucrose during acquisition. Low doses of LiCl induced a CTA that decayed within 6 hr after pairing. Central administration of the protein synthesis inhibitor cycloheximide prior to LiCl injection blocked long-term CTA expression at 6 and 48 hr, but not short-term CTA expression at 1 hr. Thus, short-term memory for CTA learning exists that is acquired rapidly and independent of protein synthesis, but labile in the absence of long-term memory formation.     

Phosphorylation of mitogen-activated protein kinase in the medial prefrontal cortex and the amygdala following memory retrieval or forgetting in developing rats

We examined neuronal correlates of forgetting in rats by detection of phosphorylated mitogen-activated protein kinase (pMAPK) in the medial prefrontal cortex (mPFC) and amygdala. In Experiment 1, postnatal day (P)23 and P16 rats received paired noise CS-shock US presentations. When tested immediately after conditioning, P23 and P16 rats exhibited similar levels of conditioned fear; when tested after 2 days, however, P16 rats showed poor CS-elicited freezing relative to P23 rats. In Experiment 2, P16 and P23 rats received either paired or unpaired CS-US presentations, and then were tested 48 h later. Consistent with Experiment 1, P16 rats showed forgetting whereas P23 rats exhibited good retention at test. Additionally, unpaired groups showed poor CS-elicited freezing at test. Immunohistochemistry showed that P23 and P16 rats given paired presentations exhibited significant elevation of pMAPK-immunoreactive (ir) neurons in the amygdala compared to rats given unpaired presentations. That is, MAPK phosphorylation in the amygdala tracked learning history rather than behavioral performance at test. In contrast, only the P23-paired group showed an elevated number of pMAPK-ir neurons in mPFC, indicating that MAPK phosphorylation in the mPFC tracks memory expression. Different test-perfusion intervals were employed in Experiment 3, which showed that the developmental dissociation in the pMAPK-ir neurons observed in the mPFC in Experiment 2 was not due to age differences in the rate of phosphorylation of MAPK. These findings provide initial evidence suggesting that while the mPFC is involved in memory retrieval, MAPK phosphorylation in the amygdala may be a persisting neural signature of fear memory.     

Predictive learning in nutrient-based flavor conditioning

This paper presents two experiments on nutrient-based flavor conditioning with rats as subjects and sucrose as the unconditioned stimulus (US). Experiment 1 was aimed at establishing an optimal control for conditioning, comparing simultaneous and serial presentations of a flavor and the US. The results showed that simultaneous, but not serial training, produced conditioning. Experiment 2 was designed to obtain evidence of summation as an index of both conditioned inhibition and predictive learning. Group Simultaneous received Pavlovian conditioned inhibition training during which flavor A was simultaneously paired with sucrose on excitatory trials (A+), and forming an unreinforced compound with flavor B on inhibitory trials (AB-). An independent excitor for the summation test was also trained by simultaneous pairings with sucrose (C+). In the control group (Blocked), the AB- trials were presented forming a block at the beginning of training to avoid a negative contingency-relationship with sucrose, and flavor A received serial rather than simultaneous pairing with sucrose (A --> +). On the summation test, only in group Simultaneous was consumption of the CB compound lower than that of flavor C alone, suggesting that, during training, flavor A activated an expectancy of the US occurrence.     

Control of appetitive and aversive taste-reactivity responses by an auditory conditioned stimulus in a devaluation task: a FOS and behavioral analysis

Through associative learning, cues for biologically significant reinforcers such as food may gain access to mental representations of those reinforcers. Here, we used devaluation procedures, behavioral assessment of hedonic taste-reactivity responses, and measurement of immediate-early gene (IEG) expression to show that a cue for food engages behavior and brain activity related to sensory and hedonic processing of that food. Rats first received a tone paired with intraoral infusion of sucrose. Then, in the absence of the tone, the value of sucrose was reduced (Devalue group) by pairing sucrose with lithium chloride (LiCl), or maintained (Maintain group) by presenting sucrose and LiCl unpaired. Finally, taste-reactivity responses to the tone were assessed in the absence of sucrose. Devalue rats showed high levels of aversive responses and minimal appetitive responses, whereas Maintain rats exhibited substantial appetitive responding but little aversive responding. Control rats that had not received tone-sucrose pairings did not display either class of behaviors. Devalue rats showed greater FOS expression than Maintain rats in several brain regions implicated in devaluation task performance and the display of aversive responses, including the basolateral amygdala, orbitofrontal cortex, gustatory cortex (GC), and the posterior accumbens shell (ACBs), whereas the opposite pattern was found in the anterior ACBs. Both Devalue and Maintain rats showed greater FOS expression than control rats in amygdala central nucleus, GC, and both subregions of ACBs. Thus, through associative learning, auditory cues for food gained access to neural processing in several brain regions importantly involved in the processing of taste memory information.     

Apparent Incentive Contrast Effects in Autoshaping with Rats

The effects of shifts in reward quality and quantity on Pavlovian acquisition were studied in rats. In Experiment 1, animals preexposed to unsignaled food pellets, 10% sucrose solution, or home cage controls subsequently received autoshaping training (response-independent lever-pellet or lever-solution pairings, in three groups each). Unsignaled preexposure to sucrose solution facilitated autoshaping for pellets (relative to unshifted controls), whereas unsignaled preexposure to pellets retarded autoshaping for sucrose solution. In Experiment 2, unsignaled preexposure to 30% sucrose solution impaired acquisition reinforced by food pellets, relative to 2% solution. Using a choice procedure, Experiment 3 demonstrated that rats prefer pellets to either 2 or 10% sucrose solutions, but they prefer the 30% solution to the pellets. Experiment 4 demonstrated the facilitatory effect after an upward shift in reward magnitude rather than quality (from 1 to 12 pellets), but provided weaker evidence for retardation following a downward magnitude shift. Experiment 5 was similar to Experiment 4, except that animals received autoshaping training from the outset. No evidence of successive positive contrast was obtained, but there was a significant successive negative contrast effect. Moreover, extinction was faster after acquisition with 12 pellets rather than 1. These results suggest the presence of incentive contrast effects under Pavlovian training conditions.     

Counterconditioning of appetitive and defensive CRs in rabbits

Two experiments examined interactions between conditioned appetitive and defensive responses in the rabbit. In Experiment I, a conditioned jaw-movement response was established by following presentations of a clicker CS with intra-oral sucrose delivery on 50% of trials. The jaw-movement response was then maintained on this partial reinforcement schedule during a counterconditioning phase. A group which received para-orbital shock paired with the CS on non-sucrose trials showed acquisition of eyeblink responding and suppression of jaw-movement responding to the CS, in comparison to control groups which received either no shock or unpaired presentations of the CS and shock. Experiment II was identical in design to Experiment I except that the roles of the sucrose and shock reinforcers were reversed. The paired group acquired a conditioned jaw-movement response when sucrose was added in the counterconditioning phase, but in contrast to Experiment I showed a slight enhancement of the previously established eyeblink response. The asymmetry of appetitive and defensive counterconditioning was discussed in relation to opponent-process theories of motivation and reinforcement.     

Memory enhancement of classical fear conditioning by post-training injections of corticosterone in rats

There is extensive evidence that post-training administration of the adrenocortical hormone corticosterone facilitates memory consolidation processes in a variety of contextual and spatial-dependent learning situations. The present experiments examine whether corticosterone can modulate memory of auditory-cue classical fear conditioning, a learning task that is not contingent on contextual or spatial representations. Male Sprague-Dawley rats received three pairings of a single-frequency auditory stimulus and footshock, followed immediately by a post-training subcutaneous injection of either corticosterone (1.0 or 3.0mg/kg) or vehicle. Retention was tested 24h later in a novel test chamber and suppression of ongoing motor behavior served as the measure of conditioned fear. Corticosterone dose-dependently facilitated suppression of motor activity during the 10-s presentation of the auditory cue. As corticosterone administration did not alter responding after unpaired presentations of tone and shock, tone alone, shock alone or absence of tone/shock, the findings indicated that corticosterone selectively facilitated memory of the tone-shock association. Furthermore, injections of corticosterone given 3h after training did not alter motor activity during retention testing, demonstrating that corticosterone enhanced time-dependent memory consolidation processes. These findings provide evidence that corticosterone modulates the consolidation of memory for auditory-cue classical fear conditioning and are consistent with a wealth of data indicating that glucocorticoids can modulate a wide variety of emotionally influenced memories.     

Posttraining handling facilitates memory for auditory-cue fear conditioning in rats

A large number of studies have indicated that stress exposure or the administration of stress hormones and other neuroactive drugs immediately after a learning experience modulates the consolidation of long-term memory. However, there has been little investigation into how arousal induced by handling of the animals in order to administer these drugs affects memory. Therefore, the present study examined whether the posttraining injection or handling procedure per se affects memory of auditory-cue classical fear conditioning. Male Sprague-Dawley rats, which had been pre-handled on three days for 1 min each prior to conditioning, received three pairings of a single-frequency auditory stimulus and footshock, followed immediately by either a subcutaneous injection of a vehicle solution or brief handling without injection. A control group was placed back into their home cages without receiving any posttraining treatment. Retention was tested 24 h later in a novel chamber and suppression of ongoing motor behavior during a 10-s presentation of the auditory-cue served as the measure of conditioned fear. Animals that received posttraining injection or handling did not differ from each other but showed significantly less stimulus-induced movement compared to the non-handled control group. These findings thus indicate that the posttraining injection or handling procedure is sufficiently arousing or stressful to facilitate memory consolidation of auditory-cue classical fear conditioning.     

Evaluative conditioning can be modulated by memory of the CS–US pairings at the time of testing

Evaluative conditioning (EC) is the valence change of a (typically neutral) stimulus (CS) that is due to the previous pairing with another (typically valent) stimulus (US). It has been repeatedly shown that EC effects are stronger or existent only if participants know which US was paired with which CS. Knowledge of the CS–US pairings is usually measured temporally close to both the conditioning phase and the CS valence measurement phase. Hence, the relation between EC and knowledge about the pairings could indicate either that participants need to become aware of the pairings at some point or that they need to remember them during the CS valence test. We isolated the impact of memory during the CS valence test in a study that encompassed two sessions. During the first session, participants were presented with CS–US pairings. The valence of the CSs was measured in a second session several days later using both a rating scale and an affective priming procedure. Memory for the pairings was measured both during the first and the second session. Using item-based multilevel analysis, we found that EC in the second session was related to memory for the pairings during the second session, but not to the memory for the pairings measured immediately after the learning phase. For the pairs that were remembered during the first session, but not during the second session, no EC effect was found. These results suggest that memory for CS–US pairings during valence measurement can be relevant for EC effects to occur.     

Is evaluative conditioning really resistant to extinction? Evidence for changes in evaluative judgements without changes in evaluative representations

Evaluative conditioning (EC) is defined as the change in the evaluation of a conditioned stimulus (CS) due to its pairing with a positive or negative unconditioned stimulus (US). Although several individual studies suggest that EC is unaffected by unreinforced presentations of the CS without the US, a recent meta-analysis indicates that EC effects are less pronounced for post-extinction measurements than post-acquisition measurements. The disparity in research findings suggests that extinction of EC may depend on yet unidentified conditions. In an attempt to uncover these conditions, three experiments (N = 784) investigated the influence of unreinforced post-acquisition CS presentations on EC effects resulting from simultaneous versus sequential pairings and pairings with single versus multiple USs. For all four types of CS–US pairings, EC effects on self-reported evaluations were reduced by unreinforced CS presentations, but only when the CSs had been rated after the initial presentation of CS–US pairings. EC effects on an evaluative priming measure remained unaffected by unreinforced CS presentations regardless of whether the CSs had been rated after acquisition. The results suggest that reduced EC effects resulting from unreinforced CS presentations are due to judgement-related processes during the verbal expression of CS evaluations rather than genuine changes in the underlying evaluative representations.     

Contextual modulation of memory consolidation

We investigate olfactory memory consolidation in honeybees. Three experiments are reported that include 1024 animals in 28 experimental groups. After one pairing of odorant and sucrose reward, retention is typically nonmonotonic with a minimum 3 min after conditioning. This corresponds to the “Kamin effect” in vertebrates; the postminimum rise in retention is usually interpreted as reflecting memory consolidation. First, we test for the generality of this effect across four different odorants. The postminimum rise in retention was reproducibly observed for 1-hexanol but not for 1-octanol, limonene, or geraniol. Second, we investigate whether previous learning about the training context modulates subsequent memory consolidation. On the day before training, a reward was applied either upon placement into the future training context for 1 min, halfway during exposure or just before removal from the context. In the latter group, the 3-min minimum in retention was eliminated; thus, in that group, forward pairings of context and reward (i.e., context exposure begins before reward is applied) lead to an associative context memory that can modulate subsequent olfactory memory consolidation. Third, we found no evidence for a modulation of olfactory memory consolidation by pre-exposure to the odorant.     

Background illumination effects upon in vitro conditioning in Hermissenda

In the marine snail Hermissenda, associative learning can be accomplished by paired presentations of light and vestibular stimulation. It is generally assumed that associative learning depends upon the intensity or salience of the conditioned or unconditioned stimulus (CS and US, respectively). Accordingly, during Hermissenda conditioning a stronger dark adaptation is expected to render the CS (the light) more salient and hence facilitate association. We studied the influence of background illumination level using an in vitro pairing procedure in Hermissenda. This procedure allows one to assess the effect of conditioning upon a single cell, the B photoreceptor, which is implicated in this learning process. After 15 min of adaptation to a dim background light, B photoreceptors maintained a basal rate of firing, while after adaptation to complete darkness, they stopped firing. Paired and unpaired groups received 10 training trials in either a completely dark or a dim light environment. Although a trial to trial cumulative increase in excitability was found in the paired group trained in darkness, only the paired group trained under dim background light showed a higher input resistance and cell excitability 10 min after training. These results suggest that the background dim illumination was not needed for the induction but played a role in the maintenance of the pairing effect. Possible mechanisms for such a modulatory effect are discussed.     

Weanlings' transfer of conditioned ethanol aversion from olfaction to ingestion depends on the unconditioned stimulus

Weanling (21-day-old) rats were exposed to an alcohol odor paired with either an interoceptive (apomorphine-induced illness) or exteroceptive (footshock-induced distress) reinforcer. Twenty-four hours later, ethanol preferences were measured in a locational olfactory test (ethanol vs lemon odor) or an ingestion test (5.6% v/v ethanol vs 0.25% v/v citric acid solution). Weanling rats expressed substantial olfactory aversions, independent of the reinforcer employed in conditioning. During the drinking test, however, only rats that had experienced the ethanol odor paired with internal malaise showed a significant reduction in the intake of the ethanol solution when compared to unpaired controls. Furthermore, rats that had experienced the ethanol odor paired with external distress drank significantly more of the ethanol solution than their controls. These results provide further evidence that olfactory experiences with ethanol can lead to changes in ethanol ingestion, and indicate that the nature of the unconditioned stimulus is critical in establishing the ingestive effect.     

Pavlovian counterconditioning: changing the suppressive properties of shock by association with food.

Two experiments are described in which rats received a series of shock and food presentations in a Pavlovian counterconditioning sequence. Subsequently, the capacity of the shock to act as a reinforcer during conditioned emotional response (CER) training was assessed. In the first experiment, following each shock during counterconditioning by a food presentation was found to retard the development of suppression during CER training relative to control conditions in which different groups of rats received either a sequence of explicitly unpaired shock and food presentations, shock presentations alone, or food presentations alone. The second experiment demonstrated that when the magnitude of the food presentations is held constant, the attenuation of the suppressive properties of shock by pairing with food depends on the magnitude of the shock. The results are interpreted as demonstrations of Pavlovian counterconditioning, which, it is suggested, modifies the general aversiveness of a noxious stimulus.     

Effects of expectancies of different reward magnitudes in transfer from noncontingent pairings to instrumental performance

In two experiments, rats received noncontingent pairings of two stimuli with food reward, one paired with small reward and the other with large reward, and received bar press training with large reward or with small reward. When the noncontingent stimuli (NS) were presented for test during subsequent rewarded bar pressing and during early extinction of bar pressing, responding for each group was faster in the presence of the NS which was paired with the same reward magnitude that group received in bar press training than to the NS which had been paired with a different reward magnitude. As extinction progressed, all groups responded more slowly in the presence of the NS which had been paired with the large reward than in the presence of the NS which had been paired with small reward. These results were interpreted as indicating that responding in the presence of an NS depends on: (i) whether the reward expectancy elicited by the NS has been conditioned to the instrumental response, and (ii) the relationship between the reward expected in the presence of the NS and that received in test.