One retrieval trial induces reconsolidation in an appetitive learning paradigm in honeybees (Apis mellifera)

Combining memory retrieval with the application of a protein synthesis-inhibitor leads to an amnestic effect that is referred to as the reconsolidation phenomenon. Several behavioural studies demonstrate that only a few or weak retrieval trials (that do not result in significant extinction) lead to this phenomenon. In contrast, many trials (that result in significant extinction) combined with a protein synthesis inhibitor result in an inhibition of the extinction memory. Based on these findings it was suggested that extinction is the boundary condition for reconsolidation: when extinction is induced the consolidation of the extinction memory is the dominant process. Recently we were not able to confirm this hypothesis in the honeybee (Apis mellifera): We did not find the reconsolidation phenomenon after one retrieval trial, but demonstrated reconsolidation after five retrieval trials that led to extinction. To exclude that this observation resembles a special case in insects we here wanted to know if one retrieval trial induces reconsolidation as it has been demonstrated before in many other species. To do so we used experimental parameters that had been used before to demonstrate consolidation in the honeybee with the exception that this time the protein synthesis-inhibitor was applied 1 h after one memory retrieval instead after acquisition. We thereby demonstrate the reconsolidation phenomenon after one retrieval trial but only when using the doubled dose of protein synthesis-inhibitor that has been used to inhibit consolidation.     

Aversive and appetitive electrodermal classical conditioning using pictures as stimuli

The principal goal of this study was to verify whether it was possible to obtain both aversive and appetitive electrodermal classical conditioning, using pictures as conditioned stimuli (CS), and unconditioned stimuli (US). Additionally, we tried to verify whether, as a consequence of such conditioning, diminution of the unconditioned response (UR) was observed. With this aim, IAPS (?International Affective Picture System?) pictures were selected as stimuli. A picture showing a burnt face was used as the aversive US (USav), and a picture showing a scene with erotic content was used as the appetitive US (USap). As the aversive CS (CSav), and appetitive CS (CSap), two images with intermediate values of valence and arousal showing male faces were selected. In the experimental group, 10 CSav/USav and 10 CSap/USap trials were presented. In the control group 10 CSav, CSap, USav, and USap trials were presented in pseudorandom order. Skin conductance response (SCR) elicited by both the CSs and the USs was scored. Results showed aversive conditioning, but neither appetitive conditioning nor UR diminution. Problems to obtain conditioning using pictures as stimuli and possible options to overcome them in future research are discussed.     

Emotional eating and Pavlovian learning: evidence for conditioned appetitive responding to negative emotional states

Appetitive learning has been demonstrated several times using neutral cues or contexts as a predictor of food intake and it has been shown that humans easily learn cued desires for foods. It has, however, never been studied whether internal cues are also capable of appetitive conditioning. In this study, we tested whether humans can learn cued eating desires to negative moods as conditioned stimuli (CS), thereby offering a potential explanation of emotional eating (EE). Female participants were randomly presented with 10 different stimuli eliciting either negative or neutral emotional states, with one of these states paired with eating chocolate. Expectancy to eat, desire to eat, salivation, and unpleasantness of experiencing negative emotions were assessed. After conditioning, participants were brought into a negative emotional state and were asked to choose between money and chocolate. Data showed differential conditioned responding on the expectancy and desire measures, but not on salivation. Specific conditioned effects were obtained for participants with a higher BMI (body mass index) on the choice task, and for participants high on EE on the unpleasantness ratings. These findings provide the first experimental evidence for the idea that negative emotions can act as conditioned stimuli, and might suggest that classical conditioning is involved in EE.     

Successive olfactory reversal learning in honeybees

Honeybees Apis mellifera can associate an originally neutral odor with a reinforcement of sucrose solution. Forward pairings of odor and reinforcement enable the odor to release the proboscis extension reflex in consecutive tests. Bees can also be conditioned differentially: They learn to respond to a reinforced odor and not to a nonreinforced one. They can also learn to reverse their choice. Here we ask whether honeybees can learn successive olfactory differential conditioning tasks involving different overlapping pairs of odors. The conditioning schedules were established in order to train the animals with 3, 2, 1, or 0 reversals previous to a last differential conditioning phase in which two additional reversals were present. We studied whether or not successive reversal learning is possible and whether or not learning olfactory discrimination reversals affects the solving of subsequent discrimination reversals. Therefore we compared the responses of bees that had experienced reversals with those of bees that had not experienced such reversals when both are confronted with a new reversal situation. In experiment 1 we showed that bees that had experienced three previous reversals were better in solving the final reversal task than bees with no previous reversal experience. In experiment 2, we showed that one reversal learning is enough for bees to perform better in the final reversal task. The successive different reversals trained in our experiments resemble the natural foraging situation in which a honeybee forager has to switch successively from an initial floral species to different ones. The fact that experiencing such changes seems to improve a bee's performance in dealing with further new exploited food sources has therefore an adaptive impact for the individual and for the colony as a whole.     

Enhancement of food-rewarded instrumental responding by an appetitive conditioned stimulus

Four experiments are reported in which a stimulus (with a minimum duration of 60 s) signalling the delivery of “free” food was presented to rats lever-pressing for food available on a variable interval schedule. It was found that responding was enhanced in the presence of the stimulus when the baseline schedule of reinforcement was lean (Experiment I) and that the enhancement was dependent upon the pairing of the stimulus with free food (Experiments II and III). Experiment IV showed that an enhancement could be found after initial training in which stimulus-food pairings were given to subjects that were not concurrently lever pressing for food. It is argued that these results are consistent with the suggestion that an appetitive conditioned stimulus can energise appetitive instrumental behaviour.     

Intertrial pellets influence the acquisition and expression of timed appetitive responding in rats

Two experiments examined temporally based changes in the conditioned magazine-entries of rats when a target food pellet arrived at a fixed time before the termination of a conditioned stimulus. Both experiments found that increasing the rate of intertrial pellets systematically interfered with the rate of acquisition. When intertrial pellets were featured in acquisition, their omission on test trials also resulted in a very poorly expressed conditioned response. On test trials preceded by intertrial pellets, however, peak times in groups trained with intertrial pellets were shifted in a rightward direction under a long but not short interstimulus interval. This rightward shift was accompanied by a smaller standard deviation. Our key findings are best accommodated by real-time associative models.     

The role of nonreinforcement in the learning of honeybees

Two series of experiments with honeybees were designed to test the assumption that inhibition is generated by nonreinforcement as a function of the excitatory value of the context. In the first series (Experiments 1-3), summation tests with B were made after A+/C-/AB- as compared to A+/C-/CB- training, with precautions taken to minimize the possibility of a masking effect of excitatory within-compound conditioning on AB trials; responding to B did not vary with training procedure. In the second series (Experiments 4-5), retardation tests rather than summation tests were used, in the belief that they might be more sensitive; after A+/AB-/CD- training, acquisition in a B+/D- problem was found to be no less rapid than in a D+/B- problem. A third series of experiments (Experiments 6-9) was designed to test the more general assumption that the effectiveness of nonreinforcement increases with the excitatory value of the context; response to B was found to be no different after A+/B+/C- training followed by A+/AB- training than after A+/B+/C- training followed by A+/CB- training. The results are compatible with the view that the role of nonreinforcement in honeybees is not to generate inhibition, but only to reduce excitation in a manner independent of the excitatory value of the context.     

Effects of acetoxycycloheximide on appetitive learning and memory

The effects of direct brain infusions of acetoxycycloheximide (an inhibitor of protein synthesis; ACXH) on acquisition, storage and recall of memory for one-trial appetitive learning were examined in five experiments. ACXH was infused into the rats' hippocampi through implanted cannulas. Control subjects received an equal volume of physiological saline. ACXH was infused (a) 5 h before acquisition, (b) 5 hr before commencement of recall tests, and (c) immediately after acquisition. Each subject's general motor activity was recording during testing. The results indicate that (1) ACXH has similar effects on appetitive and avoidance learning. (2) ACXH administered immediately after acquisition, has no effect on memory. (3) At 4 hr after acquisition memory is affected by ACXH. (4) Short-term memory is unaffected by ACXH and can exist independently of long-term memory. (5) ACXH consistently reduces general motor activity.     

Maze navigation by honeybees: learning path regularity

We investigated the ability of honeybees to learn mazes of four types: constant-turn mazes, in which the appropriate turn is always in the same direction in each decision chamber; zig-zag mazes, in which the appropriate turn is alternately left and right in successive decision chambers; irregular mazes, in which there is no readily apparent pattern to the turns; and variable irregular mazes, in which the bees were trained to learn several irregular mazes simultaneously. The bees were able to learn to navigate all four types of maze. Performance was best in the constant-turn mazes, somewhat poorer in the zig-zag mazes, poorer still in the irregular mazes, and poorest in the variable irregular mazes. These results demonstrate that bees do not navigate such mazes simply by memorizing the entire sequence of appropriate turns. Rather, performance in the various configurations depends on the existence of regularity in the structure of the maze and on the ease with which this regularity is recognized and learned.     

Color modulates olfactory learning in honeybees by an occasion-setting mechanism

A sophisticated form of nonelemental learning is provided by occasion setting. In this paradigm, animals learn to disambiguate an uncertain conditioned stimulus using alternative stimuli that do not enter into direct association with the unconditioned stimulus. For instance, animals may learn to discriminate odor rewarded from odor nonrewarded trials if these two situations are indicated by different colors that do not themselves become associated with the reward. Despite a growing interest in nonelemental learning in insects, no study has so far attempted to study occasion setting in restrained honeybees, although this would allow direct access to the neural basis of nonelemental learning. Here we asked whether colors can modulate olfactory conditioning of the proboscis extension reflex (PER) via an occasion-setting mechanism. We show that intact, harnessed bees are not capable of learning a direct association between color and sucrose. Despite this incapacity, bees solved an occasion-setting discrimination in which colors set the occasion for appropriate responding to an odor that was rewarded or nonrewarded depending on the color. We therefore provide the first controlled demonstration of bimodal (color-odor) occasion setting in harnessed honeybees, which opens the door for studying the neural basis of such bimodal, nonelemental discriminations in insects.     

Differential effects of omission contingencies on various components of Pavlovian appetitive conditioned responding in rats

Five experiments with rat subjects investigated the effects of omission and partial reinforcement contingencies on five individual behaviors evoked by visual and auditory conditioned stimuli paired with a food unconditioned stimulus. The effects of omission depended on the behavior on which that contingency was placed: One behavior was eliminated, one was unaffected, and three were reduced relative to the performance of yoked controls. Partial reinforcement resulted in lower frequencies of three behaviors and higher frequencies of two behaviors, compared with performance under consistent reinforcement. A partial reinforcement extinction effect was noted with one behavior but not with the others. These results are related to the possible role of instrumental conditioning contingencies in generating conditioned behavior in this appetitive conditioning preparation and to the independence of individual components of a complex conditioned response.     

A molecular dissociation between cued and contextual appetitive learning

In appetitive Pavlovian learning, animals learn to associate discrete cues or environmental contexts with rewarding outcomes, and these cues and/or contexts can potentiate an ongoing instrumental response for reward. Although anatomical substrates underlying cued and contextual learning have been proposed, it remains unknown whether specific molecular signaling pathways within the striatum underlie one form of learning or the other. Here, we show that while the striatum-enriched isoform of adenylyl cyclase (AC5) is required for cued appetitive Pavlovian learning, it is not required for contextual appetitive learning. Mice lacking AC5 (AC5KO) could not learn an appetitive Pavlovian learning task in which a discrete signal light predicted reward delivery, yet they could form associations between context and either natural or drug reward, which could in turn elicit Pavlovian approach behavior. However, unlike wild-type (WT) mice, AC5KO mice could not use these Pavlovian conditioned stimuli to potentiate ongoing instrumental behavior in a Pavlovian-to-instrumental transfer paradigm. These data suggest that AC5 is specifically required for learning associations between discrete cues and outcomes in which the temporal relationship between conditioned stimulus (CS) and unconditioned stimulus (US) is essential, while alternative signaling mechanisms may underlie the formation of associations between context and reward. In addition, loss of AC5 compromises the ability of both contextual and discrete cues to modulate instrumental behavior.In Pavlovian learning, animals form associations between discrete or contextual stimuli in their environment to shape their behavior and make appropriate responses. In discrete cue appetitive Pavlovian conditioning, a single cue with a defined onset and offset that typically activates one sensory modality is provided, immediately followed by reward delivery (Hall 2002; Domjan 2006; Ito et al. 2006). Alternatively, behavior can be driven by context, an assortment of stimuli activating a number of sensory modalities that contribute to the representation of environmental space (Balsam 1985; Rudy and Sutherland 1995; Smith and Mizumori 2006). Collectively, these stimuli make up a context that is paired with reward delivery in contextual appetitive learning. One important distinction between these two forms of learning is that in cued conditioning, there is a discrete temporal relationship between conditioned stimulus (CS) and unconditioned stimulus (US). Thus, an animal can effectively anticipate timing of reward delivery from onset and offset of CS. In vivo studies of dopamine (DA) neuron activity have suggested this discrete temporal relationship can be encoded by DA neurons (Schultz et al. 1997; Schultz 1998a). In contrast, in many contextual Pavlovian conditioning tasks, US delivery is not predicted, it is delivered as the animal explores the environment; thus, the temporal relationship between contextual stimuli and reinforcement is not an essential component of the learned associations (Fanselow 2000). These two types of environmental stimuli may be encoded differently and mediated by different neural substrates.Lesion studies have elucidated the anatomical dissociations between cued and contextual appetitive learning. Using a modified Y-maze procedure, it has been suggested that contextual appetitive learning is hippocampus- and nucleus-accumbens (NAc) dependent, while cued learning is dependent on the basolateral nucleus of the amygdala (BLA) and the NAc (Ito et al. 2005, 2006). In addition, as the NAc processes glutamatergic inputs from the amygdala and the hippocampus (Groenewegen et al. 1999; Goto and Grace 2008), recent studies have indicated that disconnecting the hippocampus from the NAc shell can disrupt contextual appetitive conditioning (Ito et al. 2008). In addition to glutamatergic inputs, the NAc, as part of the ventral striatum, receives dense dopaminergic input from midbrain nuclei (Groenewegen et al. 1999). Temporal shifts in phasic DA release in striatal regions has been correlated with appetitive Pavlovian learning (Day et al. 2007), and models of striatal function suggest that DA-dependent modification of glutamatergic transmission in the striatum may underlie reinforcement learning (Reynolds et al. 2001; Reynolds and Wickens 2002).The cAMP pathway has been implicated in plasticity and learning in a number of neuronal structures (Abel et al. 1997; Ferguson and Storm 2004; Pittenger et al. 2006). Adenylyl cyclase (AC), the enzyme that makes cAMP, has nine membrane-bound isoforms, each with different expression patterns and regulatory properties (Hanoune and Defer 2001). AC5 is highly enriched in the striatum, with very low levels of expression in other regions of the brain (Mons et al. 1998; Iwamoto et al. 2003; Kheirbek et al. 2008, 2009), and genetic deletion of AC5 (AC5KO) severely compromises DA''s ability to modulate cAMP levels in the striatum (Iwamoto et al. 2003). Previous studies have shown that AC5KO mice were severely impaired in acquisition of a cued appetitive Pavlovian learning task, while formation of action–outcome contingencies in instrumental learning was intact (Kheirbek et al. 2008). Yet, it remains unknown whether the cAMP pathway in the striatum underlies all forms of appetitive Pavlovian learning, or how it contributes to the ability of Pavlovian cues to modulate instrumental behavior.In this study, we asked if genetic deletion of AC5 selectively impairs cued or contextual appetitive learning. In addition, we tested whether loss of AC5 affects the ability of conditioned cues or contexts to modulate instrumental behavior. Our data indicate that although loss of AC5 abolishes cued appetitive learning, contextual learning is spared. Although contextual stimuli could elicit approach behavior in AC5KO mice, they could not potentiate an ongoing instrumental response, highlighting the importance of this isoform of AC in Pavlovian–instrumental interactions.     

A test of sex differences in instrumental appetitive learning by rats

A total of 48 laboratory rats, half male and half female, were trained and extinguished on instrumental bar-pressing for either 16% or 64% sucrose incentives. No consistent sex differences were found in either acquisition or variability of bar-press responding, or in extinction. The only statistically reliable difference indicated that males were more variable in responding for the 16% sucrose incentive, as measured by the range of scores over each week. It is concluded that no real sex differences in learning proficiency obtain under the experimental conditions employed, and that therefore the practice of using mainly or exclusively female rats in our laboratory is methodologically sound.     

Differential involvement of the central amygdala in appetitive versus aversive learning

Understanding the function of the distinct amygdaloid nuclei in learning comprises a major challenge. In the two studies described herein, we used c-Fos immunolabeling to compare the engagement of various nuclei of the amygdala in appetitive and aversive instrumental training procedures. In the first experiment, rats that had already acquired a bar-pressing response to a partial food reinforcement were further trained to learn that an acoustic stimulus signaled either continuous food reinforcement (appetitive training) or a footshock (aversive training). The first training session of the presentation of the acoustic stimulus resulted in significant increases of c-Fos immunolabeling throughout the amygdala; however, the pattern of activation of the nuclei of the amygdala differed according to the valence of motivation. The medial part of the central amygdala (CE) responded, surprisingly, to the appetitive conditioning selectively. The second experiment was designed to extend the aversive versus appetitive conditioning to mice, trained either for place preference or place avoidance in an automated learning system (INTELLICAGE). Again, much more intense c-Fos expression was observed in the medial part of the CE after the appetitive training as compared to the aversive training. These data, obtained in two species and by means of novel experimental approaches balancing appetitive versus aversive conditioning, support the hypothesis that the central nucleus of the amygdala is particularly involved in appetitively motivated learning processes.     

Configural olfactory learning in honeybees: negative and positive patterning discrimination

In an appetitive context, honeybees (Apis mellifera) learn to associate odors with a reward of sucrose solution. If an odor is presented immediately before the sucrose, an elemental association is formed that enables the odor to release the proboscis extension response (PER). Olfactory conditioning of PER was used to study whether, beyond elemental associations, honeybees are able to process configural associations. Bees were trained in a positive and anegative patterning discrimination problem. In the first problem, single odorants were nonreinforced whereas the compound was reinforced. In the second problem, single odorants were reinforced whereas the compound was nonreinforced. We studied whether bees can solve these problems and whether the ratio between the number of presentations of the reinforced stimuli and the number of presentations of the nonreinforced stimuli affects discrimination. Honeybees differentiated reinforced and nonreinforced stimuli in positive and negative patterning discriminations. They thus can process configural associations. The variation of the ratio of reinforced to nonreinforced stimuli modulated the amount of differentiation. The assignment of singular codes to complex odor blends could be implemented at the neural level: When bees are stimulated with odor mixtures, the activation patterns evoked at the primary olfactory neuropile, the antennal lobe, may be combinations of the single odorant responses that are not necessarily fully additive.     

Asymmetrical effects of Pavlovian excitatory and inhibitory aversive transfer on Pavlovian appetitive responding and acquisition

Predictions of a theory of Pavlovian motivational transfer, which incorporates principles of both the theory of reciprocal inhibition and the Rescorla-Wagner model, were tested in several Pavlovian aversive to Pavlovian appetitive transfer tasks. As predicted, the presence of a signal for an aversive event, conditioned stimulus (AV CS+), reliably suppressed performance of appetitive conditioned responses (CRs) whether imposed during acquisition or on independently established responding. Acquisition of appetitive responding to a novel CS reinforced in compound with an AV CS+, however, was enhanced (“superconditioning”). This observation suggests that the effects of a discrepancy between expectation and actual outcome on a conditioning trial are influenced by the affective value of both the expectation and the reinforcer. These transfer effects were not symmetrical for an inhibitory aversive stimulus (AV CS?). An AV CS? did not enhance appetitive responding compared to a random control condition, nor did the AV CS? reduce (i.e., block) appetitive conditioning to a novel CS when appetitive reinforcement occurred in the presence of the AV CS?. Comparison of the two shock-exposed conditions with a naive control condition suggests that previous results that were apparently consistent with inhibitory aversive enhancement and blocking of appetitive conditioning may have been due to aversive context conditioning.