The role of nucleus accumbens dopamine in outcome encoding in instrumental and Pavlovian conditioning

Considerable evidence suggests that dopamine in the core subregion of the nucleus accumbens is not only involved in Pavlovian conditioning but also supports instrumental performance. However, it is largely unknown whether NAc dopamine is required for outcome encoding which plays an important role both in Pavlovian stimulus-outcome learning and instrumental action-outcome learning. Therefore, we tested rats with 6-hydroxydopamine (6-OHDA) induced dopamine depletion of the NAc core for their sensitivity to outcome devaluation in a Pavlovian and an instrumental task. Results indicate that 6-OHDA-lesioned animals were sensitive to outcome devaluation in an instrumental task. This finding provides support to the notion that NAc core dopamine may not be crucial in encoding action-outcome associations. However, during instrumental conditioning lever pressing rates in 6-OHDA-lesioned animals were markedly lower which could reflect an impaired behavioral activation. By contrast, after outcome-specific devaluation in a Pavlovian task, performance in 6-OHDA-lesioned animals was impaired, i.e. their magazine-directed responding was non-selectively reduced. One possibility to explain non-selective responding is that NAc core DA depletion impaired the ability of conditioned stimuli to activate the memory of the current value of the reinforcer.     

Intact risk-based decision making in rats with prefrontal or accumbens dopamine depletion

The medial prefrontal cortex (mPFC) and the core region of the nucleus accumbens (AcbC) are key regions of a neural system that subserves risk-based decision making. Here, we examined whether dopamine (DA) signals conveyed to the mPFC and AcbC are critical for risk-based decision making. Rats with 6-hydroxydopamine or vehicle infusions into the mPFC or AcbC were examined in an instrumental task demanding probabilistic choice. In each session, probabilities of reward delivery after pressing one of two available levers were signaled in advance in forced trials followed by choice trials that assessed the animal??s preference. The probabilities of reward delivery associated with the large/risky lever declined systematically across four consecutive blocks but were kept constant within four subsequent daily sessions of a particular block. Thus, in a given session, rats need to assess the current value associated with the large/risky versus small/certain lever and adapt their lever preference accordingly. Results demonstrate that the assessment of within-session reward probabilities and probability discounting across blocks were not altered in rats with mPFC and AcbC DA depletions, relative to sham controls. These findings suggest that the capacity to evaluate the magnitude and likelihood of rewards associated with alternative courses of action seems not to rely on intact DA transmission in the mPFC or AcbC.     

Changes in cis(Z)-flupentixol-induced dopamine blockade produce contrast effects in rats

Two groups of rats were trained under a 45-sec variable-interval schedule. The magnitude of reinforcement used in each test session alternated daily between one and four 45-mg food pellets, the magnitude of reinforcement available at any time being signalled by lights in the operant chamber. Testing took place over two consecutive 11-day phases. During the first phase one group of rats was injected with 0.06 mg/kg of cis(Z)-flupentixol prior to testing; the other group received vehicle injections. In the second phase drug conditions were reversed. The change in drug conditions between phases produced both positive and negative successive contrast effects, consistent with the hypothesis that dopamine blockade attenuated the hedonic impact of reinforcement. Embedded within each session were two short signalled probe periods during which the reinforcement magnitude was switched to that used on the alternate days. No contrast effects were found during these brief daily probe periods.     

Higher Order Chunking in Serial Pattern Learning by Rats in the T-Maze

Rats were repeatedly exposed to two- and three-trial series consisting of different reinforced (R) and nonreinforced (N) trials in a fixed order on extended visually distinct runways in a T-maze. When initially presented with the same sequence, RR or RN in Experiment 1 and RRN or RNR in Experiment 2, on each series in a session (Separate Presentations), rats developed slower running speeds or bypassed the runway in returning to the start chamber on N more than on R trials. These serial response patterns occurred whether rats experienced each sequence on a specific runway (Associated group) or equally on different runways over sessions (Nonassociated group). When sequences were intermixed within each session, only the Associated group maintained its serial pattern responses to both sequences while the Nonassociated group only did so within the RRN sequence. When allowed to choose between runways over trials, the Associated group tended to select the runway associated with the RR sequence on the first two free-choice trials while rats in the Nonassociated group tended to alternate their choices of runways. Switching the relevancy of the runway context in the second experiment caused rats in each group to react like rats in the other group on forced-choice trials. In terms of Capaldi's (1992) model of levels of chunking in serial pattern learning, our findings indicate that rats learned to anticipate at least second trial outcomes by using higher order list or contextually cued chunking and also to anticipate third trial outcomes by using lower order series chunking.     

AMPA/kainate, NMDA, and dopamine D1 receptor function in the nucleus accumbens core: a context-limited role in the encoding and consolidation of instrumental memory

Neural integration of glutamate- and dopamine-coded signals within the nucleus accumbens (NAc) is a fundamental process governing cellular plasticity underlying reward-related learning. Intra-NAc core blockade of NMDA or D1 receptors in rats impairs instrumental learning (lever-pressing for sugar pellets), but it is not known during which phase of learning (acquisition or consolidation) these receptors are recruited, nor is it known what role AMPA/kainate receptors have in these processes. Here we show that pre-trial intra-NAc core administration of the NMDA, AMPA/KA, and D1 receptor antagonists AP-5 (1 microg/0.5 microL), LY293558 (0.01 or 0.1 microg/0.5 microL), and SCH23390 (1 microg/0.5 microL), respectively, impaired acquisition of a lever-pressing response, whereas post-trial administration left memory consolidation unaffected. An analysis of the microstructure of behavior while rats were under the influence of these drugs revealed that glutamatergic and dopaminergic signals contribute differentially to critical aspects of the initial, randomly emitted behaviors that enable reinforcement learning. Thus, glutamate and dopamine receptors are activated in a time-limited fashion-only being required while the animals are actively engaged in the learning context.     

Facilitation of a Distributed Shuttle-Box Conditioning with Posttraining Intracranial Self-Stimulation in Old Rats

Old Wistar rats (16–17 months) were trained in a two-way active avoidance task for 5 consecutive days (10 trials/day). Immediately after each training session a lateral hypothalamic intracranial self-stimulation session (ICSS group) or a sham-treatment session (Control group) was given to the animals. Long-term retention was tested 7 days after the last acquisition session. ICSS treatment led to a significant improvement in acquisition. In the long-term retention session the level of avoidance in both groups was similar to that achieved in the last acquisition session, although differences among groups failed to reach statistical significance. These results are compared with those obtained in previous experiments with young adult rats. While ICSS facilitated the process of acquisition in both young and old rats (however, it was much more powerful in young animals), further experiments are needed to elucidate whether this effect is long-lasting in old rats, as occurs in young adult subjects.     

Incubation of conditional suppression is an associatively-based phenomenon

Ten rats were deprived of water and trained to lick a tube for saccharin reinforcement. In each of the two sessions that followed, the rats received six contiguous pairings of a 30-second illumination of the houselight and a 0.75 second, 0.10 mA electric shock while licking. No sign of conditioning was observed during the first experimental session, but profound conditioning was observed on the first and subsequent trials of the second conditioning session. No comparable change in the rate of licking was observed in groups of rats that received only presentations of the visual stimulus, only presentations of the electric shock, or random presentation of the visual stimulus and electric shock during the first conditioning session. These data establish that the incubation of conditional suppression is an associative phenomenon.     

Early stimulation and patterned responding in rats

This experiment was designed to determine the extent to which performance differences on learning tasks arising from infantile handling reflect learning or emotionality differences. Handled and nonhandled rats were trained and tested in a controlled operant situation where a response on every other trial yielded reinforcement. A key light was turned on for 10 sec during each trial and was off during the 10-sec intertrial interval. The reinforced (S+) trials alternated in a regular sequence with nonreinforced (S–) trials. The pattern of S+ and S– trials recycled continuously throughout the session of 160 trials. The handled rats emitted more responses during both S+ and S– trials than did the nonhandled rats. However, when they were compared on the percentage of emitted responses that were reinforced, there was no significant differences between the two groups. These results were interpreted to suggest that infantile handling does not directly facilitate associative learning in rats.This research was aided by a grant from the National Research Council of Canada, 67-0247. I thank Linda Easton and Beverly Clark for their help in collecting and analyzing the data.     

Memories and anticipations control responding by rats (<Emphasis Type="Italic">Rattus norvegicus</Emphasis>) in a Pavlovian procedure

In Experiment 1 each rat received two different fixed series of three trials each. The unconditioned stimulus occurred on Trial 1 of one series and on Trial 3 of the other series, all other trials being nonreinforced. Previous Pavlovian investigations have shown that rats can remember the immediately prior reward outcome and anticipate the immediately subsequent reward outcome. Experiment 1 demonstrated that rats could remember and anticipate even more remote reward outcomes. In Experiment 2 two groups received a series of two nonrewarded trials followed by a rewarded trial. It was demonstrated that a change in the conditioned stimulus (CS) from Trial 2 to Trial 3, which occurred in one group, produced weaker responding than in the other group that did not experience such CS change. On the basis of these findings it was suggested that the rats organized the trials of a series into a unit or chunk. This was concluded for two reasons. First, remembering and anticipating remote reward outcomes strongly suggests that responding is being controlled by events extending beyond the current trial. Secondly, the experimental manipulations employed in the Pavlovian situation here are similar to those used in prior human learning and animal instrumental learning investigations concerned with chunking. Thus, it would appear that chunking is a ubiquitous phenomenon appearing in human serial learning (e.g., Bower and Winzenz 1969; Crowder 1976), in animal instrumental learning (e.g., Capaldi 1992; Hulse and Dorsky 1977; Terrace 1987), and now in Pavlovian learning.     

Integration and representation in rats' serial pattern learning in the T-maze

Rats were exposed to three-trial series consisting of reinforced (R) trials and one nonreinforced (N) trial in a fixed order, RRN and RNR (Experiments 1 and 2) or NRR and RRN (Experiment 3), on extended visually distinct runways in a T-maze. When initially presented with the same sequence on each series in a session (separate presentations) with the same runway on all trials within a series (Experiments 1 and 3), all the rats developed slower running speeds on N than on R trials. When a runway was sometimes changed between the first and next two trials during separate presentations training (Experiment 2) or both sequences were later intermixed within each session in each experiment, only rats exposed to each sequence on a specific runway maintained these serial running patterns. Rats displayed serial running patterns on a test RNN sequence similar to that on the RNR sequence (Experiment 2), as would be predicted by an intertrial association model of serial pattern learning (Capaldi & Molina, 1979), but responded on test RRR and NRN sequences (Experiment 3) as would be predicted by an ordinal-trial-tag/intratrial association model (Burns, Wiley, & Payne, 1986). Results from test series of free-choice trials in Experiments 1 and 2 failed to support a prediction of the intratrial association model that these rats would integrate RRN and RNR sequences. Rather than always selecting a baited runway on both the second and the third free-choice trials, the rats only selected a baited runway on the third trial on the basis of their choice on the second trial, as would be predicted by the intertrial association model. Only after experiencing all possible outcome sequences during forced-choice training in Experiment 3 did these rats predominantly select a baited runway on every free-choice trial.     

The role of nucleus accumbens and dorsolateral striatal D2 receptors in active avoidance conditioning

The role of dopamine (DA) in rewarding motivated actions is well established but its role in learning how to avoid aversive events is still controversial. Here we tested the role of D2-like DA receptors in the nucleus accumbens (NAc) and the dorsolateral striatum (DLS) of rats in the learning and performance of conditioned avoidance responses (CAR). Adult male Wistar rats received systemic, intra-NAc or intra-DLS (pre- or post-training) administration of a D2-like receptor agonist (quinpirole) or antagonist ((−)sulpiride) and were given two sessions in the two-way active avoidance task. The main effects observed were: (i) sulpiride and lower (likely pre-synaptic) doses of quinpirole decreased the number of CARs and increased the number of escape failures; (ii) higher doses of quinpirole (likely post-synaptic) increased inter-trial crossings and failures; (iii) pre-training administration of sulpiride decreased the number of CARs in both training and test sessions when infused into the NAc, but this effect was observed only in the test session when it was infused into the DLS; (iv) post-training administration of sulpiride decreased CARs in the test session when infused into the NAc but not DLS. These findings suggest that activation of D2 receptors in the NAc is critical for fast adaptation to responding to unconditioned and conditioned aversive stimuli while activation of these receptors in the DLS is needed for a slower learning of how to respond to the same stimuli based on previous experiences.     

Functional interactions between the nucleus tractus solitarius (NTS) and nucleus accumbens shell in modulating memory for arousing experiences

The shell division of the nucleus accumbens receives noradrenergic input from neurons in the nucleus of the solitary tract (NTS) that transmit information regarding fluctuations in peripheral hormonal and autonomic activity. Accumbens shell neurons also receive converging inputs from limbic areas such as the hippocampus and amygdala that process newly acquired information. However, few studies have explored whether peripheral information regarding changes in emotional arousal contributes to memory processing in the accumbens. The beneficial effects on memory produced by emotional arousal and the corresponding activation of NTS neurons may be mediated through influences on neuronal activity in the accumbens shell during memory encoding. To explore this putative relationship, Experiment 1 examined interactions between the NTS and the accumbens shell in modulating memory for responses acquired after footshock training in a water-motivated inhibitory avoidance task. Memory for the noxious shock was significantly improved by posttraining excitation of noradrenergic NTS neurons. The enhanced retention produced by activating NTS neurons was attenuated by suppressing neuronal activity in the accumbens shell with bupivacaine (0.25%/0.5 microl). Experiment 2 examined the direct involvement of accumbens shell noradrenergic activation in the modulation of memory for psychologically arousing events such as a reduction in perceived reward value. Noradrenergic activation of the accumbens shell with phenylephrine (1.0 microg/0.5 microl) produced an enhancement in memory for the frustrating experience relative to control injections as evidenced by runway performance on an extended seven-day retention test. These findings demonstrate a functional relationship between NTS neurons and the accumbens shell in modulating memory following physiological arousal and identifies a role of norepinephrine in modulating synaptic activity in the accumbens shell to facilitate this process.     

Sequence learning in pianists and nonpianists: An fMRI study of motor expertise

Previous studies of motor learning have proposed a distinction between fast and slow learning, but these mechanisms have rarely been examined simultaneously. We examined the influence of longterm motor expertise (slow learning) while pianists and nonpianists performed alternating epochs of sequenced and random keypresses in response to visual cues (fast learning) during functional neuroimaging. All of the participants demonstrated learning of the sequence as demonstrated by decreasing reaction times (RTs) on sequence trials relative to random trials throughout the session. Pianists also demonstrated faster RTs and superior sequence acquisition in comparison with nonpianists. Withinsession decreases in bilateral sensorimotor and parietal activation were observed for both groups. Additionally, there was more extensive activation throughout the session for pianists in comparison with nonpianists across a network of primarily right-lateralized prefrontal, sensorimotor, and parietal regions. These findings provide evidence that different neural systems subserve slow and fast phases of learning.     

Differential dependence of Pavlovian incentive motivation and instrumental incentive learning processes on dopamine signaling

Here we attempted to clarify the role of dopamine signaling in reward seeking. In Experiment 1, we assessed the effects of the dopamine D(1)/D(2) receptor antagonist flupenthixol (0.5 mg/kg i.p.) on Pavlovian incentive motivation and found that flupenthixol blocked the ability of a conditioned stimulus to enhance both goal approach and instrumental performance (Pavlovian-to-instrumental transfer). In Experiment 2 we assessed the effects of flupenthixol on reward palatability during post-training noncontingent re-exposure to the sucrose reward in either a control 3-h or novel 23-h food-deprived state. Flupenthixol, although effective in blocking the Pavlovian goal approach, was without effect on palatability or the increase in reward palatability induced by the upshift in motivational state. This noncontingent re-exposure provided an opportunity for instrumental incentive learning, the process by which rats encode the value of a reward for use in updating reward-seeking actions. Flupenthixol administered prior to the instrumental incentive learning opportunity did not affect the increase in subsequent off-drug reward-seeking actions induced by that experience. These data suggest that although dopamine signaling is necessary for Pavlovian incentive motivation, it is not necessary for changes in reward experience, or for the instrumental incentive learning process that translates this experience into the incentive value used to drive reward-seeking actions, and provide further evidence that Pavlovian and instrumental incentive learning processes are dissociable.     

Functional activity in the brain of socially deprivated rats produced by an active avoidance test after razobazam (Hoe 175) treatment: a 2-deoxyglucose study

The 2-deoxyglucose (2-DG) autoradiographic method was used to map metabolic activity in the brain of socially deprivated rats during an active avoidance test. The method investigated the effects of razobazam during this learning test. The animals were socially deprivated for 5 weeks. On the first experimental day the animals were trained to avoid a footshock by jumping onto a platform. During training and testing, the total number of avoidance responses was scored. On the second day during one 2-DG session of 40 min, razobazam increased the avoidance score by 18% as compared to controls. Autoradiographs were analyzed using a two-dimensional densitometric method. The analysis of the brain structures showed a 22% reduction of optical density in the nucleus habenularis lateralis, a 25% increase in the caudal part of the nucleus accumbens, and a 13% increase in the frontal cortex in rats treated with razobazam, but no change in the amygdala. These results provide a preliminary concept to explain how the new compound razobazam produced a better learning performance in socially deprivated rats.     

Trace fear conditioning is reduced in the aging rat

Auditory trace fear conditioning is a hippocampus-dependent learning task that requires animals to associate an auditory conditioned stimulus (CS) and a fear-producing shock-unconditioned stimulus (US) that are separated by an empty 20-s trace interval. Previous studies have shown that aging impairs learning performance on hippocampus-dependent tasks. This study measured heart rate (HR) and freezing fear responses to determine if aging impairs hippocampus-dependent auditory trace fear conditioning in freely moving rats. Aging and Young rats received one long-trace fear conditioning session (10 trials). Each trial consisted of a tone-CS (5 s) and a shock-US separated by an empty 20-s trace interval. The next day rats received CS-alone retention trials. Young rats showed significantly larger HR and freezing responses on the initial CS-alone retention trials compared to the Aging rats. A control group of aging rats received fear conditioning trials with a short 1-s trace interval separating the CS and US. The Aging Short-Trace Group showed HR and freezing responses on the initial CS alone retention trials that were similar to the Young Long-Trace Group, but greater than the Aging Long-Trace Group. A second aging control group received unpaired CSs and USs, and showed no HR or freezing responses on CS-alone retention trials. These data show that HR and freezing are effective measures for detecting aging-related deficits in trace fear conditioning.     

Spatial learning in rats is impaired by microinfusions of protein kinase C-gamma antisense oligodeoxynucleotide within the nucleus accumbens

The nucleus accumbens (NAcc) has been shown to play a role in motor and spatial learning. Protein kinase C (PKC) has been implicated in the mechanisms of initiation and maintenance of long-term potentiation that is thought to be involved in the storage of long-term memory. In the present study, the importance of de novo synthesis of PKC-gamma within the NAcc in the acquisition and retention of spatial discrimination learning was assessed using an antisense knockdown approach. Separate groups of Long-Evans rats were exposed to acute microinfusions (6microg/microl) of PKC-gamma antisense oligodeoxynucleotide (AS-ODN), control oligodeoxynucleotide (C-ODN) or vehicle into the NAcc at 24 and 3h before each training session. Behavioral findings showed that the blockade of NAcc-PKC-gamma translation caused impairments in the early phase of learning and retention of spatial information. Biochemical experiments showed that PKC-gamma expression was reduced and Ca(2+)/phospholipid-dependent protein kinase C (PKC) activity was blocked significantly in the AS-ODN-treated rats in comparison with control rats. The present findings suggest that NAcc-PKC-gamma plays a role during the early acquisition of spatial learning. Also, retention test results suggest that NAcc-PKC-gamma may be working as an intermediate factor involved in the onset of molecular mechanisms necessary for spatial memory consolidation within the NAcc.     

“We will work for you” – Social influence may suppress individual food preferences in a communicative situation in dogs

The level of motivation (i.e. incentive power) is thought to be one of the most important factors affecting performance and learning in various tasks. We investigated whether reward quality has an effect on the performance of family dogs in a two-way object choice test in which they can find the hidden food by relying on distal momentary human pointing cues. In three experiments we varied (1) the type of food reward according to the subjects’ own preference; (2) the quality of the reward offered at the same time in the indicated and not-indicated locations; and (3) the order of the high or low quality rewards in consecutive sessions. In Experiment 1, we first tested whether dogs prefer one kind of reward over another. Then one group was tested with the ‘preferred’ food as reward in the indicated bowl, while dogs in the other group received the ‘non-preferred’ food as reward. We did not find any difference between the performance and choice latencies of the two groups. In Experiment 2 for the first group, the indicated bowl contained a piece of carrot and the not-indicated bowl was empty. In the second group the indicated bowl contained carrot, but the not-indicated bowl contained sausage. According to a preliminary preference test, most dogs prefer sausage over carrot invariably. After 20 trials, the two groups performed surprisingly similarly. There was no difference found between groups in the number of correct choices, incorrect choices and non-choices. However, the comparison between the first and last five trials revealed that subjects who found sausage when they chose the not-indicated bowl (did not follow the pointing) chose the non-indicated bowl significantly more often toward the end of their test session. In Experiment 3, each dog received two sessions with 12 pointing trials in each. For the first session, one group was rewarded with sausage and the other with carrot upon choosing the indicated bowl. In the second session, the indicated bowl contained dry dog food for both groups. We found that correct choices and response latencies did not change over two sessions in the ‘sausage’ group. In the ‘carrot’ group, the dogs chose faster in the second session, but their performance did not improve; in fact, they chose the not-indicated bowl more often than the indicated bowl. As a conclusion, we can say that reward quality had some effect on dogs’ choice behavior in these experiments. The drop in their performance was not drastic, taking into account the general refusal to eat one of the ‘rewards’ (carrot) during the preference tests and also during the test trials. It seems that incentive contrast may play a relatively minor role in dog-human social interactions. Appropriate reward quality can be very important in asocial problem solving tasks, but, when interacting with humans, following human signals may override the effect of changed incentive power.     

Dopamine D1 and D2 receptors in the nucleus accumbens core and shell mediate Pavlovian-instrumental transfer

Pavlovian stimuli previously paired with food can markedly elevate the rate of food-reinforced instrumental responding. This effect, termed Pavlovian-instrumental transfer (PIT), depends both on general activating and specific cueing properties of Pavlovian stimuli. Recent evidence suggests that the general activating properties of Pavlovian stimuli are mediated by mesoaccumbens dopamine systems; however, the role of NAC dopamine D1 and D2 receptors is still unknown. Here we examined the effects of a selective dopamine D1 and D2 receptor blockade in the shell and core subregion of the NAC on general PIT. Rats were trained to press a single lever for food, and the effect of a single Pavlovian stimulus previously associated with the same food on performance of that lever was measured in extinction. Results reveal that PIT, that is, the increase in instrumental responding during presentation of the Pavlovian stimulus, was reduced by microinjections of the dopamine D1 receptor antagonist SCH-23390 and, less pronounced, by microinjections of the dopamine D2 receptor antagonist raclopride into the NAC core or shell, respectively. Our data suggest that dopamine D1 and D2 receptors in the NAC core and shell mediate the general activating effects of Pavlovian stimuli on instrumental behavior.