Individual differences in dopamine efflux in nucleus accumbens shell and core during instrumental learning

Combined activation of dopamine D1- and NMDA-glutamate receptors in the nucleus accumbens has been strongly implicated in instrumental learning, the process in which an individual learns that a specific action has a wanted outcome. To assess dopaminergic activity, we presented rats with two sessions (30 trials each) of a one-lever appetitive instrumental task and simultaneously measured dopamine efflux in the shell and core accumbens subareas using in vivo microdialysis. Dopamine efflux was increased during each session in all areas. The behavioral performance of the rats in the second session led us to divide them into a learning group (>90% correct trials) and a non-learning group. In the first session, the rats of the learning group showed significantly higher increases. The difference was most pronounced in the shell. In the second session, the dopamine increase was similar in both groups, although the learning groups now pressed the lever about three times more often and consequently obtained more rewards. We conclude that task-related activation of dopamine efflux is different between learning and non-learning rats only during the learning phase. These results support the pharmacological evidence that dopamine is of particular importance during the instrumental learning process.     

AMPA/kainate, NMDA, and dopamine D1 receptor function in the nucleus accumbens core: a context-limited role in the encoding and consolidation of instrumental memory

Neural integration of glutamate- and dopamine-coded signals within the nucleus accumbens (NAc) is a fundamental process governing cellular plasticity underlying reward-related learning. Intra-NAc core blockade of NMDA or D1 receptors in rats impairs instrumental learning (lever-pressing for sugar pellets), but it is not known during which phase of learning (acquisition or consolidation) these receptors are recruited, nor is it known what role AMPA/kainate receptors have in these processes. Here we show that pre-trial intra-NAc core administration of the NMDA, AMPA/KA, and D1 receptor antagonists AP-5 (1 microg/0.5 microL), LY293558 (0.01 or 0.1 microg/0.5 microL), and SCH23390 (1 microg/0.5 microL), respectively, impaired acquisition of a lever-pressing response, whereas post-trial administration left memory consolidation unaffected. An analysis of the microstructure of behavior while rats were under the influence of these drugs revealed that glutamatergic and dopaminergic signals contribute differentially to critical aspects of the initial, randomly emitted behaviors that enable reinforcement learning. Thus, glutamate and dopamine receptors are activated in a time-limited fashion-only being required while the animals are actively engaged in the learning context.     

The role of nucleus accumbens and dorsolateral striatal D2 receptors in active avoidance conditioning

The role of dopamine (DA) in rewarding motivated actions is well established but its role in learning how to avoid aversive events is still controversial. Here we tested the role of D2-like DA receptors in the nucleus accumbens (NAc) and the dorsolateral striatum (DLS) of rats in the learning and performance of conditioned avoidance responses (CAR). Adult male Wistar rats received systemic, intra-NAc or intra-DLS (pre- or post-training) administration of a D2-like receptor agonist (quinpirole) or antagonist ((−)sulpiride) and were given two sessions in the two-way active avoidance task. The main effects observed were: (i) sulpiride and lower (likely pre-synaptic) doses of quinpirole decreased the number of CARs and increased the number of escape failures; (ii) higher doses of quinpirole (likely post-synaptic) increased inter-trial crossings and failures; (iii) pre-training administration of sulpiride decreased the number of CARs in both training and test sessions when infused into the NAc, but this effect was observed only in the test session when it was infused into the DLS; (iv) post-training administration of sulpiride decreased CARs in the test session when infused into the NAc but not DLS. These findings suggest that activation of D2 receptors in the NAc is critical for fast adaptation to responding to unconditioned and conditioned aversive stimuli while activation of these receptors in the DLS is needed for a slower learning of how to respond to the same stimuli based on previous experiences.     

Motivational states influence effort-based decision making in rats: The role of dopamine in the nucleus accumbens

Decision-making policies are subject to modulation by changing motivational states. However, so far, little is known about the neurochemical mechanisms that bridge motivational states with decision making. Here we examined whether dopamine (DA) in the nucleus accumbens core (AcbC) modulates the effects of motivational states on effort-based decision making. Using a cost–benefit T-maze task in rats, we examined the effects of AcbC DA depletions on effort-based decision making, in particular on the sensitivity of effort-based decision making to a shift from a hungry to a sated state. The results demonstrated that, relative to sham controls, rats with AcbC DA depletion in a hungry as well as in a sated state had a reduced preference for effortful but large-reward action. This finding provides further support for the notion that AcbC DA regulates how much effort to invest for rewards. Importantly, our results further revealed that effort-based decision making in lesioned rats, as in sham controls, was still sensitive to a shift from a hungry to a sated state; that is, their preferences for effortful large-reward actions became lower after a shift from a restricted to a free-feeding regimen. These finding indicate that AcbC DA is not necessarily involved in mediating the effects of a shift in motivational state on decision-making policies.     

A role for CS-US contingency in Pavlovian conditioning

Two experiments evaluated the role of conditioned stimulus-unconditioned stimulus (CS-US) contingency in appetitive Pavlovian conditioning in rats. In both experiments, some groups received a positively contingent CS signaling an increased likelihood of the US relative to the absence of the CS. These groups were compared with control treatments in which the likelihood of the US was the same in the presence and absence of the CS. A trial marker served as a trial context. Experiment 1 found contingency sensitivity. There was a reciprocal relationship between responding to the CS and the trial marker. Experiment 2 showed that this result was not stimulus or response specific. These results are consistent with associative explanations and the idea that rats are sensitive to CS-US contingency.     

The role of awareness in Pavlovian conditioning: empirical evidence and theoretical implications

This article reviews research over the past decade concerning the relationship between Pavlovian conditioning and conscious awareness. The review covers autonomic conditioning, conditioning with subliminal stimuli, eyeblink conditioning, conditioning in amnesia, evaluative conditioning, and conditioning under anesthesia. The bulk of the evidence is consistent with the position that awareness is necessary but not sufficient for conditioned performance, although studies suggestive of conditioning without awareness are identified as worthy of further investigation. Many studies have used inadequate measures of awareness, and strategies for increasing validity and sensitivity are discussed. It is concluded that conditioning may depend on the operation of a propositional system associated with consciousness rather than a separate, lower level system.     

Differential dependence of Pavlovian incentive motivation and instrumental incentive learning processes on dopamine signaling

Here we attempted to clarify the role of dopamine signaling in reward seeking. In Experiment 1, we assessed the effects of the dopamine D(1)/D(2) receptor antagonist flupenthixol (0.5 mg/kg i.p.) on Pavlovian incentive motivation and found that flupenthixol blocked the ability of a conditioned stimulus to enhance both goal approach and instrumental performance (Pavlovian-to-instrumental transfer). In Experiment 2 we assessed the effects of flupenthixol on reward palatability during post-training noncontingent re-exposure to the sucrose reward in either a control 3-h or novel 23-h food-deprived state. Flupenthixol, although effective in blocking the Pavlovian goal approach, was without effect on palatability or the increase in reward palatability induced by the upshift in motivational state. This noncontingent re-exposure provided an opportunity for instrumental incentive learning, the process by which rats encode the value of a reward for use in updating reward-seeking actions. Flupenthixol administered prior to the instrumental incentive learning opportunity did not affect the increase in subsequent off-drug reward-seeking actions induced by that experience. These data suggest that although dopamine signaling is necessary for Pavlovian incentive motivation, it is not necessary for changes in reward experience, or for the instrumental incentive learning process that translates this experience into the incentive value used to drive reward-seeking actions, and provide further evidence that Pavlovian and instrumental incentive learning processes are dissociable.     

Relative validity effects with either one or two more valid cues in Pavlovian and instrumental conditioning

Two experiments investigated the relative validity effect with either 1 or 2 continuously reinforced cues in Wistar rats using appetitive Pavlovian and instrumental preparations. Discrimination training involved 3 compound cues containing a common element (1AX: 1BX: 2CX). In the first true-discrimination group (TD-1), CX was followed by food, but AX and BX were not. In the second true-discrimination group (TD-2), AX and BX but not CX were followed by food. In the third, pseudodiscrimination group (PD), food followed 50% of each compound. Compared with the PD group, there were lower levels of responding to X in Groups TD-1 and TD-2, which did not differ. That is, both TD treatments showed equivalent relative validity effects. There was evidence for a relative validity effect on the context. The Rescorla-Wagner model incorrectly predicts a smaller relative validity effect after the TD-2 than the TD-1 treatment. Comparator theory predicts these results.     

The role of the nucleus accumbens in knowing when to respond

While knowing what to expect is important, it is equally important to know when to expect it and to respond accordingly. This is apparent even in simple Pavlovian training situations in which animals learn to respond more strongly closer to reward delivery. Here we report that the nucleus accumbens core, an area well-positioned to represent information about the timing of impending rewards, plays a critical role in this timing function.     

Pavlovian conditioning and rule learning

The experiment reviewed here was an attempt to show that two differential Pavlovian conditioning designs, namely positive and negative patterning, can best be understood as rule learning. First, it is shown that positive patterning is equivalent to the logical rule of conjunction (AND) and that negative patterning is equivalent to the logical rule of exclusive disjunction (XOR). It is assumed that in order to learn both kinds of discrimination subjects learn to use the according rule. If this is the case, the observed differentiation should be independent of the number of reinforcements for each individual stimulus. Second, subjects should be able to transfer the rule to new stimuli. Forty human subjects were randomly divided into four groups (N=10 each). Two factors were manipulated independently between subjects: (1) positive vs negative patterning, and (2) 2 vs 4 pairs of trained stimuli. Second interval skin conductance responses were measured. During initial acquisition positive as well as negative patterning occurred independently of number of pairs of trained stimuli (with total amount of training kept constant). Furthermore, AND as well as XOR could be transfered to new stimuli.     

Unblocking in Pavlovian fear conditioning

Six experiments used rats to study blocking and unblocking of fear learning. An excitatory stimulus (A) blocked fear learning to a neutral stimulus (B). Unblocking of B occurred if the AB compound signaled an increase in unconditioned stimulus (US) intensity or number. Assessments of associative change during blocking showed that more was learned about B than A. Such assessments during unblocking revealed that more was learned about B than A following an increase in US intensity but not US number. These US manipulations had no differential effects on single-cue learning. The results show that variations in US intensity or number produce unblocking of fear learning, but for each there is a different profile of associative change and a potentially different mechanism.     

Naloxone and Pavlovian fear conditioning

Previous research has shown that pretreating rats with the opiate antagonist naloxone increases the freezing that follows painful electric shock. Three experiments, using freezing behavior as a dependent variable, were carried out to determine whether the drug might cause this effect by enhancing fear conditioning. Two of the studies employed a differential context fear-conditioning paradigm. Naloxone did not affect freezing behavior during the preshock adaptation period. In Experiment 1, naloxone was found to increase resistance to extinction in the S+ context. In Experiment 2, naloxone was found to increase freezing in the S+ context. This effect was dependent upon administering naloxone during training but not dependent on administering it during testing. The third study employed a generalization paradigm. It was found that naloxone's effect on postshock freezing was dependent on the place of testing; as the contextual cues of the test chamber were changed from those of the conditioning chamber, the effect of naloxone on freezing was reduced. The results of these experiments lend strong support to the hypothesis that naloxone increases freezing by enhancing the conditioning of fear to contextual stimuli associated with shock.     

Aversive CS control of instrumental avoidance as a function of selected parameters and method of Pavlovian conditioning

Effects of aversive CSs upon dogs' instrumental avoidance responding were assessed as a function of the Pavlovian conditioning parameters using off-baseline (Experiment 1) and on-baseline (Experiment 2) conditioning procedures. In each experiment, three UCS intensitites (2, 6, and 10 mA) were crossed factorially with two CS-UCS intervals (10 and 90 sec) yielding six groups. After 4 days of conditioning, the CS-produced effects on avoidance were determined primarily by an interaction between Pavlovian procedure (off- versus on-baseline) and UCS intensity. With off-baseline conditioning, CS-produced facilitation of avoidance was an increasing function of the UCS intensity. However, with on-baseline conditioning, only the CS paired with weak UCS facilitated avoidance, and the CS paired with the strong UCS suppressed responding. This reversing parametric function poses problems for two-process motivational theories of avoidance.     

Reactivation-dependent amnesia in Pavlovian approach and instrumental transfer

The theory of memory reconsolidation relates to the hypothesized restabilisation process that occurs following the reactivation of a memory through retrieval. Thus the demonstration of reactivation-dependent amnesia for a previously acquired memory is a prerequisite for showing that such a memory undergoes reconsolidation. Here we show that the appetitive Pavlovian representations that underlie Pavlovian approach and Pavlovian-instrumental transfer are destabilized following their retrieval. This reactivation-dependent amnesia demonstrates that the general motivational or incentive properties of appetitive conditioned stimuli, as well as their conditioned reinforcing properties, can be reduced by blocking memory reconsolidation.     

The pavlovian analysis of instrumental conditioning

An account was given of the development within the Russian literature of a uniprocess formulation of classical and instrumental conditioning, known as the bidirectional conditioning hypothesis. The hypothesis purports to offer a single set of Pavlovian principles to account for both paradigms, based upon a neural model which assumes that bidirectional (forward and backward) connections are formed in both classical and instrumental conditioning situations. In instrumental conditioning, the bidirectional connections are hypothesized to be simply more complex than those in classical conditioning, and any differences in empirical functions are presumed to lie not in difference in mechanism, but in the strength of the forward and backward connections. Although bidirectional connections are assumed to develop in instrumental conditioning, the experimental investigation of the bidirectional conditioning hypothesis has been essentially restricted to the classical conditioning operations of pairing two CSs (sensory preconditioning training), a US followed by a CS (backward conditioning training) and two USs. However, the paradigm involving the pairing of two USs, because of theoretical and analytical considerations, is the one most commonly employed by Russian investigators. The results of an initial experiment involving the pairing of two USs, and reference to the results of a more extensive investigation, leads us to tentatively question the validity of the bidirectional conditioning account of instrumental conditioning.     

Proactive interference between cues trained with a common outcome in first-order Pavlovian conditioning

The nature of interference between cues (X, A) trained apart with a common outcome (O; an unconditioned stimulus) was explored by assessing proactive interference in first-order Pavlovian conditioning (i.e., A-O, X-O, resulting in attenuated responding to X). Three lick-suppression studies were conducted with water-deprived rats. Posttraining extinction of the interfering cue (A) attenuated proactive interference (Experiment 1), which mirrors the observation that extinction of the competing cue can reduce competition between cues trained together (e.g., recovery from overshadowing). Proactive interference was also attenuated with manipulations known to attenuate interference between outcomes trained apart (e.g., counterconditioning), namely reminder cues (Experiment 2) and renewal (Experiment 3). The findings suggest that similar processes underlie interference between cues trained apart, between cues trained together, and between outcomes trained apart.     

Contemporary study of Pavlovian conditioning

Pavlov's first report on conditioning emphasized its role in allowing the animal to adjust to its environment. Contemporary theories have seen this adjustment in terms of developing accurate knowledge of the environment. Three aspects of that thinking are explored: how the animal acquires initial knowledge, how it changes its knowledge when conditions of the world change, and how it makes use of multiple knowledge representations.