Facilitation of instrumental behavior by a Pavlovian appetitive conditioned stimulus

Three experiments examined appetitive Pavlovian-instrumental interactions by presenting separately trained conditioned stimuli (CSs) during reinforced instrumental responding in rabbits. Intra-oral reinforcement was used to minimize interference from peripheral responses such as magazine approach. In experiment 1, the rabbits were first trained to perform an instrumental head-raising response for sucrose reward. A conditioned jaw movement response was then established to a 2-sec CS by pairing it with sucrose; a control stimulus was unpaired with sucrose. Instrumental responding maintained by a variable-interval 40-sec schedule was enhanced during 10-sec presentations of the paired, but not the unpaired, CS. Responding on a variable-ratio 15 schedule was unaffected except on trials on which the pre-CS baseline response rate was low; in such cases the paired CS caused a long-lasting acceleration of responding. Noncontingent presentation of the sucrose reinforcer itself briefly suppressed responding but had no long-term effect. In Experiment 2, a CS that had been conditioned at a 10-sec duration produced the same pattern of effects as in the first study, indicating that facilitation resulted from CS presentation rather than from the frustrative effects of non-reinforcement of the CS. In Experiment 3 an inhibitory CS blocked facilitation by the excitatory CS but did not itself affect instrumental responding. These results support the view that Pavlovian processes play a positive role in instrumental performance and suggest that previous findings of suppression by a short-duration CS reflect peripheral interference. The dependence of facilitation on the baseline level of responding is discussed in terms of associative and motivational theories of Pavlovian mediation.     

Inactivation of the ventral tegmental area abolished the general excitatory influence of Pavlovian cues on instrumental performance

Pavlovian stimuli can markedly elevate instrumental responding, an effect known as Pavlovian-instrumental transfer (PIT). As the role of the ventral tegmental area (VTA) in PIT is yet unknown, we examined the effects of transient VTA inactivation by direct microinjections of a mixture of the GABA(A) and GABA(B) receptor agonists, muscimol and baclofen. Results reveal that PIT, i.e., the increase in instrumental responding during presentation of a Pavlovian stimulus, was abolished by intra-VTA microinjections of muscimol/baclofen. These data provide the first evidence that the VTA mediates Pavlovian influences on instrumental behavior.     

Instrumental learning in hyperdopaminergic mice

In two experiments we investigated the effects of elevated dopaminergic tone on instrumental learning and performance using dopamine transporter knockdown (DAT KD) mice. In Experiment 1, we showed that both DAT KD mice and wild-type controls were similarly sensitive to outcome devaluation induced by sensory specific satiety, indicating normal action-outcome learning in both groups. In Experiment 2, we used a Pavlovian-to-instrumental transfer procedure to assess the potentiation of instrumental responding by Pavlovian conditional stimuli (CS). Although during the Pavlovian training phase the DAT KD mice entered the food magazine more frequently in the absence of the CS, when tested later both groups showed outcome-selective PIT. These results suggest that the elevated dopaminergic tone reduced the selectivity of stimulus control over conditioned behavior, but did not affect instrumental learning.     

Dopamine D1 and D2 receptors in the nucleus accumbens core and shell mediate Pavlovian-instrumental transfer

Pavlovian stimuli previously paired with food can markedly elevate the rate of food-reinforced instrumental responding. This effect, termed Pavlovian-instrumental transfer (PIT), depends both on general activating and specific cueing properties of Pavlovian stimuli. Recent evidence suggests that the general activating properties of Pavlovian stimuli are mediated by mesoaccumbens dopamine systems; however, the role of NAC dopamine D1 and D2 receptors is still unknown. Here we examined the effects of a selective dopamine D1 and D2 receptor blockade in the shell and core subregion of the NAC on general PIT. Rats were trained to press a single lever for food, and the effect of a single Pavlovian stimulus previously associated with the same food on performance of that lever was measured in extinction. Results reveal that PIT, that is, the increase in instrumental responding during presentation of the Pavlovian stimulus, was reduced by microinjections of the dopamine D1 receptor antagonist SCH-23390 and, less pronounced, by microinjections of the dopamine D2 receptor antagonist raclopride into the NAC core or shell, respectively. Our data suggest that dopamine D1 and D2 receptors in the NAC core and shell mediate the general activating effects of Pavlovian stimuli on instrumental behavior.     

Disruption in Pavlovian-Instrumental Transfer as a Function of Depression and Anxiety

The Pavlovian-Instrumental Transfer (PIT) paradigm examines probabilistic and reinforcement learning. Disruptions in mechanisms that mediate PIT (i.e., cues not triggering adaptive behaviors) are thought to be contributors to psychopathology, making the study of probabilistic and reinforcement learning clinically relevant. The current study evaluated an appetitive PIT effect and its relationship with symptom dimensions spanning depression and anxiety, with a particular focus on anhedonia. Forty young adults ranging in scores across dimensions of depression and anxiety symptoms completed the PIT paradigm and self-report symptom measures. The PIT paradigm consisted of three phases. The instrumental phase consisted of a contingent association in which participants squeezed a handgrip for monetary reward. The Pavlovian phase established a purely predictive association between three visual stimuli (CS?+?, CS-, baseline) and presence or absence of monetary reward. In the transfer phase, participants’ responses allowed for examination of whether motivational characteristics of Pavlovian predictors influenced the vigor of their handgrip squeezes (instrumental action), which were formerly independent of Pavlovian associations. Analyses revealed a baseline-reward PIT effect, whereby a reward-associated Pavlovian cue enhanced instrumental responding in the transfer phase. However, there were no significant differences between CS?+?and CS- or CS- and baseline cues, suggesting a disrupted interaction of Pavlovian and instrumental learning. Further, the appetitive PIT effect captured in this paradigm was not associated with anhedonia, fears, or general distress. Future work should investigate the influence of mood states using more specific appetitive PIT paradigms to further understanding of the implications of disrupted reflexive and instrumental responding.

     

The effects of acute stress on Pavlovian-instrumental transfer in rats

Pavlovian stimuli invigorate ongoing instrumental action, a phenomenon termed the Pavlovian–instrumental transfer (PIT) effect. Acute stressors can markedly enhance the release of corticotropin-releasing factor (CRF), and CRF injection into the nucleus accumbens increases the PIT effect. However, it is unknown whether acute stressors by themselves would amplify the PIT effect. Here, we examined the effects of acute stressors on PIT. Rats first received Pavlovian and instrumental training, and then the impact of the Pavlovian stimuli on instrumental responding was analyzed in the subsequent PIT test. Acute stressors were applied prior to the PIT test. Because the effects of acute stressors critically depend on stressor type and time of day, we used two acute stressors that involved one or several distinct stressors (denoted here as “single” vs. “multiple” stressors) applied either in the light or the dark period of the light:dark cycle. The results revealed that single and multiple stressors applied in the light period did not alter the PIT effect—that is, the ability of an appetitive Pavlovian stimulus to enhance leverpressing—or the basal leverpress rate. When applied in the dark period, single and multiple stressors also did not alter the PIT effect, but they did markedly reduce the basal leverpress rate. Diazepam pretreatment did not counteract the declines in basal instrumental responding in the PIT test that were induced by either a single or multiple stressors. Our findings suggest that acute stressors were unable to amplify the incentive salience of reward-predictive Pavlovian stimuli to activate instrumental responding, but, depending on the time of day of stressor exposure, they did reduce basal instrumental responding.     

Relations between Pavlovian-instrumental transfer and reinforcer devaluation

Relations between posttraining reinforcer devaluation and Pavlovian-instrumental transfer were examined in 2 experiments. When a single reinforcer was used, extended training of the instrumental response increased transfer but reduced devaluation effects. When multiple instrumental reinforcers were used, both reinforcer-specific transfer and devaluation effects were less influenced by the amount of instrumental training. Finally, although reinforcer devaluation decreased both Pavlovian conditioned responses and baseline instrumental responding, it had no effect on either single-reinforcer or reinforcer-specific transfer. These results indicate that transfer and reinforcer devaluation can reflect different aspects of associative learning and that the nature of associative learning can be influenced by parameters such as the amount of training and the use of multiple reinforcers.     

Effect of telencephalon ablation on the reinforcing and eliciting properties of species-specific events in Betta splendens

In male Betta splendens, aggressive behavior is drastically attenuated following telencephalon ablation. Because instrumental training and Pavlovian conditioning experiments with intact fish have suggested that associative factors may play an important role in the performance of agonistic behaviors, the effect of ablation on instrumental learning and Pavlovian conditioning was studied. In Experiment 1, ablation had no effect on the learning of the instrumental tunnel-swimming response reinforced by mirror presentation (i.e., viewing a conspecific), although the mirror presentations in yoked-control groups elicited fewer responses in ablates than in normal and sham-operated control fish. Yoked controls further established that instrumental responding was maintained by the reinforcement contingency and was not merely the result of increased motor activity. Experiment 2 studied Pavlovian conditioning of the components of the agonistic display. Unconditioned fin erection, gill erection, and tail beating (i.e., unconditioned responses, URs) to the mirror US all were less frequent in ablates than in normals or shams. Of these, only gill cover erection showed evidence of true conditioning (i.e., conditioned responses; CRs) in which responses to the conditioned stimulus (CS) are due to the pairings of CS and US (unconditioned stimulus). However, ablates suffered no impairment of conditioned gill erections. Ablates performed fewer fin erections to the CS; however, fin erection responses were not due to CS-US pairings but were attributable to pseudoconditioning. These results are related to hypotheses postulating the involvement of learning mechanisms in ablation-produced deficits and normal aggressive behavior.     

The effects upon food-reinforced pecking and treadle-pressing of auditory and visual signals for response-independent food

Seven pigeons whose key-pecking was maintained by food reinforcement on a differential-reinforcement-of-low-rates 12-sec limited-hold 4-sec schedule and 12 other pigeons whose treadle-pressing was maintained by the same schedule received appetitive Pavlovian conditioning trials superimposed upon the instrumental baseline. Half the birds in each group received a tone as the CS, and the other half received a stimulus change on the key. Each CS was 20 sec long, and was immediately followed by 10-sec access to grain. The visual CS markedly facilitated the rate of pecking on the key for the birds whose baseline response was pecking. The visual CS produced auto-shaping of the key-peck and tended to produce suppression of treadle-pressing for the birds whose baseline response was treadle-pressing. The auditory CS produced inconsistent effects across birds regardless of the baseline response. In all cases the conditioned effects extinguished when response-independent food was omitted.     

Differential effects of forward or simultaneous conditioned stimulus-unconditioned stimulus intervals on the defensive behavior system of the Norway rat (Rattus norvegicus)

To test several predictions derived from a behavior-systems approach, the authors assessed Pavlovian fear conditioning in rats after 30 trials of forward, simultaneous, or unpaired training. Direct evidence of conditioned fear was collected through observation of flight and freezing reactions during presentations of the conditioned stimulus (CS) alone. The authors also tested the CS's potential to reinforce an instrumental escape response in an escape-from-fear paradigm. On the one hand, rats that received forward training showed conditioned freezing, but no conditioned flight was observed. On the other hand, rats that received simultaneous training showed conditioned flight, but no conditioned freezing was observed. Rats that received either forward or simultaneous pairings showed instrumental learning of the escape-from-fear response. Implications for several theories of Pavlovian conditioning are discussed.     

Facilitation of food-reinforced responding by a signal for response-independent food

Five pigeons whose key pecking was maintained by 4-sec access to grain on a variable-interval 2-min schedule received Pavlovian differential conditioning trials superimposed upon the instrumental baseline. The conditioned stimuli were changes in the stimulus on the key from white to red, or to a white horizontal line against a dark background. The positive conditioned stimulus was 20 sec long, and was followed immediately by 8-sec access to grain. The negative conditioned stimulus, also 20 sec long, was never paired with response-independent food. All pigeons responded more rapidly in the presence of the positive conditioned stimulus than in the presence of the negative one. The positive conditioned stimulus produced an increase in response rate over the pre-conditioned stimulus period. The negative conditioned stimulus had no marked effect upon response rate. When the roles of the positive and negative stimuli were reversed, and the duration of the response-independent reinforcement was reduced to 4 sec, the new positive conditioned stimulus came to facilitate responding, and the new negative conditioned stimulus no longer produced facilitation. A second discrimination reversal produced similar outcomes. When a third reversal was initiated, and the duration of response-independent reinforcement was reduced to 2 sec, the difference between the effects of the positive and negative stimuli diminished.     

Post-training amphetamine administration enhances memory consolidation in appetitive Pavlovian conditioning: Implications for drug addiction

It has been suggested that some of the addictive potential of psychostimulant drugs of abuse such as amphetamine may result from their ability to enhance memory for drug-related experiences through actions on memory consolidation. This experiment examined whether amphetamine can specifically enhance consolidation of memory for a Pavlovian association between a neutral conditioned stimulus (CS-a light) and a rewarding unconditioned stimulus (US-food), as Pavlovian conditioning of this sort plays a major role in drug addiction. Male Long-Evans rats were given six training sessions consisting of 8 CS presentations followed by delivery of the food into a recessed food cup. After the 1st, 3rd, and 5th session, rats received subcutaneous injections of amphetamine (1.0 or 2.0 mg/kg) or saline vehicle immediately following training. Conditioned responding was assessed using the percentage of time rats spent in the food cup during the CS relative to a pre-CS baseline period. Both amphetamine-treated groups showed significantly more selective conditioned responding than saline controls. In a control experiment, there were no differences among groups given saline, 1.0 or 2.0 mg/kg amphetamine 2 h post-training, suggesting that immediate post-training amphetamine enhanced performance specifically through actions on memory consolidation rather than through non-mnemonic processes. This procedure modeled Pavlovian learning involved in drug addiction, in which the emotional valence of a drug reward is transferred to neutral drug-predictive stimuli such as drug paraphernalia. These data suggest that amphetamine may contribute to its addictive potential through actions specifically on memory consolidation.     

Pavlovian to instrumental transfer of control in a human learning task

Pavlovian learning tasks have been widely used as tools to understand basic cognitive and emotional processes in humans. The present studies investigated one particular task, Pavlovian-to-instrumental transfer (PIT), with human participants in an effort to examine potential cognitive and emotional effects of Pavlovian cues upon instrumentally trained performance. In two experiments, subjects first learned two separate instrumental response-outcome relationships (i.e., R1-O1 and R2-O2) and then were exposed to various stimulus-outcome relationships (i.e., S1-O1, S2-O2, S3-O3, and S4-) before the effects of the Pavlovian stimuli on instrumental responding were assessed during a non-reinforced test. In Experiment 1, instrumental responding was established using a positive-reinforcement procedure, whereas in Experiment 2, a quasi-avoidance learning task was used. In both cases, the Pavlovian stimuli exerted selective control over instrumental responding, whereby S1 and S2 selectively elevated the instrumental response with which it shared an outcome. In addition, in Experiment 2, S3 exerted a nonselective transfer of control effect, whereby both responses were elevated over baseline levels. These data identify two ways, one specific and one general, in which Pavlovian processes can exert control over instrumental responding in human learning paradigms, suggesting that this method may serve as a useful tool in the study of basic cognitive and emotional processes in human learning.     

Aversive CS control of instrumental avoidance as a function of selected parameters and method of Pavlovian conditioning

Effects of aversive CSs upon dogs' instrumental avoidance responding were assessed as a function of the Pavlovian conditioning parameters using off-baseline (Experiment 1) and on-baseline (Experiment 2) conditioning procedures. In each experiment, three UCS intensitites (2, 6, and 10 mA) were crossed factorially with two CS-UCS intervals (10 and 90 sec) yielding six groups. After 4 days of conditioning, the CS-produced effects on avoidance were determined primarily by an interaction between Pavlovian procedure (off- versus on-baseline) and UCS intensity. With off-baseline conditioning, CS-produced facilitation of avoidance was an increasing function of the UCS intensity. However, with on-baseline conditioning, only the CS paired with weak UCS facilitated avoidance, and the CS paired with the strong UCS suppressed responding. This reversing parametric function poses problems for two-process motivational theories of avoidance.     

Appetitive pavlovian-instrumental interactions: Effects of inter-stimulus interval and baseline reinforcement conditions

Experiment I manipulated two variables which appear to influence whether a signal for food enhances or suppresses food-rewarded instrumental performance: interstimulus interval (ISI) during classical conditioning and instrumental reinforcement schedule during testing. In two groups a 10-s conditioned stimulus (CS) and food were paired (10-s and 20- to 100-s ISI), while in a third group they were unpaired. During signalled reinforcement of lever-pressing (S+), the CS suppressed responding in both paired groups. During signalled extinction (S-), responding in the 10-s ISI group was suppressed during the CS and enhanced for 60 s after CS offset; responding in the 20- to 100-s ISI group was enhanced both during the CS and for 120 s after CS offset. Experiment II examined whether the long ISI enhancement effect would occur when the baseline response rate was lowered by satiation rather than extinction. A 20- to 100-s CS and food were paired in one group and unpaired in another. After near-satiation on a CRF schedule, CS presentations caused a reduction in responding in both groups, with no significant difference between the two groups. The results of the two experiments were interpreted in terms of an interaction between the expectancy of food generated by stimuli conditioned at short and long ISIs and the expectancy of food availability controlled by the instrumental schedule.     

Negative emotional appraisal selectively disrupts retrieval of expected outcome values required for goal-directed instrumental choice*

Stress induction reduces people's ability to modify their instrumental choices following changes in the value of outcomes, but the mechanisms underpinning this effect have not been specified because previous studies have lacked crucial control conditions. To address this, the current study had participants learn two instrumental responses for food and water, respectively, before water was devalued by specific satiety. Choice between these two responses was then measured in extinction, reacquisition and Pavlovian to instrumental transfer (PIT) tests. Concurrently during these tests, a negative emotional appraisal group evaluated aversive images (stress induction), whereas a control group evaluated neutral images, at the same time as choosing between the two instrumental responses. Negative emotional appraisal abolished the impact of water devaluation on instrumental choice in the extinction test, but did not affect instrumental choice in the reacquisition or PIT tests. These findings suggest that negative emotional appraisal selectively impaired participants’ ability to retrieve the expected value of outcomes required to make goal-directed instrumental choices in the extinction test, and that this effect was not due to task disengagement, nullification of the devaluation treatment or impaired knowledge of response-outcome relationships.     

Acquisition and retention of control of instrumental behavior by a cue-signaling airblast: How specific are conditioned anticipations?

The conditioned state evoked by a CS that had been paired with an aversive airblast was assessed by superimposing the CS on baselines of instrumental responding. The first experiment used a shock avoidance baseline; CS-airblast pairings were administered on the baseline, with the shock avoidance schedule in effect. The CS that signaled airblast came, over trails, to accelerate shock avoidance responding. As further CS-airblast trials were administered, however, the acceleratory effect of the CS decreased, producing an inverted-U-shaped acquisition curve. A second experiment used off-baseline CS-airblast conditioning, and the CS was tested on both shock avoidance and water reinforcement baselines. After 20 CS-airblast pairings the CS accelerated shock avoidance and suppressed water-reinforced responding. After 10o pairings, however, the acceleratory effect on shock avoidance was much attenuated, although the suppressions of the water-reinforced behavior did not differ between 20- and 100-trial groups. Two further experiments examined retention, over a 45-day interval, of the attenuation of the CS's effects on shock avoidance. Over the retention interval, this attenuation disappeared. It was, however, reinstated by giving the animals “reminder” presentations of the airblast. These results are interpreted as representing systematic changes in the specificity of the conditioned state evoked by a Pavlovian CS.     

Nicotine trained as a negative feature passes the retardation-of-acquisition and summation tests of a conditioned inhibitor

Nicotine functions as a negative feature in a Pavlovian discriminated goal-tracking task. Whether withholding of responding to the conditional stimulus (CS) reflects nicotine functioning as a conditioned inhibitor is unknown. Accordingly, the present research sought to determine whether nicotine trained as a negative feature passed the retardation-of-acquisition and summation tests, thus characterizing it as a pharmacological (interoceptive) conditioned inhibitor. In the retardation test, rats received either nicotine (0.4 mg/kg) or chlordiazepoxide (5 mg/kg) negative feature training in which the drug state signaled when a 15-sec light CS would not be paired with sucrose; light was paired with sucrose on intermixed saline sessions. Following acquisition of the discrimination, both groups received nicotine CS training in which sucrose was intermittently available on nicotine but not intermixed saline sessions. Acquisition of conditioned responding to the nicotine CS was slower in the nicotine negative feature group than in the chlordiazepoxide negative feature group. In the summation test, rats were assigned to either the nicotine negative feature group or a pseudoconditioning control. In this control, the light CS was paired with sucrose on half the nicotine and half the saline sessions. Both groups also received excitatory training in which a white noise CS was paired with sucrose. The summation test consisted of presenting the white noise in conjunction with nicotine. Conditioned responding evoked by the white noise was decreased in the negative feature but not the pseudoconditioning group. Combined, the results provide the first evidence that an interoceptive stimulus (nicotine) can become a conditioned inhibitor.     

Contextually Based Conditional Discrimination of the Rabbit Eyeblink Response

Rabbits received conditional discrimination training using contextual stimuli to set the occasion for stimulus pairings during eyelid conditioning. Specifically, animals were exposed to either the presence or the absence of an oscillating chamber light throughout the intertrial interval (50 ± 10 s). For half the animals, this light signaled paired presentations of a discrete tone conditioned stimulus (CS) and air puff unconditioned stimulus (US) while darkness signaled presentations of only the tone CS. The remaining animals experienced the opposite contextual relationship to the conditioning stimuli. These trial types occurred pseudo-randomly across a session, with all transitions between contextual settings (i.e., light or dark) taking place immediately at the CS–US offset. Under these conditions, animals successfully utilized the contextual stimuli as conditional cues for differential responding to the shared CS. Moreover, both light and dark were equally effective as discriminative stimuli. A subset of animals received further training in which the contextual contingency was removed by restricting all conditioning to the CS-alone context. Without the contingency in place, subsequent CS presentations (paired and CS-alone) evoked equivalent conditioned responding across three sessions of training. Following the reinstatement of the contextual contingencies, discriminatory responding was immediately observed and returned to previous levels within three sessions. Finally, animals appeared to use the static representation of the conditional cue, rather than the phasic transition between cues, for discriminatory responding. These findings are discussed in terms of current neurobiological models of eyelid conditioning.     

Memories and anticipations control responding by rats (<Emphasis Type="Italic">Rattus norvegicus</Emphasis>) in a Pavlovian procedure

In Experiment 1 each rat received two different fixed series of three trials each. The unconditioned stimulus occurred on Trial 1 of one series and on Trial 3 of the other series, all other trials being nonreinforced. Previous Pavlovian investigations have shown that rats can remember the immediately prior reward outcome and anticipate the immediately subsequent reward outcome. Experiment 1 demonstrated that rats could remember and anticipate even more remote reward outcomes. In Experiment 2 two groups received a series of two nonrewarded trials followed by a rewarded trial. It was demonstrated that a change in the conditioned stimulus (CS) from Trial 2 to Trial 3, which occurred in one group, produced weaker responding than in the other group that did not experience such CS change. On the basis of these findings it was suggested that the rats organized the trials of a series into a unit or chunk. This was concluded for two reasons. First, remembering and anticipating remote reward outcomes strongly suggests that responding is being controlled by events extending beyond the current trial. Secondly, the experimental manipulations employed in the Pavlovian situation here are similar to those used in prior human learning and animal instrumental learning investigations concerned with chunking. Thus, it would appear that chunking is a ubiquitous phenomenon appearing in human serial learning (e.g., Bower and Winzenz 1969; Crowder 1976), in animal instrumental learning (e.g., Capaldi 1992; Hulse and Dorsky 1977; Terrace 1987), and now in Pavlovian learning.