Control of instrumental behavior by deprivation stimuli

In Experiment 1, rats were given one trial per day in a straight alley under food deprivation on half of the trials and under water deprivation on the other half. Wet mash was available in the goal box under food deprivation for Group H and under water deprivation for Group T, the other deprivation being nonrewarded for each group. After 15--18 trials both groups ran significantly faster on their rewarded than on their nonrewarded deprivation days. A third group showed that random variation of alley color retarded formation of the discrimination. A fourth group was run in a conditional discrimination in which under food deprivation wet mash was available in a black alley, nonreward in a white alley, and vice versa under water deprivation. This group took 114 trials to begin running significantly faster in their rewarded than in their nonrewarded alley under each deprivation. In Experiment 2, it was shown that prior learning about deprivation cues "blocks" learning about alley color when alley color is subsequently presented in compound with the deprivation cue but that when both alley color and deprivation cues are relevant from the start of training, the rat learns about both cues. It is suggested that previous studies have underestimated the importance of deprivation cues by using conditional discrimination designs, choice measures rather than speeds, and parameters that are not optimal for discrimination learning.     

Self-stimulating rats combine subjective reward magnitude and subjective reward rate multiplicatively

For rats that bar pressed for intracranial electrical stimulation in a 2-lever matching paradigm with concurrent variable interval schedules of reward, the authors found that the time allocation ratio is based on a multiplicative combination of the ratio of subjective reward magnitudes and the ratio of the rates of reward. Multiplicative combining was observed in a range covering approximately 2 orders of magnitude in the ratio of the rates of reward from about 1:10 to 10:1) and an order of magnitude change in the size of rewards. After determining the relation between the pulse frequency of stimulation and subjective reward magnitude, the authors were able to predict from knowledge of the subjective magnitudes of the rewards and the obtained relative rates of reward the subject's time allocation ratio over a range in which it varied by more than 3 orders of magnitude.     

The sensory nature of reward in instrumental behavior

Improvement in the sensory state, or “better-being,” achieved either through obtaining desirable sensations or through escaping undesirable sensations, is considered the essential value which maintains the instrumental behavior. This value may vary depending on such factors as the intensity of the stimulus producing sensations, the effects of its previous occurrence, and the changes in the internal and external environment. Methods of evaluation of sensory “better-being” by means of measuring instrumental, autonomic and EEG responses are discussed.     

The effect of informational stimuli on instrumental response rate: Signalling reward versus signalling the availability of reward

Rats were trained to press a lever for food on an interval schedule and were given a brief cue (0.5 sec) between the operative response and the reward (C condition). Some control subjects in Experiments 1, 2 and 4 were given their cue either following the end of the temporal interval during which reward had been unavailable (SD condition), or randomly with respect to food (R condition). Other control subjects in Experiments 2 and 4 received both the food-correlated cue and the temporal-interval stimulus (B condition). In all experiments, rate of responding was lowest for the C subjects and for B animals when the two cues were from different modalities. Food-correlated and temporal-interval cues did not interact, suggesting that a reward-correlated signal does not affect response rate simply by enhancing the salience of the temporal interval offset.     

Orbital and ventromedial prefrontal cortex functioning in Parkinson's disease: neuropsychological evidence

A recent paper (Zald & Andreotti, 2010) reviewed neuropsychological tasks that assess the function of the orbital and ventromedial portions of the prefrontal cortex (OMPFC). Neuropathological studies have shown that the function of the OMPFC should be preserved in the early stages of Parkinson's disease (PD) but becomes affected in the advanced stages of PD. This pattern has also been suggested by studies that have shown that dopaminergic drugs impair the performance of early PD patients in OMPFC tasks that involve reinforcement learning but enhance the performance of advanced PD patients. Based on these empirical findings, we reviewed the neuropsychological evidence of OMPFC functions in PD patients to test two hypotheses regarding the following: (1) OMPFC functions at different stages of PD; (2) different effects of dopaminergic drugs on OMPFC functions based on PD stage and task demand. We focused our review only on the neuropsychological tasks that were specific and sensitive to the functions of the OMPFC and that were adopted at different stages of PD, such as reversal learning tasks, the Iowa Gambling Task and the affective Theory of Mind task. We found robust empirical evidence that in early PD, OMPFC functions are preserved and dopaminergic drugs result in a detrimental effect when the task involves reinforcement learning. Further studies are needed to verify the status of OMPFC functions in non-demented, advanced PD and to describe the longitudinal course of OMPFC functions in this clinical population.     

Involvement of the rat anterior cingulate cortex in control of instrumental responses guided by reward expectancy

The anterior cingulate cortex (ACC) plays a critical role in stimulus-reinforcement learning and reward-guided selection of actions. Here we conducted a series of experiments to further elucidate the role of the ACC in instrumental behavior involving effort-based decision-making and instrumental learning guided by reward-predictive stimuli. In Experiment 1, rats were trained on a cost-benefit T-maze task in which they could either choose to climb a barrier to obtain a high reward (four pellets) in one arm or a low reward (two pellets) in the other with no barrier present. In line with previous studies, our data reveal that rats with quinolinic acid lesions of the ACC selected the response involving less work and smaller reward. Experiment 2 demonstrates that breaking points of instrumental performance under a progressive ratio schedule were similar in sham-lesioned and ACC-lesioned rats. Thus, lesions of the ACC did not interfere with the effort a rat is willing to expend to obtain a specific reward in this test. In a subsequent task, we examined effort-based decision-making in a lever-press task where rats had the choice between pressing a lever to receive preferred food pellets under a progressive ratio schedule, or free feeding on a less preferred food, i.e. lab chow. Results show that sham- and ACC-lesioned animals had similar breaking points and ingested comparable amounts of less-preferred food. Together, the results of Experiment 1 and 2 suggest that the ACC plays a role in evaluating how much effort to expend for reward; however, the ACC is not necessary in all situations requiring an assessment of costs and benefits. In Experiment 3 we investigated learning and reversal learning of instrumental responses guided by reward predictive stimuli. A reaction time (RT) task demanding conditioned lever release was used in which the upcoming reward magnitude (five vs. one food pellet) was signalled in advance by discriminative visual stimuli. Results revealed that rats with ACC lesions were able to discriminate reward magnitude-predictive stimuli and to adapt instrumental behavior to reversed stimulus-reward magnitude contingencies. Thus, in a simple discrimination task as used here, the ACC appears not to be required to discriminate reward magnitude-predictive stimuli and to use the learned significance of the stimuli to guide instrumental behavior.     

Transfer between downshift in reward magnitude and continuous delay of reward

Two separate experiments were conducted to investigate transfer of persistence between delay and a downshift in reward magnitude. In the first experiment, experimental rats were initially downshifted in reward magnitude and later tested for persistence to continuous delay of large reward. It was found that these rats were more persistent to the effects of delay than control rats which did not receive prior experience with a downshift in reward magnitude. In the second experiment, experimental rats were first trained to receive large reward under delayed conditions and then tested for persistence to a downshift in reward magnitude. Compared to control rats which received no prior experience with delay, the experimental rats showed a significantly smaller negative contrast effect. The results were interpreted as supporting Amsel's theory of persistence as well as Capaldi's recent interpretation of contrast effects.     

Medial prefrontal cortex activation facilitates re-extinction of fear in rats

It has been suggested that reduced infralimbic (IL) cortical activity contributes to impairments of fear extinction. We therefore explored whether pharmacological activation of the IL would facilitate extinction under conditions it normally fails (i.e., immediate extinction). Rats received auditory fear conditioning 1 h before extinction training. Immediately prior to extinction, rats received microinfusions into the IL of the GABA(A) receptor antagonist, picrotoxin, or the NMDA receptor partial agonist, D-cycloserine. Although neither drug facilitated extinction, they both facilitated the subsequent re-extinction of fear when animals were trained in a drug-free state, suggesting that activating the IL primes behavioral extinction.     

Emotional conditioning to masked stimuli and modulation of visuospatial attention

Two studies investigated the effects of conditioning to masked stimuli on visuospatial attention. During the conditioning phase, masked snakes and spiders were paired with a burst of white noise, or paired with an innocuous tone, in the conditioned stimulus (CS)+ and CS- conditions, respectively. Attentional allocation to the CSs was then assessed with a visual probe task, in which the CSs were presented unmasked (Experiment 1) or both unmasked and masked (Experiment 2), together with fear-irrelevant control stimuli (flowers and mushrooms). In Experiment 1, participants preferentially allocated attention to CS+ relative to control stimuli. Experiment 2 suggested that this attentional bias depended on the perceived aversiveness of the unconditioned stimulus and did not require conscious recognition of the CSs during both acquisition and expression.     

Order of partial and consistent reward,reward magnitude shift,and resistance to extinction

In a separate-phase runway experiment with rats, four schedules involving partial (P) and consistent (C) reward (CC, CP, PC, and PP) were crossed with three reward magnitude shift conditions (upshift, nonshift, and downshift). The data revealed three major findings: (a) Reward magnitude downshift generally led to rapid extinction; (b) consistent reward prior to partial reward (CP) resulted in slower extinction than the reverse order (PC) under conditions of reward magnitude shift (particularly downshift); and (c) the relative performance of PC and CP under conditions of reward magnitude shift was reversed from postshift to extinction. On the basis of these data it was suggested that processes not presently identified by reinforcement level theory and stimulus analyzer theory influence extinction following separate-phase acquisition. A modification of reward level theory was presented to provide an account of extinction performance following separate-phase reward reduction.     

Some conditions for the dissociation of consummatory and instrumental behavior in rats

To investigate whether consummatory and instrumental behavior depend upon different motivational factors, rats were trained to press a bar for saccharin solution, then given various treatments designed to reduce the palatability of saccharin, and finally tested for bar-pressing in extinction and for free intake of saccharin. Prefeeding with dextrose, prefeeding with saccharin, and association of saccharin with injections of lithium chloride all reduced intake of saccharin compared to control treatments, but only prefeeding with dextrose also reduced barpressing in extinction. Thus, performance of an instrumental response may depend upon need for food rather than appetite for it.     

Neural coding of reward magnitude in the orbitofrontal cortex of the rat during a five-odor olfactory discrimination task

The orbitofrontal cortex (OBFc) has been suggested to code the motivational value of environmental stimuli and to use this information for the flexible guidance of goal-directed behavior. To examine whether information regarding reward prediction is quantitatively represented in the rat OBFc, neural activity was recorded during an olfactory discrimination “go”/“no-go” task in which five different odor stimuli were predictive for various amounts of reward or an aversive reinforcer. Neural correlates related to both actual and expected reward magnitude were observed. Responses related to reward expectation occurred during the execution of the behavioral response toward the reward site and within a waiting period prior to reinforcement delivery. About one-half of these neurons demonstrated differential firing toward the different reward sizes. These data provide new and strong evidence that reward expectancy, regardless of reward magnitude, is coded by neurons of the rat OBFc, and are indicative for representation of quantitative information concerning expected reward. Moreover, neural correlates of reward expectancy appear to be distributed across both motor and nonmotor phases of the task.     

The impact of orbital prefrontal cortex damage on emotional activation to unanticipated and anticipated acoustic startle stimuli

Damage to the orbital prefrontal cortex has been implicated in selectively diminishing electrodermal autonomic nervous system responses to anticipated punishing stimuli (e.g., losing money; Bechara, Damasio, & Damasio, 2000), but not to unanticipated punishing stimuli (e.g., loud noises; Damasio, Tranel, & Damasio, 1990). We extended this research by examining the effects of orbitofrontal damage on emotional responses to unanticipated and anticipated acoustic startles and collecting a more extensive set of physiological measures, emotional facial behavior, and self-reported emotional experience. Consistent with previous research, patients showed intact physiology to an unanticipated startle but failed to show appropriate anticipatory cardiovascular responses (patients’ heart rates decreased, controls’ increased). In addition, patients displayed more surprise facial behavior and reported marginally more fear than did controls in response to the unanticipated startle. Thus, orbitofrontal damage may compromise the ability to anticipate physiologically the onset of aversive stimuli, despite intact or enhanced emotional responses when such stimuli occur unexpectedly.     

Effects of rank ordering stimuli on magnitude ratings of these and other stimuli

Five experiments investigated the effects of rank ordering stimuli on subsequent magnitude ratings of these and other stimuli. In Experiments 1 and 2, subjects first rank ordered environmental issues in terms of their importance. Ranking stimuli from “most” to “least” led to more extreme ratings than ranking them from “least” to “most,” regardless of whether the rating criterion was the same as or diametrically opposite to the ranking criterion. (For example, subjects who had previously ranked them beginning with the most important issue subsequently rated these issues not only as more important, but also as more trivial, than did subjects who had ranked them beginning with the least important.) These effects generalized to stimuli other than those that had previously been ranked, and generalized over stimulus domains. (For example, ratings of environmental issues were also affected by ranking the importance of attributes of a marriage partner.) Other experiments in the series circumscribed the conditions in which these effects occur. Results suggested that rank ordering stimuli leads subjects to adopt comparative standards, the use of which generalizes to subsequent magnitude rating tasks and produces an anchoring bias similar to that identified by A. Tversky and D. Kahneman (1974, Science (Washington, D.C.), 185, 1124–1131). Implications of these results for the cognitive processes that underlie social judgment are discussed.