The effects of hippocampal lesions on learning, memory, and reward expectancies

The hippocampus appears to be critical for the formation of certain types of memories. Hippocampal-lesioned animals fail to exhibit some spatial, contextual, and relational associations. After aspiration lesions of the hippocampus and/or cortex, male rats were allowed to recover for three weeks before being trained on a matching-to-position task. The matching-to-position task was altered to influence the type of cognitive strategies a subject would use to solve the task. The main behavioral manipulation was the reinforcement contingency assignment: Use of a differential outcomes procedure (DOP) or a nondifferential outcomes procedure (NOP). The DOP involves correlating each to-be-remembered event with a distinct reward condition via Pavlovian trace conditioning, whereas the NOP results in random reward contingency. We found that hippocampal lesions did retard learning the matching rule, regardless of the reinforcement contingency assignment. However, when delay intervals were added to the task memory performance of subjects with hippocampal lesions was dramatically impaired--if subjects were not trained with the DOP. When subjects were trained with the DOP, the hippocampal lesion had a marginal effect on delayed memory performance. These findings demonstrate two important points regarding lesions of the hippocampus: (1) hippocampal lesions have a minimal effect on the on the ability of rats to use reward information to solve a delayed discrimination task; (2) rats with hippocampal lesions have the ability to learn about reward information using Pavlovian trace conditioning procedures.     

In search of the neurobiological underpinnings of the differential outcomes effect

Correlating unique rewards with to-be-remembered events (the Differential Outcomes Procedure [DOP]) enhances learning and memory performance in a range of species. Recently, we have demonstrated that the DOP can be used to reduce or eliminate the learning and memory impairments associated with animal models of amnesia and dementia. This powerful phenomenon, the Differential Outcomes Effect (DOE), has led to the question: How does such a simple manipulation exert such dramatic influence on learning and memory performance? A revised two-process account of the DOE states that using the DOP results in the activation of reward expectancies through Pavlovian mechanisms. The use of unique reward expectancies alters the nature of cognitive processing used to solve discrimination tasks. The change in cognitive processing is represented by utilization of a different memory system than that commonly used to acquire and remenber information when a Nondifferential Outcomes Procedure (NOP) is used. Using neurochemical manipulations, it has been demonstrated that different, potentially independent, brain systems modulate memory performance when subjects are trained with a NOP versus a DOP. This memory-based DOP/NOP distinction resembles other dissociative memory theories in which two psychological processes are purportedly served by distinct neurobiological mechanisms. In addition, such results have important ramifications for the treatment of memory disorders because they demonstrate that stimulus and behavioral manipulations, like drugs, can influence neurotransmitter functioning.     

In Search of the Neurobiological Underpinnings of the Differential Outcomes Effect

Correlating unique rewards with to-be-remembered events (the Differential Outcomes Procedure [DOP]) enhances learning and memory performance in a range of species. Recently, we have demonstrated that the DOP can be used to reduce or eliminate the learning and memory impairments associated with animal models of amnesia and dementia. This powerful phenomenon, the Differential Outcomes Effect (DOE), has led to the question: How does such a simple manipulation exert such dramatic influence on learning and memory performance? A revised two-process account of the DOE states that using the DOP results in the activation of reward expectancies through Pavlovian mechanisms. The use of unique reward expectancies alters the nature of cognitive processing used to solve discrimination tasks. The change in cognitive processing is represented by utilization of a different memory system than that commonly used to acquire and remember information when a Nondifferential Outcomes Procedure (NOP) is used. Using neurochemical manipulations, it has been demonstrated that different, potentially independent, brain systems modulate memory performance when subjects are trained with a NOP versus a DOP. This memory-based DOP/NOP distinction resembles other dissociative memory theories in which two psychological processes are purportedly served by distinct neurobiological mechanisms. In addition, such results have important ramifications for the treatment of memory disorders because they demonstrate that stimulus and behavioral manipulations, like drugs, can influence neurotransmitter functioning.     

The differential outcomes procedure can overcome self-bias in perceptual matching

There are biases in perceptual matching between shapes and labels referring to familiar others, compared with when the labels refer to unfamiliar people. We assessed whether these biases could be affected by differential feedback (using the differential outcomes procedure [DOP]) compared with when feedback is provided using a nondifferential outcomes procedure (NOP). Participants formed associations between simple geometric shapes and labels referring to people the participant did or did not know (self, best friend, other). Subsequently, the task was to match a label to one of two shapes shown on a trial. When feedback for correct responses was given following the NOP condition, matches were faster to known people (self and friend) compared with those to an unknown person (stranger). However, this advantage for known personal relations was eliminated when participants were given feedback for correct responses following the DOP condition. The data are consistent with prior work showing that the DOP can facilitate the learning of taxing associations (for the stranger stimuli relative to the familiar self and friend stimuli). In addition, the results suggest that the facilitated perceptual matching for stimuli associated to individuals known personally may reflect better individuation of the association between the shape stimulus and the label, a process enhanced by using a DOP for associations with unfamiliar people.     

Improving conditional discrimination learning and memory in five-year-old children: Differential outcomes effect using different types of reinforcement

Previous studies have demonstrated that discriminative learning is facilitated when a particular outcome is associated with each relation to be learned. When this training procedure is applied (the differential outcomes procedure; DOP), learning is faster and better than when the typical common outcomes procedure or nondifferential outcomes (NDO) is used. Our primary purpose in the two experiments reported here was to assess the potential advantage of DOP in 5-year-old children using three different strategies of reinforcement in which (a) children received a reinforcer following a correct choice (“ + ”), (b) children lost a reinforcer following an incorrect choice (“ ? ”), or (c) children received a reinforcer following a correct choice and lost one following an incorrect choice (“ + / ? ”). In Experiment 1, we evaluated the effects of the presence of DOP and different types of reinforcement on learning and memory of a symbolic delayed matching-to-sample task using secondary and primary reinforcers. Experiment 2 was similar to the previous one except that only primary reinforcers were used. The results from these experiments indicated that, in general, children learned the task faster and showed higher performance and persistence of learning whenever differential outcomes were arranged independent of whether it was differential gain, loss, or combinations. A novel finding was that they performed the task better when they lost a reinforcer following an incorrect choice (type of training “ ? ”) in both experiments. A further novel finding was that the advantage of the DOP over the nondifferential outcomes training increased in a retention test.     

The involvement of the orbitofrontal cortex in learning under changing task contingencies

Previous investigations examining the rat prefrontal cortex subregions in attentional-set shifting have commonly employed two-choice discriminations. To better understand how varying levels of difficulty influence the contribution of the prefrontal cortex to learning, the present studies examined the effects of orbitofrontal cortex inactivation in a two- or four-choice odor reversal learning test. Long-Evans rats were trained to dig in cups that contained distinct odors. In the two-choice odor discrimination, one odor cup was always associated with a cereal reinforcement in acquisition while the opposite odor cup was associated with a cereal reinforcement in reversal learning. In the four-choice odor discrimination, an additional two cups containing distinct odors were used that were never associated with reinforcement in acquisition or reversal learning. Bilateral infusions of the GABA-A agonist, muscimol (0.5 microg) into the orbitofrontal cortex did not impair acquisition of either the two- or four-choice discrimination task. However, muscimol infusions into the orbitofrontal cortex impaired two- and four-choice reversal learning. In the two-choice odor reversal, muscimol treatment selectively increased perseverative errors. In the four-choice odor reversal, muscimol treatment increased perseverative, regressive, as well as irrelevant errors. These findings suggest that the orbital prefrontal cortex not only enables task switching by supporting the initial inhibition of a previously relevant choice pattern, but under increasing task demands also enables the reliable execution of a new choice pattern and reduction of interference to irrelevant stimuli.     

Escalation of irritable aggression: Control by consequences and antecedents

Two experiments demonstrated that rats could be trained in a negative reinforcement paradigm to display a shock-induced aggressive response on the first shock presented. Later, rats that had been submitted to the negative reinforcement training procedure displayed more shock-induced aggression than did control groups during a test session that was situationally different from the one used during training. A third experiment demonstrated that noxious antecedent events, if presented with sufficient rapidity, can combine to increase the probability of aggressive behavior. The three experiments together suggest that aversive antecedents and reinforcement contingencies could be involved in the escalation of irritable aggression.     

Orbital prefrontal cortex and guidance of instrumental behavior of rats by visuospatial stimuli predicting reward magnitude

The orbital prefrontal cortex (OPFC) is part of a circuitry mediating the perception of reward and the initiation of adaptive behavioral responses. We investigated whether the OPFC is involved in guidance of the speed of instrumental behavior by visuospatial stimuli predictive of different reward magnitudes. Unoperated rats, sham-lesioned rats, and rats with bilateral lesions of the OPFC by N-methyl-D-aspartate (NMDA) were trained in a visuospatial discrimination task. The task required a lever press on the illuminated lever of two available to obtain a food reward. Different reward magnitudes were permanently assigned to lever presses to respective sides of the operant chamber; that is, responses to one lever (e.g., the left one) were always rewarded with one pellet and responses to the other lever with five pellets. On each trial, the position of the illuminated lever was pseudorandomly determined in advance. Results revealed that OPFC lesions did not impair acquisition of the task, as the speed of conditioned responses was significantly shorter with expectancy of a high reward magnitude. In addition, during reversal, shift and reshift of lever position–reward magnitude contingencies and under extinction conditions, performance of the OPFC-lesioned and control groups did not differ. It is concluded that the OPFC in rats might not be critical for adapting behavioral responses to changes of stimulus–reward magnitude contingencies signaled by visuospatial cues.     

The differential outcomes effect (DOE) in spatial localization: An investigation with adults

We investigated whether search accuracy of adult humans could be enhanced using differential reward contingencies in landmark-based spatial tasks conducted on a computer screen. We found that search accuracy was significantly enhanced by differential outcomes in a conditional spatial search task, in which the landmark-to-goal relationship depended on a previously presented sample object (Experiment 4). In contrast, no significant differential outcomes effect (DOE) was seen in several other variations of spatial search tasks. We interpret the pattern of significant and non-significant results in terms of the information value of outcome expectancies. To our knowledge this is the first report of a DOE in a landmark-based spatial localization task and is one of only a few demonstrations that differential outcomes can enhance memory performance in normal functioning adults.     

Histidine-decarboxylase knockout mice show deficient nonreinforced episodic object memory, improved negatively reinforced water-maze performance, and increased neo- and ventro-striatal dopamine turnover

The brain's histaminergic system has been implicated in hippocampal synaptic plasticity, learning, and memory, as well as brain reward and reinforcement. Our past pharmacological and lesion studies indicated that the brain's histamine system exerts inhibitory effects on the brain's reinforcement respective reward system reciprocal to mesolimbic dopamine systems, thereby modulating learning and memory performance. Given the close functional relationship between brain reinforcement and memory processes, the total disruption of brain histamine synthesis via genetic disruption of its synthesizing enzyme, histidine decarboxylase (HDC), in the mouse might have differential effects on learning dependent on the task-inherent reinforcement contingencies. Here, we investigated the effects of an HDC gene disruption in the mouse in a nonreinforced object exploration task and a negatively reinforced water-maze task as well as on neo- and ventro-striatal dopamine systems known to be involved in brain reward and reinforcement. Histidine decarboxylase knockout (HDC-KO) mice had higher dihydrophenylacetic acid concentrations and a higher dihydrophenylacetic acid/dopamine ratio in the neostriatum. In the ventral striatum, dihydrophenylacetic acid/dopamine and 3-methoxytyramine/dopamine ratios were higher in HDC-KO mice. Furthermore, the HDC-KO mice showed improved water-maze performance during both hidden and cued platform tasks, but deficient object discrimination based on temporal relationships. Our data imply that disruption of brain histamine synthesis can have both memory promoting and suppressive effects via distinct and independent mechanisms and further indicate that these opposed effects are related to the task-inherent reinforcement contingencies.     

The roles of the medial prefrontal cortex and hippocampus in a spatial paired-association task

Although the roles of both the hippocampus and the medial prefrontal cortex (mPFC) have been suggested in a spatial paired-associate memory task, both areas were investigated separately in prior studies. The current study investigated the relative contributions of the hippocampus and mPFC to spatial paired-associate learning within a single behavioral paradigm. In a novel behavioral task, a pair of different objects appeared repeatedly across trials, but in different arms in a radial maze, and different rules were associated with those arms for reward. Specifically, in an "object-in-place" arm, the rat was required to choose a particular object associated with the arm. In a "location-in-place" arm, the animal was required to choose a certain within-arm location (ignoring the object occupying the location). Compared to normal animals, rats with ibotenic acid-based lesions in the hippocampus showed an irrecoverable impairment in performance in both object-in-place and location-in-place arms. When the mPFC was inactivated by muscimol (GABA(A) receptor agonist) in the normal animals with intact hippocampi, they showed the same severe impairment as seen in the hippocampal lesioned rats only in object-in-place arms. The results confirm that the hippocampus is necessary for a biconditional paired-associate task when space is a critical component. The mPFC, however, is more selectively involved in the object-place paired-associate task than in the location-place paired-associate task. The current task powerfully demonstrates an experimental situation in which both the hippocampus and mPFC are required and may serve as a useful paradigm for investigating the neural mechanisms of object-place association.     

Visual perceptual learning by operant conditioning training follows rules of contingency

Visual perceptual learning (VPL) can occur as a result of a repetitive stimulus-reward pairing in the absence of any task. This suggests that rules that guide Conditioning, such as stimulus-reward contingency (e.g., that stimulus predicts the likelihood of reward), may also guide the formation of VPL. To address this question, we trained subjects with an operant conditioning task in which there were contingencies between the response to one of three orientations and the presence of reward. Results showed that VPL only occurred for positive contingencies, but not for neutral or negative contingencies. These results suggest that the formation of VPL is influenced by similar rules that guide the process of Conditioning.     

The effects of negative reinforcement for irritable aggression on resident-intruder behavior

This experiment demonstrated that rats trained to display elevated levels of shock-induced aggression in a negative reinforcement paradigm displayed more boxing behavior than yoked control groups in a later test in which intruder rats were placed in the home cage of resident rats. Resident or intruder status did not affect the influence of the negative reinforcement procedure on the observed resident-intruder behavior of trained animals; however, naive intruders paired with trained residents displayed increased defensive behavior, suggesting that negative reinforcement for shock-induced aggression affected the behavior of these residents.     

The Differential Outcomes Procedure Can Interfere or Enhance Operant Rule Learning

The differential outcomes effect--the enhancement of learning and memory performance by correlating distinct reinforcers with to-be-remembered events (sample stimuli)--has been stated to be one of the most robust phenomena in learning psychology. However, in this paper we demonstrate that the correlation between unique samples and unique reinforcers can either interfere with or enhance learning a spatial matching-rule, dependent on whether these two processes are trained concurrently or sequentially. If the Pavlovian conditioning (unique sample-reward pairings) occurs before the matching rule is learned (sequentially), the conditioned expectations of unique rewards will enhance the acquisition of the spatial matching-rule in rats (the differential outcomes effect will be observed). However, if rats are required to learn the Pavlovian associations and the matching-rule concurrently, they are impaired in acquiring the spatial matching-rule. Thus, employing the differential outcomes procedure can either enhance or detract from learning and remembering the task rule-dependent on the nature of the task and order of training. These data suggest that under some circumstances learning Pavlovian associations can compete with the formation of instrumental behavior.     

Rat hippocampus and memory for places of changing significance

Rats were tested once daily on a four-choice delayed match from sample task with a water reward. Each day the correct place changed, and a single exposure to it was provided on information trials. Lesions of the hippocampal formation that involved the fornix, or dorsal hippocampus bilaterally, produced a severe impairment in the performance of previously trained rats. By contrast, lesions of the ventral hippocampus did not preclude reacquisition of the place-memory task. Some otherwise impaired rats with fornical lesions were able to find the water when aided by nonplace cues that consistently signaled reward. Reducing the number of choices from four to two did not aid the impaired rats. Certain lesions of the hippocampal formation in the rat produce a deficit appropriately described as amnesia. The memory deficit is consistent with a role for the hippocampus in processing of place information and shows some parallels to the amnesia seen in persons with temporal lobe lesions.     

Muscimol Induces Retrograde Amnesia for Changes in Reward Magnitude

These experiments examined the effect of the GABA_A, agonist, muscimol (MUS), on memory for changes in reward magnitude. In Experiment 1 rats were trained to run a straight alley for either a large or small food reward. After reaching asymptotic performance rats in the high reward group were shifted to the small food reward. Half the animals received 1.0 or 3.0 mg/kg (ip) of MUS or the equivalent volume of saline immediately after training. Shifted training continued for 3 more days and no further injections were given. Shifted saline animals displayed an increase in response latencies compared to unshifted controls with a sharp peak on the day after the shift. Shifted MUS receiving 1.0 mg/kg performed comparably to shifted saline animals. In contrast, shifted MUS animals receiving 3.0 mg/kg displayed performance comparable to shifted saline animals on the day of the shift but displayed a sharp increase in response latencies on the second day after the shift. These findings indicate that post-training systemic MUS injections delay the peak increase in response latencies and suggest that MUS induces retrograde amnesia for reward reduction. Experiment 2 examined the effect of MUS on the memory of a reward increase. Rats were first trained as in Experiment 1 and rats under the high reward condition were then shifted to the small reward. On the next training session, the large food reward was reinstated. Immediately after the session all animals were injected with saline or 3.0 mg/kg of MUS. The large food reward was continued for the remainder of training and no further injections were given. On the following session, the performance of the shifted saline animals was comparable to that of the unshifted controls while shifted MUS animals displayed significantly higher response latencies. The findings that MUS prevented the reduction in response latencies seen in saline-injected animals suggest that MUS also induces retrograde amnesia for reward increases.     

The differential outcomes procedure can interfere or enhance operant rule learning

The differential outcomes effect—the enhancement of learning and memory performance by correlating distinct reinforcers with to-be-remembered events (sample stimuli)—has been stated to be one of the most robust phenomena in learning psychology. However, in this paper we demonstrate that the correlation between unique samples and unique reinforcers can either interfere with or enhance learning a spatial matching-rule, dependent on whether these two processes are trained concurrently or sequentially. If the Pavlovian conditioning (unique sample-reward pairings) occurs before the matching rule is learned (sequentially), the conditioned expectations of unique rewards will enhance the acquisition of the spatial matching-rule in rats (the differential outcomes effect will be observed). However, if rats are required to learn the Pavlovian associations and the matching-rule concurrently, they are impaired in acquiring the spatial matching-rule. Thus, employing the differential outcomes procedure can either enhance or detract from learning and remembering the task rule—dependent on the nature of the task and order of training. These data suggest that under some circumstances learning Pavlovian associations can compete with the formation of instrumental behavior.     

Involvement of the rat anterior cingulate cortex in control of instrumental responses guided by reward expectancy

The anterior cingulate cortex (ACC) plays a critical role in stimulus-reinforcement learning and reward-guided selection of actions. Here we conducted a series of experiments to further elucidate the role of the ACC in instrumental behavior involving effort-based decision-making and instrumental learning guided by reward-predictive stimuli. In Experiment 1, rats were trained on a cost-benefit T-maze task in which they could either choose to climb a barrier to obtain a high reward (four pellets) in one arm or a low reward (two pellets) in the other with no barrier present. In line with previous studies, our data reveal that rats with quinolinic acid lesions of the ACC selected the response involving less work and smaller reward. Experiment 2 demonstrates that breaking points of instrumental performance under a progressive ratio schedule were similar in sham-lesioned and ACC-lesioned rats. Thus, lesions of the ACC did not interfere with the effort a rat is willing to expend to obtain a specific reward in this test. In a subsequent task, we examined effort-based decision-making in a lever-press task where rats had the choice between pressing a lever to receive preferred food pellets under a progressive ratio schedule, or free feeding on a less preferred food, i.e. lab chow. Results show that sham- and ACC-lesioned animals had similar breaking points and ingested comparable amounts of less-preferred food. Together, the results of Experiment 1 and 2 suggest that the ACC plays a role in evaluating how much effort to expend for reward; however, the ACC is not necessary in all situations requiring an assessment of costs and benefits. In Experiment 3 we investigated learning and reversal learning of instrumental responses guided by reward predictive stimuli. A reaction time (RT) task demanding conditioned lever release was used in which the upcoming reward magnitude (five vs. one food pellet) was signalled in advance by discriminative visual stimuli. Results revealed that rats with ACC lesions were able to discriminate reward magnitude-predictive stimuli and to adapt instrumental behavior to reversed stimulus-reward magnitude contingencies. Thus, in a simple discrimination task as used here, the ACC appears not to be required to discriminate reward magnitude-predictive stimuli and to use the learned significance of the stimuli to guide instrumental behavior.     

Enhancing recognition memory in adults through differential outcomes

It has been widely demonstrated that the differential outcomes procedure (DOP) facilitates both the learning of conditional relationships and the memory for the conditional stimuli in animal subjects. For conditional discriminations in humans, the DOP also produces an increase in the speed of acquisition and/or final accuracy. However, the potential facilitative effects of differential outcomes in human memory have not been fully assessed. In the present study, we aimed to test whether this procedure improves performance on a recognition memory task in healthy adults. Participants showed significantly better delayed face recognition when differential outcomes were used. This novel finding is discussed in the light of other studies on the differential outcomes effect (DOE) in both animals and humans, and implications for future research are presented.     

Matching- and nonmatching-to-sample concept learning in rats using olfactory stimuli

Previous research has shown that rats can learn matching-to-sample relations with olfactory stimuli; however, the specific characteristics of this relational control are unclear. In Experiment 1, 6 rats were trained to either match or nonmatch to sample in a modified operant chamber using common household spices as olfactory stimuli. After matching or nonmatching training with 10 exemplars, the contingencies were reversed with five new stimuli such that subjects trained on matching were shifted to nonmatching and vice versa. Following these reversed contingencies, the effects of the original training persisted for many trials with new exemplars. In Experiment 2, 9 rats were trained with matching procedures in an arena that provided for 18 different spatial locations for comparison stimuli. Five subjects were trained with differential reinforcement outcomes and 4 with only one type of reinforcer. Differential outcomes and multiple exemplars facilitated learning, and there was strong evidence for generalization to new stimuli for most rats that acquired several conditional discriminations. Performances with novel samples were generally above chance, but rarely reached the high levels obtained during baseline with well-trained stimulus relations. However, taken together, the data from the two experiments extend previous work, show that rats can learn both match and nonmatch relations with different experimental protocols, and demonstrate generalization to novel sample stimuli.