Hippocampus, cortex, and basal ganglia: insights from computational models of complementary learning systems

We present a framework for understanding how the hippocampus, neocortex, and basal ganglia work together to support cognitive and behavioral function in the mammalian brain. This framework is based on computational tradeoffs that arise in neural network models, where achieving one type of learning function requires very different parameters from those necessary to achieve another form of learning. For example, we dissociate the hippocampus from cortex with respect to general levels of activity, learning rate, and level of overlap between activation patterns. Similarly, the frontal cortex and associated basal ganglia system have important neural specializations not required of the posterior cortex system. Taken together, this overall cognitive architecture, which has been implemented in functioning computational models, provides a rich and often subtle means of explaining a wide range of behavioral and cognitive neuroscience data. Here, we summarize recent results in the domains of recognition memory, contextual fear conditioning, effects of basal ganglia lesions on stimulus-response and place learning, and flexible responding.     

Generalization through the recurrent interaction of episodic memories: a model of the hippocampal system

In this article, we present a perspective on the role of the hippocampal system in generalization, instantiated in a computational model called REMERGE (recurrency and episodic memory results in generalization). We expose a fundamental, but neglected, tension between prevailing computational theories that emphasize the function of the hippocampus in pattern separation (Marr, 1971; McClelland, McNaughton, & O'Reilly, 1995), and empirical support for its role in generalization and flexible relational memory (Cohen & Eichenbaum, 1993; Eichenbaum, 1999). Our account provides a means by which to resolve this conflict, by demonstrating that the basic representational scheme envisioned by complementary learning systems theory (McClelland et al., 1995), which relies upon orthogonalized codes in the hippocampus, is compatible with efficient generalization-as long as there is recurrence rather than unidirectional flow within the hippocampal circuit or, more widely, between the hippocampus and neocortex. We propose that recurrent similarity computation, a process that facilitates the discovery of higher-order relationships between a set of related experiences, expands the scope of classical exemplar-based models of memory (e.g., Nosofsky, 1984) and allows the hippocampus to support generalization through interactions that unfold within a dynamically created memory space.     

Conjunctive representations in learning and memory: principles of cortical and hippocampal function

The authors present a theoretical framework for understanding the roles of the hippocampus and neocortex in learning and memory. This framework incorporates a theme found in many theories of hippocampal function: that the hippocampus is responsible for developing conjunctive representations binding together stimulus elements into a unitary representation that can later be recalled from partial input cues. This idea is contradicted by the fact that hippocampally lesioned rats can learn nonlinear discrimination problems that require conjunctive representations. The authors' framework accommodates this finding by establishing a principled division of labor, where the cortex is responsible for slow learning that integrates over multiple experiences to extract generalities whereas the hippocampus performs rapid learning of the arbitrary contents of individual experiences. This framework suggests that tasks involving rapid, incidental conjunctive learning are better tests of hippocampal function. The authors implement this framework in a computational neural network model and show that it can account for a wide range of data in animal learning.     

Integrating incremental learning and episodic memory models of the hippocampal region

By integrating previous computational models of corticohippocampal function, the authors develop and test a unified theory of the neural substrates of familiarity, recollection, and classical conditioning. This approach integrates models from 2 traditions of hippocampal modeling, those of episodic memory and incremental learning, by drawing on an earlier mathematical model of conditioning, SOP (A. Wagner, 1981). The model describes how a familiarity signal may arise from parahippocampal cortices, giving a novel explanation for the finding that the neural response to a stimulus in these regions decreases with increasing stimulus familiarity. Recollection is ascribed to the hippocampus proper. It is shown how the properties of episodic representations in the neocortex, parahippocampal gyrus, and hippocampus proper may explain phenomena in classical conditioning. The model reproduces the effects of hippocampal, septal, and broad hippocampal region lesions on contextual modulation of classical conditioning, blocking, learned irrelevance, and latent inhibition.     

Episodic memory and the hippocampus: it's about time

Several recent studies have sought to develop animal models of episodic memory, the capacity to recollect unique personal experiences. However, these studies have not yet provided unambiguous evidence that this capacity is based on recollection of the learning episodes. A recent study that examined memory for the ordering of events within unique experiences, and demonstrated a critical and selective role for the hippocampus, suggests a new and promising model for neurobiological analyses of episodic memory.     

Dissociating Basal Forebrain and Medial Temporal Amnesic Syndromes: Insights from Classical Conditioning

In humans, anterograde amnesia can result from damage to the medial temporal (MT) lobes (including hippocampus), as well as to other brain areas such as basal forebrain. Results from animal classical conditioning studies suggest that there may be qualitative differences in the memory impairment following MT vs. basal forebrain damage. Specifically, delay eyeblink conditioning is spared after MT damage in animals and humans, but impaired in animals with basal forebrain damage. Recently, we have likewise shown delay eyeblink conditioning impairment in humans with amnesia following anterior communicating artery (ACoA) aneurysm rupture, which damages the basal forebrain. Another associative learning task, a computer-based concurrent visual discrimination, also appears to be spared in MT amnesia while ACoA amnesics are slower to learn the discriminations. Conversely, animal and computational models suggest that, even though MT amnesics may learn quickly, they may learn qualitatively differently from controls, and these differences may result in impaired transfer when familiar information is presented in novel combinations. Our initial data suggests such a two-phase learning and transfer task may provide a double dissociation between MT amnesics (spared initial learning but impaired transfer) and ACoA amnesics (slow initial learning but spared transfer). Together, these emerging data suggest that there are subtle but dissociable differences in the amnesic syndrome following damage to the MT lobes vs. basal forebrain, and that these differences may be most visible in non-declarative tasks such as eyeblink classical conditioning and simple associative learning.     

Dissociating basal forebrain and medial temporal amnesic syndromes: Insights from classical conditioning

In humans, anterograde amnesia can result from damage to the medical temporal (MT) lobes (including hippocampus), as well as to other brain areas such as basal forebrain. Results from animal classical conditioning studies suggest that there may be qualitative differences in the memory impairment following MT vs. basal forebrain damage. Specifically, delay eyeblink conditioning is spared after MT damage in animals and humans, but impaired in animals with basal forebrain damage. Recently, we have likewise shown delay eyeblink conditioning impairment in humans with amnesia following anterior communicating artery (ACoA) aneurysm rupture, which damages the basal forebrain. Another associative learning task, a computer-based concurrent visual discrimination, also appears to be spared in MT amnesia while ACoA amnesics are slower to learn the discriminations. Conversely, animal and computational models suggest that, even though MT amnesics may learn quickly, they may learn qualitatively differently from controls, and these differences may result in impaired transfer when familiar information is presented in novel combinations. Our initial data suggests such a two-phase learning and transfer task may provide a double dissociation between MT amnesics (spared initial learning but impaired transfer) and ACoA amnesics (slow initial learning but spared transfer). Together, these merging data suggest that there are subtle but dissociable differences in the amnesic syndrome following damage to the MT lobes vs. basal forebrain, and that these differences may be most visible in non-declarative tasks such as eyeblink classical conditioning and simple associative learning.     

颞叶梗塞致学习记忆障碍的机制研究──微透析分析与超微结构观察

应用光化学的方法建立了颞叶梗塞性学习记忆障碍动物模型。海马区神经介质类氨基酸的变化用微透析技术进行活体动态观测、超微结构改变用ＴＥＭ电镜观察,并同时观察光镜下的神经细胞变化及鼠的学习记忆障碍。结果发现：（１）颞叶梗塞性痴呆鼠其海马区有大量兴奋性和抑制性氨基酸释放;（２）痴呆鼠海马区神经突触有大量囊泡聚集,晚期大小不均;（３）痴呆鼠脑新皮层及海马锥体细胞均有选择性坏死;（４）氨基酸的释放量与选择性神经细胞损伤及学习记忆障碍有关。     

Neurogenesis interferes with the retrieval of remote memories: Forgetting in neurocomputational terms

In contrast to models and theories that relate adult neurogenesis with the processes of learning and memory, almost no solid hypotheses have been formulated that involve a possible neurocomputational influence of adult neurogenesis on forgetting. Based on data from a previous study that implemented a simple but complete model of the main hippocampal circuitry (Weisz & Argibay, 2009), we now test this model under different situations to better study the case of remote memories. The results of this work show that following neurogenesis, the new, ongoing memories in the hippocampus are better retained than when no neurogenesis occurs at all, while the older memories are affected (to a lesser extent) by a special type of interference that is different from interference that occurs with an increasing number of memories per se. This work adds a new point of analysis in support of the interference view that might lead to the forgetting of memories in the hippocampus as they are transferred to neocortex for long-term storage, consistent with the Complementary Learning Systems models of system-level consolidation. Attention should be directed to the specific causes of interference; the results of this work signal a type of distortion of remote memories that is produced by the birth and the growth of new processing units, which results in a subtly impoverished retrieval as new neurons become active. The proposals of this model fit well with some empirical findings that are related to the issue. In the future, as new evidence emerges, we believe that this biological process, which is largely related to learning and memory, will also help to shape our ideas about normal forgetting and its possible contributions to system consolidation.     

The neurobiology of learning and memory

The study of memory is a great challenge, perhaps the greatest in biological sciences. Memory involves changes in a tiny fraction of an extremely large pool of elements, a conclusion that makes the task of finding those changes using current technologies formidable. What can be done about this roadblock to neurological investigations of learning? One response that has become particularly productive in recent years is to study learning or learning-like phenomena in relatively simple "model" systems. The idea is to extract basic principles from these models in which molecular and anatomical details can be studied and then to use these in analyzing learning in higher regions of the brain. In this article we discuss current progress and emerging concepts derived from the simple system approach using animal models.     

Towards learning agents with personality traits: Modeling Openness to Experience

Recent advances in neurosciences and cognitive sciences show us that the human neocortex is not a slave to the experiences from our perception and that the memories stored in hippocampus are goal weighted during the replay of the experiences for the purpose of re-learning from them. Temporal difference reinforcement learning systems that use neural networks as function approximators rely on an experience replay memory structure similar to the hippocampus. We bring forward this similarity and present a novel way of using a goal weighted prioritization of the memory that is biologically inspired. Furthermore, we introduce a novel prioritization criteria called Variety of Experience Index, or VEI, for weighting the selection of the experiences that are stored in the replay memory. Weighting the experiences based on two different extremes of VEI can behaviourally modify the agent’s learning process, generating different types of learning agents that exhibit different personality traits along the dimension of Openness to Experience.     

Endogenous BDNF is required for long-term memory formation in the rat parietal cortex

Information storage in the brain is a temporally graded process involving different memory phases as well as different structures in the mammalian brain. Cortical plasticity seems to be essential to store stable long-term memories, although little information is available at the moment regarding molecular and cellular events supporting memory consolidation in the neocortex. Brain-derived neurotrophic factor (BDNF) modulates both short-term synaptic function and activity-dependent synaptic plasticity in hippocampal and cortical neurons. We have recently demonstrated that endogenous BDNF in the hippocampus is involved in memory formation. Here we examined the role of BDNF in the parietal cortex (PCx) in short-term (STM) and long-term memory (LTM) formation of a one-trial fear-motivated learning task in rats. Bilateral infusions of function-blocking anti-BDNF antibody into the PCx impaired both STM and LTM retention scores and decreased the phosphorylation state of cAMP response element-binding protein (CREB). In contrast, intracortical administration of recombinant human BDNF facilitated LTM and increased CREB activation. Moreover, inhibitory avoidance training is associated with a rapid and transient increase in phospho-CREB/total CREB ratio in the PCx. Thus, our results indicate that endogenous BDNF is required for both STM and LTM formation of inhibitory avoidance learning, possibly involving CREB activation-dependent mechanisms. The present data support the idea that early sensory areas constitute important components of the networks subserving memory formation and that information processing in neocortex plays an important role in memory formation.     

Perirhinal cortex lesions produce retrograde amnesia for spatial information in rats: consolidation or retrieval?

Several lines of evidence in humans and experimental animals suggest that the hippocampus is critical for the formation and retrieval of spatial memory. However, although the hippocampus is reciprocally connected to adjacent cortices within the medial temporal lobe and they, in turn, are connected to the neocortex, little is known regarding the function of these cortices in memory. Here, using a reference spatial memory task in the radial maze, we show that neurotoxic perirhinal cortex lesions produce a profound retrograde amnesia when learning-surgery intervals of 1 or 50 d are used (Experiment 1). With the aim of dissociating between consolidation and retrieval processes, we injected lidocaine either daily after training (Experiment 2) or before a retention test once the learning had been completed (Experiment 3). Results show that reversible perirhinal inactivation impairs retrieval but not consolidation. However, the same procedure followed in Experiment 2 disrupted consolidation when the lidocaine was injected into the dorsal hippocampus. The results of Experiment 4 rule out the possibility that the deficit in retrieval is due to a state-dependent effect. These findings demonstrate the differential contribution of various regions of the medial temporal lobe to memory, suggesting that the perirhinal cortex plays a key role in the retrieval of spatial information for a long period of time.     

A nonlinear dynamic artificial neural network model of memory

Nonlinearity and dynamics in psychology are found in various domains such as neuroscience, cognitive science, human development, etc. However, the models that have been proposed are mostly of a computational nature and ignore dynamics. In those models that do include dynamic properties, only fixed points are used to store and retrieve information, leaving many principles of nonlinear dynamic systems (NDS) aside; for instance, chaos is often perceived as a nuisance. This paper considers a nonlinear dynamic artificial neural network (NDANN) that implements NDS principles while also complying with general neuroscience constraints. After a theoretical presentation, simulation results will show that the model can exhibit multi-valued, fixed-point, region-constrained attractors and aperiodic (including chaotic) behaviors. Because the capabilities of NDANN include the modeling of spatiotemporal chaotic activities, it may be an efficient tool to help bridge the gap between biological memory neural models and behavioral memory models.     

Acetylcholine in the orbitofrontal cortex is necessary for the acquisition of a socially transmitted food preference

The social transmission of food preference task (STFP) has been used to examine the involvement of the hippocampus in learning and memory for a natural odor-odor association. However, cortical involvement in STFP has not been extensively studied. The orbitofrontal cortex (OFC) is important in odor-guided learning, and cholinergic depletion of the entire neocortex results in impairments in STFP. Here we examined the specific role of cholinergic modulation in the OFC by assessing the effect of 192 immunoglobulin G-saporin infusion directly into OFC prior to training on STFP. Cholinergic depletion in the OFC impaired expression of the socially transmitted odor association measured 2 d after training, indicating that cholinergic function in the OFC is essential for this form of associative learning.     

Different types of environmental enrichment have discrepant effects on spatial memory and synaptophysin levels in female mice

Environmental enrichment paradigms that incorporate cognitive stimulation, exercise, and motor learning benefit memory and synaptic plasticity across the rodent lifespan. However, the contribution each individual element of the enriched environment makes to enhancing memory and synaptic plasticity has yet to be delineated. Therefore, the current study tested the effects of three of these elements on memory and synaptic protein levels. Young female C57BL/6 mice were given 3h of daily exposure to either rodent toys (cognitive stimulation) or running wheels (exercise), or daily acrobatic training for 6 weeks prior to and throughout behavioral testing. Controls were group housed, but did not receive enrichment. Spatial working and reference memory were tested in a water-escape motivated radial arm maze. Levels of the presynaptic protein synaptophysin were then measured in frontoparietal cortex, hippocampus, striatum, and cerebellum. Exercise, but not cognitive stimulation or acrobat training, improved spatial working memory relative to controls, despite the fact that both exercise and cognitive stimulation increased synaptophysin levels in the neocortex and hippocampus. These data suggest that exercise alone is sufficient to improve working memory, and that enrichment-induced increases in synaptophysin levels may not be sufficient to improve working memory in young females. Spatial reference memory was unaffected by enrichment. Acrobat training had no effect on memory or synaptophysin levels, suggesting a minimal contribution of motor learning to the mnemonic and neuronal benefits of enrichment. These results provide the first evidence that different elements of the enriched environment have markedly distinct effects on spatial memory and synaptic alterations.     

Using hippocampal-striatal loops for spatial navigation and goal-directed decision-making

The hippocampus plays a central role in spatial representation, declarative and episodic memory. In this area, so-called place cells possess high spatial selectivity, firing preferentially when the individual is within a small area of the environment. Interestingly, it has been found in rats that these cells can be active also when the animal is outside the location or context of their corresponding place field producing so-called “forward sweeps”. These typically occur at decision points during task execution and seem to be utilized, among other things, for the evaluation of potential alternative paths. Anticipatory firing is also found in the ventral striatum, a brain area that is strongly interconnected with the hippocampus and is known to encode value and reward. In this paper, we describe a biologically based computational model of the hippocampal-ventral striatum circuit that implements a goal-directed mechanism of choice, with the hippocampus primarily involved in the mental simulation of possible navigation paths and the ventral striatum involved in the evaluation of the associated reward expectancies. The model is validated in a navigation task in which a rat is placed in a complex maze with multiple rewarding sites. We show that the rat mentally activates place cells to simulate paths, estimate their value, and make decisions, implementing two essential processes of model-based reinforcement learning algorithms of choice: look-ahead prediction and the evaluation of predicted states.     

Using hippocampal-striatal loops for spatial navigation and goal-directed decision-making

The hippocampus plays a central role in spatial representation, declarative and episodic memory. In this area, so-called place cells possess high spatial selectivity, firing preferentially when the individual is within a small area of the environment. Interestingly, it has been found in rats that these cells can be active also when the animal is outside the location or context of their corresponding place field producing so-called "forward sweeps". These typically occur at decision points during task execution and seem to be utilized, among other things, for the evaluation of potential alternative paths. Anticipatory firing is also found in the ventral striatum, a brain area that is strongly interconnected with the hippocampus and is known to encode value and reward. In this paper, we describe a biologically based computational model of the hippocampal-ventral striatum circuit that implements a goal-directed mechanism of choice, with the hippocampus primarily involved in the mental simulation of possible navigation paths and the ventral striatum involved in the evaluation of the associated reward expectancies. The model is validated in a navigation task in which a rat is placed in a complex maze with multiple rewarding sites. We show that the rat mentally activates place cells to simulate paths, estimate their value, and make decisions, implementing two essential processes of model-based reinforcement learning algorithms of choice: look-ahead prediction and the evaluation of predicted states.