Conjunctive representations in learning and memory: principles of cortical and hippocampal function

The authors present a theoretical framework for understanding the roles of the hippocampus and neocortex in learning and memory. This framework incorporates a theme found in many theories of hippocampal function: that the hippocampus is responsible for developing conjunctive representations binding together stimulus elements into a unitary representation that can later be recalled from partial input cues. This idea is contradicted by the fact that hippocampally lesioned rats can learn nonlinear discrimination problems that require conjunctive representations. The authors' framework accommodates this finding by establishing a principled division of labor, where the cortex is responsible for slow learning that integrates over multiple experiences to extract generalities whereas the hippocampus performs rapid learning of the arbitrary contents of individual experiences. This framework suggests that tasks involving rapid, incidental conjunctive learning are better tests of hippocampal function. The authors implement this framework in a computational neural network model and show that it can account for a wide range of data in animal learning.     

Basal ganglia output and cognition: evidence from anatomical, behavioral, and clinical studies

The traditional view that the basal ganglia are simply involved in the control of movement has been challenged in recent years. Three lines of evidence indicate that the basal ganglia also are involved in nonmotor operations. First, the results of anatomical studies clearly indicate that the basal ganglia participate in multiple circuits or 'loops' with cognitive areas of the cerebral cortex. Second, the activity of neurons within selected portions of the basal ganglia is more related to cognitive or sensory operations than to motor functions. Finally, in some instances basal ganglia lesions cause primarily cognitive or sensory disturbances without gross motor impairments. In this report, we briefly review some of these data and present a new anatomical framework for understanding the basal ganglia contributions to nonmotor function.     

Interactions between frontal cortex and basal ganglia in working memory: a computational model

The frontal cortex and the basal ganglia interact via a relatively well understood and elaborate system of interconnections. In the context of motor function, these interconnections can be understood as disinhibiting, or “releasing the brakes,” on frontal motor action plans: The basal ganglia detect appropriate contexts for performing motor actions and enable the frontal cortex to execute such actions at the appropriate time. We build on this idea in the domain of working memory through the use of computational neural network models of this circuit. In our model, the frontal cortex exhibits robust active maintenance, whereas the basal ganglia contribute a selective, dynamic gating function that enables frontal memory representations to be rapidly updated in a task-relevant manner. We apply the model to a novel version of the continuous performance task that requires subroutine-like selective working memory updating and compare and contrast our model with other existing models and theories of frontal-cortex-basal-ganglia interactions.     

标量计时模型中的神经机制

标量计时模型中各阶段的神经机制有重叠也有分离。从当今认知神经科学的研究结果看,与内部时钟有关的神经结构有小脑、基底神经节、前额皮质、前运动辅助皮质及顶叶下回皮质等;与记忆阶段有关的神经结构有基底神经节、背外侧前额皮质、右侧额下皮质及外侧前运动皮质等;与决策阶段有关的神经结构有背外侧前额皮质、前扣带回、高级颞叶皮质和基底神经节等。文章还从神经机制角度论证了计时的标量特性,讨论了今后研究值得注意的三个问题,即研究结果的确定性、研究手段的局限性以及该模型的适用性     

Conditional visuo-motor learning and dimension reduction

Conditional visuo-motor learning consists in learning by trial and error to associate visual cues with correct motor responses, that have no direct link. Converging evidence supports the role of a large brain network in this type of learning, including the prefrontal and the premotor cortex, the basal ganglia (BG) and the hippocampus. In this paper we focus on the role of a major structure of the BG, the striatum. We first present behavioral results and electrophysiological data recorded from this structure in monkeys engaged in learning new visuo-motor associations. Visual stimuli were presented on a video screen and the animals had to learn, by trial and error, to select the correct movement of a joystick, in order to receive a liquid reward. Behavioral results revealed that the monkeys used a sequential strategy, whereby they learned the associations one by one although they were presented randomly. Human subjects, tested on the same task, also used a sequential strategy. Neuronal recordings in monkeys revealed learning-related modulations of neural activity in the striatum. We then present a mathematical model inspired by viability theory developed to implement the use of strategies during learning. This model complements existing models of the BG based on reinforcement learning (RL), which do not take into account the use of strategies to reduce the dimension of the learning space.     

Individual Differences in Reward-Based Learning Predict Fluid Reasoning Abilities

The ability to reason and problem-solve in novel situations, as measured by the Raven's Advanced Progressive Matrices (RAPM), is highly predictive of both cognitive task performance and real-world outcomes. Here we provide evidence that RAPM performance depends on the ability to reallocate attention in response to self-generated feedback about progress. We propose that such an ability is underpinned by the basal ganglia nuclei, which are critically tied to both reward processing and cognitive control. This hypothesis was implemented in a neurocomputational model of the RAPM task, which was used to derive novel predictions at the behavioral and neural levels. These predictions were then verified in one neuroimaging and two behavioral experiments. Furthermore, an effective connectivity analysis of the neuroimaging data confirmed a role for the basal ganglia in modulating attention. Taken together, these results suggest that individual differences in a neural circuit related to reward processing underpin human fluid reasoning abilities.     

A neural model of hippocampal–striatal interactions in associative learning and transfer generalization in various neurological and psychiatric patients

Building on our previous neurocomputational models of basal ganglia and hippocampal region function (and their modulation by dopamine and acetylcholine, respectively), we show here how an integration of these models can inform our understanding of the interaction between the basal ganglia and hippocampal region in associative learning and transfer generalization across various patient populations. As a common test bed for exploring interactions between these brain regions and neuromodulators, we focus on the acquired equivalence task, an associative learning paradigm in which stimuli that have been associated with the same outcome acquire a functional similarity such that subsequent generalization between these stimuli increases. This task has been used to test cognitive dysfunction in various patient populations with damages to the hippocampal region and basal ganglia, including studies of patients with Parkinson’s disease (PD), schizophrenia, basal forebrain amnesia, and hippocampal atrophy. Simulation results show that damage to the hippocampal region—as in patients with hippocampal atrophy (HA), hypoxia, mild Alzheimer’s (AD), or schizophrenia—leads to intact associative learning but impaired transfer generalization performance. Moreover, the model demonstrates how PD and anterior communicating artery (ACoA) aneurysm—two very different brain disorders that affect different neural mechanisms—can have similar effects on acquired equivalence performance. In particular, the model shows that simulating a loss of dopamine function in the basal ganglia module (as in PD) leads to slow acquisition learning but intact transfer generalization. Similarly, the model shows that simulating the loss of acetylcholine in the hippocampal region (as in ACoA aneurysm) also results in slower acquisition learning. We argue from this that changes in associative learning of stimulus–action pathways (in the basal ganglia) or changes in the learning of stimulus representations (in the hippocampal region) can have similar functional effects.     

Neural-network models of learning and memory: leading questions and an emerging framework

Real-time neural-network models provide a conceptual framework for formulating questions about the nature of cognition, an architectural framework for mapping cognitive functions to brain regions, a semantic framework for defining terms, and a computational framework for testing hypotheses. This article considers key questions about how a physical system might simultaneously support one-trial learning and lifetime memories, in the context of neural models that test possible solutions to the problems posed. Model properties point to partial answers, and model limitations lead to new questions. Placing individual system components in the context of a unified real-time network allows analysis to move from the level of neural processes, including learning laws and rules of synaptic transmission, to cognitive processes, including attention and consciousness.     

From cybernetics to brain theory,and more: A memoir

While structured as an autobiography, this memoir exemplifies ways in which classic contributions to cybernetics (e.g., by Wiener, McCulloch & Pitts, and von Neumann) have fed into a diversity of current research areas, including the mathematical theory of systems and computation, artificial intelligence and robotics, computational neuroscience, linguistics, and cognitive science. The challenges of brain theory receive special emphasis. Action-oriented perception and schema theory complement neural network modeling in analyzing cerebral cortex, cerebellum, hippocampus, and basal ganglia. Comparative studies of frog, rat, monkey, ape and human not only deepen insights into the human brain but also ground an EvoDevoSocio view of “how the brain got language.” The rapprochement between neuroscience and architecture provides a recent challenge. The essay also assesses some of the social and theological implications of this broad perspective.     

Hierarchically organized behavior and its neural foundations: a reinforcement learning perspective

Research on human and animal behavior has long emphasized its hierarchical structure—the divisibility of ongoing behavior into discrete tasks, which are comprised of subtask sequences, which in turn are built of simple actions. The hierarchical structure of behavior has also been of enduring interest within neuroscience, where it has been widely considered to reflect prefrontal cortical functions. In this paper, we reexamine behavioral hierarchy and its neural substrates from the point of view of recent developments in computational reinforcement learning. Specifically, we consider a set of approaches known collectively as hierarchical reinforcement learning, which extend the reinforcement learning paradigm by allowing the learning agent to aggregate actions into reusable subroutines or skills. A close look at the components of hierarchical reinforcement learning suggests how they might map onto neural structures, in particular regions within the dorsolateral and orbital prefrontal cortex. It also suggests specific ways in which hierarchical reinforcement learning might provide a complement to existing psychological models of hierarchically structured behavior. A particularly important question that hierarchical reinforcement learning brings to the fore is that of how learning identifies new action routines that are likely to provide useful building blocks in solving a wide range of future problems. Here and at many other points, hierarchical reinforcement learning offers an appealing framework for investigating the computational and neural underpinnings of hierarchically structured behavior.     

Neural substrates of individual differences in human fear learning: Evidence from concurrent fMRI, fear-potentiated startle, and US-expectancy data

To provide insight into individual differences in fear learning, we examined the emotional and cognitive expressions of discriminative fear conditioning in direct relation to its neural substrates. Contrary to previous behavioral-neural (fMRI) research on fear learning--in which the emotional expression of fear was generally indexed by skin conductance--we used fear-potentiated startle, a more reliable and specific index of fear. While we obtained concurrent fear-potentiated startle, neuroimaging (fMRI), and US-expectancy data, healthy participants underwent a fear-conditioning paradigm in which one of two conditioned stimuli (CS(+) but not CS(-)) was paired with a shock (unconditioned stimulus [US]). Fear learning was evident from the differential expressions of fear (CS(+) > CS(-)) at both the behavioral level (startle potentiation and US expectancy) and the neural level (in amygdala, anterior cingulate cortex, hippocampus, and insula). We examined individual differences in discriminative fear conditioning by classifying participants (as conditionable vs. unconditionable) according to whether they showed successful differential startle potentiation. This revealed that the individual differences in the emotional expression of discriminative fear learning (startle potentiation) were reflected in differential amygdala activation, regardless of the cognitive expression of fear learning (CS-US contingency or hippocampal activation). Our study provides the first evidence for the potential of examining startle potentiation in concurrent fMRI research on fear learning.     

A neural model of normal and abnormal learning and memory consolidation: adaptively timed conditioning,hippocampus, amnesia,neurotrophins, and consciousness

How do the hippocampus and amygdala interact with thalamocortical systems to regulate cognitive and cognitive-emotional learning? Why do lesions of thalamus, amygdala, hippocampus, and cortex have differential effects depending on the phase of learning when they occur? In particular, why is the hippocampus typically needed for trace conditioning, but not delay conditioning, and what do the exceptions reveal? Why do amygdala lesions made before or immediately after training decelerate conditioning while those made later do not? Why do thalamic or sensory cortical lesions degrade trace conditioning more than delay conditioning? Why do hippocampal lesions during trace conditioning experiments degrade recent but not temporally remote learning? Why do orbitofrontal cortical lesions degrade temporally remote but not recent or post-lesion learning? How is temporally graded amnesia caused by ablation of prefrontal cortex after memory consolidation? How are attention and consciousness linked during conditioning? How do neurotrophins, notably brain-derived neurotrophic factor (BDNF), influence memory formation and consolidation? Is there a common output path for learned performance? A neural model proposes a unified answer to these questions that overcome problems of alternative memory models.     

Learning and production of movement sequences: behavioral, neurophysiological, and modeling perspectives

A wave of recent behavioral studies has generated a new wealth of parametric observations about serial order behavior. What was a trickle of neurophysiological studies has grown to a steady stream of probes of neural sites and mechanisms underlying sequential behavior. Moreover, simulation models of serial behavior generation have begun to open a channel to link cellular dynamics with cognitive and behavioral dynamics. Here we review major results from prominent sequence learning and performance tasks, namely immediate serial recall, typing, 2 x N, discrete sequence production, and serial reaction time. These tasks populate a continuum from higher to lower degrees of internal control of sequential organization and probe important contemporary issues such as the nature of working-memory representations for sequential behavior, and the development and role of chunks in hierarchical control. The main movement classes reviewed are speech and keypressing, both involving small amplitude movements amenable to parametric study. A synopsis of serial order models, vis-a-vis major empirical findings leads to a focus on competitive queuing (CQ) models. Recently, the many behavioral predictive successes of CQ models have been complemented by successful prediction of distinctively patterned electrophysiological recordings. In lateral prefrontal cortex, parallel activation dynamics of multiple neural ensembles strikingly matches the parallel dynamics predicted by CQ theory. An extended CQ simulation model--the N-STREAMS neural network model--exemplifies ongoing attempts to accommodate a broad range of both behavioral and neurobiological data within a CQ-consistent theory.     

Inhibition of action, thought, and emotion: A selective neurobiological review

The neural bases of inhibitory function are reviewed, covering data from paradigms assessing inhibition of motor responses (antisaccade, go/nogo, stop-signal), cognitive sets (e.g., Wisconsin Card Sort Test), and emotion (fear extinction). The frontal cortex supports performance on these paradigms, but the specific neural circuitry varies: response inhibition depends upon fronto-basal ganglia networks, inhibition of cognitive sets is supported by orbitofrontal cortex, and retention of fear extinction reflects ventromedial prefrontal cortex–amygdala interactions. Inhibition is thus neurobiologically heterogeneous, although right ventrolateral prefrontal cortex may support a general inhibitory process. Dysfunctions in these circuits may contribute to psychopathological conditions marked by inhibitory deficits.     

Neural dynamics of autistic behaviors: cognitive, emotional, and timing substrates

What brain mechanisms underlie autism, and how do they give rise to autistic behavioral symptoms? This article describes a neural model, called the Imbalanced Spectrally Timed Adaptive Resonance Theory (iSTART) model, that proposes how cognitive, emotional, timing, and motor processes that involve brain regions such as the prefrontal and temporal cortex, amygdala, hippocampus, and cerebellum may interact to create and perpetuate autistic symptoms. These model processes were originally developed to explain data concerning how the brain controls normal behaviors. The iSTART model shows how autistic behavioral symptoms may arise from prescribed breakdowns in these brain processes, notably a combination of underaroused emotional depression in the amygdala and related affective brain regions, learning of hyperspecific recognition categories in the temporal and prefrontal cortices, and breakdowns of adaptively timed attentional and motor circuits in the hippocampal system and cerebellum. The model clarifies how malfunctions in a subset of these mechanisms can, through a systemwide vicious circle of environmentally mediated feedback, cause and maintain problems with them all.     

Hippocampal and neocortical contributions to memory: advances in the complementary learning systems framework

The complementary learning systems framework provides a simple set of principles, derived from converging biological, psychological and computational constraints, for understanding the differential contributions of the neocortex and hippocampus to learning and memory. The central principles are that the neocortex has a low learning rate and uses overlapping distributed representations to extract the general statistical structure of the environment, whereas the hippocampus learns rapidly using separated representations to encode the details of specific events while minimizing interference. In recent years, we have instantiated these principles in working computational models, and have used these models to address human and animal learning and memory findings, across a wide range of domains and paradigms. Here, we review a few representative applications of our models, focusing on two domains: recognition memory and animal learning in the fear-conditioning paradigm. In both domains, the models have generated novel predictions that have been tested and confirmed.     

Hierarchical models of behavior and prefrontal function

The recognition of hierarchical structure in human behavior was one of the founding insights of the cognitive revolution. Despite decades of research, however, the computational mechanisms underlying hierarchically organized behavior are still not fully understood. Recent findings from behavioral and neuroscientific research have fueled a resurgence of interest in the problem, inspiring a new generation of computational models. In addition to developing some classic proposals, these models also break fresh ground, teasing apart different forms of hierarchical structure, placing a new focus on the issue of learning and addressing recent findings concerning the representation of behavioral hierarchies within the prefrontal cortex. In addition to offering explanations for some key aspects of behavior and functional neuroanatomy, the latest models also pose new questions for empirical research.     

Introduction to the special section on theory and data in categorization: Integrating computational, behavioral, and cognitive neuroscience approaches

This special section brings together behavioral, computational, mathematical, and neuroimaging approaches to understand the processes underlying category learning. Over the past decade, there has been growing convergence in research on categorization, with computational-mathematical models influencing the interpretation of brain imaging and neuropsychological data, and with cognitive neuroscience findings influencing the development and refinement of models. Classic debates between single-system and multiple-memory-system theories have become more nuanced and focused. Multiple brain areas and cognitive processes contribute to categorization, but theories differ markedly in whether and when those neurocognitive components are recruited for different aspects of categorization. The articles in this special section approach this issue from several diverse angles.     

Spiking neural networks and hippocampal function: A web-accessible survey of simulations,modeling methods,and underlying theories

Computational modeling has contributed to hippocampal research in a wide variety of ways and through a large diversity of approaches, reflecting the many advanced cognitive roles of this brain region. The intensively studied neuron type circuitry of the hippocampus is a particularly conducive substrate for spiking neural models. Here we present an online knowledge base of spiking neural network simulations of hippocampal functions. First, we overview theories involving the hippocampal formation in subjects such as spatial representation, learning, and memory. Then we describe an original literature mining process to organize published reports in various key aspects, including: (i) subject area (e.g., navigation, pattern completion, epilepsy); (ii) level of modeling detail (Hodgkin-Huxley, integrate-and-fire, etc.); and (iii) theoretical framework (attractor dynamics, oscillatory interference, self-organizing maps, and others). Moreover, every peer-reviewed publication is also annotated to indicate the specific neuron types represented in the network simulation, establishing a direct link with the Hippocampome.org portal. The web interface of the knowledge base enables dynamic content browsing and advanced searches, and consistently presents evidence supporting every annotation. Moreover, users are given access to several types of statistical reports about the collection, a selection of which is summarized in this paper. This open access resource thus provides an interactive platform to survey spiking neural network models of hippocampal functions, compare available computational methods, and foster ideas for suitable new directions of research.