Hippocampus, cortex, and basal ganglia: insights from computational models of complementary learning systems

We present a framework for understanding how the hippocampus, neocortex, and basal ganglia work together to support cognitive and behavioral function in the mammalian brain. This framework is based on computational tradeoffs that arise in neural network models, where achieving one type of learning function requires very different parameters from those necessary to achieve another form of learning. For example, we dissociate the hippocampus from cortex with respect to general levels of activity, learning rate, and level of overlap between activation patterns. Similarly, the frontal cortex and associated basal ganglia system have important neural specializations not required of the posterior cortex system. Taken together, this overall cognitive architecture, which has been implemented in functioning computational models, provides a rich and often subtle means of explaining a wide range of behavioral and cognitive neuroscience data. Here, we summarize recent results in the domains of recognition memory, contextual fear conditioning, effects of basal ganglia lesions on stimulus-response and place learning, and flexible responding.     

Ability, breadth, and parsimony in computational models of higher-order cognition

Computational models will play an important role in our understanding of human higher‐order cognition. How can a model's contribution to this goal be evaluated? This article argues that three important aspects of a model of higher‐order cognition to evaluate are (a) its ability to reason, solve problems, converse, and learn as well as people do; (b) the breadth of situations in which it can do so; and (c) the parsimony of the mechanisms it posits. This article argues that fits of models to quantitative experimental data, although valuable for other reasons, do not address these criteria. Further, using analogies with other sciences, the history of cognitive science, and examples from modern‐day research programs, this article identifies five activities that have been demonstrated to play an important role in our understanding of human higher‐order cognition. These include modeling within a cognitive architecture, conducting artificial intelligence research, measuring and expanding a model's ability, finding mappings between the structure of different domains, and attempting to explain multiple phenomena within a single model.     

The role of response mechanisms in determining reaction time performance: Piéron’s law revisited

A response mechanism takes evaluations of the importance of potential actions and selects the most suitable. Response mechanism function is a nontrivial problem that has not received the attention it deserves within cognitive psychology. In this article, we make a case for the importance of considering response mechanism function as a constraint on cognitive processes and emphasized links with the wider problem of behavioral action selection. First, we show that, contrary to previous suggestions, a well-known model of the Stroop task (Cohen, Dunbar, & McClelland, 1990) relies on the response mechanism for a key feature of its results—the interference—facilitation asymmetry. Second, we examine a variety of response mechanisms (including that in the model of Cohen et al., 1990) and show that they all follow a law analogous to Piéron’s law in relating their input to reaction time. In particular, this is true of a decision mechanism not designed to explain RT data but based on a proposed solution to the general problem of action selection and grounded in the neurobiology of the vertebrate basal ganglia. Finally, we show that the dynamics of simple artificial neurons also support a Piéron-like law.     

Individual Differences in Reward-Based Learning Predict Fluid Reasoning Abilities

The ability to reason and problem-solve in novel situations, as measured by the Raven's Advanced Progressive Matrices (RAPM), is highly predictive of both cognitive task performance and real-world outcomes. Here we provide evidence that RAPM performance depends on the ability to reallocate attention in response to self-generated feedback about progress. We propose that such an ability is underpinned by the basal ganglia nuclei, which are critically tied to both reward processing and cognitive control. This hypothesis was implemented in a neurocomputational model of the RAPM task, which was used to derive novel predictions at the behavioral and neural levels. These predictions were then verified in one neuroimaging and two behavioral experiments. Furthermore, an effective connectivity analysis of the neuroimaging data confirmed a role for the basal ganglia in modulating attention. Taken together, these results suggest that individual differences in a neural circuit related to reward processing underpin human fluid reasoning abilities.     

The role of falsification in the development of cognitive architectures: insights from a lakatosian analysis

It has been suggested that the enterprise of developing mechanistic theories of the human cognitive architecture is flawed because the theories produced are not directly falsifiable. Newell attempted to sidestep this criticism by arguing for a Lakatosian model of scientific progress in which cognitive architectures should be understood as theories that develop over time. However, Newell's own candidate cognitive architecture adhered only loosely to Lakatosian principles. This paper reconsiders the role of falsification and the potential utility of Lakatosian principles in the development of cognitive architectures. It is argued that a lack of direct falsifiability need not undermine the scientific development of a cognitive architecture if broadly Lakatosian principles are adopted. Moreover, it is demonstrated that the Lakatosian concepts of positive and negative heuristics for theory development and of general heuristic power offer methods for guiding the development of an architecture and for evaluating the contribution and potential of an architecture's research program.     

Confronting Many‐Many Problems: Attention and Agentive Control

I argue that when perception plays a guiding role in intentional bodily action, it is a necessary part of that action. The argument begins with a challenge that necessarily arises for embodied agents, what I call the Many‐Many Problem. The Problem is named after its most common case where agents face too many perceptual inputs and too many possible behavioral outputs. Action requires a solution to the Many‐Many Problem by selection of a specific linkage between input and output. In bodily action the agent perceptually selects, and in this way perceptually attends to, relevant information so as to guide the execution of specific movements. Since perceptual attention is a necessary part of solving the Many‐Many Problem, it is a necessary part of bodily action. Indeed, the process of implementing a solution to the Many‐Many Problem, as constrained by the agent's motivational state, just is the agent's performing an intentional bodily action in the relevant way.     

The social foundation of executive function

In this study, we propose that infant social cognition may ‘bootstrap' the successive development of domain‐general cognition in line with the cultural intelligence hypothesis. Using a longitudinal design, 6‐month‐old infants (N = 118) were assessed on two basic social cognitive tasks targeting the abilities to share attention with others and understanding other peoples' actions. At 10 months, we measured the quality of the child's social learning environment, indexed by parent's abilities to provide scaffolding behaviors during a problem‐solving task. Eight months later, the children were followed up with a cognitive test‐battery, including tasks of inhibitory control and working memory. Our results showed that better infant social action understanding interacted with better parental scaffolding skills in predicting simple inhibitory control in toddlerhood. This suggests that infants' who are better at understanding other's actions are also better equipped to make the most of existing social learning opportunities, which in turn may benefit future non‐social cognitive outcomes.     

A neural model of hippocampal–striatal interactions in associative learning and transfer generalization in various neurological and psychiatric patients

Building on our previous neurocomputational models of basal ganglia and hippocampal region function (and their modulation by dopamine and acetylcholine, respectively), we show here how an integration of these models can inform our understanding of the interaction between the basal ganglia and hippocampal region in associative learning and transfer generalization across various patient populations. As a common test bed for exploring interactions between these brain regions and neuromodulators, we focus on the acquired equivalence task, an associative learning paradigm in which stimuli that have been associated with the same outcome acquire a functional similarity such that subsequent generalization between these stimuli increases. This task has been used to test cognitive dysfunction in various patient populations with damages to the hippocampal region and basal ganglia, including studies of patients with Parkinson’s disease (PD), schizophrenia, basal forebrain amnesia, and hippocampal atrophy. Simulation results show that damage to the hippocampal region—as in patients with hippocampal atrophy (HA), hypoxia, mild Alzheimer’s (AD), or schizophrenia—leads to intact associative learning but impaired transfer generalization performance. Moreover, the model demonstrates how PD and anterior communicating artery (ACoA) aneurysm—two very different brain disorders that affect different neural mechanisms—can have similar effects on acquired equivalence performance. In particular, the model shows that simulating a loss of dopamine function in the basal ganglia module (as in PD) leads to slow acquisition learning but intact transfer generalization. Similarly, the model shows that simulating the loss of acetylcholine in the hippocampal region (as in ACoA aneurysm) also results in slower acquisition learning. We argue from this that changes in associative learning of stimulus–action pathways (in the basal ganglia) or changes in the learning of stimulus representations (in the hippocampal region) can have similar functional effects.     

The Application of Cognitive‐Developmental or Mediated Cognitive Learning Strategies in Online College Coursework

This research article explores the active use of cognitive‐developmental or mediated cognitive learning strategies in undergraduate online courses. Examples and applications are drawn from two online sessions integrating online interaction, essay and discussion assignments, as well as a variety of multimedia components conducted during the spring of 2008. While focus on the interaction among students remains an important aspect of the online discussion environment, particular attention is given to the interaction between the student and the instructor. This paper argues that while online learning environments are ultimately student‐controlled, they should be teacher‐centered. The findings of this research suggest that students are more directly influenced by an instructor's intentional effort to mediate the learning process than by the course objectives, material, or subject matter. Successful use of online technologies requires deliberate action on the part of the instructor to integrate various mediated cognitive learning strategies: (a) student participation and response is significantly increased, and (b) student motivation and morale is dramatically influenced.     

A neural cognitive architecture

It is difficult to study the mind, but cognitive architectures are one tool. As the mind emerges from the behaviour of the brain, neuropsychological methods are another method to study the mind, though a rather indirect method. A cognitive architecture that is implemented in spiking neurons is a method of studying the mind that can use neuropsychological evidence directly. A neural cognitive architecture, based on rule based systems and associative memory, can be readily implemented, and would provide a good bridge between standard cognitive architectures, such as Soar, and neuropsychology. This architecture could be implemented in spiking neurons, and made available via the Human Brain Project, which provides a good collaborative environment. The architecture could be readily extended to use spiking neurons for subsystems, such as spatial reasoning, and could evolve over time toward a complete architecture. The theory behind this architecture could evolve over time. Simplifying assumptions, made explicit, such as those behind the rule based system, could gradually be replaced by more neuropsychologically accurate behaviour. The overall task of collaborative architecture development would be eased by direct evidence of the actual neural cognitive architectures in human brains. While the initial architecture is biologically inspired, the ultimate goal is a biological cognitive architecture.     

Basal ganglia output and cognition: evidence from anatomical, behavioral, and clinical studies

The traditional view that the basal ganglia are simply involved in the control of movement has been challenged in recent years. Three lines of evidence indicate that the basal ganglia also are involved in nonmotor operations. First, the results of anatomical studies clearly indicate that the basal ganglia participate in multiple circuits or 'loops' with cognitive areas of the cerebral cortex. Second, the activity of neurons within selected portions of the basal ganglia is more related to cognitive or sensory operations than to motor functions. Finally, in some instances basal ganglia lesions cause primarily cognitive or sensory disturbances without gross motor impairments. In this report, we briefly review some of these data and present a new anatomical framework for understanding the basal ganglia contributions to nonmotor function.     

Action mechanisms for social cognition: behavioral and neural correlates of developing Theory of Mind

Many psychological theories posit foundational links between two fundamental constructs: (1) our ability to produce, perceive, and represent action; and (2) our ability to understand the meaning and motivation behind the action (i.e. Theory of Mind; ToM). This position is contentious, however, and long‐standing competing theories of social‐cognitive development debate roles for basic action‐processing in ToM. Developmental research is key to investigating these hypotheses, but whether individual differences in neural and behavioral measures of motor action relate to social‐cognitive development is unknown. We examined 3‐ to 5‐year‐old children's (N = 26) EEG mu‐desynchronization during production of object‐directed action, and explored associations between mu‐desynchronization and children's behavioral motor skills, behavioral action‐representation abilities, and behavioral ToM. For children with high (but not low) mu‐desynchronization, motor skill related to action‐representation abilities, and action‐representation mediated relations between motor skill and ToM. Results demonstrate novel foundational links between action‐processing and ToM, suggesting that basic motor action may be a key mechanism for social‐cognitive development, thus shedding light on the origins and emergence of higher social cognition.     

Memory systems within a cognitive architecture

This article addresses the division of memory systems in relation to an overall cognitive architecture. As understanding the architecture is essential to understanding the mind, developing computational cognitive architectures is an important enterprise in computational psychology (computational cognitive modeling). The article proposes a set of hypotheses concerning memory systems from the standpoint of a cognitive architecture, in particular, the four-way division of memory (including explicit and implicit procedural memory and explicit and implicit declarative memory). It then discusses in detail how these hypotheses may be validated through examining qualitatively the literature on memory. A quick review follows of computational simulations of a variety of quantitative data (which are not limited to narrowly conceived “memory tasks”). Results of accounting for both qualitative and quantitative data point to the promise of this approach.     

Cognitive architectures: Research issues and challenges

In this paper, we examine the motivations for research on cognitive architectures and review some candidates that have been explored in the literature. After this, we consider the capabilities that a cognitive architecture should support, some properties that it should exhibit related to representation, organization, performance, and learning, and some criteria for evaluating such architectures at the systems level. In closing, we discuss some open issues that should drive future research in this important area.     

A Review and Taxonomy of Choice Architecture Techniques

We present a taxonomy of choice architecture techniques that focus on intervention design, as opposed to the underlying cognitive processes that make an intervention work. We argue that this distinction will facilitate further empirical testing and will assist practitioners in designing interventions. The framework is inductively derived from empirically tested examples of choice architecture and consists of nine techniques targeting decision information, decision structure, and decision assistance. An inter‐rater reliability test demonstrates that these techniques can be used in an intersubjectively replicable way to describe sample choice architectures. We conclude by discussing limitations of the framework and key issues concerning the use of the techniques in the development of new choice architectures. Copyright © 2015 John Wiley & Sons, Ltd.     

Separating goals from behavioral control: Implications from learning predictive modularizations

The brain may be regarded as an anticipatory machine whose behavior strongly depends on its current predictive knowledge. Behavioral decision making depends on anticipated goal states as well as on the current internal motivations of the organism. Behavioral control, on the other hand, is guided by the goals currently chosen along with additional constraints. Both, decision making and control are thus anticipatory processes. Moreover, they are mutually dependent: while action control depends on currently selected goals, goal selection depends on achievability estimates, which must be based on the system's current action control competence. An autonomous, adaptive system thus faces the challenge of learning goal representations that are suitable for both, action selection and action control. We propose that a goal processing pathway should be separated from but also strongly interact with a sensorimotor control pathway. We investigate the encoding structures expectable along these two pathways for realizing effective and flexible action decision making and control. While the goal processing pathway needs to be able to distinguish motivation-oriented relevancies for decision making, the sensorimotor pathway needs to provide control-oriented encodings. We use an evolutionary machine learning technique to investigate how important modularity may be for realizing particular sensorimotor mappings. Next, we survey the results obtained by a neural network architecture, which show that enforcing multiplicative interactions between self-organizing sensorimotor control-oriented encodings and goal-oriented interaction selection encodings enables the learning of highly flexible decision making and action control structures. Furthermore, we show that the emerging goal-oriented encodings exhibit pre-linguistic compositional structures. We conclude that for bootstrapping higher-level cognitive capabilities it may be essential on the one hand to separate sensorimotor, anticipatory, control-oriented spatial encodings from compositional, goal-oriented spaces, and on the other hand to enable bidirectional, multiplicative interactions between these two sets of spatial encodings.     

To Accept or to Reject: The Effect of Framing on Attitudes Toward Affirmative Action1

Two experiments examined the effect of framing on attitudes toward an affirmative‐action program of preferential treatment. Participants' attitudes were consistently more favorable toward the affirmative‐action program presented in a positive frame—preferring a target group's applicant over a majority group's applicant—than when the very same program was presented in a negative frame—rejecting the majority group's applicant in favor of the target group's applicant. Similar effects were evident for 3 target groups in the context of higher education selection and personnel selection. Two theoretical explanations for the effect of framing on attitudes toward affirmative‐action programs are suggested. The implications of this effect are discussed, and the challenges facing future research of this phenomenon are outlined.     

Rule‐Following and Externalism*

John McDowell has suggested recently that there is a route from his favoured solution to Kripke's Wittgenstein's “sceptical paradox” about rule‐following to a particular form of cognitive externalism. In this paper, 1 argue that this is not the case: even granting McDowell his solution to the rule‐following paradox, his preferred version of cognitive externalism does not follow.     

From recurrent choice to skill learning: a reinforcement-learning model

The authors propose a reinforcement-learning mechanism as a model for recurrent choice and extend it to account for skill learning. The model was inspired by recent research in neurophysiological studies of the basal ganglia and provides an integrated explanation of recurrent choice behavior and skill learning. The behavior includes effects of differential probabilities, magnitudes, variabilities, and delay of reinforcement. The model can also produce the violation of independence, preference reversals, and the goal gradient of reinforcement in maze learning. An experiment was conducted to study learning of action sequences in a multistep task. The fit of the model to the data demonstrated its ability to account for complex skill learning. The advantages of incorporating the mechanism into a larger cognitive architecture are discussed.