A computational model of sequential movement learning with a signal mimicking dopaminergic neuron activities

We present a computational model of approach learning in a simulated maze environment. Our maze environment and training method mimics those used in the experimental literature. We show that our model learns the correct sequence of six decisions that lead to the location of positive reinforcement and in a manner consistent with experimental observations. Our model exhibits many properties that are characteristic of animal learning in maze environments including delay conditioning, secondary conditioning, and backward chaining. Finally, we map our model to the basal ganglia and show that a signal in our model that is responsible for learning has the same temporal properties as dopamine, the neurotransmitter believed to play an important part in learning decision sequences.     

厌恶加工的神经基础

厌恶是由令人不愉悦、反感的事物诱发的情绪。根据刺激类型的不同, 厌恶可以分为不同的类型。脑岛和基底节是厌恶加工的主要脑区, 前扣带回、杏仁核、丘脑、内侧前额叶也参与厌恶加工。对已有研究的总结发现, 不同类型的厌恶、不同感觉通道的厌恶加工可能具有不同的神经基础。在未来的研究中, 应当注重研究厌恶加工的认知机制、神经基础以及与厌恶相关的神经递质等问题。     

Variability in subcortical aphasia is due to variable sites of cortical hypoperfusion

A variety of fluent and nonfluent aphasias have been reported after left basal ganglia stroke. It has been speculated that this heterogeneity may reflect variations in cortical hypoperfusion resulting from large vessel stenosis. To test this hypothesis, a consecutive series of 24 patients with left caudate infarct identified with diffusion-weighted imaging underwent language testing and perfusion-weighted imaging < 24h from onset of symptoms. Specific regions in perisylvian cortex were rated for the percentage of the region that was hypoperfused. Aphasia type was determined on the basis of speech fluency, comprehension, and repetition performance on the language tests. Association between aphasia type/language impairment and regions of hypoperfusion were identified with Fisher's exact tests. Results demonstrated that in patients with acute left caudate infarct, the presence and type of aphasia reflected regions of hypoperfusion, and generally followed predictions based on chronic lesion studies, regarding anatomical lesions associated with classic aphasia types.     

Conditional visuo-motor learning and dimension reduction

Conditional visuo-motor learning consists in learning by trial and error to associate visual cues with correct motor responses, that have no direct link. Converging evidence supports the role of a large brain network in this type of learning, including the prefrontal and the premotor cortex, the basal ganglia (BG) and the hippocampus. In this paper we focus on the role of a major structure of the BG, the striatum. We first present behavioral results and electrophysiological data recorded from this structure in monkeys engaged in learning new visuo-motor associations. Visual stimuli were presented on a video screen and the animals had to learn, by trial and error, to select the correct movement of a joystick, in order to receive a liquid reward. Behavioral results revealed that the monkeys used a sequential strategy, whereby they learned the associations one by one although they were presented randomly. Human subjects, tested on the same task, also used a sequential strategy. Neuronal recordings in monkeys revealed learning-related modulations of neural activity in the striatum. We then present a mathematical model inspired by viability theory developed to implement the use of strategies during learning. This model complements existing models of the BG based on reinforcement learning (RL), which do not take into account the use of strategies to reduce the dimension of the learning space.     

The Declarative/Procedural Model of Lexicon and Grammar

Our use of language depends upon two capacities: a mental lexicon of memorized words and a mental grammar of rules that underlie the sequential and hierarchical composition of lexical forms into predictably structured larger words, phrases, and sentences. The declarative/procedural model posits that the lexicon/grammar distinction in language is tied to the distinction between two well-studied brain memory systems. On this view, the memorization and use of at least simple words (those with noncompositional, that is, arbitrary form-meaning pairings) depends upon an associative memory of distributed representations that is subserved by temporal-lobe circuits previously implicated in the learning and use of fact and event knowledge. This declarative memory system appears to be specialized for learning arbitrarily related information (i.e., for associative binding). In contrast, the acquisition and use of grammatical rules that underlie symbol manipulation is subserved by frontal/basal-ganglia circuits previously implicated in the implicit (nonconscious) learning and expression of motor and cognitive skills and habits (e.g., from simple motor acts to skilled game playing). This procedural system may be specialized for computing sequences. This novel view of lexicon and grammar offers an alternative to the two main competing theoretical frameworks. It shares the perspective of traditional dual-mechanism theories in positing that the mental lexicon and a symbol-manipulating mental grammar are subserved by distinct computational components that may be linked to distinct brain structures. However, it diverges from these theories where they assume components dedicated to each of the two language capacities (that is, domain-specific) and in their common assumption that lexical memory is a rote list of items. Conversely, while it shares with single-mechanism theories the perspective that the two capacities are subserved by domain-independent computational mechanisms, it diverges from them where they link both capacities to a single associative memory system with broad anatomic distribution. The declarative/procedural model, but neither traditional dual- nor single-mechanism models, predicts double dissociations between lexicon and grammar, with associations among associative memory properties, memorized words and facts, and temporal-lobe structures, and among symbol-manipulation properties, grammatical rule products, motor skills, and frontal/basal-ganglia structures. In order to contrast lexicon and grammar while holding other factors constant, we have focused our investigations of the declarative/procedural model on morphologically complex word forms. Morphological transformations that are (largely) unproductive (e.g., in go—went, solemn—solemnity) are hypothesized to depend upon declarative memory. These have been contrasted with morphological transformations that are fully productive (e.g., in walk—walked, happy—happiness), whose computation is posited to be solely dependent upon grammatical rules subserved by the procedural system. Here evidence is presented from studies that use a range of psycholinguistic and neurolinguistic approaches with children and adults. It is argued that converging evidence from these studies supports the declarative/procedural model of lexicon and grammar.     

Testosterone reactivity to infant crying and caregiving in women: The role of oral contraceptives and basal cortisol

Several studies have shown that mothers and fathers have significant lower levels of testosterone (T) than non-mothers and non-fathers, and that in men caregiving is related to a decrease in T. To date, only a few studies have examined T in women. We examined T reactivity to a crying infant simulator in 160 women. Use of oral contraceptives (OC), basal cortisol (CORT) levels and childhood experiences of maternal love withdrawal were taken into account. T levels were consistently significantly higher in women not using OC. In women not using OC, high basal CORT was related to higher initial T levels and larger decreases of T during caregiving. No effect of basal CORT was found in women with OC use. Childhood experiences of maternal love withdrawal did not affect T levels. This is the first study to show support for a decrease of T in women while taking care of a crying infant, supporting the Challenge hypothesis and the Steroid/Peptide Theory of Social Bonds.     

Open and closed cortico-subcortical loops: A neuro-computational account of access to consciousness in the distractor-induced blindness paradigm

How the brain decides which information to process ‘consciously’ has been debated over for decades without a simple explanation at hand. While most experiments manipulate the perceptual energy of presented stimuli, the distractor-induced blindness task is a prototypical paradigm to investigate gating of information into consciousness without or with only minor visual manipulation. In this paradigm, subjects are asked to report intervals of coherent dot motion in a rapid serial visual presentation (RSVP) stream, whenever these are preceded by a particular color stimulus in a different RSVP stream. If distractors (i.e., intervals of coherent dot motion prior to the color stimulus) are shown, subjects’ abilities to perceive and report intervals of target dot motion decrease, particularly with short delays between intervals of target color and target motion.We propose a biologically plausible neuro-computational model of how the brain controls access to consciousness to explain how distractor-induced blindness originates from information processing in the cortex and basal ganglia. The model suggests that conscious perception requires reverberation of activity in cortico-subcortical loops and that basal-ganglia pathways can either allow or inhibit this reverberation. In the distractor-induced blindness paradigm, inadequate distractor-induced response tendencies are suppressed by the inhibitory ‘hyperdirect’ pathway of the basal ganglia. If a target follows such a distractor closely, temporal aftereffects of distractor suppression prevent target identification. The model reproduces experimental data on how delays between target color and target motion affect the probability of target detection.     

Individual Differences in Reward-Based Learning Predict Fluid Reasoning Abilities

The ability to reason and problem-solve in novel situations, as measured by the Raven's Advanced Progressive Matrices (RAPM), is highly predictive of both cognitive task performance and real-world outcomes. Here we provide evidence that RAPM performance depends on the ability to reallocate attention in response to self-generated feedback about progress. We propose that such an ability is underpinned by the basal ganglia nuclei, which are critically tied to both reward processing and cognitive control. This hypothesis was implemented in a neurocomputational model of the RAPM task, which was used to derive novel predictions at the behavioral and neural levels. These predictions were then verified in one neuroimaging and two behavioral experiments. Furthermore, an effective connectivity analysis of the neuroimaging data confirmed a role for the basal ganglia in modulating attention. Taken together, these results suggest that individual differences in a neural circuit related to reward processing underpin human fluid reasoning abilities.