Learning in a simple motor system

Motor learning is a very basic, essential form of learning that appears to share common mechanisms across different motor systems. We evaluate and compare a few conceptual models for learning in a relatively simple neural system, the vestibulo-ocular reflex (VOR) of vertebrates. We also compare the different animal models that have been used to study the VOR. In the VOR, a sensory signal from the semicircular canals is transformed into a motor signal that moves the eyes. The VOR can modify the transformation under the guidance of vision. The changes are persistent and share some characteristics with other types of associative learning. The cerebellar cortex is directly linked to the VOR reflex circuitry in a partnership that is present in all vertebrates, and which is necessary for motor learning. Early theories of Marr, Albus, and Ito, in which motor memories are stored solely in the cerebellar cortex, have not explained the bulk of the experimental data. Many studies appear to indicate a site of learning in the vestibular nuclei, and the most successful models have incorporated long-term memory storage in both the cerebellar cortex and the brainstem. Plausible cellular mechanisms for learning have been identified in both structures. We propose that short-term motor memory is initially stored in the cerebellar cortex, and that during consolidation of the motor memory the locus of storage shifts to include a brainstem site. We present experimental results that support our hypothesis.     

A model of cerebellar metaplasticity

The term "learning rule" in neural network theory usually refers to a rule for the plasticity of a given synapse, whereas metaplasticity involves a "metalearning algorithm" describing higher level control mechanisms for apportioning plasticity across a population of synapses. We propose here that the cerebellar cortex may use metaplasticity, and we demonstrate this by introducing the Cerebellar Adaptive Rate Learning (CARL) algorithm that concentrates learning on those Purkinje cell synapses whose adaptation is most relevant to learning an overall pattern. Our results show that this biologically plausible metalearning algorithm not only improves significantly the learning capability of the cerebellum but is very robust. Finally, we identify several putative neurochemicals that could be involved in a cascade of events leading to adaptive learning rates in Purkinje cell synapses.     

Reversal of motor learning in the vestibulo-ocular reflex in the absence of visual input

Motor learning in the vestibulo-ocular reflex (VOR) and eyeblink conditioning use similar neural circuitry, and they may use similar cellular plasticity mechanisms. Classically conditioned eyeblink responses undergo extinction after prolonged exposure to the conditioned stimulus in the absence of the unconditioned stimulus. We investigated the possibility that a process similar to extinction may reverse learned changes in the VOR. We induced a learned alteration of the VOR response in rhesus monkeys using magnifying or miniaturizing goggles, which caused head movements to be accompanied by visual image motion. After learning, head movements in the absence of visual stimulation caused a loss of the learned eye movement response. When the learned gain was low, this reversal of learning occurred only when head movements were delivered, and not when the head was held stationary in the absence of visual input, suggesting that this reversal is mediated by an active, extinction-like process.     

Global VOR gain adaptation during near fixation to foveal targets

Long-term rotational vestibulo-ocular (VOR) adaptation occurs during systematic dysmetria between visual and vestibular afferents, adjusting eye-rotation angular velocity to re-establish retinal stability of the visual field. Due to translational motion of the eyes during head rotation, VOR gain is higher when fixating near objects. The current study measures VOR in humans before and after 6 min of exposure to a foveal near-target during sinusoidal whole-body rotation at 0.45 Hz. All of six participants showed post-exposure increases in open-loop VOR gain after fixating near targets, demonstrating a mean modulation increase of open-loop VOR gain from 0.86 before adaptation to 1.2 after adaptation. We discuss a number of theoretical and applied implications.     

Hippocampal Activations during Repetitive Learning and Recall of Geometric Patterns

Hippocampal activation is required for episodic memory. Encoding and retrieval of novel and memorable items have been related to different locations in the hippocampus; however, the data remain ambiguous. The application of a newly designed keyboard allowed investigation of brain activation during encoding and free immediate and delayed recall with functional magnetic resonance imaging (fMRI) in young healthy controls (n=12). Because of the repetitive learning and recall conditions, an individual learning gradient was used to contrast neural activity at different individual levels of novelty. During learning, subjects were asked to memorize 10 geometric patterns requiring the establishment of intra-item associations for memorization. After learning, subjects were asked to recall the items actively via the keyboard. Learning and recall were alternated five times. Delayed recall was scanned about 15 min after the fifth immediate recall condition without subjects having seen the items again. Left-sided anterior hippocampal activity was observed during conditions of initial learning as well as maximum recall. Neural activity during delayed recall did not reveal hippocampal responses and was characterized by a transition of neural activity from occipitoparietal regions to bilateral temporal cortices. We conclude that both lateralization and segregation depend on the specific relational characteristics of the stimuli requiring establishment of intra-item associations for encoding as well as retrieval. The absence of hippocampal activation during delayed recall together with the increase of lateral temporal involvement possibly corresponds with an emerging transition from episodic to long-term memory.     

Ontogenetic changes in the neural mechanisms of eyeblink conditioning

The rodent eyeblink conditioning paradigm is an ideal model system for examining the relationship between neural maturation and the ontogeny of associative learning. Elucidation of the neural mechanisms underlying the ontogeny of learning is tractable using eyeblink conditioning because the necessary neural circuitry (cerebellum and interconnected brainstem nuclei) underlying the acquisition and retention of the conditioned response (CR) has been identified in adult organisms. Moreover, the cerebellum exhibits substantial postnatal anatomical and physiological maturation in rats. The eyeblink CR emerges developmentally between postnatal day (PND) 17 and 24 in rats. A series of experiments found that the ontogenetic emergence of eyeblink conditioning is related to the development of associative learning and not related to changes in performance. More recent studies have examined the relationship between the development of eyeblink conditioning and the physiological maturation of the cerebellum, a brain structure that is necessary for eyeblink conditioning in adult organisms. Disrupting cerebellar development with lesions or antimitotic treatments impairs the ontogeny of eyeblink conditioning. Studies of the development of physiological processes within the cerebellum have revealed striking ontogenetic changes in stimulus-elicited and learning-related neuronal activity. Neurons in the interpositus nucleus and Purkinje cells in the cortex exhibit developmental increases in neuronal discharges following the unconditioned stimulus (US) and in neuronal discharges that model the amplitude and time-course of the eyeblink CR. The developmental changes in CR-related neuronal activity in the cerebellum suggest that the ontogeny of eyeblink conditioning depends on the development of mechanisms that estavlish cerebellar plasticity. Learning and the induction of neural plasticity depend on the magnitude of the US input to the cerebellum. The role of developmental changes in the efficacy of the US pathway has been investigated by monitoring neuronal activity in the inferior olive and with stimulation techniques. The results of these experiments indicate that the development of the conditioned eyeblink response may depend on dynamic interactions between multiple developmental processes within the eyeblink neural circuitry.     

Purkinje cell activity in the cerebellar anterior lobe after rabbit eyeblink conditioning

The cerebellar anterior lobe may play a critical role in the execution and proper timing of learned responses. The current study was designed to monitor Purkinje cell activity in the rabbit cerebellar anterior lobe after eyeblink conditioning, and to assess whether Purkinje cells in recording locations may project to the interpositus nucleus. Rabbits were trained in an interstimulus interval discrimination procedure in which one tone signaled a 250-msec conditioned stimulus-unconditioned stimulus (CS-US) interval and a second tone signaled a 750-msec CS-US interval. All rabbits showed conditioned responses to each CS with mean onset and peak latencies that coincided with the CS-US interval. Many anterior lobe Purkinje cells showed significant learning-related activity after eyeblink conditioning to one or both of the CSs. More Purkinje cells responded with inhibition than with excitation to CS presentation. In addition, when the firing patterns of all conditioning-related Purkinje cells were pooled, it appeared that the population showed a pattern of excitation followed by inhibition during the CS-US interval. Using cholera toxin-conjugated horseradish peroxidase, Purkinje cells in recording areas were found to project to the interpositus nucleus. These data support previous studies that have suggested a role for the anterior cerebellar cortex in eyeblink conditioning as well as models of cerebellar-mediated CR timing that postulate that Purkinje cell activity inhibits conditioned response (CR) generation during the early portion of a trial by inhibiting the deep cerebellar nuclei and permits CR generation during the later portion of a trial through disinhibition of the cerebellar nuclei.     

Neural circuitry and plasticity mechanisms underlying delay eyeblink conditioning

Pavlovian eyeblink conditioning has been used extensively as a model system for examining the neural mechanisms underlying associative learning. Delay eyeblink conditioning depends on the intermediate cerebellum ipsilateral to the conditioned eye. Evidence favors a two-site plasticity model within the cerebellum with long-term depression of parallel fiber synapses on Purkinje cells and long-term potentiation of mossy fiber synapses on neurons in the anterior interpositus nucleus. Conditioned stimulus and unconditioned stimulus inputs arise from the pontine nuclei and inferior olive, respectively, converging in the cerebellar cortex and deep nuclei. Projections from subcortical sensory nuclei to the pontine nuclei that are necessary for eyeblink conditioning are beginning to be identified, and recent studies indicate that there are dynamic interactions between sensory thalamic nuclei and the cerebellum during eyeblink conditioning. Cerebellar output is projected to the magnocellular red nucleus and then to the motor nuclei that generate the blink response(s). Tremendous progress has been made toward determining the neural mechanisms of delay eyeblink conditioning but there are still significant gaps in our understanding of the necessary neural circuitry and plasticity mechanisms underlying cerebellar learning.     

Associative representational plasticity in the auditory cortex: a synthesis of two disciplines

Historically, sensory systems have been largely ignored as potential loci of information storage in the neurobiology of learning and memory. They continued to be relegated to the role of "sensory analyzers" despite consistent findings of associatively induced enhancement of responses in primary sensory cortices to behaviorally important signal stimuli, such as conditioned stimuli (CS), during classical conditioning. This disregard may have been promoted by the fact that the brain was interrogated using only one or two stimuli, e.g., a CS(+) sometimes with a CS(-), providing little insight into the specificity of neural plasticity. This review describes a novel approach that synthesizes the basic experimental designs of the experimental psychology of learning with that of sensory neurophysiology. By probing the brain with a large stimulus set before and after learning, this unified method has revealed that associative processes produce highly specific changes in the receptive fields of cells in the primary auditory cortex (A1). This associative representational plasticity (ARP) selectively facilitates responses to tonal CSs at the expense of other frequencies, producing tuning shifts toward and to the CS and expanded representation of CS frequencies in the tonotopic map of A1. ARPs have the major characteristics of associative memory: They are highly specific, discriminative, rapidly acquired, exhibit consolidation over hours and days, and can be retained indefinitely. Evidence to date suggests that ARPs encode the level of acquired behavioral importance of stimuli. The nucleus basalis cholinergic system is sufficient both for the induction of ARPs and the induction of specific auditory memory. Investigation of ARPs has attracted workers with diverse backgrounds, often resulting in behavioral approaches that yield data that are difficult to interpret. The advantages of studying associative representational plasticity are emphasized, as is the need for greater behavioral sophistication.     

Behavioural context and a distributed system: metabolic mapping studies of the basal ganglia.

Behavioural context is known to affect neural activity in the striatum. Responses of single cells increase to rewarding stimuli, or drop out as a bar press or saccade is learned. Networks that can accomplish a unique response to changing contexts are of particular interest to systems neuroscience and were a part of Hebb's interest in perception and learning. An overall map of the striatum that localizes changes related to this remarkable phenomenon of contextual responses contributes to our understanding of anatomical substrates of neural systems that integrate information, and may lead us to new striatal regions to study synaptic mechanisms of learning.     

Ontogenetic Changes in the Neural Mechanisms of Eyeblink Conditioning

The rodent eyeblink conditioning paradigm is an ideal model system for examining the relationship between neural maturation and the ontogeny of associative learning. Elucidation of the neural mechanisms underlying the ontogeny of learning is tractable using eyeblink conditioning because the necessary neural circuitry (cerebellum and interconnected brainstem nuclei) underlying the acquisition and retention of the conditioned response (CR) has been identified in adult organisms. Moreover, the cerebellum exhibits substantial postnatal anatomical and physiological maturation in rats. The eyeblink CR emerges developmentally between postnatal day (PND) 17 and 24 in rats. A series of experiments found that the ontogenetic emergence of eyeblink conditioning is related to the development of associative learning and not related to changes in performance. More recent studies have examined the relationship between the development of eyeblink conditioning and the physiological maturation of the cerebellum, a brain structure that is necessary for eyeblink conditioning in adult organisms. Disrupting cerebellar development with lesions or antimitotic treatments impairs the ontogeny of eyeblink conditioning. Studies of the development of physiological processes within the cerebellum have revealed striking ontogenetic changes in stimulus-elicited and learning-related neuronal activity. Neurons in the interpositus nucleus and Purkinje cells in the cortex exhibit developmental increases in neuronal discharges following the unconditioned stimulus (US) and in neuronal discharges that model the amplitude and time-course of the eyeblink CR. The developmental changes in CR-related neuronal activity in the cerebellum suggest that the ontogeny of eyeblink conditioning depends on the development of mechanisms that establish cerebellar plasticity. Learning and the induction of neural plasticity depend on the magnitude of the US input to the cerebellum. The role of developmental changes in the efficacy of the US pathway has been investigated by monitoring neuronal activity in the inferior olive and with stimulation techniques. The results of these experiments indicate that the development of the conditioned eyeblink response may depend on dynamic interactions between multiple developmental processes within the eyeblink neural circuitry.     

Functional specificity of the visual word form area: general activation for words and symbols but specific network activation for words

The functional specificity of the brain region known as the Visual Word Form Area (VWFA) was examined using fMRI. We explored whether this area serves a general role in processing symbolic stimuli, rather than being selective for the processing of words. Brain activity was measured during a visual 1-back task to English words, meaningful symbols (e.g., $, %), digits, words in an unfamiliar language (Hebrew), and geometric control stimuli. Mean activity in the functionally defined VWFA, as well as a pattern of whole-brain activity identified using a multivariate technique, did not differ for words and symbols, but was distinguished from that seen with other stimuli. However, functional connectivity analysis of this region identified a network of regions that was specific to words, including the left hippocampus, left lateral temporal, and left prefrontal cortex. Results support the hypothesis that activity in the VWFA plays a general role in processing abstract stimuli; however, the left VWFA is part of a unique network of brain regions active only during the word condition. These findings suggest that it is the neural "context" of the VWFA, i.e., the broader activity distributed in the brain that is correlated with VWFA, that is specific for visual word representation, not activity in this brain region per se.     

Categories in the pigeon brain: A reverse engineering approach

Pigeons are well known for their visual capabilities as well as their ability to categorize visual stimuli at both the basic and superordinate level. We adopt a reverse engineering approach to study categorization learning: Instead of training pigeons on predefined categories, we simply present stimuli and analyze neural output in search of categorical clustering on a solely neural level. We presented artificial stimuli, pictorial and grating stimuli, to pigeons without the need of any differential behavioral responding while recording from the nidopallium frontolaterale (NFL), a higher visual area in the avian brain. The pictorial stimuli differed in color and shape; the gratings differed in spatial frequency and amplitude. We computed representational dissimilarity matrices to reveal categorical clustering based on both neural data and pecking behavior. Based on neural output of the NFL, pictorial and grating stimuli were differentially represented in the brain. Pecking behavior showed a similar pattern, but to a lesser extent. A further subclustering within pictorial stimuli according to color and shape, and within gratings according to frequency and amplitude, was not present. Our study gives proof‐of‐concept that this reverse engineering approach—namely reading out categorical information from neural data—can be quite helpful in understanding the neural underpinnings of categorization learning.     

Functional specialization within the striatum along both the dorsal/ventral and anterior/posterior axes during associative learning via reward and punishment

The goal of the present study was to elucidate the role of the human striatum in learning via reward and punishment during an associative learning task. Previous studies have identified the striatum as a critical component in the neural circuitry of reward-related learning. It remains unclear, however, under what task conditions, and to what extent, the striatum is modulated by punishment during an instrumental learning task. Using high-resolution functional magnetic resonance imaging (fMRI) during a reward- and punishment-based probabilistic associative learning task, we observed activity in the ventral putamen for stimuli learned via reward regardless of whether participants were correct or incorrect (i.e., outcome). In contrast, activity in the dorsal caudate was modulated by trials that received feedback--either correct reward or incorrect punishment trials. We also identified an anterior/posterior dissociation reflecting reward and punishment prediction error estimates. Additionally, differences in patterns of activity that correlated with the amount of training were identified along the anterior/posterior axis of the striatum. We suggest that unique subregions of the striatum--separated along both a dorsal/ventral and anterior/posterior axis--differentially participate in the learning of associations through reward and punishment.     

Eyeblink Classical Conditioning Differentiates Normal Aging from Alzheimer's Disease

Eyeblink classical conditioning is a useful paradigm for the study of the neurobiology of learning, memory, and aging, which also has application in the differential diagnosis of neurodegenerative diseases expressed in advancing age. Converging evidence from studies of eyeblink conditioning in neurological patients and brain imaging in normal adults document parallels in the neural substrates of this form of associative learning in humans and non-human mammals. Age differences in the short-delay procedure (400 ms CS-US interval) appear in middle age in humans and may be caused at least in part by cerebellar cortical changes such as loss of Purkinje cells. Whereas the hippocampus is not essential for conditioning in the delay procedure, disruption of hippocampal cholinergic neurotransmission impairs acquisition and slows the rate of learning. Alzheimer's disease (AD) profoundly disrupts the hippocampaL cholinergic system, and patients with AD consistently perform poorly in eyeblink conditioning. We hypothesize that disruption of hippocampal cholinergic pathways in AD in addition to age-associated Purkinje cell loss results in severely impaired eyeblink conditioning. The earliest pathology in AD occurs in entorhinal cortical input to hippocampus, and eyeblink conditioning may detect this early disruption before declarative learning and memory circuits become impaired. A case study is presented in which eyeblink conditioning detected impending dementia six years before changes on other screening tests indicated impairment. Because eyeblink conditioning is simple, non-threatening, and non-invasive, it may become a useful addition to test batteries designed to differentiate normal aging from mild cognitive impairment that progresses to AD and AD from other types of dementia.     

Synaptic plasticity in the lateral amygdala: a cellular hypothesis of fear conditioning

Fear conditioning is a form of associative learning in which subjects come to express defense responses to a neutral conditioned stimulus (CS) that is paired with an aversive unconditioned stimulus (US). Considerable evidence suggests that critical neural changes mediating the CS-US association occur in the lateral nucleus of the amygdala (LA). Further, recent studies show that associative long-term potentiation (LTP) occurs in pathways that transmit the CS to LA, and that drugs that interfere with this LTP also disrupt behavioral fear conditioning when infused into the LA, suggesting that associative LTP in LA might be a mechanism for storing memories of the CS-US association. Here, we develop a detailed cellular hypothesis to explain how neural responses to the CS and US in LA could induce LTP-like changes that store memories during fear conditioning. Specifically, we propose that the CS evokes EPSPs at sensory input synapses onto LA pyramidal neurons, and that the US strongly depolarizes these same LA neurons. This depolarization, in turn, causes calcium influx through NMDA receptors (NMDARs) and also causes the LA neuron to fire action potentials. The action potentials then back-propagate into the dendrites, where they collide with CS-evoked EPSPs, resulting in calcium entry through voltage-gated calcium channels (VGCCs). Although calcium entry through NMDARs is sufficient to induce synaptic changes that support short-term fear memory, calcium entry through both NMDARs and VGCCs is required to initiate the molecular processes that consolidate synaptic changes into a long-term memory.     

The level of cholinergic nucleus basalis activation controls the specificity of auditory associative memory

Learning involves not only the establishment of memory per se, but also the specific details of its contents. In classical conditioning, the former concerns whether an association was learned while the latter discloses what was learned. The neural bases of associativity have been studied extensively while neural mechanisms of memory specificity have been neglected. Stimulation of the cholinergic nucleus basalis (NBs) paired with a preceding tone induces CS-specific associative memory. As different levels of acetylcholine may be released naturally during different learning situations, we asked whether the level of activation of the cholinergic neuromodulatory system can control the degree of detail that is encoded and retrieved. Adult male rats were tested pre- and post-training for behavioral responses (interruption of ongoing respiration) to tones of various frequencies (1-15 kHz, 70 dB, 2 s). Training consisted of 200 trials/day of tone (8.0 kHz, 70 dB, 2 s) either paired or unpaired with NBs (CS-NBs = 1.8 s) at moderate (65.7+/-9.0 microA, one day) or weak (46.7+/-12.1 microA, three training days) levels of stimulation, under conditions of controlled behavioral state (pre-trial stable respiration rate). Post-training (24 h) responses to tones revealed that moderate activation induced both associative and CS-specific behavioral memory, whereas weak activation produced associative memory lacking frequency specificity. The degree of memory specificity 24 h after training was positively correlated with the magnitude of CS-elicited increase in gamma activity within the EEG during training, but only in the moderate NBs group. Thus, a low level of acetylcholine released by the nucleus basalis during learning is sufficient to induce associativity whereas a higher level of release enables the storage of greater experiential detail. gamma waves, which are thought to reflect the coordinated activity of cortical cells, appear to index the encoding of CS detail. The findings demonstrate that the amount of detail in memory can be directly controlled by neural intervention.     

Tracing trajectories of audio‐visual learning in the infant brain

Although infants begin learning about their environment before they are born, little is known about how the infant brain changes during learning. Here, we take the initial steps in documenting how the neural responses in the brain change as infants learn to associate audio and visual stimuli. Using functional near‐infrared spectroscopy (fNRIS) to record hemodynamic responses in the infant cortex (temporal, occipital, and frontal cortex), we find that across the infant brain, learning is characterized by an increase in activation followed by a decrease. We take this U‐shaped response as evidence of repetition enhancement during early stages of learning and repetition suppression during later stages, a result that mirrors the Hunter and Ames model of infant visual preference. Furthermore, we find that the neural response to violations of the learned associations can be predicted by the shape of the learning curve in temporal and occipital cortex. These data provide the first look at the shape of the neural response during audio‐visual associative learning in infancy establishing that diverse regions of the infant brain exhibit systematic changes across the time‐course of learning.