Tracing trajectories of audio‐visual learning in the infant brain

Although infants begin learning about their environment before they are born, little is known about how the infant brain changes during learning. Here, we take the initial steps in documenting how the neural responses in the brain change as infants learn to associate audio and visual stimuli. Using functional near‐infrared spectroscopy (fNRIS) to record hemodynamic responses in the infant cortex (temporal, occipital, and frontal cortex), we find that across the infant brain, learning is characterized by an increase in activation followed by a decrease. We take this U‐shaped response as evidence of repetition enhancement during early stages of learning and repetition suppression during later stages, a result that mirrors the Hunter and Ames model of infant visual preference. Furthermore, we find that the neural response to violations of the learned associations can be predicted by the shape of the learning curve in temporal and occipital cortex. These data provide the first look at the shape of the neural response during audio‐visual associative learning in infancy establishing that diverse regions of the infant brain exhibit systematic changes across the time‐course of learning.     

Hemodynamic and electrophysiological connectivity in the language system: Simultaneous near-infrared spectroscopy and electrocorticography recordings during cortical stimulation

We applied near-infrared spectroscopy (NIRS) and electrocorticography (ECoG) recordings during cortical stimulation to a temporal lobe epilepsy patient who underwent subdural electrode implantation. Using NIRS, changes in blood concentrations of oxyhemoglobin (HbO(2)) and deoxyhemoglobin (HbR) during cortical stimulation of the left language areas were measured in each hemisphere. NIRS revealed that 2Hz stimulation with 5mA produced no significant hemodynamic changes in either hemisphere. By contrast, 50Hz stimulation elicited significant increases in both HbO(2) and HbR at the stimulation site. Furthermore, with 50Hz stimulation of the left superior temporal gyrus, the increases in HbO(2) and HbR were observed not only at the stimulation site but also concurrently at the left inferior frontal gyrus. This suggests the existence of functional connectivity in the language system. The present study demonstrates that simultaneous NIRS and ECoG studies during cortical stimulation allow a novel analysis of cerebral connectivity.     

Frontotemporal neural systems supporting semantic processing in Alzheimer’s disease

We hypothesized that semantic memory for object concepts involves both representations of visual feature knowledge in modality-specific association cortex and heteromodal regions that are important for integrating and organizing this semantic knowledge so that it can be used in a flexible, contextually appropriate manner. We examined this hypothesis in an fMRI study of mild Alzheimer’s disease (AD). Participants were presented with pairs of printed words and asked whether the words matched on a given visual–perceptual feature (e.g., guitar, violin: SHAPE). The stimuli probed natural kinds and manufactured objects, and the judgments involved shape or color. We found activation of bilateral ventral temporal cortex and left dorsolateral prefrontal cortex during semantic judgments, with AD patients showing less activation of these regions than healthy seniors. Moreover, AD patients showed less ventral temporal activation than did healthy seniors for manufactured objects, but not for natural kinds. We also used diffusion-weighted MRI of white matter to examine fractional anisotropy (FA). Patients with AD showed significantly reduced FA in the superior longitudinal fasciculus and inferior frontal-occipital fasciculus, which carry projections linking temporal and frontal regions of this semantic network. Our results are consistent with the hypothesis that semantic memory is supported in part by a large-scale neural network involving modality-specific association cortex, heteromodal association cortex, and projections between these regions. The semantic deficit in AD thus arises from gray matter disease that affects the representation of feature knowledge and processing its content, as well as white matter disease that interrupts the integrated functioning of this large-scale network.     

Cortical integration of audio–visual speech and non-speech stimuli

Using fMRI we investigated the neural basis of audio–visual processing of speech and non-speech stimuli using physically similar auditory stimuli (speech and sinusoidal tones) and visual stimuli (animated circles and ellipses). Relative to uni-modal stimuli, the different multi-modal stimuli showed increased activation in largely non-overlapping areas. Ellipse-Speech, which most resembles naturalistic audio–visual speech, showed higher activation in the right inferior frontal gyrus, fusiform gyri, left posterior superior temporal sulcus, and lateral occipital cortex. Circle-Tone, an arbitrary audio–visual pairing with no speech association, activated middle temporal gyri and lateral occipital cortex. Circle-Speech showed activation in lateral occipital cortex, and Ellipse-Tone did not show increased activation relative to uni-modal stimuli. Further analysis revealed that middle temporal regions, although identified as multi-modal only in the Circle-Tone condition, were more strongly active to Ellipse-Speech or Circle-Speech, but regions that were identified as multi-modal for Ellipse-Speech were always strongest for Ellipse-Speech. Our results suggest that combinations of auditory and visual stimuli may together be processed by different cortical networks, depending on the extent to which multi-modal speech or non-speech percepts are evoked.     

客体在视觉工作记忆中的存储机制

采用变化觉察范式,通过两个实验对不同类型视觉信息在工作记忆中的存储机制进行了探讨。实验一中要求记忆的客体由颜色和形状两基本特征构成;实验二中的客体为具有不同颜色及开口朝向的兰道环。通过观察检测项变化方式对记忆绩效的影响,获得以下结论:（1）对颜色和形状两基本特征的存储以整合客体为单位;（2）兰道环的颜色与开口朝向信息难以以整合客体方式存储。笔者推测,视觉工作记忆中存储的整合客体并非如现有理论假设的,是在集中注意参与下创建的,而是由并行加工阶段所获信息构成;客体中所含需集中注意提取的细节特征信息难以与其基本特征信息整合于该客体表征     

Rapid development of feature binding in visual short-term memory

The binding of object identity (color) and location in visual short-term memory (VSTM) was examined in 6.5- to 12.5-month-old infants (N= 144). Although we previously found that by age 6.5 months, infants can represent both color and location in VSTM, in the present study we observed that 6.5-month-old infants could not remember trivially simple color-location combinations across a 300-ms delay. However, 7.5-month-old infants could bind color and location as effectively as 12.5-month-old infants. Control conditions confirmed that the failure of 6.5-month-old infants was not a result of perceptual or attentional limitations. This rapid development of VSTM binding between 6.5 and 7.5 months occurs during a period of rapid increase in VSTM storage capacity and just after a period of dramatic neuroanatomical changes in parietal cortex. Thus, the ability to bind features and the ability to store multiple objects may both depend on a process that is mediated by posterior parietal cortex and is perhaps related to focused attention.     

The Development of Infants' Sensitivity to Arbitrary Intermodal Relations

The development of infants' ability to detect the arbitrary relation between the color/shape of an object and the pitch of its impact sound was investigated using an infant-control habituation procedure. Ninety-six infants of 3, 5, or 7 months were habituated to films of two objects differing in color and shape, striking a surface in an erratic pattern. One object produced an impact sound of a high pitch, and the other object produced a low pitch. During test trials, infants in the experimental conditions received a change in the pairing of pitch with color/shape, whereas controls received no change. Results indicate that visual recovery to the change in pitch-color/shape relations was significantly greater than that of age-matched controls at 7 months, but not at 3 or 5 months. A prior study demonstrated that by 3 months, infants were able to discriminate the color/shape and pitch changes of these events. However, it is not until 7 months that they show evidence of detecting the arbitrary relation between these attributes.     

Inhibition is picky: Shape difference is a necessary condition for attentional inhibition of irrelevant objects

The present study investigated the conditions for attentional amplification in the processing of relevant objects and for attentional inhibition in the processing of irrelevant objects. Participants reported the color of one of two superimposed objects that was specified by occlusion. Irrelevant color words were presented as part of the relevant object, as part of the irrelevant object, or in the background; the words were either congruent or incongruent with the color of the relevant object. The size of the congruency effects provided a measure of the (relative) strength of the processing of the objects and the background. Finally, the two objects had the same shape in one session and different shapes in another session. In both sessions, results showed larger Stroop effects from words belonging to the relevant object than from words in the background, indicating attentional amplification of the relevant object that was unaffected by object similarity. In contrast, smaller Stroop effects from words belonging to the irrelevant object than from words in the background occurred only when the two objects differed in shape. The latter result suggests that a shape difference is both a necessary and a sufficient condition for inhibiting the processing of an irrelevant visual object.     

Fast temporal dynamics of visual cue integration

The integration of information from different sensors, cues, or modalities lies at the very heart of perception. We are studying adaptive phenomena in visual cue integration. To this end, we have designed a visual tracking task, where subjects track a target object among distractors and try to identify the target after an occlusion. Objects are defined by three different attributes (color, shape, size) which change randomly within a single trial. When the attributes differ in their reliability (two change frequently, one is stable), our results show that subjects dynamically adapt their processing. The results are consistent with the hypothesis that subjects rapidly re-weight the information provided by the different cues by emphasizing the information from the stable cue. This effect seems to be automatic, ie not requiring subjects' awareness of the differential reliabilities of the cues. The hypothesized re-weighting seems to take place in about 1 s. Our results suggest that cue integration can exhibit adaptive phenomena on a very fast time scale. We propose a probabilistic model with temporal dynamics that accounts for the observed effect.     

Binocular vision and spatial perception in 4- and 5-month-old infants

Four experiments investigated the relation between the development of binocular vision and infant spatial perception. Experiments 1 and 2 compared monocular and binocular depth perception in 4- and 5-month-old infants. Infants in both age groups reached more consistently for the nearer of two objects under binocular viewing conditions than under monocular viewing conditions. Experiments 3 and 4 investigated whether the superiority of binocular depth perception in 4-month-olds is related to the development of sensitivity to binocular disparity. Under binocular viewing conditions in Experiment 3, infants identified as disparity-sensitive reached more consistently for the nearer object than did infants identified as disparity-insensitive. The two groups' performances did not differ under monocular viewing conditions. These results suggest that, binocularly, the disparity-sensitive infants perceived the objects' distances more accurately than did the disparity-insensitive infants. In Experiment 4, infants were habituated to an object, then presented with the same object and a novel object that differed only in size. Disparity-sensitive infants showed size constancy by recovering from habituation when viewing the novel object. Disparity-insensitive infants did not show clear evidence of size constancy. These findings suggest that the development of sensitivity to binocular disparity is accompanied by a substantial increase in the accuracy of infant spatial perception.     

Synchronous change and perception of object unity: evidence from adults and infants.

Adults and infants display a robust ability to perceive the unity of a center-occluded object when the visible ends of the object undergo common motion (e.g. Kellman, P.J., Spelke, E.S., 1983. Perception of partly occluded objects in infancy. Cognitive Psychology 15, 483-524). Ecologically oriented accounts of this ability focus on the primary of motion in the perception of segregated objects, but Gestalt theory suggests a broader possibility: observers may perceive object unity by detecting patterns of synchronous change, of which common motion is a special case. We investigated this possibility with observations of adults and 4-month-old infants. Participants viewed a center-occluded object whose visible surfaces were either misaligned or aligned, stationary or moving, and unchanging or synchronously changing in color or brightness in various temporal patterns (e.g. flashing). Both alignment and common motion contributed to adults' perception of object unity, but synchronous color changes did not. For infants, motion was an important determinant of object unity, but other synchronous changes and edge alignment were not. When a stationary object with aligned edges underwent synchronous changes in color or brightness, infants showed high levels of attention to the object, but their perception of its unity appeared to be indeterminate. An inherent preference for fast over slow flash rates, and a novelty preference elicited by a change in rate, both indicated that infants detected the synchronous changes, although they failed to use them as information for object unity. These findings favor ecologically oriented accounts of object perception in which surface motion plays a privileged role.     

Multisensory cortical processing of object shape and its relation to mental imagery

Here, we used functional magnetic resonance imaging to investigate the multisensory processing of object shape in the human cerebral cortex and explored the role of mental imagery in such processing. Regions active bilaterally during both visual and haptic shape perception, relative to texture perception in the respective modality, included parts of the superior parietal gyrus, the anterior intraparietal sulcus, and the lateral occipital complex. Of these bimodal regions, the lateral occipital complexes preferred visual over haptic stimuli, whereas the parietal areas preferred haptic over visual stimuli. Whereas most subjects reported little haptic imagery during visual shape perception, experiences of visual imagery during haptic shape perception were common. Across subjects, ratings of the vividness of visual imagery strongly predicted the amount of haptic shape-selective activity in the right, but not in the left, lateral occipital complex. Thus, visual imagery appears to contribute to activation of some, but not all, visual cortical areas during haptic perception.     

Visual imagery and knowledge about the visual appearance of objects in patients with posterior cerebral artery lesions

The purpose of this study was to explore whether patients with left posterior cerebral artery (PCA) lesions have defective mental visual imagery and to differentiate whether such a deficit stems from a loss of knowledge about the visual appearance of objects or from an inability to create mental visual images out of preserved visual knowledge. Normal controls and patients with either left or right PCA lesions were asked to verify low- and high-imagery sentences and then to verify pictorial representations of the predicates of high-imagery sentences. High-imagery questions concerned either the shape or the color of objects. In addition, perceptual discrimination of shape and color was assessed. Patients with right PCA lesions were impaired on the perceptual discrimination tasks. Left PCA patients did not significantly differ from controls on low-imagery sentences but scored significantly lower on shape and color questions. Their impairment was distinctly more severe with the pictorial than with the verbal versions of shape and color questions. In comparison to patients with occipital left PCA lesions, patients with temporo-occipital left PCA lesions were selectively impaired on verbal and visual color questions. It is concluded that patients with left PCA lesions lack knowledge about shape and color of objects, but that their ability to convert visual knowledge into mental images is preserved.     

The effects of joint attention on object processing in 4- and 9-month-old infants

In this study, we examined the effects of joint attention on object processing in 4- and 9-month-old infants. An adult experimenter differed social cues while speaking to infants about a novel object. Only 9-month-olds showed evidence of enhanced object processing following a joint attention interaction.     

Functional brain organization for number processing in pre‐verbal infants

Humans are born with the ability to mentally represent the approximate numerosity of a set of objects, but little is known about the brain systems that sub‐serve this ability early in life and their relation to the brain systems underlying symbolic number and mathematics later in development. Here we investigate processing of numerical magnitudes before the acquisition of a symbolic numerical system or even spoken language, by measuring the brain response to numerosity changes in pre‐verbal infants using functional near‐infrared spectroscopy (fNIRS). To do this, we presented infants with two types of numerical stimulus blocks: number change blocks that presented dot arrays alternating in numerosity and no change blocks that presented dot arrays all with the same number. Images were carefully constructed to rule out the possibility that responses to number changes could be due to non‐numerical stimulus properties that tend to co‐vary with number. Interleaved with the two types of numerical blocks were audio‐visual animations designed to increase attention. We observed that number change blocks evoked an increase in oxygenated hemoglobin over a focal right parietal region that was greater than that observed during no change blocks and during audio‐visual attention blocks. The location of this effect was consistent with intra‐parietal activity seen in older children and adults for both symbolic and non‐symbolic numerical tasks. A distinct set of bilateral occipital and middle parietal channels responded more to the attention‐grabbing animations than to either of the types of numerical stimuli, further dissociating the specific right parietal response to number from a more general bilateral visual or attentional response. These results provide the strongest evidence to date that the right parietal cortex is specialized for numerical processing in infancy, as the response to number is dissociated from visual change processing and general attentional processing.     

Understanding the abstract role of speech in communication at 12 months

Adult humans recognize that even unfamiliar speech can communicate information between third parties, demonstrating an ability to separate communicative function from linguistic content. We examined whether 12-month-old infants understand that speech can communicate before they understand the meanings of specific words. Specifically, we test the understanding that speech permits the transfer of information about a Communicator's target object to a Recipient. Initially, the Communicator selectively grasped one of two objects. In test, the Communicator could no longer reach the objects. She then turned to the Recipient and produced speech (a nonsense word) or non-speech (coughing). Infants looked longer when the Recipient selected the non-target than the target object when the Communicator had produced speech but not coughing (Experiment 1). Looking time patterns differed from the speech condition when the Recipient rather than the Communicator produced the speech (Experiment 2), and when the Communicator produced a positive emotional vocalization (Experiment 3), but did not differ when the Recipient had previously received information about the target by watching the Communicator's selective grasping (Experiment 4). Thus infants understand the information-transferring properties of speech and recognize some of the conditions under which others' information states can be updated. These results suggest that infants possess an abstract understanding of the communicative function of speech, providing an important potential mechanism for language and knowledge acquisition.     

Longitudinal expressions of infant's prehension as a function of object properties

This longitudinal study examined the prehensile development of infants (9-37 weeks) under different task constraints (object shape, size and texture). At 9 weeks of age, the infants did not reach or make contact with the objects, but all 10 infants showed goal directed prehensile movement by about 17 weeks. As they continued to age the infants further differentiated an adaptive prehensile grip configuration to the object constraints. The pattern of longitudinal findings provides further evidence that: (1) the constraints of object properties play an important role in channeling the expressions of infant's prehensile functioning and (2) the classic observation by Halverson [Halverson, H. M. (1931). An experimental study of prehension in infants by means of systematic cinema records. Genetic Psychology Monographs, 10, 107-283] of an apparent order to the development of the fundamental prehensile sequence is task dependent.     

Brain activity differentiates face and object processing in 6-month-old infants.

Event-related potentials were used to determine whether infants, like adults, show differences in spatial and temporal characteristics of brain activation during face and object recognition. Three aspects of visual processing were identified: (a) differentiation of face vs. object (P400 at occipital electrode was shorter latency for faces), (b) recognition of familiar identity (Nc, or negative component, at fronto-temporal electrodes [FTEs] was of larger amplitude for familiar stimuli), and (c) encoding novelty (slow wave at FTEs was larger for unfamiliar stimuli). The topography of the Nc was influenced by category type: Effects of familiarity were limited to the midline and right anterior temporal electrodes for faces but extended to all temporal electrodes for objects. Results show that infants' experience with specific examples within categories and their general category knowledge influence the neural correlates of visual processing.     

特征变化的连续性影响客体连续表征的维持

本研究采用客体回溯范式考察了特征变化的连续性对维持客体连续表征的作用。实验1和实验2分别探索了形状维度上的变化方式（不变、渐变、突变）和亮度维度上的变化方式（不变、渐变、随机变化）对客体预览利化效应的影响。在特征连续条件下（不变或渐变）,两个实验都获得了客体预览利化效应。而在特征不连续变化条件下（突变或随机变化）,该效应消失。这些结果说明特征变化的连续性同样影响客体连续表征的维持。     

Neural recruitment related to threat perception differs as a function of adolescent sleep

Detecting threat cues in the environment is an important aspect of social functioning. This is particularly true for adolescents as social threats become more salient and they navigate increasingly complex relationships outside of the family. Sleep relates to socioemotional processing throughout development, but the neurobiological relevance of sleep for threat perceptions in adolescence remains unknown. In the present study, 46 human adolescents (aged 14–18 years; 26 female) made judgments while undergoing a brain scan about whether unfamiliar, affectively neutral, computer‐generated faces were threatening. Prior to the scan, several indices of sleep were assessed nightly for two‐weeks using actigraphy. Sleep duration and poor sleep quality (defined as less efficiency, more awakenings, longer awakenings), factors influenced by biological and psychosocial changes during adolescence, elicited distinct neural activation patterns. Sleep duration was positively associated with activation in visual and face processing regions (occipital cortex, occipital fusiform gyrus), and this activation was linked to increased threat detection during the threat perception task. Sleep quality was negatively related to dorsolateral prefrontal cortex activation, which moderated the relation between reaction time (RT) and exposure to faces. Findings suggest reduced threat perception for adolescents with shorter sleep durations and more impulsive responding (as evinced by less consistent RT) for adolescents experiencing worse quality sleep. This study identifies an association between sleep and neural functioning relevant for socioemotional decision making during adolescence, a time when these systems undergo significant development.