Synchrony,complexity and directiveness in mothers’ interactions with infants pre- and post-cochlear implantation

This study investigated effects of profound hearing loss on mother–infant interactions before and after cochlear implantation with a focus on maternal synchrony, complexity, and directiveness. Participants included two groups of mother–infant dyads: 9 dyads of mothers and infants with normal hearing; and 9 dyads of hearing mothers and infants with profound hearing loss. Dyads were observed at two time points: Time 1, scheduled to occur before cochlear implantation for infants with profound hearing loss (mean age = 13.6 months); and Time 2 (mean age = 23.3 months), scheduled to occur approximately six months after cochlear implantation. Hearing infants were age-matched to infants with hearing loss at both time points. Dependent variables included the proportion of maternal utterances that overlapped infant vocalizations, maternal mean length of utterance, infant word use, and combined maternal directives and prohibitions. Results showed mothers’ utterances overlapped the vocalizations of infants with hearing loss more often before cochlear implantation than after, mothers used less complex utterances with infants with cochlear implants compared to hearing peers (Time 2), and mothers of infants with profound hearing loss used frequent directives and prohibitions both before and after cochlear implantation. Together, mothers and infants adapted relatively quickly to infants’ access to cochlear implants, showing improved interactional synchrony, increased infant word use, and levels of maternal language complexity compatible with infants’ word use, all within seven months of cochlear implant activation.     

Is neocortex essentially multisensory?

Although sensory perception and neurobiology are traditionally investigated one modality at a time, real world behaviour and perception are driven by the integration of information from multiple sensory sources. Mounting evidence suggests that the neural underpinnings of multisensory integration extend into early sensory processing. This article examines the notion that neocortical operations are essentially multisensory. We first review what is known about multisensory processing in higher-order association cortices and then discuss recent anatomical and physiological findings in presumptive unimodal sensory areas. The pervasiveness of multisensory influences on all levels of cortical processing compels us to reconsider thinking about neural processing in unisensory terms. Indeed, the multisensory nature of most, possibly all, of the neocortex forces us to abandon the notion that the senses ever operate independently during real-world cognition.     

The development of multisensory processes

To understand the development of sensory processes, it is necessary not only to look at the maturation of each of the sensory systems in isolation, but also to study the development of the nervous systems capacity to integrate information across the different senses. It is through such multisensory integration that a coherent perceptual gestalt of the world comes to be generated. In the adult brain, multisensory convergence and integration take place at a number of brainstem and cortical sites, where individual neurons have been found that respond to multisensory stimuli with patterns of activation that depend on the nature of the stimulus complex and the intrinsic properties of the neuron. Parallels between the responses of these neurons and multisensory behavior and perception suggest that they are the substrates that underlie these cognitive processes. In both cat and monkey models, the development of these multisensory neurons and the appearance of their integrative capacity is a gradual postnatal process. For subcortical structures (i.e., the superior colliculus) this maturational process appears to be gated by the appearance of functional projections from regions of association cortex. The slow postnatal maturation of multisensory processes, coupled with its dependency on functional corticotectal connections, suggested that the development of multisensory integration may be tied to sensory experiences acquired during postnatal life. In support of this, eliminating experience in one sensory modality (i.e., vision) during postnatal development severely compromises the integration of multisensory cues. Research is ongoing to better elucidate the critical development antecedents for the emergence of normal multisensory capacity.Edited by Marie-Hélène Giard and Mark WallaceThis revised version was published in May 2004 with corrections to Fig. 1.     

THE IMPACT OF AUDITION ON THE DEVELOPMENT OF VISUAL ATTENTION

Interactions between audition and vision were investigated in two experiments In the first experiment, school-age hearing children, deaf children with cochlear implants, and deaf children without implants participated in a task in which they were to respond to some visual signals and not others This task did not involve sound at all Deaf children without implants performed much more poorly than hearing children Deaf children with cochlear implants performed considerably better than deaf children without implants The second experiment employed a longitudinal design and showed that the rate of development in visual selective attention was faster for deaf children with cochlear implants than deaf children without implants Moreover, the gams were rapid–occurring within 2 years post-implant surgery The results suggest that a history of experience with sounds matters in the development of visual attention The results are discussed in terms of multimodal developmental processes     

Cognitive and linguistic skills in Swedish children with cochlear implants - measures of accuracy and latency as indicators of development

The purpose of the present study was to examine working memory (WM) capacity, lexical access and phonological skills in 19 children with cochlear implants (CI) (5;7-13;4 years of age) attending grades 0-2, 4, 5 and 6 and to compare their performance with 56 children with normal hearing. Their performance was also studied in relation to demographic factors. The findings indicate that children with CI had visuospatial WM capacities equivalent to the comparison group. They had lower performance levels on most of the other cognitive tests. Significant differences between the groups were not found in all grades and a number of children with CI performed within 1 SD of the mean of their respective grade-matched comparison group on most of the cognitive measures. The differences between the groups were particularly prominent in tasks of phonological WM. The results are discussed with respect to the effects of cochlear implants on cognitive development.     

Audiotactile multisensory interactions in human information processing

Abstract:  The last few years has seen a very rapid growth of interest in how signals from different sensory modalities are integrated in the brain to form the unified percepts that fill our daily lives. Research on multisensory interactions between vision, touch, and proprioception has revealed the existence of multisensory spatial representations that code the location of external events relative to our own bodies. In this review, we highlight recent converging evidence from both human and animal studies that has revealed that spatially-modulated multisensory interactions also occur between hearing and touch, especially in the space immediately surrounding the head. These spatial audiotactile interactions for stimuli presented close to the head can affect not only the spatial aspects of perception, but also various other non-spatial aspects of audiotactile information processing. Finally, we highlight some of the most important questions for future research in this area.     

Language development in profoundly deaf children with cochlear implants

Although cochlear implants improve the ability of profoundly deaf children to understand speech, critics claim that the published literature does not document even a single case of a child who has developed a linguistic system based on input from an implant. Thus, it is of clinical and scientific importance to determine whether cochlear implants facilitate the development of English language skills. The English language skills of prelingually deaf children with cochlear implants were measured before and after implantation. We found that the rate of language development after implantation exceeded that expected from unimplanted deaf children ( p < .001) and was similar to that of children with normal hearing. Despite a large amount of individual variability, the best performers in the implanted group seem to be developing an oral linguistic system based largely on auditory input obtained from a cochlear implant.     

Grape expectations: The role of cognitive influences in color–flavor interactions

Color conveys critical information about the flavor of food and drink by providing clues as to edibility, flavor identity, and flavor intensity. Despite the fact that more than 100 published papers have investigated the influence of color on flavor perception in humans, surprisingly little research has considered how cognitive and contextual constraints may mediate color–flavor interactions. In this review, we argue that the discrepancies demonstrated in previously-published color–flavor studies may, at least in part, reflect differences in the sensory expectations that different people generate as a result of their prior associative experiences. We propose that color–flavor interactions in flavor perception cannot be understood solely in terms of the principles of multisensory integration (the currently dominant theoretical framework) but that the role of higher-level cognitive factors, such as expectations, must also be considered.     

The mathematical abilities of children with cochlear implants

Research has shown that cochlear implants give rise to improvements in speech recognition and production in children with profound hearing loss but very few studies have explored mathematical abilities in these children. The current study compared the mathematical abilities of 24 children with cochlear implants (mean age 10 years 1 month) to a control group of 22 hearing children (mean age 9 years 8 months). The math questions were categorized into questions that tapped into arithmetic or geometrical reasoning. It was predicted that the cochlear implant group would perform below the hearing group on the arithmetic questions but not the geometrical reasoning questions. Unexpectedly, the results showed that the cochlear implant group performed significantly below the hearing group on both types of math questions, but that this difference was mediated by language skill as assessed by vocabulary knowledge. The clinical implications of these results and possible future research results are considered.     

A rational analysis of the acquisition of multisensory representations

How do people learn multisensory, or amodal, representations, and what consequences do these representations have for perceptual performance? We address this question by performing a rational analysis of the problem of learning multisensory representations. This analysis makes use of a Bayesian nonparametric model that acquires latent multisensory features that optimally explain the unisensory features arising in individual sensory modalities. The model qualitatively accounts for several important aspects of multisensory perception: (a) it integrates information from multiple sensory sources in such a way that it leads to superior performances in, for example, categorization tasks; (b) its performances suggest that multisensory training leads to better learning than unisensory training, even when testing is conducted in unisensory conditions; (c) its multisensory representations are modality invariant; and (d) it predicts ‘‘missing” sensory representations in modalities when the input to those modalities is absent. Our rational analysis indicates that all of these aspects emerge as part of the optimal solution to the problem of learning to represent complex multisensory environments.     

Audio-visual simultaneity judgments

The relative spatiotemporal correspondence between sensory events affects multisensory integration across a variety of species; integration is maximal when stimuli in different sensory modalities are presented from approximately the same position at about the same time. In the present study, we investigated the influence of spatial and temporal factors on audio-visual simultaneity perception in humans. Participants made unspeeded simultaneous versus successive discrimination responses to pairs of auditory and visual stimuli presented at varying stimulus onset asynchronies from either the same or different spatial positions using either the method of constant stimuli (Experiments 1 and 2) or psychophysical staircases (Experiment 3). The participants in all three experiments were more likely to report the stimuli as being simultaneous when they originated from the same spatial position than when they came from different positions, demonstrating that the apparent perception of multisensory simultaneity is dependent on the relative spatial position from which stimuli are presented.     

Play enhances visual form perception in infancy–an active training study

Motor experiences and active exploration during early childhood may affect individual differences in a wide range of perceptual and cognitive abilities. In the current study, we suggest that active exploration of objects facilitates the ability to process object forms and magnitudes, which in turn impacts the development of numerosity perception. We tested our hypothesis by conducting a preregistered active exploration intervention with 59 8‐month‐old infants. The minimal intervention consisted of actively playing with and exploring blocks once a day for 8 weeks. In order to control for possible training effects on attention, we used book reading as a control condition. Pre‐ and post‐test assessments using eye‐tracking showed that block play improved visual form perception, where infants became better at detecting a deviant shape. Furthermore, using three control tasks, we showed that the intervention specifically improved infants' ability to process visual forms and the effect could not be explained by a domain general improvement in attention or visual perception. We found that the intervention did not improve numerosity perception and suggest that because of the sequential nature of our hypothesis, a longer time frame might be needed to see improvements in this ability. Our findings indicate that if infants are given more opportunities for play and exploration, it will have positive effects on their visual form perception, which in turn could help their understanding of geometrical concepts.     

Neuroimaging of multisensory processing in vision,audition, touch,and olfaction

The development of neuroimaging methods has had a significant impact on the study of the human brain. Functional MRI, with its high spatial resolution, provides investigators with a method to localize the neuronal correlates of many sensory and cognitive processes. Magneto- and electroencephalography, in turn, offer excellent temporal resolution allowing the exact time course of neuronal processes to be investigated. Applying these methods to multisensory processing, many research laboratories have been successful in describing cross-sensory interactions and their spatio-temporal dynamics in the human brain. Here, we review data from selected neuroimaging investigations showing how vision can influence and interact with other senses, namely audition, touch, and olfaction. We highlight some of the similarities and differences in the cross-processing of the different sensory modalities and discuss how different neuroimaging methods can be applied to answer specific questions about multisensory processing.Edited by: Marie-Hélène Giard and Mark Wallace     

Developmental changes in the multisensory temporal binding window persist into adolescence

We live in a world rich in sensory information, and consequently the brain is challenged with deciphering which cues from the various sensory modalities belong together. Determinations regarding the relatedness of sensory information appear to be based, at least in part, on the spatial and temporal relationships between the stimuli. Stimuli that are presented in close spatial and temporal correspondence are more likely to be associated with one another and thus 'bound' into a single perceptual entity. While there is a robust literature delineating behavioral changes in perception induced by multisensory stimuli, maturational changes in multisensory processing, particularly in the temporal realm, are poorly understood. The current study examines the developmental progression of multisensory temporal function by analyzing responses on an audiovisual simultaneity judgment task in 6- to 23-year-old participants. The overarching hypothesis for the study was that multisensory temporal function will mature with increasing age, with the developmental trajectory for this change being the primary point of inquiry. Results indeed reveal an age-dependent decrease in the size of the 'multisensory temporal binding window', the temporal interval within which multisensory stimuli are likely to be perceptually bound, with changes occurring over a surprisingly protracted time course that extends into adolescence.     

Development of visuo-haptic transfer for object recognition in typical preschool and school-aged children

Object recognition is a long and complex adaptive process and its full maturation requires combination of many different sensory experiences as well as cognitive abilities to manipulate previous experiences in order to develop new percepts and subsequently to learn from the environment. It is well recognized that the transfer of visual and haptic information facilitates object recognition in adults, but less is known about development of this ability. In this study, we explored the developmental course of object recognition capacity in children using unimodal visual information, unimodal haptic information, and visuo-haptic information transfer in children from 4 years to 10 years and 11 months of age. Participants were tested through a clinical protocol, involving visual exploration of black-and-white photographs of common objects, haptic exploration of real objects, and visuo-haptic transfer of these two types of information. Results show an age-dependent development of object recognition abilities for visual, haptic, and visuo-haptic modalities. A significant effect of time on development of unimodal and crossmodal recognition skills was found. Moreover, our data suggest that multisensory processes for common object recognition are active at 4 years of age. They facilitate recognition of common objects, and, although not fully mature, are significant in adaptive behavior from the first years of age. The study of typical development of visuo-haptic processes in childhood is a starting point for future studies regarding object recognition in impaired populations.     

Origins of Self-Perception in Infancy

Most accounts of the origins of the self-concept in humans rely on the mirror self-recognition (rouge removal) task whereby the infant is credited with self-awareness at about 15 months, once it is able to use the mirror reflection to locate a dab of rouge on the nose. But mirror self- recognition may require relatively advanced cognitive abilities and may reveal relatively little about the ontogenetic origins of self-knowledge. The aim of this article is to consider the antecedents of self-knowledge in processes of sensory perception during infancy. J. J. Gibson' s ecological approach to sensory perception asserts that there is information for the distinction between self and nonself inherent in perception. Evidence from human infants who are too young to recognize themselves in mirrors is reviewed for a sensory perceptual basis for the existential self (the I) and for the categorical self (the me) in William James' s terminology. Studies of the visual proprioceptive control of posture in babies may be interpreted to support an inherent distinction between self and nonself in infantperception, rather than the traditional account of an "adualistic confusion." Similarly, various aspects of bodily self-awareness manifested even by fetuses demonstrate some basis for a categorical self as an original aspect of experience. Self-specification in perception is also indicated in recent research on imitation in very young infants, a possible mechanism for the essentially social component of self-concept development. Although a case for early self-specification in perception can readily be made, it is much more difficult to explain how self-perception gives rise to self-conception. One possibility briefly discussed is that a process of representation and re-representation of information originally obtained through interaction with physical and social objects gives rise to reflective self- awareness and the particularly autobiographical knowledge of self which we take to be species- typical of humans.     

多感觉整合范式中潜在的跨通道转换效应

大脑可以对来自不同感觉通道的信息进行处理与整合。与单一感觉通道相比, 个体对同时呈现在不同感觉通道的目标信号的响应会更快。对于这种现象的一种主要理论解释是共同激活模型, 该模型认为来自不同通道的刺激在特定的脑区汇聚整合, 比如顶叶内沟、颞上沟和前额叶皮层区域。整合后的信号强度更大, 可以更快地触发反应, 但是信号的整合发生在认知加工的哪一阶段目前尚未有明确结论。当个体对出现在不同感觉通道之间的任务转换进行加工时, 产生与感觉通道相关的任务转换的损失小于跨感觉通道转换损失与任务转换损失的总和, 这为与感觉通道相关的转换代价来源于任务设置的惯性和干扰提供了证据。而在单通道和多通道之间发生转换时, 跨通道转换代价会减小甚至消失, 这是由于同时发生的多感觉整合抵消了一部分损失, 这种现象支持了共同激活模型理论。然而, 多感觉信号整合对任务转换的神经加工过程产生怎样的影响并不清楚, 在未来的研究中可以把多感觉整合范式同经典的任务转换范式结合改进, 进而确定跨通道转换的加工机制和多感觉信号整合的发生阶段。     

Neural correlates of perceptual narrowing in cross‐species face‐voice matching

Integrating the multisensory features of talking faces is critical to learning and extracting coherent meaning from social signals. While we know much about the development of these capacities at the behavioral level, we know very little about the underlying neural processes. One prominent behavioral milestone of these capacities is the perceptual narrowing of face–voice matching, whereby young infants match faces and voices across species, but older infants do not. In the present study, we provide neurophysiological evidence for developmental decline in cross‐species face–voice matching. We measured event‐related brain potentials (ERPs) while 4‐ and 8‐month‐old infants watched and listened to congruent and incongruent audio‐visual presentations of monkey vocalizations and humans mimicking monkey vocalizations. The ERP results indicated that younger infants distinguished between the congruent and the incongruent faces and voices regardless of species, whereas in older infants, the sensitivity to multisensory congruency was limited to the human face and voice. Furthermore, with development, visual and frontal brain processes and their functional connectivity became more sensitive to the congruence of human faces and voices relative to monkey faces and voices. Our data show the neural correlates of perceptual narrowing in face–voice matching and support the notion that postnatal experience with species identity is associated with neural changes in multisensory processing ( Lewkowicz & Ghazanfar, 2009 ).     

Timing of human cortical functions during cognition: role of MEG

Understanding of sensory and cognitive brain processes requires information about activation timing within and between different brain sites. Such data can be obtained by magnetoencephalography (MEG) that tracks cortical activation sequences with a millisecond temporal accuracy. MEG is gaining a well-established role in human neuroscience, complementing with its excellent temporal resolution the spatially more focused brain imaging methods. As examples of MEG's role in cognitive neuroscience, we discuss time windows related to cortical processing of sensory and multisensory stimuli, effects of the subject's own voice on the activity of their auditory cortex, timing of brain activation in reading, and cortical dynamics of the human mirror-neuron system activated when the subject views another person's movements.     

First‐hand sensory experience plays a limited role in children's early understanding of seeing and hearing as sources of knowledge: Evidence from typically hearing and deaf children

One early‐developing component of theory of mind is an understanding of the link between sensory perception and knowledge formation. We know little about the extent to which children's first‐hand sensory experiences drive the development of this understanding, as most tasks capturing this early understanding target vision, with less attention paid to the other senses. In this study, 64 typically hearing children (M_age = 4.0 years) and 21 orally educated deaf children (M_age = 5.44 years) were asked to identify which of two informants knew the identity of a toy animal when each had differing perceptual access to the animal. In the ‘seeing’ condition, one informant saw the animal and the other did not; in the ‘hearing’ condition, one informant heard the animal and the other did not. For both hearing and deaf children, there was no difference between performance on hearing and seeing trials, but deaf children were delayed in both conditions. Further, within both the hearing and deaf groups, older children outperformed younger children on these tasks, indicating that there is a developmental progression. Taken together, the pattern of results suggests that experiences other than first‐hand sensory experiences drive children's developing understanding that sensory perception is associated with knowledge.