Late‐ but not early‐onset blindness impairs the development of audio‐haptic multisensory integration

Integrating different senses to reduce sensory uncertainty and increase perceptual precision can have an important compensatory function for individuals with visual impairment and blindness. However, how visual impairment and blindness impact the development of optimal multisensory integration in the remaining senses is currently unknown. Here we first examined how audio‐haptic integration develops and changes across the life span in 92 sighted (blindfolded) individuals between 7 and 70 years of age. We used a child‐friendly task in which participants had to discriminate different object sizes by touching them and/or listening to them. We assessed whether audio‐haptic performance resulted in a reduction of perceptual uncertainty compared to auditory‐only and haptic‐only performance as predicted by maximum‐likelihood estimation model. We then compared how this ability develops in 28 children and adults with different levels of visual experience, focussing on low‐vision individuals and blind individuals that lost their sight at different ages during development. Our results show that in sighted individuals, adult‐like audio‐haptic integration develops around 13–15 years of age, and remains stable until late adulthood. While early‐blind individuals, even at the youngest ages, integrate audio‐haptic information in an optimal fashion, late‐blind individuals do not. Optimal integration in low‐vision individuals follows a similar developmental trajectory as that of sighted individuals. These findings demonstrate that visual experience is not necessary for optimal audio‐haptic integration to emerge, but that consistency of sensory information across development is key for the functional outcome of optimal multisensory integration.     

Integration of audio‐visual information for spatial decisions in children and adults

In adults, decisions based on multisensory information can be faster and/or more accurate than those relying on a single sense. However, this finding varies significantly across development. Here we studied speeded responding to audio‐visual targets, a key multisensory function whose development remains unclear. We found that when judging the locations of targets, children aged 4 to 12 years and adults had faster and less variable response times given auditory and visual information together compared with either alone. Comparison of response time distributions with model predictions indicated that children at all ages were integrating (pooling) sensory information to make decisions but that both the overall speed and the efficiency of sensory integration improved with age. The evidence for pooling comes from comparison with the predictions of Miller's seminal ‘race model’, as well as with a major recent extension of this model and a comparable ‘pooling’ (coactivation) model. The findings and analyses can reconcile results from previous audio‐visual studies, in which infants showed speed gains exceeding race model predictions in a spatial orienting task (Neil et al., 2006) but children below 7 years did not in speeded reaction time tasks (e.g. Barutchu et al., 2009). Our results provide new evidence for early and sustained abilities to integrate visual and auditory signals for spatial localization from a young age.     

Facilitating Effect of Multisensory Letter Encoding on Reading and Spelling in 5‐Year‐Old Children

This study aims at identifying the effect of training in the acquisition of the alphabetic principle in 5‐year‐old children. We compared the effect of multisensory training of letters in visual, haptic, graphomotor, visuo‐haptic, and visuo‐graphomotor groups. For each training type, we contrasted trained versus untrained letters in reading and spelling tasks. First, visuo‐haptic and visuo‐graphomotor training improved letter‐sound correspondence acquisition scores more than the other types of training, and this improvement persisted in the second post‐test. A cross‐modal transfer was revealed by the fact that scores increased after blindfold haptic and graphomotor experiences. Moreover, performance on untrained letters also improved, suggesting an indirect effect following the specific trained letters training. The results argue in favor of a facilitating effect of multisensory encoding on acquisition of the alphabetic principle. Practical implications for the prevention of future reading difficulties are discussed. Copyright © 2015 John Wiley & Sons, Ltd.     

How to rapidly construct a spatial–numerical representation in preliterate children (at least temporarily)

Spatial processing of numbers has emerged as one of the basic properties of humans’ mathematical thinking. However, how and when number–space relations develop is a highly contested issue. One dominant view has been that a link between numbers and left/right spatial directions is constructed based on directional experience associated with reading and writing. However, some early forms of a number–space link have been observed in preschool children who cannot yet read and write. As literacy experience is evidently not necessary for number–space effects, we are searching for other potential sources of this association. Here we propose and test a hypothesis that the number–space link can be quickly constructed in preschool children's cognition on the basis of spatially oriented visuo‐motor activities. We trained 3‐ and 4‐year‐old children with a non‐numerical spatial movement task (left‐to‐right or right‐to‐left), where via touch screen children had to move a frog across a pond. After the training, children had to perform a numerosity comparison task. After left‐to‐right training, we observed a SNARC‐like effect (reactions to smaller numbers were faster on the left side, and reactions to larger numbers on the right side), and after right‐to‐left training a reverse effect. These results are the first to show a causal link between visuo‐motor activities and number–space associations in children before they learn to read and write. We argue that simple activities, such as manual games, dominant in a given society, might shape number–space associations in children in a way similar to lifelong reading training.     

Visuo‐spatial play experience: forerunner of visuo‐spatial achievement in preadolescent and adolescent boys and girls?

This cross‐sectional study explored whether participation, from early childhood, in play involving different cognitive abilities predicts visuo‐spatial achievement at ages 9, 12, and 15. Based on parental assessment, prior and present practice of spatial manipulation play was found to be consistently more frequent in boys than in girls; the reverse held true for verbal expression play. Whereas boys did not significantly outperform girls in three visuo‐spatial tasks, girls were superior on a contrastive vocabulary task. In general, with IQ statistically controlled, regression analyses showed that estimated past and present spatial manipulation play predicted both genders' proficiency in the water‐level task and Embedded Figures Test, as did mothers' socioeconomic status for Block Design performance. Contrastingly, a negative relation was established between spatial manipulation play and vocabulary scores. Similar to the activity‐ability association often identified among adults, the relation established here between spatial play experience and visuo‐spatial ability was only modest. Further research should aim at more definitive conclusions through augmenting both diversity in the visuo‐spatial skills measured and sophistication in play behaviour appraisal. Copyright © 2003 John Wiley & Sons, Ltd.     

Temporal representation impairment in developmental dyslexia for unisensory and multisensory stimuli

Dyslexia has been associated with a problem in visual–audio integration mechanisms. Here, we investigate for the first time the contribution of unisensory cues on multisensory audio and visual integration in 32 dyslexic children by modelling results using the Bayesian approach. Non‐linguistic stimuli were used. Children performed a temporal task: they had to report whether the middle of three stimuli was closer in time to the first one or to the last one presented. Children with dyslexia, compared with typical children, exhibited poorer unimodal thresholds, requiring greater temporal distance between items for correct judgements, while multisensory thresholds were well predicted by the Bayesian model. This result suggests that the multisensory deficit in dyslexia is due to impaired audio and visual inputs rather than impaired multisensory processing per se. We also observed that poorer temporal skills correlated with lower reading skills in dyslexic children, suggesting that this temporal capability can be linked to reading abilities.     

Young children combine sensory cues with learned information in a statistically efficient manner: But task complexity matters

Human adults are adept at mitigating the influence of sensory uncertainty on task performance by integrating sensory cues with learned prior information, in a Bayes‐optimal fashion. Previous research has shown that young children and infants are sensitive to environmental regularities, and that the ability to learn and use such regularities is involved in the development of several cognitive abilities. However, it has also been reported that children younger than 8 do not combine simultaneously available sensory cues in a Bayes‐optimal fashion. Thus, it remains unclear whether, and by what age, children can combine sensory cues with learned regularities in an adult manner. Here, we examine the performance of 6‐ to 7‐year‐old children when tasked with localizing a ‘hidden’ target by combining uncertain sensory information with prior information learned over repeated exposure to the task. We demonstrate that 6‐ to 7‐year‐olds learn task‐relevant statistics at a rate on par with adults, and like adults, are capable of integrating learned regularities with sensory information in a statistically efficient manner. We also show that variables such as task complexity can influence young children's behavior to a greater extent than that of adults, leading their behavior to look sub‐optimal. Our findings have important implications for how we should interpret failures in young children's ability to carry out sophisticated computations. These ‘failures’ need not be attributed to deficits in the fundamental computational capacity available to children early in development, but rather to ancillary immaturities in general cognitive abilities that mask the operation of these computations in specific situations.     

Training and transfer effects of executive functions in preschool children

Executive functions, including working memory and inhibition, are of central importance to much of human behavior. Interventions intended to improve executive functions might therefore serve an important purpose. Previous studies show that working memory can be improved by training, but it is unknown if this also holds for inhibition, and whether it is possible to train executive functions in preschoolers. In the present study, preschool children received computerized training of either visuo‐spatial working memory or inhibition for 5 weeks. An active control group played commercially available computer games, and a passive control group took part in only pre‐ and posttesting. Children trained on working memory improved significantly on trained tasks; they showed training effects on non‐trained tests of spatial and verbal working memory, as well as transfer effects to attention. Children trained on inhibition showed a significant improvement over time on two out of three trained task paradigms, but no significant improvements relative to the control groups on tasks measuring working memory or attention. In neither of the two interventions were there effects on non‐trained inhibitory tasks. The results suggest that working memory training can have significant effects also among preschool children. The finding that inhibition could not be improved by either one of the two training programs might be due to the particular training program used in the present study or possibly indicate that executive functions differ in how easily they can be improved by training, which in turn might relate to differences in their underlying psychological and neural processes.     

Creativity and involvement in art in different types of synaesthesia

The aim of the present study was to test the relationship between different types of synaesthesia and their involvement in art, creative, and visual abilities. We tested 20 grapheme‐colour, 18 sound‐colour, 19 grapheme‐colour‐and‐sound‐colour, 20 sequence‐space synaesthetes, and the same number of controls matched by age, gender, and education. We assessed the number of artistic professions, involvement in art, and the performance in psychometric tests of divergent and convergent creativity, as well as visual and visuo‐spatial abilities. Results show a higher prevalence of artists among synaesthetes, especially sound‐colour synaesthetes. Sound‐colour synaesthetes also showed a higher involvement in artistic activities overall while sequence‐space synaesthetes showed higher involvement in visual art. Only grapheme‐colour‐and‐sound‐colour synaesthetes showed significantly higher divergent creativity compared to matched controls. Additionally, overall, synaesthetes scored higher in visuo‐spatial abilities (i.e., mental rotation). For synaesthetes and controls, visuo‐spatial abilities correlated with divergent creativity. We discuss that synaesthetes’ higher involvement in art is not necessarily reflected in their basic creative abilities.     

The relationship between multisensory integration and IQ in children

It is well accepted that multisensory integration has a facilitative effect on perceptual and motor processes, evolutionarily enhancing the chance of survival of many species, including humans. Yet, there is limited understanding of the relationship between multisensory processes, environmental noise, and children's cognitive abilities. Thus, this study investigated the relationship between multisensory integration, auditory background noise, and the general intellectual abilities of school-age children (N = 88, mean age = 9 years, 7 months) using a simple audiovisual detection paradigm. We provide evidence that children with enhanced multisensory integration in quiet and noisy conditions are likely to score above average on the Full-Scale IQ of the Wechsler Intelligence Scale for Children-Fourth Edition (WISC-IV). Conversely, approximately 45% of tested children, with relatively low verbal and nonverbal intellectual abilities, showed reduced multisensory integration in either quiet or noise. Interestingly, approximately 20% of children showed improved multisensory integration abilities in the presence of auditory background noise. The findings of the present study suggest that stable and consistent multisensory integration in quiet and noisy environments is associated with the development of optimal general intellectual abilities. Further theoretical implications are discussed.     

Learning a map of environment: The role of visuo‐spatial abilities in young and older adults

The present study examines age‐related differences between young and older adults in spatial mental representation derived from learning a realistic city map. A sample of 30 young (20–30‐years) and 30 older (60–72 years) adults learned a simplified map of a city; afterwards participants performed map‐drawing and pointing‐aligned and counter‐aligned tasks. Tasks measuring visuo‐spatial abilities were also administered to explore their relationship with map learning. Results showed an age‐related impairment in older adults in both map tasks, as well as in visuo‐spatial ones. Furthermore, performance on counter‐aligned pointing was poorer than on aligned pointing in young and older adults, and its relationship with visuo‐spatial abilities changed as a function of age group: The performance of counter‐aligned pointing in older adults was related to all visuo‐spatial abilities, and in young adults with perspective‐taking measures only. Copyright © 2011 John Wiley & Sons, Ltd.     

On the relationship between children's perspective taking in complex scenes and their spatial drawing ability

Depicting space and volume in drawings is challenging for young children in particular. It has been assumed that several cognitive skills may contribute to children's drawing. In the present study, we investigated the relationship between perspective‐taking skills in complex scenes and the spatial characteristics in drawings of 5‐ to 9‐year‐olds (N= 121). Perspective taking was assessed by two tasks: (a) a visual task similar to the three‐mountains task, in which the children had to select a three‐dimensional model that showed the view on a scene from particular perspective and (b) a spatial construction task, in which children had to plastically reconstruct a three‐dimensional scene as it would appear from a new point of view. In the drawing task, the children were asked to depict a three‐dimensional scene exactly as it looked like from their own point of view. Several spatial features in the drawings were coded. The results suggested that children's spatial drawing and their perspective‐taking skills were related. The axes system and the spatial relations between objects in the drawings in particular were predicted, beyond age, by certain measures of the two perspective‐taking tasks. The results are discussed in the light of particular demands that might underlay both perspective taking and spatial drawing.     

Racing dragons and remembering aliens: Benefits of playing number and working memory games on kindergartners' numerical knowledge

Sources that contribute to variation in mathematical achievement include both numerical knowledge and general underlying cognitive processing abilities. The current study tested the benefits of tablet‐based training games that targeted each of these areas for improving the mathematical knowledge of kindergarten‐age children. We hypothesized that playing a number‐based game targeting numerical magnitude knowledge would improve children's broader numerical skills. We also hypothesized that the benefits of playing a working memory (WM) game would transfer to children's numerical knowledge given its important underlying role in mathematics achievement. Kindergarteners from diverse backgrounds (n = 148; 52% girls; M_age = 71.87 months) were randomly assigned to either play a number‐based game, a WM game, or a control game on a tablet for 10 sessions. Structural equation modeling was used to model children's learning gains in mathematics and WM across time. Overall, our results suggest that playing the number game improved kindergarten children's numerical knowledge at the latent level, and these improvements remained stable as assessed 1 month later. However, children in the WM group did not improve their numerical knowledge compared to children in the control condition. Playing both the number game and WM game improved children's WM at the latent level. Importantly, the WM group continued to improve their WM for at least a month after playing the games. The results demonstrate that computerized games that target both domain‐specific and domain‐general skills can benefit a broad range of kindergarten‐aged children.     

Parental socioeconomic status and the neural basis of arithmetic: differential relations to verbal and visuo‐spatial representations

We examined the relation of parental socioeconomic status (SES) to the neural bases of subtraction in school‐age children (9‐ to 12‐year‐olds). We independently localized brain regions subserving verbal versus visuo‐spatial representations to determine whether the parental SES‐related differences in children's reliance on these neural representations vary as a function of math skill. At higher SES levels, higher skill was associated with greater recruitment of the left temporal cortex, identified by the verbal localizer. At lower SES levels, higher skill was associated with greater recruitment of right parietal cortex, identified by the visuo‐spatial localizer. This suggests that depending on parental SES, children engage different neural systems to solve subtraction problems. Furthermore, SES was related to the activation in the left temporal and frontal cortex during the independent verbal localizer task, but it was not related to activation during the independent visuo‐spatial localizer task. Differences in activation during the verbal localizer task in turn were related to differences in activation during the subtraction task in right parietal cortex. The relation was stronger at lower SES levels. This result suggests that SES‐related differences in the visuo‐spatial regions during subtraction might be based in SES‐related verbal differences.     

Multisensory integration in multiple reference frames in the posterior parietal cortex

Spatial information processing takes place in different brain regions that receive converging inputs from several sensory modalities. Because of our own movements—for example, changes in eye position, head rotations, and so forth—unimodal sensory representations move continuously relative to one another. It is generally assumed that for multisensory integration to be an orderly process, it should take place between stimuli at congruent spatial locations. In the monkey posterior parietal cortex, the ventral intraparietal (VIP) area is specialized for the analysis of movement information using visual, somatosensory, vestibular, and auditory signals. Focusing on the visual and tactile modalities, we found that in area VIP, like in the superior colliculus, multisensory signals interact at the single neuron level, suggesting that this area participates in multisensory integration. Curiously, VIP does not use a single, invariant coordinate system to encode locations within and across sensory modalities. Visual stimuli can be encoded with respect to the eye, the head, or halfway between the two reference frames, whereas tactile stimuli seem to be prevalently encoded relative to the body. Hence, while some multisensory neurons in VIP could encode spatially congruent tactile and visual stimuli independently of current posture, in other neurons this would not be the case. Future work will need to evaluate the implications of these observations for theories of optimal multisensory integration.Edited by: Marie-Hélène Giard and Mark Wallace     

The race that precedes coactivation: development of multisensory facilitation in children

Rationale: The facilitating effect of multisensory integration on motor responses in adults is much larger than predicted by race‐models and is in accordance with the idea of coactivation. However, the development of multisensory facilitation of endogenously driven motor processes and its relationship to the development of complex cognitive skills in school‐age children is largely unexplored. Method: Twenty adults and 95 children where allocated into six age groups: 6, 7, 8, 9, 10–11 and adults. Participants’ motor reaction times (MRTs) and accuracy in response to the detection of auditory, visual and audiovisual stimuli were recorded. Children's reading accuracy and nonverbal IQ were also assessed. Results: In general, MRTs of children were significantly slower with greater variability than those of adults. Although the average level of multisensory facilitation was similar for all age groups, mean cumulative density functions (CDFs) showed that multisensory facilitation in 6 and 10–11‐year‐olds is within the predictive limits of race‐models. Where coactivation was seen in the CDF of individual children it was not as strong or as consistent as that in adults. The degree of multisensory facilitation did not correlate with age, reading accuracy or IQ. Conclusion: The average level of multisensory facilitation to endogenously driven motor responses does not change gradually with age nor is it related to intelligence or reading accuracy. In general, multisensory integration remains immature until 10–11 years of age and lies within the predicted confines of race‐models.     

The social foundation of executive function

In this study, we propose that infant social cognition may ‘bootstrap' the successive development of domain‐general cognition in line with the cultural intelligence hypothesis. Using a longitudinal design, 6‐month‐old infants (N = 118) were assessed on two basic social cognitive tasks targeting the abilities to share attention with others and understanding other peoples' actions. At 10 months, we measured the quality of the child's social learning environment, indexed by parent's abilities to provide scaffolding behaviors during a problem‐solving task. Eight months later, the children were followed up with a cognitive test‐battery, including tasks of inhibitory control and working memory. Our results showed that better infant social action understanding interacted with better parental scaffolding skills in predicting simple inhibitory control in toddlerhood. This suggests that infants' who are better at understanding other's actions are also better equipped to make the most of existing social learning opportunities, which in turn may benefit future non‐social cognitive outcomes.     

Hearing visuo‐tactile synchrony – Sound‐induced proprioceptive drift in the invisible hand illusion

The rubber hand illusion (RHI) and its variant the invisible hand illusion (IHI) are useful for investigating multisensory aspects of bodily self‐consciousness. Here, we explored whether auditory conditioning during an RHI could enhance the trisensory visuo‐tactile‐proprioceptive interaction underlying the IHI. Our paradigm comprised of an IHI session that was followed by an RHI session and another IHI session. The IHI sessions had two parts presented in counterbalanced order. One part was conducted in silence, whereas the other part was conducted on the backdrop of metronome beats that occurred in synchrony with the brush movements used for the induction of the illusion. In a first experiment, the RHI session also involved metronome beats and was aimed at creating an associative memory between the brush stroking of a rubber hand and the sounds. An analysis of IHI sessions showed that the participants’ perceived hand position drifted more towards the body‐midline in the metronome relative to the silent condition without any sound‐related session differences. Thus, the sounds, but not the auditory RHI conditioning, influenced the IHI. In a second experiment, the RHI session was conducted without metronome beats. This confirmed the conditioning‐independent presence of sound‐induced proprioceptive drift in the IHI. Together, these findings show that the influence of visuo‐tactile integration on proprioceptive updating is modifiable by irrelevant auditory cues merely through the temporal correspondence between the visuo‐tactile and auditory events.     

Individual differences in the acquisition of non‐linguistic audio‐visual associations in 5 year olds

Audio‐visual associative learning – at least when linguistic stimuli are employed – is known to rely on core linguistic skills such as phonological awareness. Here we ask whether this would also be the case in a task that does not manipulate linguistic information. Another question of interest is whether executive skills, often found to support learning, may play a larger role in a non‐linguistic audio‐visual associative task compared to a linguistic one. We present a new task that measures learning when having to associate non‐linguistic auditory signals with novel visual shapes. Importantly, our novel task shares with linguistic processes such as reading acquisition the need to associate sounds with arbitrary shapes. Yet, rather than phonemes or syllables, it uses novel environmental sounds – therefore limiting direct reliance on linguistic abilities. Five‐year‐old French‐speaking children (N = 76, 39 girls) were assessed individually in our novel audio‐visual associative task, as well as in a number of other cognitive tasks evaluating linguistic abilities and executive functions. We found phonological awareness and language comprehension to be related to scores in the audio‐visual associative task, while no correlation with executive functions was observed. These results underscore a key relation between foundational language competencies and audio‐visual associative learning, even in the absence of linguistic input in the associative task.