Infants' intermodal perception of two levels of temporal structure in natural events

Infants' intermodal perception of two levels of temporal structure uniting the visual and acoustic stimulation from natural, complex events was investigated in four experiments. Films depicting a single object (single, large marble) and a compound object (group of smaller marbles) colliding against a surface in an erratic pattern were presented to infants between 3 and months of age using an intermodal preference and search method. These stimulus events portrayed two levels of invariant temporal structure: (a) temporal synchrony united the sights and sounds of object impact, and (b) temporal microstructure, the internal temporal structure of each impact sound and motion, specified the composition of the object (single vs. compound). Experiment 1 demonstrated that by 6 months infants detected a relation between the audible and visible stimulation from these events when both levels of invariant temporal structure guided their intermodal exploration. Experiment 2 revealed that by 6 months infants detected the bimodal temporal microstructure specifying object composition. They looked predominantly to the film whose natural soundtrack was played even though the motions of objects in both films were synchronized with the soundtrack. Experiment 3 assessed infants' sensitivity to temporal synchrony relations. Two films depicting objects of the same composition were presented while the motions of only one of them was synchronized with the appropriate soundtrack. Both 6-month-olds showed evidence of detecting temporal synchrony relations under some conditions. Experiment 4 examined how temporal synchrony and temporal microstructure interact in directing intermodal exploration. The natural soundtrack to one of the objects was played out-of-synchrony with the motions of both. In contrast with the results of Experiment 2, infants at 6 months showed no evidence of detecting a relationship between the film and its appropriate soundtrack. This suggests that the temporal asynchrony disrupted their detection of the temporal microstructure specifying object composition. Results of these studies support on invariant-detection view of the development of intermodal perception.     

Learning and discrimination of audiovisual events in human infants: the hierarchical relation between intersensory temporal synchrony and rhythmic pattern cues

This study examined 4- to 10-month-old infants' perception of audio-visual (A-V) temporal synchrony cues in the presence or absence of rhythmic pattern cues. Experiment 1 established that infants of all ages could successfully discriminate between two different audiovisual rhythmic events. Experiment 2 showed that only 10-month-old infants detected a desynchronization of the auditory and visual components of a rhythmical event. Experiment 3 showed that 4- to 8-month-old infants could detect A-V desynchronization but only when the audiovisual event was nonrhythmic. These results show that initially in development infants attend to the overall temporal structure of rhythmic audiovisual events but that later in development they become capable of perceiving the embedded intersensory temporal synchrony relations as well.     

Infants' perceptual differentiation of amodal and modality-specific audio-visual relations.

Ninety-six infants of 3 1/2 months were tested in an infant-control habituation procedure to determine whether they could detect three types of audio-visual relations in the same events. The events portrayed two amodal invariant relations, temporal synchrony and temporal microstructure specifying the composition of the objects, and one modality-specific relation, that between the pitch of the sound and the color/shape of the objects. Subjects were habituated to two events accompanied by their natural, synchronous, and appropriate sounds and then received test trials in which the relation between the visual and the acoustic information was changed. Consistent with Gibson's increasing specificity hypothesis, it was expected that infants would differentiate amodal invariant relations prior to detecting arbitrary, modality-specific relations. Results were consistent with this prediction, demonstrating significant visual recovery to a change in temporal synchrony and temporal microstructure, but not to a change in the pitch-color/shape relations. Two subsequent discrimination studies demonstrated that infants' failure to detect the changes in pitch-color/shape relations could not be attributed to an inability to discriminate the pitch or the color/shape changes used in Experiment 1. Infants showed robust discrimination of the contrasts used.     

Infants' learning,memory, and generalization of learning for bimodal events

Study 1 investigated whether infants 3 and 7 months of age show differential learning of and memory for sight-sound pairs depending on whether or not temporal synchrony was present; memory was assessed after a 10-min and 1-week interval. Study 2 examined whether 7-month-olds show generalization of learning when they encounter novel bimodal events that are similar (changes in size, orientation, or color, and spectral sound properties) to the sight-sound pairs learned 1 week earlier based on temporal synchrony. For Study 1, infants received a familiarization phase followed by a paired-comparison preference procedure to assess for learning of the sight-sound pairs. One week later a memory test was given. Results confirmed that 7-month-olds had no difficulty learning auditory-visual pairings regardless of whether or not events were temporally synchronous, and they remembered these 10 min and 1 week later. In contrast, 3-month-olds showed poorer learning of sight-sound associations in the no-synchrony than synchrony conditions, and memory for sight-sound pairs 1 week later was shown only for the synchrony conditions. Results for Study 2 revealed generalization of learning of bimodal pairings under all stimulus conditions after a 1-week interval at 7 months of age. Implications of these findings for development of intersensory knowledge are discussed.     

Perception of serial order in infants

Serial order is fundamental to perception, cognition and behavioral action. Three experiments investigated infants' perception, learning and discrimination of serial order. Four- and 8-month-old infants were habituated to three sequentially moving objects making visible and audible impacts and then were tested on separate test trials for their ability to detect auditory, visual or auditory-visual changes in their ordering. The 4-month-old infants did not respond to any order changes and instead appeared to attend to the 'local' audio-visual synchrony part of the event. When this local part of the event was blocked from view, the 4-month-olds did perceive the serial order feature of the event but only when it was specified multimodally. In contrast, the 8-month-old infants perceived all three kinds of order changes regardless of whether the synchrony part of the event was visible or not. The findings show that perception of spatiotemporal serial order emerges early in infancy and that its perception is initially facilitated by multimodal specification.     

Development of perception of the unity of musical events

Infants' and young children's perception of the unity of musical events was investigated in three studies. In the first two, children watched video displays of two musicians playing different musical instruments side by side in synchrony, and heard a soundtrack in synchrony with both instruments but specific to one. The children judged which instrument was producing the music they heard. Three- to 4-year-olds differentiated instruments from different families but not instruments from the same family. Five- to 7-year-olds additionally differentiated instrument pairs differing in size and pitch range (e.g., violin, cello). In the third study, infants were presented some of the same musical events in order to assess whether specific experience with the instruments is necessary for perceiving the unity of musical events. Looking times revealed that 7- to 9-month-olds detected the correspondence of the sight and sound of some musical instruments. Specific experience with a variety of instruments is evidently not necessary for detecting correspondences of audible and visible properties and for differentiating instruments from different families.     

Developmental changes in associations between auditory-visual events

The goal of the present study was twofold: to examine the influence of two amodal properties, co-location and temporal synchrony, on infants' associating a sight with a sound, and to determine if the relative influence of these properties on crossmodal learning changes with age. During familiarization 2-, 4-, 6- and 8-month-olds were presented two toys and a sound, with sights and sounds varying with respect to co-location and temporal synchrony. Following each familiarization phase infants were given a paired preference test to assess their learning of sight-sound associations. Measures of preferential looking revealed age-related changes in the influence of co-location and temporal synchrony on infants' learning sight-sound associations. At all ages, infants could use temporal synchrony and co-location as a basis for associating an auditory with a visual event and, in the absence of temporal synchrony, co-location was sufficient to support crossmodal learning. However, when these cues conflicted there were developmental changes in the influence of these cues on infants' learning auditory-visual associations. At 2 and 4 months infants associated the sounds with the toy that moved in synchrony with the sound's rhythm despite extreme violation of co-location of this sight and sound. In contrast, 6- and 8-month-olds did not associate a specific toy with the sound when co-location and synchrony information conflicted. The findings highlight the unique and interactive effects of distinct amodal properties on infants' learning arbitrary crossmodal relations. Possible explanations for the age shift in performance are discussed.     

Intermodal perception of expressive behaviors by human infants

Four experiments demonstrate that infants of 5 and 7 months can detect information that is invariant across the acoustic and optic presentations of a single affective expression. Infants were presented simultaneously with two filmed facial expressions accompanied by a single vocal expression characteristic of one of the facial expressions. The infants increased their looking time to a facial expression when it was sound-specified, as compared to when that filmed expression was projected silently. Even when synchrony relations were disrupted, infants looked proportionately longer to the film that was sound-specified, indicating that some factor other than temporal synchrony guided the infants' looking behavior. When infants viewed the filmed facial expressions either in a normal orientation or upside-down, those infants viewing the facial expressions in the normal orientation looked appropriately, while those viewing the inverted films did not. These findings support the view that infants are sensitive to amodal, potentially meaningful invariant relations in expressive behaviors. These results are discussed in the context of J. J. Gibson's theory of affordances.     

Developments in young infants' reasoning about occluded objects

Eight experiments were conducted to examine 3- and 3.5-month-old infants' responses to occlusion events. The results revealed two developments, one in infants' knowledge of when objects should and should not be occluded and the other in infants' ability to posit additional objects to make sense of events that would otherwise violate their occlusion knowledge. The first development is that, beginning at about 3 months of age, infants expect an object to become temporarily visible when passing behind an occluder with an opening extending from its lower edge. The second development is that, beginning at about 3.5 months of age, infants generate a two-object explanation when shown a violation in which an object fails to become visible when passing behind an occluder with an opening in its lower edge. Unless given information contradicting such an explanation, infants infer that two identical objects are involved in the event, one traveling to the left and one to the right of the opening. These and related findings provide the basis for a model of young infants' responses to occlusion events; alternative models are also discussed.     

The infant's acquisition of knowledge of bimodally specified events

Four-month-old infants viewed, for a duration of several minutes, two objects that bounced in synchrony with two percussion sounds. This synchrony was the only information tying each sound to its respective object. During the viewing the infants learned about the relationships between sound and object. Learning was revealed in two ways. In a search test, infants looked for an object when its sound was played. In a transfer test, infants' declining interest in a sound presented alone generalized to the visible object that the sound specified. Studies that reversed the spatial locations of the objects revealed that sound-object learning, rather than place or response learning, guided infants' perceptual exploration.     

The ratio principle holds over a million-to-one range of illumination

Infant sensitivity to kinetic information specifying three-dimensional object shape was assessed using computer-generated random-dot displays. Four-month-old infants were habituated to displays of an object oscillating about two different axes on alternating trials. Following habituation, the infants were tested for recovery from habituation to a display of the same object and a novel object. Both test displays employed a new axis of rotation. The infants generalized habituation to the same object and increased their looking to the new object. These results provide evidence that infants are sensitive to motion-carried information specifying three-dimensional object shape, since the random-dot displays minimized static information that differentiated the two objects. These findings suggest that, at least by 4 months of age, infants can detect subtle differences in shape from purely kinetic information.     

When I move,you move: Associations between automatic and person-coded measures of infant-mother synchrony during free-play using virtual in-home data collection

The relationship between movement synchrony and global scores of behavioral synchrony were investigated during a naturalistic free-play between 33 infants (ages 12- to 24-months) and their mothers using a video-conferencing platform. We assessed movement synchrony by applying an automatic tool, motion-energy analysis (MEA), to the obtained video data. Dyadic movement synchrony was associated with higher levels of maternal sensitivity, infant involvement, dyadic reciprocity, and a child-led interaction pattern. This demonstrates the feasibility of using MEA as an automatic tool for assessing movement synchrony in mother-infant dyads and its application for investigating naturalistic at-home free play sessions in a remote setting.     

Age Changes in Interlimb Coupling and the Development of Bimanual Coordination

Changes in interlimb coupling, and their role in the development of bimanual coordination, were studied longitudinally in 6- to 12-month-old infants (N = 6). Infants were observed while they were reaching for simple objects of 2 different sizes. Their use of a uni-versus bimanual strategy for reaching as well as the coupling of their bimanual movements were compared; progress in bimanual coordination of complementary movements was evaluated on 8 different bimanual tasks. The bimanual tasks involved an asymmetrical cooperation between the 2 hands. Although spatiotemporal coupling of bimanual reaching movements did not decrease during the age period studied, infants around 7 months of age used their 2 hands infrequently for reaching. Occurrences of bimanual reaching were particularly low at the session preceding the first bimanual success at a bimanual task. This suggests that the temporal coincidence between greater independence of the 2 hands and progress in bimanual coordination of complementary movements acts in 2 directions: Infants may be more at ease when using their 2 hands in differentiated patterns as the hands move less in synchrony, but, in turn, they may be less likely to move their hands in synchrony as the anticipate mirror manipulations of the object less. The frequency of bimanual reaches increased toward the end of the 1st year. This might have been caused by an increase in the repertoire of bimanual asymmetrical object manipulations and by the fact that the development of bimanual coordination allows infants to manipulate objects with complementary movements even after a bimanual approach toward the object.     

The sound of darkness: why do auditory cues aid infants' search for objects hidden by darkness but not by visible occluders?

In manual search tasks designed to assess infants' knowledge of the object concept, why does search for objects hidden by darkness precede search for objects hidden by visible occluders by several months? A graded representations account explains this décalage by proposing that the conflicting visual input from occluders directly competes with object representations, whereas darkness merely weakens representations. This study tests the prediction that representations of objects hidden by darkness are strong enough for infants to bind auditory cues to them and support search, whereas representations of objects hidden by occluders are not. Six-and-half-month-olds were presented with audible or silent objects that remained visible, became hidden by darkness, or became hidden by a visible occluder. Search required engaging in the same means-end action in all conditions. As predicted, auditory cues increased search when objects were hidden by darkness but not when they were hidden by a visible occluder. Results are discussed in the context of different facets of object concept development highlighted by graded representations perspectives and core knowledge perspectives and in relation to other work on multimodal object representations.     

Motor training at 3 months affects object exploration 12 months later

The development of new motor skills alters how infants interact with objects and people. Consequently, it has been suggested that motor skills may initiate a cascade of events influencing subsequent development. However, only correlational evidence for this assumption has been obtained thus far. The current study addressed this question experimentally by systematically varying reaching experiences in 40 three‐month‐old infants who were not reaching on their own yet and examining their object engagement in a longitudinal follow‐up assessment 12 months later. Results revealed increased object exploration and attention focusing skills in 15‐month‐old infants who experienced active reaching at 3 months of age compared to untrained infants or infants who only passively experienced reaching. Further, grasping activity after – but not before – reaching training predicted infants’ object exploration 12 months later. These findings provide evidence for the long‐term effects of reaching experiences and illustrate the cascading effects initiated by early motor skills.     

Familiarity breeds searching: infants reverse their novelty preferences when reaching for hidden objects

What infants appear to know depends heavily on how they are tested. For example, infants seem to understand object permanence (that objects continue to exist when no longer perceptible) within the first few months of life when this understanding is assessed through looking measures, but not until several months later when it is assessed through search measures. One explanation of such results is that infants gradually develop stronger representations of objects through experience, and that stronger representations are required for some tasks than for others. The current study confirms one prediction from this account: Stronger representations of familiar objects (relative to novel objects) should support greater sensitivity to their continued existence. After seeing objects hidden, infants reached more for familiar than novel objects, in striking contrast to their robust novelty preferences with visible objects. Theoretical implications concerning the origins of knowledge are discussed.     

Emergence of sensitivity to pictorial depth cues: Charting development in individual infants

Three experiments were conducted to explore the emergence of sensitivity to the pictorial depth cues of texture gradient and linear perspective. In experiment 1, an initial longitudinal study explored the emergence of sensitivity to pictorial depth information between 5 and 7 months of age. In experiment 2, a cross-sectional study with 5–7-month-olds assessed revised methods designed to study development of pictorial depth sensitivity in individual infants. Experiment 3 applied these methods to a second sample of infants studied longitudinally. The results showed that: (a) a reliable method for assessing sensitivity in individual infants has been constructed; (b) there is variability in the age at which infants begin to use linear perspective and texture gradient for perceiving depth (22–28 weeks of age); and (c) sensitivity emerges across 2–8 weeks.     

Infant perception of audio-visual speech synchrony in familiar and unfamiliar fluent speech

We investigated the effects of linguistic experience and language familiarity on the perception of audio-visual (A-V) synchrony in fluent speech. In Experiment 1, we tested a group of monolingual Spanish- and Catalan-learning 8-month-old infants to a video clip of a person speaking Spanish. Following habituation to the audiovisually synchronous video, infants saw and heard desynchronized clips of the same video where the audio stream now preceded the video stream by 366, 500, or 666 ms. In Experiment 2, monolingual Catalan and Spanish infants were tested with a video clip of a person speaking English. Results indicated that in both experiments, infants detected a 666 and a 500 ms asynchrony. That is, their responsiveness to A-V synchrony was the same regardless of their specific linguistic experience or familiarity with the tested language. Compared to previous results from infant studies with isolated audiovisual syllables, these results show that infants are more sensitive to A-V temporal relations inherent in fluent speech. Furthermore, the absence of a language familiarity effect on the detection of A-V speech asynchrony at eight months of age is consistent with the broad perceptual tuning usually observed in infant response to linguistic input at this age.     

Catching audiovisual mice: Predicting the arrival time of auditory-visual motion signals

We investigated the extent to which auditory and visual motion signals are combined when observers are asked to predict the location of a virtually moving target. In Condition 1, the unimodal and bimodal signals were noisy, but the target object was continuously visible and audible; in Condition 2, the virtually moving object was hidden (invisible and inaudible) for a short period prior to its arrival at the target location. Our main finding was that the facilitation due to simultaneous visual and auditory input is very different for the two conditions. When the target is continuously visible and audible (Condition 1), the bimodal performance is twice as good as the unimodal performances, thus suggesting a very effective integration mechanism. On the other hand, if the object is hidden for a short period (Condition 2) and the task therefore requires the extrapolation of motion speed over a temporal and spatial period, the facilitation due to both sensory inputs is almost absent, and the bimodal performance is limited by the visual performance.     

The effects of adults' affective expression and direction of visual gaze on 12‐month‐olds' visual preferences for an object following a 5‐minute, 1‐day,or 1‐month delay

Between 12‐ and 14 months of age infants begin to use another's direction of gaze and affective expression in learning about various objects and events. What is not well understood is how long infants' behaviour towards a previously unfamiliar object continues to be influenced following their participation in circumstances of social referencing. In this experiment, we examined infants' sensitivity to an adult's direction of gaze and their visual preference for one of two objects following a 5‐min, 1‐day, or 1‐month delay. Ninety‐six 12‐month‐olds participated. For half of the infants during habituation (i.e., familiarization), the adults' direction of gaze was directed towards an unfamiliar object (look condition). For the remaining half of the infants during habituation, the adults' direction of gaze was directed away from the unfamiliar object (look‐away condition). All infants were habituated to two events. One event consisted of an adult looking towards (look condition) or away from (look‐away condition) an object while facially and vocally conveying a positive affective expression. The second event consisted of the same adult looking towards or away from a different object while conveying a disgusted affective expression. Following the habituation phase and a 5‐min, 1‐day, or 1‐month delay, infants' visual preference was assessed. During the visual preference phase, infants saw the two objects side by side where the adult conveying the affective expression was not visible. Results of the visual preference phase indicate that infants in the look condition showed a significant preference for object previously paired with the positive affect following a 5‐min and 1‐day delay. No significant visual preference was found in the look condition following a 1‐month delay. No significant preferences were found at any retention interval in the look‐away condition. Results are discussed in terms of early learning, social referencing, and early memory.