Spatial co-location and infants'' learning of auditory-visual associations

The role of spatial co-location between sight and sound in infants' cross-modal learning was examined in three experiments. Four-and 6-month-old infants were familiarized with a toy and an accompanying soundtrack. Across conditions, spatial congruity between sight and sound was varied. Following familiarization, infants were tested to determine under which conditions they learned to associate the toy with the sound. Results indicated age-related differences in how discrepant in location a sight and sound could be for infants to form a cross-modal association based on the amodal invariant of co-location. Specifically, 4-month-olds formed cross-modal associations under conditions of less precise co-location than did 6-month-olds. Parallel improvements in infants' sound localization abilities across this age span are likely a contributing factor to the observed developmental trend in cross-modal learning.     

Infants' learning,memory, and generalization of learning for bimodal events

Study 1 investigated whether infants 3 and 7 months of age show differential learning of and memory for sight-sound pairs depending on whether or not temporal synchrony was present; memory was assessed after a 10-min and 1-week interval. Study 2 examined whether 7-month-olds show generalization of learning when they encounter novel bimodal events that are similar (changes in size, orientation, or color, and spectral sound properties) to the sight-sound pairs learned 1 week earlier based on temporal synchrony. For Study 1, infants received a familiarization phase followed by a paired-comparison preference procedure to assess for learning of the sight-sound pairs. One week later a memory test was given. Results confirmed that 7-month-olds had no difficulty learning auditory-visual pairings regardless of whether or not events were temporally synchronous, and they remembered these 10 min and 1 week later. In contrast, 3-month-olds showed poorer learning of sight-sound associations in the no-synchrony than synchrony conditions, and memory for sight-sound pairs 1 week later was shown only for the synchrony conditions. Results for Study 2 revealed generalization of learning of bimodal pairings under all stimulus conditions after a 1-week interval at 7 months of age. Implications of these findings for development of intersensory knowledge are discussed.     

The importance of "what": Infants use featural information to index events

Dynamic spatial indexing is the ability to encode, remember, and track the location of complex events. For example, in a previous study, 6-month-old infants were familiarized to a toy making a particular sound in a particular location, and later they fixated that empty location when they heard the sound presented alone (Journal of Experimental Psychology: General, 2004, Vol. 133, pp. 46-62). The basis and developmental trajectory of this ability are currently unclear. We investigated dynamic spatial indexing across the first year after birth and tested the hypothesis that the structure of visual cues supports infants' learning of sound and location associations. In our study, 3-, 6-, and 10-month-olds were tested in a dynamic spatial indexing eye movement paradigm that paired two sounds with two locations. In one condition, these were reliably paired with two sets of visual features (two toys condition), replicating the original studies. We also presented a single set of visual cues in both locations (one toy condition) and multiple sets of visual features in both locations (six toys condition). Infants from 3months of age onward showed evidence of dynamic spatial indexing in the two toys condition, but only the 10-month-olds succeeded in the one toy and six toys conditions. We argue that this may reflect a broader developmental trajectory, whereby infants first make use of multiple cue integration but with age are able to learn from a narrow set of cues.     

Infants' perceptual differentiation of amodal and modality-specific audio-visual relations.

Ninety-six infants of 3 1/2 months were tested in an infant-control habituation procedure to determine whether they could detect three types of audio-visual relations in the same events. The events portrayed two amodal invariant relations, temporal synchrony and temporal microstructure specifying the composition of the objects, and one modality-specific relation, that between the pitch of the sound and the color/shape of the objects. Subjects were habituated to two events accompanied by their natural, synchronous, and appropriate sounds and then received test trials in which the relation between the visual and the acoustic information was changed. Consistent with Gibson's increasing specificity hypothesis, it was expected that infants would differentiate amodal invariant relations prior to detecting arbitrary, modality-specific relations. Results were consistent with this prediction, demonstrating significant visual recovery to a change in temporal synchrony and temporal microstructure, but not to a change in the pitch-color/shape relations. Two subsequent discrimination studies demonstrated that infants' failure to detect the changes in pitch-color/shape relations could not be attributed to an inability to discriminate the pitch or the color/shape changes used in Experiment 1. Infants showed robust discrimination of the contrasts used.     

Crossmodal learning in newborn infants: Inferences about properties of auditory-visual events

The majority of research on infant crossmodal perception has addressed the question of whether infants 4 months and older demonstrate the ability. By contrast, the focus in the present study was on newborns and the primary goal was twofold: to determine if newborns can learn arbitrary sight-sound pairings and to evaluate possible inferences that neonates make when repeatedly presented arbitrary sight-sound pairings. Alert neonates were familiarized with toy-sound pairs using an infant-controlled habituation procedure. They were then given a violation-of-expectancy test to determine if they had learned these sight-sound pairings and drawn any inferences about certain properties of these bimodal events. Results supported the conclusion that infants just a few hours old can learn sight-sound pairings. Furthermore, the findings suggested that newborns expected sight-sound pairs to remain colocated even when they move to a new location, they recognized that the sound was an attribute of a specific object, and they recognized that a specific spatial location was not an attribute of the sight-sound pair. Thus, infants made important and correct inferences based on limited experiences with specific auditory-visual events, and they did so just after birth. Implications of these findings for theories of crossmodal perceptual development are discussed.     

Infants' coordination of auditory and visual depth information.

Using the preferential-looking procedure infants 5, 7, and 9 months of age were presented two videoimages side by side on separate monitors accompanied by a soundtrack that matched one of the images. Each infant was presented: (1) a stationary drum-beating toy paired with the same toy approaching and receding in depth, to assess infants' recognition both that changing sound amplitude is a property of an object that is moving in space and that constancy in amplitude is a property of a stationary object; (2) a drum-beating toy moving horizontally paired with one approaching and receding in depth, to assess infants' recognition that systematic increases and decreases in amplitude accompany object movement in a particular dimension, namely depth; and (3) two identical toys alternately approaching and receding in depth but out of phase (i.e., one approaching while the other is receding), to assess infants' recognition that increases and decreases in amplitude accompany a particular type of object movement in depth. Measures of mean duration of looking time indicated that the 5-month-olds looked reliably to the correct videoimage only for the stationary toy paired with the constant amplitude sound. The 7-month-olds recognized that changes in amplitude accompany object movement in depth but did not coordinate auditory with visual depth information as well as older infants. The 9-month-olds looked reliably to the correct videoimages in all conditions. Possible contributing factors to these developmental trends in performance are discussed.     

No two cues are alike: Depth of learning during infancy is dependent on what orients attention

Human infants develop a variety of attentional mechanisms that allow them to extract relevant information from a cluttered multimodal world. We know that both social and nonsocial cues shift infants’ attention, but not how these cues differentially affect learning of multimodal events. Experiment 1 used social cues to direct 8- and 4-month-olds’ attention to two audiovisual events (i.e., animations of a cat or dog accompanied by particular sounds) while identical distractor events played in another location. Experiment 2 directed 8-month-olds’ attention with colorful flashes to the same events. Experiment 3 measured baseline learning without attention cues both with the familiarization and test trials (no cue condition) and with only the test trials (test control condition). The 8-month-olds exposed to social cues showed specific learning of audiovisual events. The 4-month-olds displayed only general spatial learning from social cues, suggesting that specific learning of audiovisual events from social cues may be a function of experience. Infants cued with the colorful flashes looked indiscriminately to both cued locations during test (similar to the 4-month-olds learning from social cues) despite attending for equal duration to the training trials as the 8-month-olds with the social cues. Results from Experiment 3 indicated that the learning effects in Experiments 1 and 2 resulted from exposure to the different cues and multimodal events. We discuss these findings in terms of the perceptual differences and relevance of the cues.     

Infants' intermodal perception of two levels of temporal structure in natural events

Infants' intermodal perception of two levels of temporal structure uniting the visual and acoustic stimulation from natural, complex events was investigated in four experiments. Films depicting a single object (single, large marble) and a compound object (group of smaller marbles) colliding against a surface in an erratic pattern were presented to infants between 3 and months of age using an intermodal preference and search method. These stimulus events portrayed two levels of invariant temporal structure: (a) temporal synchrony united the sights and sounds of object impact, and (b) temporal microstructure, the internal temporal structure of each impact sound and motion, specified the composition of the object (single vs. compound). Experiment 1 demonstrated that by 6 months infants detected a relation between the audible and visible stimulation from these events when both levels of invariant temporal structure guided their intermodal exploration. Experiment 2 revealed that by 6 months infants detected the bimodal temporal microstructure specifying object composition. They looked predominantly to the film whose natural soundtrack was played even though the motions of objects in both films were synchronized with the soundtrack. Experiment 3 assessed infants' sensitivity to temporal synchrony relations. Two films depicting objects of the same composition were presented while the motions of only one of them was synchronized with the appropriate soundtrack. Both 6-month-olds showed evidence of detecting temporal synchrony relations under some conditions. Experiment 4 examined how temporal synchrony and temporal microstructure interact in directing intermodal exploration. The natural soundtrack to one of the objects was played out-of-synchrony with the motions of both. In contrast with the results of Experiment 2, infants at 6 months showed no evidence of detecting a relationship between the film and its appropriate soundtrack. This suggests that the temporal asynchrony disrupted their detection of the temporal microstructure specifying object composition. Results of these studies support on invariant-detection view of the development of intermodal perception.     

Does category-training facilitate 11-month-olds’ acquisition of unfamiliar category-property associations?

The ability to form category-property links allows infants to extend a property from one category member to another. In two experiments, we examined whether orienting infants to the demands of the task, through categorization training, would facilitate 11-month-old infants’ category-property extensions when familiarized with a single exemplar of an unfamiliar animal category. In Experiment 1, 11-month-olds (N = 35) were trained with two familiar animal-sound pairings (i.e., dog-bark, cat-meow), familiarized with two unfamiliar animal-sound pairings and then tested on their learning and generalization of the unfamiliar animal-sound associations. Across two conditions, Experiment 2 familiarized 11-month-olds (N = 69) to one familiar (i.e., dog-bark) and one novel animal-sound pairing. Conditions differed in their presentation of familiarization trials (i.e., random or blocked). Infants were then tested on their learning and extension of the animal-sound associations. In both experiments, infants did not demonstrate learning of the original animal sound pairing, nor generalization of the sound property to new members of the animal categories. These results indicate that the two category training paradigms implemented in the current studies did not facilitate 11-month-olds’ ability to learn or generalize an unfamiliar animal-sound association, when familiarized with a single exemplar.     

Increasing specificity in perceptual development: infants' detection of nested levels of multimodal stimulation

This research assessed the development of infants' sensitivity to two nested amodal temporal relations in audible and visible events. Their detection of global temporal synchrony between visible and audible impacts and internal temporal structure nested within each impact specifying object composition (single versus compound objects) was assessed. Infants of 4, 7, and 11 weeks of age were habituated to a single and a compound object striking a surface and then received test trials depicting a change in synchrony or object composition. Results indicated an interaction between age and condition where sensitivity to synchrony was present by 4 weeks and remained stable across age, whereas sensitivity to composition emerged later, by 7 weeks, and increased dramatically with age. These findings converge with other recent findings to illustrate a pattern of increasing specificity in the development of perception, where infants first detect global and later detect embedded relations. The early sensitivity to global relations may provide an organizational framework for development by focusing infant attention on unitary events, guiding and constraining further exploration, and buffering infants from learning incongruent relations.     

Mediated imitation in 6-month-olds: remembering by association

In two experiments with 72 6-month-olds, we examined whether associating an imitation task with an operant task affects infants' memory for either task. In Experiment 1, infants who imitated target actions that were modeled for 60 s on a hand puppet remembered them for only 1 day. We hypothesized that if infants associated the puppet imitation task with a longer-remembered operant task, then they might remember it longer too. In Experiment 2, infants learned to press a lever to activate a miniature train-a task 6-month-olds remember for 2 weeks-and saw the target actions modeled immediately afterward. These infants successfully imitated for up to 2 weeks, but only if the train memory was retrieved first. A follow-up experiment revealed that the learned association was bidirectional. This is the first demonstration of mediated imitation in 6-month-olds across two very different paradigms and reveals that associations are an important means of protracting memories.     

Learning how to learn from social feedback: The origins of early vocal development

Infants’ prelinguistic vocalizations reliably organize vocal turn-taking with social partners, creating opportunities for learning to produce the sound patterns of the ambient language. This social feedback loop supporting early vocal learning is well-documented, but its developmental origins have yet to be addressed. When do infants learn that their non-cry vocalizations influence others? To test developmental changes in infant vocal learning, we assessed the vocalizations of 2- and 5-month-old infants in a still-face interaction with an unfamiliar adult. During the still-face, infants who have learned the social efficacy of vocalizing increase their babbling rate. In addition, to assess the expectations for social responsiveness that infants build from their everyday experience, we recorded caregiver responsiveness to their infants’ vocalizations during unstructured play. During the still-face, only 5-month-old infants showed an increase in vocalizing (a vocal extinction burst) indicating that they had learned to expect adult responses to their vocalizations. Caregiver responsiveness predicted the magnitude of the vocal extinction burst for 5-month-olds. Because 5-month-olds show a vocal extinction burst with unfamiliar adults, they must have generalized the social efficacy of their vocalizations beyond their familiar caregiver. Caregiver responsiveness to infant vocalizations during unstructured play was similar for 2- and 5-month-olds. Infants thus learn the social efficacy of their vocalizations between 2 and 5 months of age. During this time, infants build associations between their own non-cry sounds and the reactions of adults, which allows learning of the instrumental value of vocalizing.     

Beyond the distributional input? A developmental investigation of asymmetry in infants’ categorization of cats and dogs

Two experiments are reported using a visual familiarization categorization procedure. In both experiments, infants were familiarized with sets of stimuli previously shown to contain asymmetric feature distributions that support an asymmetry in young infants' categorization of cats and dogs (i.e. infants' cat category excludes dogs but their dog category includes cats). In Experiment 1, the asymmetry was replicated in 4-month-old infants. In contrast, 10-month-old infants demonstrated exclusive category representations for both cats and dogs. In Experiment 2, an additional group of 10-month-olds demonstrated exclusive representations for both cats and dogs under conditions of very limited within-task category familiarization. Potential mechanisms underlying the shift from an asymmetric to a symmetric pattern of categorization in the first year are discussed.     

Rapid acquisition of phonological alternations by infants

We explore whether infants can learn novel phonological alternations on the basis of distributional information. In Experiment 1, two groups of 12-month-old infants were familiarized with artificial languages whose distributional properties exhibited either stop or fricative voicing alternations. At test, infants in the two exposure groups had different preferences for novel sequences involving voiced and voiceless stops and fricatives, suggesting that each group had internalized a different familiarization alternation. In Experiment 2, 8.5-month-olds exhibited the same patterns of preference. In Experiments 3 and 4, we investigated whether infants' preferences were driven solely by preferences for sequences of high transitional probability. Although 8.5-month-olds in Experiment 3 were sensitive to the relative probabilities of sequences in the familiarization stimuli, only 12-month-olds in Experiment 4 showed evidence of having grouped alternating segments into a single functional category. Taken together, these results suggest a developmental trajectory for the acquisition of phonological alternations using distributional cues in the input.     

The interplay between perceptual organization and categorization in the representation of complex visual patterns by young infants

The relation between perceptual organization and categorization processes in 3- and 4-month-olds was explored. The question was whether an invariant part abstracted during category learning could interfere with Gestalt organizational processes. A 2003 study by Quinn and Schyns had reported that an initial category familiarization experience in which infants were presented with visual patterns consisting of a pacman shape and a complex polygon could interfere with infants' subsequent good continuation-based parsing of a circle from visual patterns consisting of a circle and a complex polygon. However, an alternative noninterference explanation for the results was possible because the pacman had been presented with greater frequency and duration than had the circle. The current study repeated Quinn and Schyns's procedure but provided an equivalent number of familiarization trials and duration of study time for the infants to process the pacman during initial familiarization and the circle during subsequent familiarization. The results replicated the previous findings of Quinn and Schyns. The data are consistent with the interference account and suggest that a cognitive system of adaptable feature creation can take precedence over organizational principles with which a perceptual system comes pre-equipped.     

The infant's acquisition of knowledge of bimodally specified events

Four-month-old infants viewed, for a duration of several minutes, two objects that bounced in synchrony with two percussion sounds. This synchrony was the only information tying each sound to its respective object. During the viewing the infants learned about the relationships between sound and object. Learning was revealed in two ways. In a search test, infants looked for an object when its sound was played. In a transfer test, infants' declining interest in a sound presented alone generalized to the visible object that the sound specified. Studies that reversed the spatial locations of the objects revealed that sound-object learning, rather than place or response learning, guided infants' perceptual exploration.     

The development of infant discrimination of affect in multimodal and unimodal stimulation: The role of intersensory redundancy

This research examined the developmental course of infants' ability to perceive affect in bimodal (audiovisual) and unimodal (auditory and visual) displays of a woman speaking. According to the intersensory redundancy hypothesis (L. E. Bahrick, R. Lickliter, & R. Flom, 2004), detection of amodal properties is facilitated in multimodal stimulation and attenuated in unimodal stimulation. Later in development, however, attention becomes more flexible, and amodal properties can be perceived in both multimodal and unimodal stimulation. The authors tested these predictions by assessing 3-, 4-, 5-, and 7-month-olds' discrimination of affect. Results demonstrated that in bimodal stimulation, discrimination of affect emerged by 4 months and remained stable across age. However, in unimodal stimulation, detection of affect emerged gradually, with sensitivity to auditory stimulation emerging at 5 months and visual stimulation at 7 months. Further temporal synchrony between faces and voices was necessary for younger infants' discrimination of affect. Across development, infants first perceive affect in multimodal stimulation through detecting amodal properties, and later their perception of affect is extended to unimodal auditory and visual stimulation. Implications for social development, including joint attention and social referencing, are considered.     

Intersensory redundancy guides attentional selectivity and perceptual learning in infancy

This study assessed an intersensory redundancy hypothesis, which holds that in early infancy information presented redundantly and in temporal synchrony across two sense modalities selectively recruits attention and facilitates perceptual differentiation more effectively than does the same information presented unimodally. Five-month-old infants' sensitivity to the amodal property of rhythm was examined in 3 experiments. Results revealed that habituation to a bimodal (auditory and visual) rhythm resulted in discrimination of a novel rhythm, whereas habituation to the same rhythm presented unimodally (auditory or visual) resulted in no evidence of discrimination. Also, temporal synchrony between the bimodal auditory and visual information was necessary for rhythm discrimination. These findings support an intersensory redundancy hypothesis and provide further evidence for the importance of redundancy for guiding and constraining early perceptual learning.     

The effect of positive emotion on infants' gaze shift

Infants often voluntarily glance at their social partner during their toy play, disengaging their gaze from a toy and selecting a caregiver as their new looking target. This study posed two research questions: Do positive emotions disengage infants' gaze from their point of fixation, and do positive emotions facilitate the selection of the caregiver as their next looking target? The rate of gaze shifts was calculated for neutral and positive emotional states during their toy play. Across all ages, infants exhibited more disengagement from their point of fixation in the positive state than in the neutral one. However, 6- and 9-month-old infants revealed no difference in selecting a caregiver or a physical object as their next looking target in the positive state, but 12-month-olds increased gazing at caregivers in the positive state. These results were discussed with regard to the role of positive emotions on the development of infants' initiating joint attention.     

Self-experience as a mechanism for learning about others: a training study in social cognition

Using a gaze-following task, the authors assessed whether self-experience with the view-obstructing properties of blindfolds influenced infants' understanding of this effect in others. In Experiment 1, 12-month-olds provided with blindfold self-experience behaved as though they understood that a person wearing a blindfold cannot see. When a blindfolded adult turned to face an object, these infants gaze followed significantly less than control infants who had either (a) seen and felt the blindfold but whose view had not been obstructed by it or (b) experienced a windowed blindfold through which they could see. In Experiment 2, 18-month-olds experienced either (a) a trick blindfold that looked opaque but could be seen through, (b) an opaque blindfold, or (c) baseline familiarization. Infants receiving trick-blindfold experience now followed a blindfolded adult's gaze significantly more than controls. The authors propose 3 mechanisms underlying infants' capacity to use self-experience as a framework for understanding the visual perception of others.