Effects of multimodal presentation and stimulus familiarity on auditory and visual processing

Two experiments examined the effects of multimodal presentation and stimulus familiarity on auditory and visual processing. In Experiment 1, 10-month-olds were habituated to either an auditory stimulus, a visual stimulus, or an auditory-visual multimodal stimulus. Processing time was assessed during the habituation phase, and discrimination of auditory and visual stimuli was assessed during a subsequent testing phase. In Experiment 2, the familiarity of the auditory or visual stimulus was systematically manipulated by prefamiliarizing infants to either the auditory or visual stimulus prior to the experiment proper. With the exception of the prefamiliarized auditory condition in Experiment 2, infants in the multimodal conditions failed to increase looking when the visual component changed at test. This finding is noteworthy given that infants discriminated the same visual stimuli when presented unimodally, and there was no evidence that multimodal presentation attenuated auditory processing. Possible factors underlying these effects are discussed.     

Learning to utilize information presented over two sensory channels

In three separate experiments, Ss were provided with auditory, visual, or simultaneous auditory and visual information in a classification task. Difficulty of classification was manipulated by varying the stimulus exposure duration. Consistent bisensory facilitation effects were noted for later trials, with interference evident on earlier trials. Exposure duration influenced rate and not amount of learning, with bisensory performance being most affected by duration. A transfer paradigm was used in Experiment III, and little if any transfer was noted between unisensory and bisensory stimulus conditions. It was concluded that Ss were extracting the most salient bisensory stimulus components from the auditory and visual modes of information into a unidimensional information configuration.     

The effects of stimulus familiarization on patterns of visual selection

Patterns of visual selection were recorded as Ss viewed pairs of stimulus drawings in which the two members were either both incongruous or both banal. Prior to presenting the paired stimuli, S was preexposed to either one member of the stimulus pair or to the incongruous or banal counterpart of one member of the stimulus pair. The results indicate that: (1) preexposure to a stimulus reduced its potential to elicit looking responses, and the magnitude of that reduction was greater for incongruous stimuli than for banal stimuli; (2) preexposure to an incongruous stimulus affected the potential of its banal counterpart to elicit looking responses, but preexposure to a banal stimulus did not affect the potential of its incongruous counterpart to elicit similar responses; and (3) the reduced potential of the preexposed member of a stimulus pair to elicit looking responses waned after 10 sec. These results were discussed in light of an “information-conflict resolution” model of visual selection.     

Fearful faces modulate looking duration and attention disengagement in 7-month-old infants

The present study investigated whether facial expressions modulate visual attention in 7-month-old infants. First, infants' looking duration to individually presented fearful, happy, and novel facial expressions was compared to looking duration to a control stimulus (scrambled face). The face with a novel expression was included to examine the hypothesis that the earlier findings of greater allocation of attention to fearful as compared to happy faces could be due to the novelty of fearful faces in infants' rearing environment. The infants looked longer at the fearful face than at the control stimulus, whereas no such difference was found between the other expressions and the control stimulus. Second, a gap/overlap paradigm was used to determine whether facial expressions affect the infants' ability to disengage their fixation from a centrally presented face and shift attention to a peripheral target. It was found that infants disengaged their fixation significantly less frequently from fearful faces than from control stimuli and happy faces. Novel facial expressions did not have a similar effect on attention disengagement. Thus, it seems that adult-like modulation of the disengagement of attention by threat-related stimuli can be observed early in life, and that the influence of emotionally salient (fearful) faces on visual attention is not simply attributable to the novelty of these expressions in infants' rearing environment.     

The influence of infant-directed speech on 12-month-olds’ intersensory perception of fluent speech

The present study examined whether infant-directed (ID) speech facilitates intersensory matching of audio–visual fluent speech in 12-month-old infants. German-learning infants’ audio–visual matching ability of German and French fluent speech was assessed by using a variant of the intermodal matching procedure, with auditory and visual speech information presented sequentially. In Experiment 1, the sentences were spoken in an adult-directed (AD) manner. Results showed that 12-month-old infants did not exhibit a matching performance for the native, nor for the non-native language. However, Experiment 2 revealed that when ID speech stimuli were used, infants did perceive the relation between auditory and visual speech attributes, but only in response to their native language. Thus, the findings suggest that ID speech might have an influence on the intersensory perception of fluent speech and shed further light on multisensory perceptual narrowing.     

Attention to repeated events in human infants (Homo sapiens): effects of joint visual attention versus stimulus change

The goal of this study was to examine the effect of joint visual attention on infants' behavior during subsequent events. Thirty-seven mother-infant (aged from 9 to 13 months) pairs were twice shown a pair of line drawing stimuli on a computer screen. For the control group, the mother never paid attention to the stimuli. For experimental group 1, the mother pointed to one stimulus in the first presentation but did not point to it in the second presentation. The infants gazed longer at the stimulus pointed to by their mothers in the first presentation. In the second presentation, during which mothers did not attend to the stimuli, infants gazed longer at the stimulus which had been pointed to by the mothers in the first presentation. In experimental group 2, one of two stimuli blinked during the first presentation but not the second presentation. Infants gazed for longer at the blinking stimulus in the first presentation, but there was no difference in looking time toward the two stimuli in the second presentation. These results suggest that joint visual attention affects infants' looking behavior during subsequent events, and that simple stimulus change does not. Accepted after revision: 2 May 2001 Electronic Publication     

Intermodal perception at birth: Intersensory redundancy guides newborn infants’ learning of arbitrary auditory−visual pairings

In this study the ability of newborn infants to learn arbitrary auditory–visual associations in the absence versus presence of amodal (redundant) and contingent information was investigated. In the auditory-noncontingent condition 2-day-old infants were familiarized to two alternating visual stimuli (differing in colour and orientation), each accompanied by its ‘own’ sound: when the visual stimulus was presented the sound was continuously presented, independently of whether the infant looked at the visual stimulus. In the auditory-contingent condition the auditory stimulus was presented only when the infant looked at the visual stimulus: thus, presentation of the sound was contingent upon infant looking. On the post-familiarization test trials attention recovered strongly to a novel auditory–visual combination in the auditory-contingent condition, but remained low, and indistinguishable from attention to the familiar combination, in the auditory-noncontingent condition. These findings are a clear demonstration that newborn infants’ learning of arbitrary auditory–visual associations is constrained and guided by the presence of redundant (amodal) contingent information. The findings give strong support to Bahrick’s theory of early intermodal perception.     

Parallel scanning of auditory and visual information

Two experiments involving memory retrieval of auditorilv and visually presented materials were performed. In Experiment I, subjects were presented with memory sets of 1, 2, or 4 stimuli and then with a test item to be classified as belonging or not belonging to the memory set. In Condition 1, each memory stimulus was a single, auditorily presented letter. In Condition 2, each memory stimulus was a visually presented letter. In Conditions 3 and 4, each memory stimulus was a pair of letters, one presented visually and the other auditorily. Mean reaction time (RT) for the classification task increased as a function of number of memory stimuli at equal rates for all four conditions. This was interpreted as evidence for a parallel scanning process in Conditions 3 and 4 where the auditory item and visual item of each memory stimulus pair can be scanned simultaneously. Experiment II compared memory retrieval for a simultaneous condition in which auditory and visual memory items were presented as pairs with a sequential condition in which mixed auditory-visual memory sets were presented one item at a time. RTs were shorter for the simultaneous condition. This was interpreted as evidence that parallel scanning may depend upon memory input parameters.     

Infants' responsiveness to the auditory and visual attributes of a sounding/moving stimulus.

Responses to unimodal and multimodal attributes of a compound auditory/visual stimulus were investigated in 4-, 6-, 8-, and 10-month-old infants. First, infants were habituated to a compound stimulus consisting of a visual stimulus that moved up and down on a video monitor and a sound that occurred each time the visual stimulus reversed direction at the bottom. Once each infant met a habituation criterion, a series of test trials was administered to assess responsiveness to the components of the compound stimulus. Response was defined as the total duration of visual fixation in each trial. In the two unimodal test trials, the rate at which the component was presented was changed while the rate of the other component remained the same, whereas in the bimodal test trial the rate of both components was changed simultaneously. Results indicated that infants at each age successfully discriminated the bimodal and the two unimodal changes and that regression to the mean did not account for the results. Results also showed that disruption of the temporal relationship that accompanied the change in rate in the two unimodal test trials was also discriminable, but rate changes appeared to play a greater role in responsiveness than did synchrony changes. Considered together with results from similar prior studies, the current results are consistent with the modality appropriateness hypothesis in showing that discrimination of temporal changes in the auditory and visual modalities is dependent on the specialization of the sensory modalities.     

Stimulus spacing effects in duration perception are larger for auditory stimuli: Data and a model

Models of duration bisection have focused on the effects of stimulus spacing and stimulus modality. However, interactions between stimulus spacing and stimulus modality have not been examined systematically. Two duration bisection experiments that address this issue are reported. Experiment 1 showed that stimulus spacing influenced the classification of auditory, but not visual, stimuli. Experiment 2 used a wider stimulus range, and showed stimulus spacing effects for both visual and auditory stimuli, although the effects were larger for auditory stimuli. A version of Temporal Range Frequency Theory was applied to the data, and was used to demonstrate that the qualitative pattern of results can be captured with the single assumption that the durations of visual stimuli are less discriminable from one another than are the durations of auditory stimuli.     

Development of infant reaching in the dark to luminous objects and 'invisible sounds'

Two studies were conducted to investigate the existence of an unusual U-shaped developmental function described by Wishart et al (1978) for human infants reaching towards invisible sounds. In study 1, 2-7 month olds were presented with four conditions: (i) an invisible auditory stimulus alone, (ii) a glowing visual stimulus alone, (iii) auditory and visual stimuli on the same side (ie combined), and (iv) auditory and visual stimuli on opposite sides (ie in conflict). Study 2 was designed to examine the effects of practice and possible associations made when using the 'combined conflict' paradigm. Infants of 5 and 7 months of age were given five trials with the auditory stimulus, with or without prior visual experience, and five trials with the visual stimulus, with the position of the stimulus varied on each trial. Stimuli were presented individually at the midline, and +/- 30 and +/- 60 degrees from the midline. In both studies testing was conducted in complete darkness. Results indicated that the auditory-alone condition was slower to elicit a reach from the infants, relative to the visual-alone one, and reaches were least frequent to the auditory target. No U-shaped function was obtained, and reaching for auditory targets occurred later in age than for visual targets, but even at 7 months of age did not occur as often and was achieved by fewer infants. In both studies the quality of the reach was significantly poorer to auditory than to visual targets, but there were some accurate reaches. This research adds to our understanding of the development of auditory-manual coordination in sighted infants and is relevant to theories of auditory localization, visually guided reaching, and programming for the blind.     

Perception of internal elements of compound figures by one-month-old infants

Previous research has shown that infants younger than 2 months frequently concentrate their attention on peripheral or external features of visual stimuli and that they do not appear to process the internal elements of a compound visual stimulus composed of a simple geometric figure enclosed in a larger figure. A visual habituation-dishabituation paradigm was used to determine whether 1-month-old infants would process internal information if the internal elements of the stimulus were patterns often found to be preferred by young infants. Four groups of eight infants each were habituated to a compound stimulus consisting of a square or triangle surrounding a checkerboard or bull's-eye. Following habituation each group was presented with one of four types of stimulus: the identical stimulus (control group), a stimulus differing only in the internal component, a stimulus differing only in the external component, or a stimulus differing in both components. The results of the present study suggest that when the internal element of a compound stimulus is a highly preferred or salient stimulus, the young infant will process information about its characteristics.     

Effects of preexposure on stimulus discrimination: an investigation of the mechanisms responsible for human perceptual learning

The effect of preexposure on human perceptual learning was investigated in four experiments. In Experiments 1a and 1b, participants were preexposed to one pair of visual stimuli on an intermixed schedule (AX/BX) and one on a blocked schedule (CX_DX). The ability to discriminate between AX and BX and between CX and DX was then assessed by examining the extent to which key presses assigned to each member of a stimulus pair generalized to the other member (Experiment 1a) and by looking at the accuracy of same-different responses (Experiment 1b). Stimuli were more easily discriminated following intermixed than following blocked preexposure on both the generalization and same-different tasks. This suggests that two stimuli are more perceptually distinct after intermixed preexposure. Experiments 2a and 2b investigated the mechanisms responsible for perceptual learning using same-different tasks. The results support the suggestion that the enhanced discrimination observed after intermixed preexposure is due to increases in the salience of the unique elements.     

Young infants'' sensitivity to movement information specifying social causality

Adults and 3- to 6-month-old infants were tested for their visual preference for two different dynamic displays presented simultaneously on two side-by-side computer monitors. Each display consisted of a pair of colored discs moving either independently (the independent display) or in systematic interaction (the “chase” display), never actually contacting one another. Except for the relative spatio-temporal dependence of the discs' movements, all dynamic parameters on the two displays were controlled and maintained equal. Analysis of looking behavior showed that adults as well as infants looked differentially at the displays. Patterns of preference depended on age. For the infants who completed the experiment, there was a significant transition from more looking at the chase to more looking at the independent display as a function of age. Adults as well as the older, attentive infants, showed enhanced visual attention to the independent display. These results provide first evidence of young infants' sensitivity to movement information specifying social causality for adult observers.     

Perception of motion transparency in 5-month-old infants

We investigated the perceptual development of motion transparency in 3- to 5-month-old infants. In two experiments we tested a total of 55 infants and examined their preferential looking behaviour. In experiment 1, we presented transparent motion as a target, and uniform motion as a non-target consisting of random-dot motions. We measured the time during which infants looked at the target and non-target stimuli. In experiment 2, we used paired-dot motions (Qian et al, 1994 Journal of Neuroscience 14 7357-7366) as non-targets and also measured target looking time. We calculated the ratio of the target looking time to the total target and no-target looking time. In both experiments we controlled the dot size, speed, the horizontal travel distance of the dots, and the motion pattern of the dots. The results demonstrated that 5-month-old infants showed a statistically significant preference for motion transparency in almost all stimulus conditions, whereas the preference in 3- and 4-month-old infants depended on stimulus conditions. These results suggest that the sensitivity to motion transparency was robust in 5-month-olds, but not in 3- and 4-month-olds.     

Intermodal enhancement of stimulus localisation in infants born prematurely

The interaction of auditory and visual modalities in the enhancement of orientation was examined in premature and near-term infants by presenting them auditory or visual stimuli or auditory-visual stimulus combinations at various positions in sensory space. In 4.5--15-mo-olds, brisk orienting responses could be elicited to very peripheral stimulus positions but only when the stimulus consisted of a spatially coherent auditory-visual combination (i.e., where a sound and a light occurred at the same point in space). This occurred for all infants, irrespective of age or gestational age at birth. First, the result shows that infants can respond to visual stimuli at eccentric positions, beyond the supposed limits of their effective visual fields as measured by standard perimetry. Second, the result extends earlier studies showing that intersensory integration and stimulus localisation develop relatively normally in prematurely born infants. The auditory-visual enhancement test as used here may have a number of further uses and applications in the clinic and laboratory.     

Infants’ responsiveness to the auditory and visual attributes of a sounding/moving stimulus

Responses to unimodal and multimodal attributes of a compouMauxlitoryMsuat stimulus were investigated in 4-, 6-, 8-, and 10-month-old infants. First, infants were habituated to a compound stimulus consisting of a visual stimulus that moved up anddown on a video monitor and a sound that occurred each time the visual stimulus reversed direction at the bottom. Once each infant met a habituation criterion, a series of test trials was administered to assess responsiveness to the components of the compound stimulus. Response was defined as the total duration of visual fixation in each trial. In the two unimodal test trials, the rate at which the component was presented was changed while the rate of the other component remained the same, whereas in the bimodal test trial the rate of both components was changed simultaneously. Results indicated that infants at each age successfully discriminated the bimodal and the two unimodal changes and that regression to the mean did not account for the results. Results also showed that disruption of the temporal relationship that accompanied the change in rate in the two unimodal test trials was also discriminable, but rate changes appeared to play a greater role in responsiveness than did synchrony changes. Considered together with results from similar prior studies, the current results are consistent with the modality appropriateness hypothesis in showing that discrimination of temporal changes in the auditory and visual modalities is dependent on the specialization of the sensory modalities.     

Pattern preferences at birth and their interaction with habituation-induced novelty preferences

Three experiments are described which relate to models of infant visual preferences, and to the ways in which preferences can be modified or created by habituation. In all experiments newborn babies were used as subjects. In Experiments equated 1 and 2 infants were presented with pairs of stimuli that were equated for contour density but which differed in spatial frequency components. The preferences obtained give support to Banks and Salapatek's (1981, Journal of Experimental Child Psychology, 31, 1-45) model of infant preferences which predicts that the maximally preferred stimulus will be that which contains high amplitude spatial frequency components falling within the age group's peak contrast sensitivity. In Experiment 3 an infant-controlled habituation procedure was used. The results obtained suggest that strong natural preferences based on the infants' peak contrast sensitivity cannot be changed by habituating infants either to the preferred or to the nonpreferred member of a stimulus pair. However, where no prior preference exists between two stimuli that are perceptually highly discriminable, very strong novelty preferences are found after habituating newborns to either stimulus. The results suggest that the contrast sensitivity model can be a powerful predictor of preferential looking in newborns, and in addition are further evidence that preferences based on experience can be found from birth.     

The perception of identity by 6 1/2-month-old infants.

In Study 1, sixteen 6 1/2-month-olds were habituated to a Reversible stimulus (an upright face that could be perceived as an entirely different upright face when it was rotated 180 degrees) and to a Nonreversible stimulus (a face that could be perceived as upright in only one orientation). Following habituation for each type of stimulus, test trials paired the habituated face with a novel stimulus (an inversion of the same face). For both Reversible and Nonreversible stimuli, the physical difference between the old and new test stimuli was the same (a 180 degrees rotation); however, infants devoted more visual attention to the 180 degrees rotation only when it was a Reversible face, suggesting that the identity change was detected. Experiment 2 ruled out the explanation that infants might have failed to dishabituate to the inversion of the Nonreversible stimulus because they could not remember it. Results are interpreted as evidence that 6 1/2-month-old infants are not limited to face recognition based on similarity in pattern arrangement alone, but are capable of processing faces at a representational level.     

Effects of preexposure on stimulus discrimination: An investigation of the mechanisms responsible for human perceptual learning

The effect of preexposure on human perceptual learning was investigated in four experiments. In Experiments 1a and 1b, participants were preexposed to one pair of visual stimuli on an intermixed schedule (AX/BX) and one on a blocked schedule (CX_DX). The ability to discriminate between AX and BX and between CX and DX was then assessed by examining the extent to which key presses assigned to each member of a stimulus pair generalized to the other member (Experiment 1a) and by looking at the accuracy of same–different responses (Experiment 1b). Stimuli were more easily discriminated following intermixed than following blocked preexposure on both the generalization and same–different tasks. This suggests that two stimuli are more perceptually distinct after intermixed preexposure. Experiments 2a and 2b investigated the mechanisms responsible for perceptual learning using same–different tasks. The results support the suggestion that the enhanced discrimination observed after intermixed preexposure is due to increases in the salience of the unique elements.