Auditory apparent motion under binaural and monaural listening conditions

This investigation examined the ability of listeners to perceive apparent motion under binaural and monaural listening conditions. Fifty-millisecond broadband noise sources were presented through two speakers separated in space by either 10 degrees, 40 degrees, or 160 degrees, centered about the subject's midline. On each trial, the sources were temporally separated by 1 of 12 interstimulus onset intervals (ISOIs). Six listeners were asked to place their experience of these sounds into one of five categories (single sound, simultaneous sounds, continuous motion, broken motion, or successive sounds), and to indicate either the proper temporal sequence of presentation or the direction of motion, depending on whether or not motion was perceived. Each listener was tested at all spatial separations under binaural and monaural listening conditions. Motion was perceived in the binaural listening condition at all spatial separations tested for ISOIs between 20 and 130 msec. In the monaural listening condition, motion was reliably heard by all subjects at 10 degrees and 40 degrees for the same range of ISOIs. At 160 degrees, only 3 of the 6 subjects consistently reported motion. However, when motion was perceived in the monaural condition, the direction of motion could not be determined.     

Auditory apparent motion in the free field: the effects of stimulus duration and separation.

The effects of stimulus duration and spatial separation on the illusion of apparent motion in the auditory modality were examined. Two narrow-band noise sources (40 dB, A-weighted) were presented through speakers separated in space by 2.5 degrees, 5 degrees, or 10 degrees, centered about the subject's midline. The duration of each stimulus was 5, 10, or 50 msec. On each trial, the sound pair was temporally separated by 1 of 10 interstimulus onset intervals (ISOIs): 0, 2, 4, 6, 8, 10, 15, 20, 50, or 70 msec. Five subjects were tested in nine trial block; each block represented a particular spatial-separation-duration combination. Within a trial block, each ISOI was presented 30 times each, in random order. Subjects were instructed to listen to the stimulus sequence and classify their perception of the sound into one of five categories: single sound, simultaneous sounds, continuous motion, broken motion, or successive sounds. Each subject was also required to identify the location of the first-occurring stimulus (left or right). The percentage of continuous-motion responses was significantly affected by the ISOI [F(9,36) = 5.67, p less than .001], the duration x ISOI interaction [F(18,72) = 3.54, p less than .0001], and the separation x duration x ISOI interaction [F(36,144) = 1.51, p less than .05]. The results indicate that a minimum duration is required for the perception of auditory apparent motion. Little or no motion was reported at durations of 10 msec or less. At a duration of 50 msec, motion was reported most often for ISOIs of 20-50 msec.(ABSTRACT TRUNCATED AT 250 WORDS)     

Auditory apparent motion in the free field: The effects of stimulus duration and separation

The effects of stimulus duration and spatial separation on the illusion of apparent motion in the auditory modality were examined. Two narrow-band noise sources (40 dB, A-weighted) were presented through speakers separated in space by 2.5°, 5°, or 100, centered about the subject’s midline. The duration of each stimulus was 5, 10, or 50 msec. On each trial, the sound pair was temporally separated by 1 of 10 interstimulus onset intervals (ISOIs): 0, 2, 4, 6, 8, 10, 15, 20, 50, or 70 msec. Five subjects were tested in nine trial blocks; each block represented a particular spatial-separation-duration combination. Within a trial block, each ISOI was presented 30 times each, in random order. Subjects were instructed to listen to the stimulus sequence and classify their perception of the sound into one of five categories: single sound, simultaneous sounds, continuous motion, broken motion, or successive sounds. Each subject was also required to identify the location of the first-occurring stimulus (left or right). The percentage of continuous-motion responses was significantly affected by the ISOI [F(9,36) = 5.67,p < .001], the duration × ISOI interaction [F(18,72) = 3.54,p < .0001], and the separation × duration × ISOI interaction [F(36,144) = 1.51,p < .05]. The results indicate that a minimum duration is required for the perception of auditory apparent motion. Little or no motion was reported at durations of 10 msec or less. At a duration of 50 msec, motion was reported most often for ISOIs of 20–50 msec. The effect of separation appeared to be limited to durations and-ISOIs during which little motion was perceived.     

Cross-modal dynamic capture: congruency effects in the perception of motion across sensory modalities

This study investigated multisensory interactions in the perception of auditory and visual motion. When auditory and visual apparent motion streams are presented concurrently in opposite directions, participants often fail to discriminate the direction of motion of the auditory stream, whereas perception of the visual stream is unaffected by the direction of auditory motion (Experiment 1). This asymmetry persists even when the perceived quality of apparent motion is equated for the 2 modalities (Experiment 2). Subsequently, it was found that this visual modulation of auditory motion is caused by an illusory reversal in the perceived direction of sounds (Experiment 3). This "dynamic capture" effect occurs over and above ventriloquism among static events (Experiments 4 and 5), and it generalizes to continuous motion displays (Experiment 6). These data are discussed in light of related multisensory phenomena and their support for a "modality appropriateness" interpretation of multisensory integration in motion perception.     

Effects of spatial separation between stimuli in whole report from brief visual displays

Direct measurements of the effects of spatial separation between stimuli in whole report from brief visual displays are reported. The stimuli were presented on the periphery of an imaginary circle centered on fixation. In Experiment 1, each display showed two capital letters (letter height approximately equal 1.3 degrees, width approximately equal 0.9 degree, eccentricity approximately equal 5.5 degrees). The proportion of correctly reported letters was a strictly increasing, decelerating function of the spatial separation between the letters for center-to-center separations ranging from less than 2 degrees to more than 10 degrees of visual angle. Experiment 2 yielded similar results with triples of letters. Experiment 3 showed that accuracy increased with spatial separation for report of two short words, and Experiment 4 showed the same result for words presented upside down. The results are explained by a model of lateral masking (crowding) based on competitive interactions within receptive fields of cortical neurons.     

A comparison of visual and auditory representational momentum in spatial tasks

Similarities have been observed in the localization of the final position of moving visual and moving auditory stimuli: Perceived endpoints that are judged to be farther in the direction of motion in both modalities likely reflect extrapolation of the trajectory, mediated by predictive mechanisms at higher cognitive levels. However, actual comparisons of the magnitudes of displacement between visual tasks and auditory tasks using the same experimental setup are rare. As such, the purpose of the present free-field study was to investigate the influences of the spatial location of motion offset, stimulus velocity, and motion direction on the localization of the final positions of moving auditory stimuli (Experiment 1 and 2) and moving visual stimuli (Experiment 3). To assess whether auditory performance is affected by dynamically changing binaural cues that are used for the localization of moving auditory stimuli (interaural time differences for low-frequency sounds and interaural intensity differences for high-frequency sounds), two distinct noise bands were employed in Experiments 1 and 2. In all three experiments, less precise encoding of spatial coordinates in paralateral space resulted in larger forward displacements, but this effect was drowned out by the underestimation of target eccentricity in the extreme periphery. Furthermore, our results revealed clear differences between visual and auditory tasks. Displacements in the visual task were dependent on velocity and the spatial location of the final position, but an additional influence of motion direction was observed in the auditory tasks. Together, these findings indicate that the modality-specific processing of motion parameters affects the extrapolation of the trajectory.     

The ventriloquist effect does not depend on the direction of deliberate visual attention

It is well known that discrepancies in the location of synchronized auditory and visual events can lead to mislocalizations of the auditory source, so-called ventriloquism. In two experiments, we tested whether such cross-modal influences on auditory localization depend on deliberate visual attention to the biasing visual event. In Experiment 1, subjects pointed to the apparent source of sounds in the presence or absence of a synchronous peripheral flash. They also monitored for target visual events, either at the location of the peripheral flash or in a central location. Auditory localization was attracted toward the synchronous peripheral flash, but this was unaffected by where deliberate visual attention was directed in the monitoring task. In Experiment 2, bilateral flashes were presented in synchrony with each sound, to provide competing visual attractors. When these visual events were equally salient on the two sides, auditory localization was unaffected by which side subjects monitored for visual targets. When one flash was larger than the other, auditory localization was slightly but reliably attracted toward it, but again regardless of where visual monitoring was required. We conclude that ventriloquism largely reflects automatic sensory interactions, with little or no role for deliberate spatial attention.     

Adaptation to auditory motion in the horizontal plane: effect of prior exposure to motion on motion detectability.

Thresholds for auditory motion detectability were measured in a darkened anechoic chamber while subjects were adapted to horizontally moving sound sources of various velocities. All stimuli were 500-Hz lowpass noises presented at a level of 55 dBA. The threshold measure employed was the minimum audible movement angle (MAMA)--that is, the minimum angle a horizontally moving sound must traverse to be just discriminable from a stationary sound. In an adaptive, two-interval forced-choice procedure, trials occurred every 2-5 sec (Experiment 1) or every 10-12 sec (Experiment 2). Intertrial time was "filled" with exposure to the adaptor--a stimulus that repeatedly traversed the subject's front hemifield at ear level (distance: 1.7 m) at a constant velocity (-150 degrees/sec to +150 degrees/sec) during a run. Average MAMAs in the control condition, in which the adaptor was stationary (0 degrees/sec,) were 2.4 degrees (Experiment 1) and 3.0 degrees (Experiment 2). Three out of 4 subjects in each experiment showed significantly elevated MAMAs (by up to 60%), with some adaptors relative to the control condition. However, there were large intersubject differences in the shape of the MAMA versus adaptor velocity functions. This loss of sensitivity to motion that most subjects show after exposure to moving signals is probably one component underlying the auditory motion aftereffect (Grantham, 1989), in which judgments of the direction of moving sounds are biased in the direction opposite to that of a previously presented adaptor.     

Effects of similarity on apparent motion and perceptual grouping

Effects of similarity in colour, luminance, size, and shape on apparent motion and perceptual grouping were examined in part 1 in two parallel experiments on the same seven subjects. In both experiments, the effect of similarity was compared with that of proximity in competitive, bistable stimulus situations. A combination of a larger horizontal separation between the homogeneous stimulus elements and a smaller constant vertical separation between heterogeneous stimulus elements produced two kinds of apparent motion (or perceptual grouping) with equal probabilities. Such matched separations between homogeneous stimulus elements were obtained by the double staircase method in various stimulus conditions. In both experiments on apparent motion and perceptual grouping matched separation was found to increase as the difference between the heterogeneous stimulus elements increased. High correlations (0.71 to 0.94) of matched separations were found between apparent motion and perceptual grouping in four stimulus series: colour, luminance, size, and shape. Six of the seven subjects were also tested in part 2. Here, the effects of differences were found to work additively across different perceptual attributes in both phenomena, when multiple differences were combined in heterogeneous elements. The experimental results are discussed from the point of view that apparent motion is an example of perceptual constancy.     

Does visual experience influence the spatial distribution of auditory attention?

Sighted individuals are less accurate and slower to localize sounds coming from the peripheral space than sounds coming from the frontal space. This specific bias in favour of the frontal auditory space seems reduced in early blind individuals, who are particularly better than sighted individuals at localizing sounds coming from the peripheral space. Currently, it is not clear to what extent this bias in the auditory space is a general phenomenon or if it applies only to spatial processing (i.e. sound localization). In our approach we compared the performance of early blind participants with that of sighted subjects during a frequency discrimination task with sounds originating either from frontal or peripheral locations. Results showed that early blind participants discriminated faster than sighted subjects both peripheral and frontal sounds. In addition, sighted subjects were faster at discriminating frontal sounds than peripheral ones, whereas early blind participants showed equal discrimination speed for frontal and peripheral sounds. We conclude that the spatial bias observed in sighted subjects reflects an unbalance in the spatial distribution of auditory attention resources that is induced by visual experience.     

Head-centred meridian effect on auditory spatial attention orienting

Six experiments examined the issue of whether one single system or separate systems underlie visual and auditory orienting of spatial attention. When auditory targets were used, reaction times were slower on trials in which cued and target locations were at opposite sides of the vertical head-centred meridian than on trials in which cued and target locations were at opposite sides of the vertical visual meridian or were not separated by any meridian. The head-centred meridian effect for auditory stimuli was apparent when targets were cued by either visual (Experiments 2, 3, and 6) or auditory cues (Experiment 5). Also, the head-centred meridian effect was found when targets were delivered either through headphones (Experiments 2, 3, and 5) or external loudspeakers (Experiment 6). Conversely, participants showed a visual meridian effect when they were required to respond to visual targets (Experiment 4). These results strongly suggest that auditory and visual spatial attention systems are indeed separate, as far as endogenous orienting is concerned.     

The effect of brief auditory stimuli on visual apparent motion

When two discrete stimuli are presented in rapid succession, observers typically report a movement of the lead stimulus toward the lag stimulus. The object of this study was to investigate crossmodal effects of irrelevant sounds on this illusion of visual apparent motion. Observers were presented with two visual stimuli that were temporally separated by interstimulus onset intervals from 0 to 350 ms. After each trial, observers classified their impression of the stimuli using a categorisation system. The presentation of short sounds intervening between the visual stimuli facilitated the impression of apparent motion relative to baseline (visual stimuli without sounds), whereas sounds presented before the first and after the second visual stimulus as well as simultaneously presented sounds reduced the motion impression. The results demonstrate an effect of the temporal structure of irrelevant sounds on visual apparent motion that is discussed in light of a related multisensory phenomenon, 'temporal ventriloquism', on the assumption that sounds can attract lights in the temporal dimension.     

Temporal constraints on apparent motion in auditory space

The hypothesis that the extent of spatial separation between successive sound events directly affects the perception of time intervals between these events was tested using an apparent motion paradigm. Subjects listened to four-tone pitch patterns whose individual tones were sounded alternately at one of two loudspeaker positions, and they adjusted the alternation rate until they could no longer distinguish the four-tone ordering of the pattern. Four horizontal and two vertical loudspeaker separations were tested. Results indicate a direct relation between horizontal separation and the critical stimulus onset asynchrony (SOA) between successive tones within a pattern. At the critical SOA, subjects reported hearing not a four-tone pattern, but two pairs of two-note groups overlapping in time. The findings are discussed in the context .of auditory spatial processing mechanisms and possible sensory-specific representational constraints.     

Auditory stimulation affects apparent motion1

Abstract: In two experiments, we investigated how the number of auditory stimuli affected the apparent motion induced by visual stimuli. The multiple visual stimuli that induced the apparent motion on the front parallel plane, or in the depth dimension in terms of the binocular disparity cue, were accompanied by multiple auditory stimuli. Observers reported the number of visual stimuli (Experiments 1 and 2) and the displacement of the apparent motion that was defined by the distance between the first and last visual stimuli (Experiment 2). When the number of auditory stimuli was more/less than that of the visual stimuli, observers tended to perceive more/less visual stimuli and a larger/smaller displacement than when the numbers of the auditory and visual stimuli were the same, regardless of the dimension of motion. These results suggest that auditory stimulation may modify the visual processing of motion by modulating the spatiotemporal resolution and extent of the displacement.     

Visual motion interferes with tactile motion perception

Previous studies have demonstrated that visual apparent motion can alter the judgment of auditory apparent motion. We investigated the effect of visual apparent motion on judgments of the direction of tactile apparent motion. When visual motion was presented at the same time as, but in a direction opposite to, tactile motion, accuracy in judging the direction of tactile apparent motion was substantially reduced. This reduction in performance is referred to as 'the congruency effect'. Similar effects were observed when the visual display was placed either near to the tactile display or at some distance from the tactile display (experiment 1). In experiment 2, the relative alignment between the visual and tactile directions of motion was varied. The size of the congruency effect was similar at 0 degrees and 45 degrees alignments but much reduced at a 90 degrees alignment. In experiment 3, subjects made confidence ratings of their judgments of the direction of the tactile motion. The results indicated that the congruency effect was not due to subjects being unsure of the direction of motion and being forced to guess. In experiment 4, static visual stimuli were shown to have no effect on the judgments of direction of the tactile stimuli. The extent to which the congruency effect reflects capture effects and is the result of perceptual versus post-perceptual processes is discussed.     

Auditory psychomotor coordination and visual search performance

In Experiments 1 and 2, the time to locate and identify a visual target (visual search performance in a two-alternative forced-choice paradigm) was measured as a function of the location of the target relative to the subject's initial line of gaze. In Experiment 1, tests were conducted within a 260 degree region on the horizontal plane at a fixed elevation (eye level). In Experiment 2, the position of the target was varied in both the horizontal (260 degrees) and the vertical (+/- 46 degrees from the initial line of gaze) planes. In both experiments, and for all locations tested, the time required to conduct a visual search was reduced substantially (175-1,200 msec) when a 10-Hz click train was presented from the same location as that occupied by the visual target. Significant differences in latencies were still evident when the visual target was located within 10 degrees of the initial line of gaze (central visual field). In Experiment 3, we examined head and eye movements that occur as subjects attempt to locate a sound source. Concurrent movements of the head and eyes are commonly encountered during auditorily directed search behavior. In over half of the trials, eyelid closures were apparent as the subjects attempted to orient themselves toward the sound source. The results from these experiments support the hypothesis that the auditory spatial channel has a significant role in regulating visual gaze.     

Perception of symmetry in infancy: the salience of vertical symmetry and the perception of pattern wholes

Four experiments were conducted to assess converging aspects of 4-month-old infants' perception of symmetry in visual patterns. Experiments 1 and 2 manipulated the structure and orientation of comparable patterns in order to evaluate the specialty of vertical symmetry. Infants showed no preference among vertically symmetrical, vertically repeated, and obliquely symmetrical patterns, but they processed vertically symmetrical patterns more efficiently than either vertically repeated patterns or obliquely symmetrical patterns. Experiment 3 manipulated the spatial separation of pattern components in order to determine the ability of young infants to integrate and coalesce information in visual patterns that is distributed in space. Infants processed vertically symmetrical patterns whose components were contiguous or nearly contiguous about the vertical axis (0 to 2.5 degrees separations) more efficiently than discontiguous patterns (5 and 10 degrees separations). Thus, extreme spatial separation about the vertical meridian caused infants to lose the advantage for vertical symmetry, and by inference their holistic perception of the visual pattern. Experiment 4 manipulated the organization of individual components of a vertical pattern in order to examine further infants' sensitivity to perceptual organization and synthesis of pattern form. Infants discriminated vertically symmetrical patterns from asymmetrical patterns with a vertical organization, thereby demonstrating sensitivity to the symmetrical organization of the pattern above their perception of components in the pattern. The results of these four experiments together corroborate and extend previous findings that vertical symmetry has a special status in early perceptual development and that infants can perceive pattern wholes.     

Automatic visual bias of perceived auditory location

Studies of reactions to audiovisual spatial conflict (alias “ventriloquism”) are generally presented as informing on the processes of intermodal coordination. However, most of the literature has failed to isolate genuine perceptual effects from voluntary postperceptual adjustments. A new approach, based on psychophysical staircases, is applied to the case of the immediate visual bias of auditory localization. Subjects have to judge the apparent origin of stereophonically controlled sound bursts as left or right of a median reference line. Successive trials belong to one of two staircases, starting respectively at extreme left and right locations, and are moved progressively toward the median on the basis of the subjects’ responses. Response reversals occur for locations farther away from center when a central lamp is flashed in synchrony with the bursts than without flashes (Experiment 1), revealing an attraction of the sounds toward the flashes. The effect cannot originate in voluntary postperceptual decision, since the occurrence of response reversal implies that the subject is uncertain concerning the direction of the target sound. The attraction is contingent on sound-flash synchronization, for early response reversals did no longer occur when the inputs from the two modalities were desynchronized (Experiment 2). Taken together, the results show that the visual bias of auditory localization observed repeatedly in less controlled conditions is due partly at least to an automatic attraction of the apparent location of sound by spatially discordant but temporally correlated visual inputs.     

Do you remember where sounds,pictures and words came from? The role of the stimulus format in object location memory

Contrasting results in visual and auditory spatial memory stimulate the debate over the role of sensory modality and attention in identity-to-location binding. We investigated the role of sensory modality in the incidental/deliberate encoding of the location of a sequence of items. In 4 separated blocks, 88 participants memorised sequences of environmental sounds, spoken words, pictures and written words, respectively. After memorisation, participants were asked to recognise old from new items in a new sequence of stimuli. They were also asked to indicate from which side of the screen (visual stimuli) or headphone channel (sounds) the old stimuli were presented in encoding. In the first block, participants were not aware of the spatial requirement while, in blocks 2, 3 and 4 they knew that their memory for item location was going to be tested. Results show significantly lower accuracy of object location memory for the auditory stimuli (environmental sounds and spoken words) than for images (pictures and written words). Awareness of spatial requirement did not influence localisation accuracy. We conclude that: (a) object location memory is more effective for visual objects; (b) object location is implicitly associated with item identity during encoding and (c) visual supremacy in spatial memory does not depend on the automaticity of object location binding.     

偏差声音干扰刺激对视觉注意网络系统的影响

结合注意网络测量工具和听视ODDBALL干扰范式,考察偏差声音干扰刺激的出现对注意网络系统不同功能的影响。结果发现,声音刺激序列下,视觉线索唤醒效应均消失; 标准声音刺激下,不存在线索的脱离和转移效应; 偏差声音刺激下,出现线索脱离和转移效应。研究表明,注意网络中的警觉功能具有超通道加工机制,偏差刺激下视觉注意系统中注意定向和线索转移效应的出现表明偏差干扰效应主要由偏差刺激引起的注意脱离和注意转移损耗部分引起。