3-D localization of virtual sound sources: Effects of visual environment, pointing method, and training

The ability to localize sound sources in three-dimensional space was tested in humans. In Experiment 1, naive subjects listened to noises filtered with subject-specific head-related transfer functions. The tested conditions included the pointing method (head or manual pointing) and the visual environment (VE; darkness or virtual VE). The localization performance was not significantly different between the pointing methods. The virtual VE significantly improved the horizontal precision and reduced the number of front-back confusions. These results show the benefit of using a virtual VE in sound localization tasks. In Experiment 2, subjects were provided with sound localization training. Over the course of training, the performance improved for all subjects, with the largest improvements occurring during the first 400 trials. The improvements beyond the first 400 trials were smaller. After the training, there was still no significant effect of pointing method, showing that the choice of either head- or manual-pointing method plays a minor role in sound localization performance. The results of Experiment 2 reinforce the importance of perceptual training for at least 400 trials in sound localization studies.     

Localization of moving sound

The final position of a moving sound source usually appears to be displaced in the direction of motion. We tested the hypothesis that this phenomenon, termed auditory representational momentum, is already emerging during, not merely after, the period of motion. For this purpose, we investigated the localization of a moving sound at different points in time. In a dark anechoic environment, an acoustic target moved along the frontal horizontal plane. In the initial, middle, or final phase of the motion trajectory, subjects received a tactile stimulus and determined the current position of the moving target at the moment of the stimulus by performing either relative-judgment or pointing tasks. Generally, in the initial phase of the auditory motion, the position was perceived to be displaced in the direction of motion, but this forward displacement disappeared in the further course of the motion. When the motion stimulus had ceased, however, its final position was again shifted in the direction of motion. The latter result suggests that representational momentum in spatial hearing is a phenomenon specific to the final point of motion. Mental extrapolation of past trajectory information is discussed as a potential source of this perceptual displacement.     

Localization of moving sound

The final position of a moving sound source usually appears to be displaced in the direction of motion. We tested the hypothesis that this phenomenon, termed auditory representational momentum, is already emerging during, not merely after, the period of motion. For this purpose, we investigated the localization of a moving sound at different points in time. In a dark anechoic environment, an acoustic target moved along the frontal horizontal plane. In the initial, middle, or final phase of the motion trajectory, subjects received a tactile stimulus and determined the current position of the moving target at the moment of the stimulus by performing either relative-judgment or pointing tasks. Generally, in the initial phase of the auditory motion, the position was perceived to be displaced in the direction of motion, but this forward displacement disappeared in the further course of the motion. When the motion stimulus had ceased, however, its final position was again shifted in the direction of motion. The latter result suggests that representational momentum in spatial hearing is a phenomenon specific to the final point of motion. Mental extrapolation of past trajectory information is discussed as a potential source of this perceptual displacement.     

Auditory localization: The importance of eye movements and a textured visual environment

Two experiments were designed to investigate the factors involved in the visual facilitation of auditory localization. In both experiments, adult human Ss pointed to targets in a variety of visual conditions. The results of the first experiment showed that target-directed eye movements were important. In the second experiment, eye localization was assessed, along with pointing localization. Both eye and hand localization of the hidden auditory targets were better when target-directed eye movements were made in a lighted environment than when made in the dark. Data also suggested that Ss have better knowledge of their eye position in the light. Possible mechanisms for the involvement of eye movements were suggested, and the theoretical importance of the results was discussed.     

Effect of visual surrounding motion on body sway in a three-dimensional environment

Unidirectional motion of a uniplanar background induces a codirectional postural sway. It has been shown recently that fixation of a stationary foreground object induces a sway response in the opposite direction (Bronstein & Buckwell, 1997) when the background moves transiently. The present study investigated factors determining this contradirectional postural response. In the experiments presented, center of foot pressure and head displacements were recorded from normal subjects. The subjects faced a visual background of 2 x 3 m, at a distance of 1.5 m, which could be moved parallel to the interaural axis. Results showed that when the visual scene consisted solely of a moving background, the conventional codirectional postural response was elicited. When subjects were asked to fixate an earth-fixed foreground (window frame) placed between them and the moving background, a consistent postural response in the opposite direction to background motion was observed. In addition, we showed that this contradirectional postural response was not transient but was sustained for the 11 sec of background motion. We investigated whether this contradirectional postural response was the consequence of the induced movement of the foreground by background motion. Although induced movement was verbally reported by subjects when viewing an earth-fixed target projected onto the moving background, the contradirectional sway did not occur. These results indicate that foreground-background separation in depth was necessary for the contradirectional postural response to occur rather than induced movement. Another experiment showed that, when the fixated foreground was attached to the head of the observer, the contradirectional sway was not observed and was therefore unrelated to vergence. Finally, results showed that the contradirectional postural response was, in the main, monocularly mediated. We conclude that the direction of the postural sway produced by a moving background in a three-dimensional environment is determined primarily by motion parallax.     

Development of a device to measure human sound localization

Measurements of human sound discrimination and localization are important for basic empirical and clinical applications. After a short survey of other methods such as evoked potentials, the development of a new device to measure human sound localization is described and its use illustrated with some examples. Built from a polyacrylic hemisphere or--in a later version--from an orbicular aluminum frame, the apparatus uses multiple speakers to emit auditory stimuli. The patient sits in the middle of the perimeter and has to press a button when a sound is perceived. In addition, the participant has to identify the correct speaker as the source of the sound. With this method it is possible to map the auditory field.     

The effect of sound on visual apparent movement

Twelve subjects found the longest possible interstimulus interval (ISI) at which they perceived continuous apparent motion of one light instead of partial motion or succession between two lights. In the visual condition, two lights only were presented. In the bimodal conditions, binaurally presented tones were presented synchronously with the lights, and the lights and tones were either spatially congruent (in phase) or incongruent (180 degrees out of phase). Bimodal presentations lowered the upper ISI threshold for the perception of visual apparent motion, and the reduction was greater when the tones and lights were spatially congruent. The threshold reduction may be caused by a perceptual inference about localization of the lights in space or by a change in visual persistence.     

Event memory and moving in a well-known environment

Research in narrative comprehension has repeatedly shown that when people read about characters moving in well-known environments, the accessibility of object information follows a spatial gradient. That is, the accessibility of objects is best when they are in the same room as the protagonist, and it becomes worse the farther away they are see, e.g., Morrow, Greenspan, & Bower, (Journal of Memory and Language, 26, 165–187, 1987). In the present study, we assessed this finding using an interactive environment in which we had people memorize a map and navigate a virtual simulation of the area. During navigation, people were probed with pairs of object names and indicated whether both objects were in the same room. In contrast to the narrative studies described above, several experiments showed no evidence of a clear spatial gradient. Instead, memory for objects in currently occupied locations (e.g., the location room) was more accessible, especially after a small delay, but no clear decline was evident in the accessibility of information in memory with increased distance. Also, memory for objects along the pathway of movement (i.e., rooms that a person only passed through) showed a transitory suppression effect that was present immediately after movement, but attenuated over time. These results were interpreted in light of the event horizon model of event cognition.     

Stereokinesis with moving visual phantoms

When a flat pattern composed of a solid ellipse with two symmetrical semirings (corresponding to the visible parts of a contour ellipse whose major axis is perpendicular to that of the solid ellipse) is slowly rotated in the frontoparallel plane, a compelling three-dimensional impression occurs. Subjects report seeing an egg-shaped object that is inserted into a circular ring: the two objects move solidly into three-dimensional space and a moving visual phantom is generated so that the ring appears completed by an illusory curved bar in the region closest to the observer during rotation. A number of variations of this illusion are presented. It is shown that stereokinetic phantoms (i) maintain the shape of the inducing elements; (ii) appear only after the stereokinetic transformation has taken place; and (iii) depend on the organization of the three-dimensional percept as a whole. Relations between stereokinetic phantoms and other completion phenomena are presented and discussed.     

Dynamic visual capture: apparent auditory motion induced by a moving visual target

Apparent motion of a sound source can be induced by a moving visual target. The direction of the perceived motion of the sound source is the same as that of the visual target, but the subjective velocity of the sound source is 25-50% of that of the visual target measured under the same conditions. Eye tracking of the light target tends to enhance the apparent motion of the sound, but is not a prerequisite for its occurrence. The findings are discussed in connection with the 'visual capture' or 'ventriloquism' effect.     

Perception of visual speed while moving

During self-motion, the world normally appears stationary. In part, this may be due to reductions in visual motion signals during self-motion. In 8 experiments, the authors used magnitude estimation to characterize changes in visual speed perception as a result of biomechanical self-motion alone (treadmill walking), physical translation alone (passive transport), and both biomechanical self-motion and physical translation together (walking). Their results show that each factor alone produces subtractive reductions in visual speed but that subtraction is greatest with both factors together, approximating the sum of the 2 separately. The similarity of results for biomechanical and passive self-motion support H. B. Barlow's (1990) inhibition theory of sensory correlation as a mechanism for implementing H. Wallach's (1987) compensation for self-motion.     

Relative effectiveness of three stimulus variables for locating a moving sound source

A study is reported in which it is shown that observers can use at least three types of acoustic variables that indicate reliably when a moving sound source is passing: interaural temporal differences, the Doppler effect, and amplitude change. Each of these variables was presented in isolation and each was successful in indicating when a (stimulated) moving sound source passed an observer. These three variables were put into competition (with each indicating that closest passage occurred at a different time) in an effort to determine their relative importance. It was found that amplitude change dominated interaural temporal differences which, in turn, dominated the Doppler effect stimulus variable. The results are discussed in terms of two interpretations. First, it is possible that subjects based their judgements on the potential discriminability of each stimulus variable. However, because the stimuli used involved easily discriminable changes, subjects may instead have based their judgements on the independence of a stimulus variable from different environmental situation conditions. The dominance ordering obtained supports the second interpretation.     

Available response choices affect localization of sound

Successful replication of an experiment by Butler and Humanski (1992) showed that listeners are able to proficiently localize sources on a lateral vertical plane on the basis of interaural differences alone. When a lateral horizontal array was included in the test setup, that finding was replicated only for a broadband signal interacting with the pinna, not for ones (lowpass and pure tone) providing only interaural differences. Cross-plane errors conforming to “cones of confusion” were observed for those latter sounds. In a second experiment, response options were made more unconstrained, which clarified the nature of the cross-plane confusions. Lowpass signals from lateral vertical plane sources tend to be heard at or close to the horizon. Measurement of cue values needs to take account of the response options available to listeners, as well as signal properties.