Visual motion interferes with tactile motion perception

Previous studies have demonstrated that visual apparent motion can alter the judgment of auditory apparent motion. We investigated the effect of visual apparent motion on judgments of the direction of tactile apparent motion. When visual motion was presented at the same time as, but in a direction opposite to, tactile motion, accuracy in judging the direction of tactile apparent motion was substantially reduced. This reduction in performance is referred to as 'the congruency effect'. Similar effects were observed when the visual display was placed either near to the tactile display or at some distance from the tactile display (experiment 1). In experiment 2, the relative alignment between the visual and tactile directions of motion was varied. The size of the congruency effect was similar at 0 degrees and 45 degrees alignments but much reduced at a 90 degrees alignment. In experiment 3, subjects made confidence ratings of their judgments of the direction of the tactile motion. The results indicated that the congruency effect was not due to subjects being unsure of the direction of motion and being forced to guess. In experiment 4, static visual stimuli were shown to have no effect on the judgments of direction of the tactile stimuli. The extent to which the congruency effect reflects capture effects and is the result of perceptual versus post-perceptual processes is discussed.     

The trajectory effect in intermodal temporal order judgments

Subjects judged which one of two patterns, a visual or a tactile pattern, had been presented first. The visual and tactile displays were placed in close spatial proximity. The patterns appeared to move across their respective displays. Although irrelevant to the temporal order judgment (TOJ), the direction of motion of the patterns--the trajectory--affected the judgments. When the leading pattern was moving towards the trailing pattern (consistent movement), subjects tended to judge it, correctly, as leading. When the leading pattern was moving away from the trailing pattern (inconsistent movement), subjects tended to judge it, incorrectly, as trailing. Changing the spatial position of the arrays such that the pattern trajectories were no longer towards one another eliminated the effect of movement on TOJs. Although there was a substantial difference in performance on consistent and inconsistent trials, there were no differences in subjects' ratings of their performances. The results demonstrate that the trajectory effect can be obtained multimodally. The issues whether the effect of motion alters the perceived temporal separation between the visual and tactile patterns, and whether the visual and tactile patterns are represented by a common framework, are discussed.     

The effect of motion on tactile and visual temporal order judgments

Four experiments were conducted, three with tactile stimuli and one with visual stimuli, in which subjects made temporal order judgments (TOJs). The tactile stimuli were patterns that moved laterally across the fingerpads. The subject's task was to judge which finger received the pattern first. Even though the movement was irrelevant to the task, the subjects' TOJs were greatly affected by the direction of movement of the patterns. Accuracy in judging temporal order was enhanced when the patterns moved in a direction that was consistent with the temporal order of presentation--for example, when the movement on each fingerpad was from right to left and the temporally leading site of stimulation was to the right of the temporally trailing site of stimulation. When movement was inconsistent with the temporal order of presentation, accuracy was considerably reduced, often well below chance.The bias in TOJs was unaffected by training or by presenting the stimuli to fingers on opposite hands. In a fourth experiment, subjects judged the temporal order of visual stimuli that, like the tactile stimuli, moved in a direction that was either consistent or inconsistent with the TOJ. The results were similar to those obtained with tactile stimuli. It is suggested that the bias may be affected by attentional mechanisms and by apparent motion generated between the two sites on the skin.     

Tactile discrimination of competing sounds

Discriminably different sounds, concurrently presented from the left and right of the medial plane, were reduced in angular separation until subjects could no longer detect which sound was “left” and which was “right.” The procedure was repeated with hearing masked and judgments made on the basis of the tactile signals at two fingertip vibrators that received their inputs from two miniature microphones bilaterally located on the subject’s head. Auditory and tactile performance were compared under active (head movements permitted) and passive (head held still) conditions. Active and passive performance were not significantly different. Auditory and tactile performance became no better than chance at angular separations of 2.7° and 4.4°, respectively. Touch compared sufficiently well with audition to support arguments for the inclusion of sound localization information in devices which use the skin as a substitute for the ear. nt]mis|This research was conducted at the Smith-Kettlewell Institute of Visual Sciences, San Francisco, California. This institute provided the apparatus for the experiment. B. L. Richardson was on Staff Development Leave from the Applied Psychology Department of the Caulfield Institute of Technology, Melbourne, Australia.     

Task-irrelevant sounds influence both temporal order and apparent-motion judgments about tactile stimuli applied to crossed and uncrossed hands

It has been suggested that judgments about the temporal–spatial order of successive tactile stimuli depend on the perceived direction of apparent motion between them. Here we manipulated tactile apparent-motion percepts by presenting a brief, task-irrelevant auditory stimulus temporally in-between pairs of tactile stimuli. The tactile stimuli were applied one to each hand, with varying stimulus onset asynchronies (SOAs). Participants reported the location of the first stimulus (temporal order judgments: TOJs) while adopting both crossed and uncrossed hand postures, so we could scrutinize skin-based, anatomical, and external reference frames. With crossed hands, the sound improved TOJ performance at short (≤300 ms) and at long (>300 ms) SOAs. When the hands were uncrossed, the sound induced a decrease in TOJ performance, but only at short SOAs. A second experiment confirmed that the auditory stimulus indeed modulated tactile apparent motion perception under these conditions. Perceived apparent motion directions were more ambiguous with crossed than with uncrossed hands, probably indicating competing spatial codes in the crossed posture. However, irrespective of posture, the additional sound tended to impair potentially anatomically coded motion direction discrimination at a short SOA of 80 ms, but it significantly enhanced externally coded apparent motion perception at a long SOA of 500 ms. Anatomically coded motion signals imply incorrect TOJ responses with crossed hands, but correct responses when the hands are uncrossed; externally coded motion signals always point toward the correct TOJ response. Thus, taken together, these results suggest that apparent-motion signals are likely taken into account when tactile temporal–spatial information is reconstructed.

     

The effect of hand position and pattern motion on temporal order judgments

Subjects made temporal order judgments (TOJs) of tactile stimuli presented to the fingerpads. The subjects judged which one of two locations had been stimulated first. The tactile stimuli were patterns that simulated movement across the fingerpads. Although irrelevant to the task, the direction of movement of the patterns biased the TOJs. If the pattern at one location moved in the direction of the second location, the subjects tended to judge the first location as leading the second location. If the pattern moved in the opposite direction, that location was judged as trailing. In a series of experiments, the effect of the spatial position of the hands and fingers on TOJs and the perception of the direction of pattern movement were examined. Changing the position of the hands so that the patterns no longer moved directly toward each other reduced or eliminated the effect of motion on TOJs. In a variation of Aristotle's illusion, the moving patterns were presented to crossed and uncrossed fingers. The results indicated that, contrary to Aristotle's illusion, the subjects processed the moving patterns relative to an environmental framework, rather than to the local direction of motion on the fingerpads. Presenting the patterns to crossed hands produced results similar to those obtained with crossed fingers: The subjects processed the patterns according to an environmental framework.     

Congruency effects between auditory and tactile motion: Extending the phenomenon of cross-modal dynamic capture

Behavioral studies of multisensory integration in motion perception have focused on the particular case of visual and auditory signals. Here, we addressed a new case: audition and touch. In Experiment 1, we tested the effects of an apparent motion stream presented in an irrelevant modality (audition or touch) on the perception of apparent motion streams in the other modality (touch or audition, respectively). We found significant congruency effects (lower performance when the direction of motion in the irrelevant modality was incongruent with the direction of the target) for the two possible modality combinations. This congruency effect was asymmetrical, with tactile motion distractors having a stronger influence on auditory motion perception than vice versa. In Experiment 2, we used auditory motion targets and tactile motion distractors while participants adopted one of two possible postures: arms uncrossed or arms crossed. The effects of tactile motion on auditory motion judgments were replicated in the arms-uncrossed posture, but they dissipated in the arms-crossed posture. The implications of these results are discussed in light of current findings regarding the representation of tactile and auditory space.     

On the interplay of visuospatial and audiotemporal dominance: Evidence from a multimodal kappa effect

When participants judge multimodal audiovisual stimuli, the auditory information strongly dominates temporal judgments, whereas the visual information dominates spatial judgments. However, temporal judgments are not independent of spatial features. For example, in the kappa effect, the time interval between two marker stimuli appears longer when they originate from spatially distant sources rather than from the same source. We investigated the kappa effect for auditory markers presented with accompanying irrelevant visual stimuli. The spatial sources of the markers were varied such that they were either congruent or incongruent across modalities. In two experiments, we demonstrated that the spatial layout of the visual stimuli affected perceived auditory interval duration. This effect occurred although the visual stimuli were designated to be task-irrelevant for the duration reproduction task in Experiment 1, and even when the visual stimuli did not contain sufficient temporal information to perform a two-interval comparison task in Experiment 2. We conclude that the visual and auditory marker stimuli were integrated into a combined multisensory percept containing temporal as well as task-irrelevant spatial aspects of the stimulation. Through this multisensory integration process, visuospatial information affected even temporal judgments, which are typically dominated by the auditory modality.     

Audiotactile temporal order judgments

We report a series of three experiments in which participants made unspeeded 'Which modality came first?' temporal order judgments (TOJs) to pairs of auditory and tactile stimuli presented at varying stimulus onset asynchronies (SOAs) using the method of constant stimuli. The stimuli were presented from either the same or different locations in order to explore the potential effect of redundant spatial information on audiotactile temporal perception. In Experiment 1, the auditory and tactile stimuli had to be separated by nearly 80 ms for inexperienced participants to be able to judge their temporal order accurately (i.e., for the just noticeable difference (JND) to be achieved), no matter whether the stimuli were presented from the same or different spatial positions. More experienced psychophysical observers (Experiment 2) also failed to show any effect of relative spatial position on audiotactile TOJ performance, despite having much lower JNDs (40 ms) overall. A similar pattern of results was found in Experiment 3 when silent electrocutaneous stimulation was used rather than vibrotactile stimulation. Thus, relative spatial position seems to be a less important factor in determining performance for audiotactile TOJ than for other modality pairings (e.g., audiovisual and visuotactile).     

Further experiments on the perception of growth in three dimensions

Mark and Todd (1983) reported an experiment in which the cardioidal strain transformation was extended to three dimensions and applied to a three-dimensional (3-D) representation of the head of a 15-year-old girl in a direction that made the transformed head appear younger to the vast majority of their subjects. The experiments reported here extend this research in order to examine whether subjects are indeed detecting cardioidal strain in three dimensions, rather than detecting changes in head slant or making 2-D comparisons of the shape of the occluding contour. Three-dimensional surfaces were obtained by measuring a real head manually (Experiment 1) and with a laser scanner (Experiment 2), and transformed to different age levels using the 3-D strain transformation described by Mark and Todd (1983). There were no statistically significant differences in the accuracy with which relative age judgments could be made in response to pairs of profiles, pairs of 3/4 views, or pairs of mixed views (profile plus 3/4 view), suggesting that subjects can indeed extract the cardioidal strain level of the head in three dimensions. However, an additional effect that emerged in these studies was that judgments were crucially affected by the instructions given to subjects, which suggests that factors other than cardioidal strain are important in making judgments about rich data structures.     

Learning spatial dimensions with a visual sensory aid: Molyneux revisited

The relationship between sensory aid research and several areas of perceptual learning has been explored with five experiments on learning the use of the Binaural Sensory Aid, an electronic sensor in which pitch specifies distance and interaural amplitude difference (IAD) specifies direction. The training task required reaching to objects in near space, with tactile error feedback. Perceptual learning for both dimensions was demonstrated within 72 trials, giving a level of performance comparable to the use of a natural sound source, although performance with the direction cue did not reach asymptote until a second training session. Training was unaffected by various kinds of regularity in the spatial target sequences, or by a reduction in the number of spatial target locations until only two locations were used; at this point directional accuracy declines. Training only one dimension at a time did not produce additional improvement of performance on that dimension, but did impair generalization of the direction cue. Learning of the pitch-distance dimension was generally better than that of the IAD dimension, possibly because of its greater discriminability with this device. Generally, the pattern of results indicates that in learning to use such devices subjects readily determine the sensory dimensions of the codes and have considerable ability to generalize to new locations.     

Psychophysics of perceiving eye-gaze and head direction with peripheral vision: implications for the dynamics of eye-gaze behavior

Two psychophysical experiments are reported, one dealing with the visual perception of the head orientation of another person (the 'looker') and the other dealing with the perception of the looker's direction of eye gaze. The participant viewed the looker with different retinal eccentricities, ranging from foveal to far-peripheral viewing. On average, judgments of head orientation were reliable even out to the extremes of peripheral vision (90 degrees eccentricity), with better performance at the extremes when the participant was able to view the looker changing head orientation from one trial to the next. In sharp contrast, judgments of eye-gaze direction were reliable only out to 4 degrees eccentricity, signifying that the eye-gaze social signal is available to people only when they fixate near the looker's eyes. While not unexpected, this vast difference in availability of information about head direction and eye direction, both of which can serve as indicators of the looker's focus of attention, is important for understanding the dynamics of eye-gaze behavior.     

Tactile "capture" of audition

Previous research has demonstrated that the localization of auditory or tactile stimuli can be biased by the simultaneous presentation of a visual stimulus from a different spatial position. We investigated whether auditory localization judgments could also be affected by the presentation of spatially displaced tactile stimuli, using a procedure designed to reveal perceptual interactions across modalities. Participants made left-right discrimination responses regarding the perceived location of sounds, which were presented either in isolation or together with tactile stimulation to the fingertips. The results demonstrate that the apparent location of a sound can be biased toward tactile stimulation when it is synchronous, but not when it is asynchronous, with the auditory event. Directing attention to the tactile modality did not increase the bias of sound localization toward synchronous tactile stimulation. These results provide the first demonstration of the tactile capture of audition.     

Eye position and visual attention influence perceived auditory direction

The singular and joint effects of eccentric visual fixation and peripheral visual attention upon the perceived auditory median plane (PAMP) of the head were examined in four subjects. They positioned a hidden, horizontally-moving sound source to the perceived median plane of the head, while fixating a light 20 degrees or 45 degrees to the right or left of the actual median plane, and attending to another light in peripheral vision. Analysis by multiple linear regression yielded a model for each subject which indicated the nature and weighting of those factors contributing to variance reduction in PAMP judgments. Three subjects showed a strong linear attention effect, resulting in shifts of the PAMP away from the locus of attention. Similarly, a strong linear fixation effect was discovered in two subjects, a weak effect in another, and a nonlinear, inverse fixation effect in the last. Additional factors were noted for some subjects. Possible explanations of the observed effects were discussed. The study indicates that both visual attention and visual fixation are critical factors in experiments on auditory space perception although inter-individual differences do exist in the nature and strength of the effects.     

Crossmodal links between audition and touch in covert endogenous spatial attention

We report three experiments designed to investigate the nature of any crossmodal links between audition and touch in sustained endogenous covert spatial attention, using the orthogonal spatial cuing paradigm. Participants discriminated the elevation (up vs. down) of auditory and tactile targets presented to either the left or the right of fixation. In Experiment 1, targets were expected on a particular side in just one modality; the results demonstrated that the participants could spatially shift their attention independently in both audition and touch. Experiment 2 demonstrated that when the participants were informed that targets were more likely to be on one side for both modalities, elevation judgments were faster on that side in both audition and touch. The participants were also able to "split" their auditory and tactile attention, albeit at some cost, when targets in the two modalities were expected on opposite sides. Similar results were also reported in Experiment 3 when participants adopted a crossed-hands posture, thus revealing that crossmodal links in audiotactile attention operate on a representation of space that is updated following posture change. These results are discussed in relation to previous findings regarding crossmodal links in audiovisual and visuotactile covert spatial attentional orienting.     

The role of intonation cues in aphasic patients' performance of the grammaticality judgment task

This study investigated the possibility that the reported success of agrammatic aphasic patients in performing auditory grammaticality judgments results from their use of intonational cues to sentence well-formedness. Two agrammatic aphasics, two anomic aphasics, and two nonagrammatic patients with comprehension deficits for semantically reversible sentences were asked to judge grammatical well-formedness in three conditions. Results in a "natural" listening condition replicated the finding of M.C. Linebarger, M.F. Schwartz, and E. M. Saffran (1983, Cognition, 13, 361-392) that agrammatic patients are sensitive to grammaticality despite poor ability to use syntatic cues in comprehension tasks. Two additional listening conditions employed signal-processed auditory stimuli in which the fundamental frequency was normal or was set at a constant level. Right-hemisphere-damaged and normal controls showed no performance deficit for judgments of monotone sentences. The aphasic patients' performance was slightly worse for both signal-processed conditions, but there was little apparent effect of removing sentence intonation on their ability to judge sentence grammaticality. These results indicate that the high levels of performance on grammaticality judgement tasks have not resulted from patients' detection of intonational cues in ungrammatical sentences.     

Tactile “capture” of audition

Previous research has demonstrated that the localization of auditory or tactile stimuli can be biased by the simultaneous presentation of a visual stimulus from a different spatial position. We investigated whether auditory localization judgments could also be affected by the presentation of spatially displaced tactile stimuli, using a procedure designed to reveal perceptual interactions across modalities. Participants made left—right discrimination responses regarding the perceived location of sounds, which were presented either in isolation or together with tactile stimulation to the fingertips. The results demonstrate that the apparent location of a sound can be biased toward tactile stimulation when it is synchronous, but not when it is asynchronous, with the auditory event. Directing attention to the tactile modality did not increase the bias of sound localization toward synchronous tactile stimulation. These results provide the first demonstration of the tactilecapture of audition.     

The influence of the acoustic context on vertical sound localization in the median plane

The influence of background sounds (frames) on vertical localization of single sound sources (targets) was examined in four experiments. Loudspeakers (five targets and four frames) were positioned in the median plane, ranging from +30 degrees to -30 degrees above and below the subject's ear level. The subjects determined the vertical position of the targets by either verbal judgments or manual pointing. Frame and target sounds were presented concurrently or successively with a 1-sec interval; both consisted of (1) 300-Hz square waves, (2) noise, or (3) targets of noise and frames of 300-Hz square waves. Particularly in the second condition, the subjects consistently shifted the apparent target positions away from the frame locations. This contrast effect persisted even 1 sec after the offset of the frames. No effect was found with different waveforms for the frame and the target. Results are related to recent findings indicating a similar effect in the azimuthal dimension. Possibly the effect is based on a mechanism in which the auditory system adapts to recently heard sound source positions.     

Representing the egocentric auditory space: Relationships of surrounding region concepts

We investigated the representation of azimuthal directions of sound sources under two different conditions. In the first experiment, we examined the participants' mental representation of sound source directions via similarity judgments. Auditory stimuli originating from sixteen loudspeakers positioned equidistantly around the participant were presented in pairs, with the first stimulus serving as the anchor, and thereby providing the context for the second stimulus. For each pair of stimuli, participants had to rate the sound source directions as either similar or dissimilar. In the second experiment, the same participants categorized single sound source directions using verbal direction labels (front, back, left, right, and combinations of any two of these). In both experiments, the directions within the front and back regions were more distinctively categorized than those on the sides, and the sides' categories included more directions than those of the front or back. Furthermore, we found evidence that the left-right decision comprises the basic differentiation of the surrounding regions. These findings illustrate what seem to be central features of the representation of directions in auditory space.     

Modifying an underlying component of perceived arm length: adaptation of tactile location induced by spatial discordance

In four experiments we examined the adaptation and aftereffect that resulted from a treatment yielding tactile/kinesthetic length discordance between the arms. Perceived discordance diminished with trials and tended to zero. Subsequent visual/tactile cross-modal judgments of distance showed the aftereffect to be a change in the perceived location of an unseen probed spot on each hand with respect to the location of a truly coincident visual marker. This occurred toward the body for the probed spot on one arm and away from the body on the other. There were three other main findings: (a) Arm movement was not a necessary condition for adaptation or aftereffect; (b) with intrinsic length information about the right arm present, but touch information from the right index finger absent during treatment, adaptation and aftereffect were abolished; (c) aftereffects of tactile location that were manifest at the hand and wrist tended to zero when a point close to the elbow was tested with a cross-modal procedure. The experiments provide evidence that the mapping of the tactile sheet onto an internal length domain had been modified by the treatment. The sensory consequences of the treatment led many subjects to report spontaneously that their arms felt to be of different lengths.