Temporal ventriloquism: sound modulates the flash-lag effect

A sound presented in close temporal proximity to a visual stimulus can alter the perceived temporal dimensions of the visual stimulus (temporal ventriloquism). In this article, the authors demonstrate temporal ventriloquism in the flash-lag effect (FLE), a visual illusion in which a flash appears to lag relative to a moving object. In Experiment 1, the magnitude and the variability of the FLE were reduced, relative to a silent condition, when a noise burst was synchronized with the flash. In Experiment 2, the sound was presented before, at, or after the flash (+/- approximately 100 ms), and the size of the FLE varied linearly with the delay of the sound. These findings demonstrate that an isolated sound can sharpen the temporal boundaries of a flash and attract its temporal occurrence.     

闪光滞后效应的机制：时间错觉与空间错觉

闪光滞后效应（flash—lag effect,FLE）是指在与某运动物体相同的位置上呈现的闪光（flash）知觉上滞后于该运动物体的视错觉。本文简要介绍了有关闪光滞后效应机制的理论模型。笔者认为,目前各种理论的争议可归结为时间错觉假说与空间错觉假说之争,并在此基础上就今后进一步研究的方向提出了建议。     

Discrimination of spatial and temporal intervals defined by three light flashes: Effects of spacing on temporal judgments and of timing on spatial judgments

Observers were presented stimulus patterns consisting of a sequence of three laterally displaced light flashes, which defined two spatial intervals and two temporal intervals. The position and time of the second flash were varied factorially, and observers were asked to make relative judgments of either the two spatial intervals or the two temporal intervals. “Induction” effects of stimulus timing on spatial judgments and of stimulus spacing on temporal judgments were both found; however, the directionality of these effects differed between subjects. The results are inconsistent with the hypothesis, derived from previous findings, that such effects are determined primarily by a tendency toward perceiving constant velocity of apparent motion; it is proposed that the directionality of the induction effects is determined largely by the strategy adopted by the observer for combining spatial and temporal stimulus information.     

Object-based anisotropies in the flash-lag effect

The relative visual position of a briefly flashed stimulus is systematically modified in the presence of motion signals. We investigated the two-dimensional distortion of the positional representation of a flash relative to a moving stimulus. Analysis of the spatial pattern of mislocalization revealed that the perceived position of a flash was not uniformly displaced, but instead shifted toward a single point of convergence that followed the moving object from behind at a fixed distance. Although the absolute magnitude of mislocalization increased with motion speed, the convergence point remained unaffected. The motion modified the perceived position of a flash, but had little influence on the perceived shape of a spatially extended flash stimulus. These results demonstrate that motion anisotropically distorts positional representation after the shapes of objects are represented. Furthermore, the results imply that the flash-lag effect may be considered a special case of two-dimensional anisotropic distortion.     

Spatial mislocalization as a consequence of sequential coding of stimuli

In three experiments, we tested whether sequentially coding two visual stimuli can create a spatial misperception of a visual moving stimulus. In Experiment 1, we showed that a spatial misperception, the flash-lag effect, is accompanied by a similar temporal misperception of first perceiving the flash and only then a change of the moving stimulus, when in fact the two events were exactly simultaneous. In Experiment 2, we demonstrated that when the spatial misperception of a flash-lag effect is absent, the temporal misperception is also absent. In Experiment 3, we extended these findings and showed that if the stimulus conditions require coding first a flash and subsequently a nearby moving stimulus, a spatial flash-lag effect is found, with the position of the moving stimulus being misperceived as shifted in the direction of its motion, whereas this spatial misperception is reversed so that the moving stimulus is misperceived as shifted in a direction opposite to its motion when the conditions require coding first the moving stimulus and then the flash. Together, the results demonstrate that sequential coding of two stimuli can lead to a spatial misperception whose direction can be predicted from the order of coding the moving object versus the flash. We propose an attentional sequential-coding explanation for the flash-lag effect and discuss its explanatory power with respect to related illusions (e.g., the Fr?hlich effect) and other explanations.     

Illusory temporal order for stimuli at different depth positions

We used four experiments to examine how the perceived temporal order of two visual stimuli depends on the depth position of the stimuli specified by a binocular disparity cue. When two stimuli were presented simultaneously at different depth positions in front of or around a fixation point, the observer perceived the more distant stimulus before the nearer stimulus (Experiments 1 and 2). This illusory temporal order was found only for sudden stimulus presentation (Experiment 3). These results suggest that a common processing, which is triggered by sudden luminance change, underlies this illusion. The strength of the illusion increased with the disparity gradient and the disparity size (Experiment 4). We propose that this illusion has a basis in the processing of motion in depth, which would alert the observer to a potential collision with an object that suddenly emerges in front of the observer.     

Visual grouping by motion precedes the relative localization between moving and flashed stimuli

A flashed stimulus is perceived as spatially lagging behind a moving stimulus when they are spatially aligned. When several elements are perceptually grouped into a unitary moving object, a flash presented at the leading edge of the moving stimulus suffers a larger spatial lag than a flash presented at the trailing edge (K. Watanabe. R. Nijhawan. B. Khurana, & S. Shimojo. 2001). By manipulation of the flash onset relative to the motion onset, the present study investigated the order of perceptual operations of visual motion grouping and relative visual localization. It was found that the asymmetric mislocalization was observed irrespective of physical and/or perceptual temporal order between the motion and flash onsets. Thus, grouping by motion must be completed to define the leading-trailing relation in a moving object before the visual system explicitly represents the relative positions of moving and flashed stimuli.     

Spatial but not temporal cueing influences the mislocalisation of a target flashed during smooth pursuit

Human subjects misjudge the position of a target that is flashed during a pursuit eye movement. Their judgments are biased in the direction in which the eyes are moving. We investigated whether this bias can be reduced by making the appearance of the flash more predictable. In the normal condition, subjects pursued a moving target that flashed somewhere along its trajectory. After the presentation, they indicated where they had seen the flash. The mislocalisations in this condition were compared to mislocalisations in conditions in which the subjects were given information about when or where the flash would come. This information consisted of giving two warning flashes spaced at equal intervals before the target flash, of giving two warning beeps spaced at equal intervals before the target flash, or of showing the same stimulus twice. Showing the same stimulus twice significantly reduced the mislocalisation. The other conditions did not. We interpret this as indicating that it is not predictability as such that influences the performance, but the fact that the target appears at a spatially cued position. This was supported by a second experiment, in which we examined whether subjects make smaller misjudgments when they have to determine the distance between a target flashed during pursuit and a reference seen previously, than when they have to determine the distance between the flashed target and a reference seen afterwards. This was indeed the case, presumably because the reference provided a spatial cue for the flash when it was presented first. We conclude that a spatial cue reduces the mislocalisation of targets that are flashed during pursuit eye movements. The cue does not have to be exactly at the same position as the flash.     

Saccades reveal that allocentric coding of the moving object causes mislocalization in the flash-lag effect

The flash-lag effect is a visual misperception of a position of a flash relative to that of a moving object: Even when both are at the same position, the flash is reported to lag behind the moving object. In the present study, the flash-lag effect was investigated with eye-movement measurements: Subjects were required to saccade to either the flash or the moving object. The results showed that saccades to the flash were precise, whereas saccades to the moving object showed an offset in the direction of motion. A further experiment revealed that this offset in the saccades to the moving object was eliminated when the whole background flashed. This result indicates that saccadic offsets to the moving stimulus critically depend on the spatially distinctive flash in the vicinity of the moving object. The results are incompatible with current theoretical explanations of the flash-lag effect, such as the motion extrapolation account. We propose that allocentric coding of the position of the moving object could account for the flash-lag effect.     

Temporal ventriloquism in a purely temporal context

This study examines how audiovisual signals are combined in time for a temporal analogue of the ventriloquist effect in a purely temporal context, that is, no spatial grounding of signals or other spatial facilitation. Observers were presented with two successive intervals, each defined by a 1250-ms tone, and indicated in which interval a brief audiovisual stimulus (visual flash + noise burst) occurred later. In "test" intervals, the audiovisual stimulus was presented with a small asynchrony, while in "probe" intervals it was synchronous and presented at various times guided by an adaptive staircase to find the perceived temporal location of the asynchronous stimulus. As in spatial ventriloquism, and consistent with maximum likelihood estimation (MLE), the asynchronous audiovisual signal was shifted toward the more reliably localized component (audition, for all observers). Moreover, these temporal shifts could be forward or backward in time, depending on the asynchrony order, suggesting perceived timing is not entirely determined by physical timing. However, the critical signature of MLE combination--better bimodal than unimodal precision--was not found. Regardless of the underlying model, these results demonstrate temporal ventriloquism in a paradigm that is defined in a purely temporal context.     

Judgments of synchrony between auditory and moving or still visual stimuli.

The flash-lag effect is a visual illusion wherein intermittently flashed, stationary stimuli seem to trail after a moving visual stimulus despite being flashed synchronously. We tested hypotheses that the flash-lag effect is due to spatial extrapolation, shortened perceptual lags, or accelerated acquisition of moving stimuli, all of which call for an earlier awareness of moving visual stimuli over stationary ones. Participants judged synchrony of a click either to a stationary flash of light or to a series of adjacent flashes that seemingly bounced off or bumped into the edge of the visual display. To be judged synchronous with a stationary flash, audio clicks had to be presented earlier--not later--than clicks that went with events, like a simulated bounce (Experiment 1) or crash (Experiments 2-4), of a moving visual target. Click synchrony to the initial appearance of a moving stimulus was no different than to a flash, but clicks had to be delayed by 30-40 ms to seem synchronous with the final (crash) positions (Experiment 2). The temporal difference was constant over a wide range of motion velocity (Experiment 3). Interrupting the apparent motion by omitting two illumination positions before the last one did not alter subjective synchrony, nor did their occlusion, so the shift in subjective synchrony seems not to be due to brightness contrast (Experiment 4). Click synchrony to the offset of a long duration stationary illumination was also delayed relative to its onset (Experiment 5). Visual stimuli in motion enter awareness no sooner than do stationary flashes, so motion extrapolation, latency difference, and motion acceleration cannot explain the flash-lag effect.     

Perceived localizations and eye movements with action‐generated and computer‐generated vanishing points of moving stimuli

When observers localize the vanishing point of a moving target, localizations are reliably displaced beyond the final position, in the direction the stimulus was travelling just prior to its offset. We examined modulations of this phenomenon through eye movements and action control over the vanishing point. In Experiment 1 with pursuit eye movements, localization errors were in movement direction, but less pronounced when the vanishing point was self‐determined by a key press of the observer. In contrast, in Experiment 2 with fixation instruction, localization errors were opposite movement direction and independent from action control. This pattern of results points at the role of eye movements, which were gathered in Experiment 3. That experiment showed that the eyes lagged behind the target at the point in time, when it vanished from the screen, but that the eyes continued to drift on the targets' virtual trajectory. It is suggested that the perceived target position resulted from the spatial lag of the eyes and of the persisting retinal image during the drift.     

Predictability and the dynamics of position processing in the flash-lag effect

Several models have been proposed to account for the flash-lag effect. One criterion for evaluating alternative models is to consider the separate effects of motion predictability and flash predictability. We first established that flash predictability has an impact on the size of the perceived spatial offset in the flash-lag illusion. We then examined motion predictability by varying the consistency of the motion trajectory. Both manipulations affected the magnitude of the flash-lag illusion. These outcomes suggest that the perception of position is a dynamic process that can be modulated by explicit cues in advance of the flash and by the temporal integration of position information over a consistent motion trajectory. A complete explanation of the flash-lag effect must specify how flash predictability and motion predictability modulate position-processing mechanisms.     

Measurement of perceived oscillation

In earlier studies of perceived oscillation a comparison stimulus placed at a different distance and in a different direction from the observer than the Standard stimulus was used. The usual indicators were angles relating the turning positions, produced on a comparison stimulus, to a physical frame of reference, common for Standard and comparison. In three experiments where both Standard and comparison were “full-cuc,” the effect of different spatial arrangements was studied. Difference in distance did not affect the angles describing the response, nor did monocular or binocular vision, but difference in direction from the observer and the order of giving the two turning positions, combined with the position of the comparison stimulus before responding, had significant effects. The difference between Standard and comparison stimulus, which in an earlier series of experiments with artificial Standard stimuli had been rather large, was not very large for most conditions in the present experiments, where a real object was used as Standard stimulus. The correspondence was better, however, if another indicator, the angle describing the perceived extent of oscillation, was used. The spatial arrangements of Standard and comparison stimulus affect this indicator less than the indicators relating the perceived turning positions to the axes of the physical space.     

The duration of a brief event in the mind's eye.

A new illusion of perceived duration associated with focused spatial attention is reported. Brief flashes in attended locations were perceived to last longer than the same flashes in unattended locations. That illusion was shown to be completely independent of another illusion concerning the perceived onset of a flash, ruling out the possibility that the effect on perceived duration is derivative of a comparison between perceived onset and offset. The illusion also occurred when the event duration was composed of a temporal gap rather than a brief flash, ruling out low-level visible persistence as a basis for the illusion. Taken together, the results point to cortical connections from higher brain centers' both speeding and prolonging the visual signals occurring in lower sensory regions. Those temporal consequences could easily subserve many of the perceptual benefits ascribed to attention for spatial and intensive properties.     

Representational momentum and the flash‐lag effect

Representational momentum (RM) is a distortion where the final orientation of a moving object is misremembered as further along its trajectory. Experiments reported here examine RM when an additional object was flashed just as the moving object disappeared. When the task was to judge the flashed object, participants reported that the flash appeared to lag behind (flash‐lag effect; FLE). When the task was to judge the moving object, larger forward distortions for the moving object were found when the flash was present, despite previous evidence that the FLE depends on the moving object's continued presence. The results suggest that some part of the FLE depends upon what precedes the flash. In addition, equivalent RM was observed for implied and smoothly animated events, a possible limit to the velocity effect for RM was found, and larger positive distortions were found for downward rotations.     

Comparing mislocalizations with moving stimuli: The Fröhlich effect,the flash-lag,and representational momentum

When observers are asked to localize the onset or the offset position of a moving target, they typically make localization errors in the direction of movement. Similarly, when observers judge a moving target that is presented in alignment with a flash, the target appears to lead the flash. These errors are known as the Fröhlich effect, representational momentum, and flash-lag effect, respectively. This study compared the size of the three mislocalization errors. In Experiment 1, a flash appeared either simultaneously with the onset, the mid-position, or the offset of the moving target. Observers then judged the position where the moving target was located when the flash appeared. Experiments 2 and 3 are exclusively concerned with localizing the onset and the offset of the moving target. When observers localized the position with respect to the point in time when the flash was presented, a clear mislocalization in the direction of movement was observed at the initial position and the mid-position. In contrast, a mislocalization opposite to movement direction occurred at the final position. When observers were asked to ignore the flash (or when no flash was presented at all), a reduced error (or no error) was observed at the initial position and only a minor error in the direction of the movement occurred at the final position. An integrative model is proposed, which suggests a common underlying mechanism, but emphasizes the specific processing components of the Fröhlich effect, flash-lag effect, and representational momentum.     

Continuous measurement of visible persistence

In the synchrony judgment paradigm, observers judge whether a click precedes or follows the onset of a light flash and, on other trials, whether or not a click precedes light termination. The interclick interval defines the duration of visible persistence. An elaboration of this method consists of two phases: In Phase 1, the luminance of a reference stimulus is psychophysically matched to the peak brightness of the test flash. Five luminance values between .1 and 1.0 of the reference stimulus are used subsequently. In Phase 2, a random one of the five reference stimuli, a test flash, and a click are presented; the observer judges whether the click occurred before or after the brightness of test flash reached the reference value (on onset trials) or decayed below it (on termination trials). This method was validated on 3 subjects with test stimuli whose luminance rises and decays slowly in time, and then was used to trace out the precise subjective rise and decay (temporal brightness response function) of brief flashes.     

Simple reaction times to the onset,onset, and contrast reversal of sinusoidal grating stimuli

Response latencies to the onset, offset, and contrast reversal of sinusoidal gratings over a range of spatial frequencies were measured. For gratings of constant physical contrast, RT was monotonically related to spatial frequency regardless of presentation mode. Comparison of RTs to 1.0- and 9.0-cycle/deg gratings adjusted to equal apparent contrast showed that the RT shifts cannot be directly attributed to contrast sensitivity differences. It is concluded that spatial-frequency-dependent processing delays occur regardless of which temporal property of the stimulus the subject must respond to.     

The position of moving objects.

Moving objects occupy a range of positions during the period of integration of the visual system. Nevertheless, a unique position is usually observed. We investigate how the trajectory of a stimulus influences the position at which the object is seen. It has been shown before that moving objects are perceived ahead of static objects shown at the same place and time. We show here that this perceived position difference builds up over the first 500 ms of a visible trajectory. Discontinuities in the visual input reduce this buildup when the presentation frequency of a stimulus with a duration of 42 ms falls below 16 Hz. We interpret this relative mislocalization in terms of a spatiotemporal-filtering model. This model fits well with the data, given two assumptions. First, the position signal persists even though the objects are no longer visible and, second, the perceived distance is a 500 ms average of the difference of these position signals.