Effects of the orientation of moving objects on the perception of streaming/bouncing motion displays

In this study, we examined the contribution of the orientation of moving objects to perception of a streaming/bouncing motion display. In three experiments, participants reported which of the two types of motion, streaming or bouncing, they perceived. The following independent variables were used: orientation differences between Gabor micropatterns (Gabors) and their path of motion (all the experiments) and the presence/absence of a transient tone (Experiment 1), transient visual flash (Experiment 2), or concurrent secondary task (Experiment 3) at the coincidence of Gabors. The results showed that the events at coincidence generally biased responses toward the perception of bouncing. On the other hand, alignment of Gabors with their motion axes significantly reduced the frequency of bounce perception. The results also indicated that an object whose orientation was parallel to its motion path strengthened the spatiotemporal integration of local motion signals along a straight motion path, resulting in the perception of streaming. We suggest that the effect of collinearity between Gabors and their motion path is relatively free from the effect of attention distraction.     

Postcoincidence trajectory duration affects motion event perception

In a two-dimensional display, identical visual targets moving toward and across each other with equal, constant speed can be perceived either to reverse their motion directions at the coincidence point (bouncing percept) or to stream through one another (streaming percept). Although there is a strong tendency to perceive the streaming percept, various factors have been reported to induce the bouncing percept, such as a sound or a visual flash at the moment of the visual target coincidence. By changing duration of the postcoincidence trajectory (PCT), we investigated how long it would take for such bounce-inducing factors to be maximally effective after the visual coincidence. With bounce-inducing factors, the percentage of the bouncing percept did not reach its maximal level immediately after the coincidence but increased as a function of PCT duration up to 150-200 msec. The results clearly reject the possibility of the cognitive-bias hypothesis about the bounce-inducing effect and suggest rather that the bounce-inducing factors have to interact with the PCT for some period after the coincidence to be maximally effective.     

Occluded motion alters event perception

We employed audiovisual stream/bounce displays, in which two moving objects with crossing trajectories are more likely to be perceived as bouncing off, rather than streaming through, each other when a brief sound is presented at the coincidence of the two objects. However, Kawachi and Gyoba (Perception 35:1289–1294, 2006b) reported that the presence of an additional moving object near the two objects altered the perception of a bouncing event to that of a streaming event. In this study, we extended this finding and examined whether alteration of the event perception could be induced by the visual context, such as by occluded object motion near the stream/bounce display. The results demonstrated that even when the sound was presented, the continuous occluded motion strongly biased observers’ percepts toward the streaming percept during a short occlusion interval (approximately 100 ms). In contrast, when the continuous occluded motion was disrupted by introducing a spatiotemporal gap in the motion trajectory or by removing occlusion cues such as deletion/accretion, the bias toward the streaming percept declined. Thus, we suggest that a representation of object motion generated under a limited occlusion interval interferes with audiovisual event perception.     

Presentation of a visual nearby moving object alters stream/bounce event perception

Two identical visual objects moving across each other in a two-dimensional display can be perceived as either streaming through or bouncing off each other. The bouncing event percept is promoted by the presentation of a brief sound at the point of coincidence of the two objects. In this study, we examined the effect of the presence of a moving object near the two objects as well as the brief sound on the stream/bounce event perception. When both the nearby moving object and brief sound were presented, a streaming event, not a bouncing event, was robustly perceived (experiment 1). The percentage of the streaming percept was also systematically affected by the proximity of the nearby object (experiment 2). These results suggest that the processing of intramodal grouping between a nearby moving object and either of the two objects in the stream/bounce display interferes with crossmodal (audiovisual) processing. Moreover, we demonstrated that, depending on the trajectory of the nearby moving object, the processing of intramodal grouping can promote the bouncing percept, just as crossmodal processing does (experiment 3).     

Visual motion disambiguation by a subliminal sound

There is growing interest in the effect of sound on visual motion perception. One model involves the illusion created when two identical objects moving towards each other on a two-dimensional visual display can be seen to either bounce off or stream through each other. Previous studies show that the large bias normally seen toward the streaming percept can be modulated by the presentation of an auditory event at the moment of coincidence. However, no reports to date provide sufficient evidence to indicate whether the sound bounce-inducing effect is due to a perceptual binding process or merely to an explicit inference resulting from the transient auditory stimulus resembling a physical collision of two objects. In the present study, we used a novel experimental design in which a subliminal sound was presented either 150 ms before, at, or 150 ms after the moment of coincidence of two disks moving towards each other. The results showed that there was an increased perception of bouncing (rather than streaming) when the subliminal sound was presented at or 150 ms after the moment of coincidence compared to when no sound was presented. These findings provide the first empirical demonstration that activation of the human auditory system without reaching consciousness affects the perception of an ambiguous visual motion display.     

Stream/bounce perception and the effect of depth cues in chimpanzees (Pan troglodytes)

The stream/bounce display represents an ambiguous motion event in which two identical visual objects move toward one another and the objects overlap completely before they pass each another. In our perception, they can be interpreted as either streaming past one another or bouncing off each other. Previous studies have shown that the streaming percept of the display is generic for humans, suggesting the inertial nature of the motion integration process. In this study, chimpanzees took part in behavioral experiments using an object-tracking task to reveal the characteristics of their stream/bounce perception. Chimpanzees did not show a tendency toward a dominant "stream" perception of the stream/bounce stimulus. However, depth cues, such as X-junctions and local motion coherence, did promote the stream percept in chimpanzees. These results suggest both similarities and differences between chimpanzees and humans with respect to motion integration and object individuation processes.     

Tactile stimulation disambiguates the perception of visual motion paths

Although visual perception traditionally has been considered to be impenetrable by non-visual information, there are a rising number of reports discussing cross-modal influences on visual perception. In two experiments, we investigated how coinciding vibrotactile stimulation affects the perception of two discs that move toward each other, superimpose in the center of the screen, and then move apart. Whereas two discs streaming past each other was the dominant impression when the visual event was presented in isolation, a brief coinciding vibrotactile stimulation at the moment of overlap biased the visual impression toward two discs bouncing off each other (Experiment 1). Further, the vibrotactile stimulation actually changed perceptual processing by reducing the amount of perceived overlap between the discs (Experiment 2), which has been demonstrated to be associated with a higher proportion of bouncing impressions. We propose that tactile-induced quantitative changes in the visual percept might alter the quality of the visual percept (from streaming to bouncing), thereby adding to the understanding of how cross-modal information interacts with early visual perception and how this interaction influences subsequent visual impressions.     

Perceived object trajectories during occlusion constrain visual statistical learning

Visual statistical learning of shape sequences was examined in the context of occluded object trajectories. In a learning phase, participants viewed a sequence of moving shapes whose trajectories and speed profiles elicited either a bouncing or a streaming percept: the sequences consisted of a shape moving toward and then passing behind an occluder, after which two different shapes emerged from behind the occluder. At issue was whether statistical learning linked both object transitions equally, or whether the percept of either bouncing or streaming constrained the association between pre- and postocclusion objects. In familiarity judgments following the learning, participants reliably selected the shape pair that conformed to the bouncing or streaming bias that was present during the learning phase. A follow-up experiment demonstrated that differential eye movements could not account for this finding. These results suggest that sequential statistical learning is constrained by the spatiotemporal perceptual biases that bind two shapes moving through occlusion, and that this constraint thus reduces the computational complexity of visual statistical learning.     

Individuation of visual objects over time

How does an observer decide that a particular object viewed at one time is actually the same object as one viewed at a different time? We explored this question using an experimental task in which an observer views two objects as they simultaneously approach an occluder, disappear behind the occluder, and re-emerge from behind the occluder, having switched paths. In this situation the observer either sees both objects continue straight behind the occluder (called "streaming") or sees them collide with each other and switch directions ("bouncing"). This task has been studied in the literature on motion perception, where interest has centered on manipulating spatiotemporal aspects of the motion paths (e.g. velocity, acceleration). Here we instead focus on featural properties (size, luminance, and shape) of the objects. We studied the way degrees and types of featural dissimilarity between the two objects influence the percept of bouncing vs. streaming. When there is no featural difference, the preference for straight motion paths dominates, and streaming is usually seen. But when featural differences increase, the preponderance of bounce responses increases. That is, subjects prefer the motion trajectory in which each continuously existing individual object trajectory contains minimal featural change. Under this model, the data reveal in detail exactly what magnitudes of each type of featural change subjects implicitly regard as reasonably consistent with a continuously existing object. This suggests a simple mathematical definition of "individual object:" an object is a path through feature-trajectory space that minimizes feature change, or, more succinctly, an object is a geodesic in Mahalanobis feature space.     

When sound affects vision: effects of auditory grouping on visual motion perception

Two identical visual targets moving across each other can be perceived either to bounce off or to stream through each other. A brief sound at the moment the targets coincide biases perception toward bouncing. We found that this bounce-inducing effect was attenuated when other identical sounds (auditory flankers) were presented 300 ms before and after the simultaneous sound. The attenuation occurred only when the simultaneous sound and auditory flankers had similar acoustic characteristics and the simultaneous sound was not salient. These results suggest that there is an aspect of auditory-grouping (saliency-assigning) processes that is context-sensitive and can be utilized by the visual system for solving ambiguity. Furthermore, control experiments revealed that such auditory context did not affect the perceptual qualities of the simultaneous sound. Because the attenuation effect is not manifest in the perception of acoustic characteristics of individual sound elements, we conclude that it is a genuine cross-modal effect.     

Audiovisual bounce-inducing effect: When sound congruence affects grouping in vision

Two disks moving from opposite points in space, overlapping, and stopping at one another’s starting point can be seen as either bouncing off one another or streaming through one another. With silent displays, observers report streaming, whereas, if a sound is played when the disks are in the overlap region, observers report bouncing. The change in perception is thought to be modulated by a lack of attention that inhibits the integration of the motion signal when disks overlap and by the sound that increases the congruence of the display, in comparison with a real elastic bounce. Here, we accompanied the disks’ motion with either a bounce-congruent sound (a billiard ball) or with bounce-incongruent sounds (a water drop, a firework). When the sound was switched on 200 msec before the disks’ overlap, (1) all the audiovisual displays induced more bounce responses than did the silent display, but (2) the bounce-congruent sound induced more bounce responses than did the bounceincongruent sounds. However, when the sound was switched on at the disks’ overlap, only the first result was observed. These results highlight both the role of attention and that of sound congruence.     

Tracking without perceiving: a dissociation between eye movements and motion perception

Can people react to objects in their visual field that they do not consciously perceive? We investigated how visual perception and motor action respond to moving objects whose visibility is reduced, and we found a dissociation between motion processing for perception and for action. We compared motion perception and eye movements evoked by two orthogonally drifting gratings, each presented separately to a different eye. The strength of each monocular grating was manipulated by inducing adaptation to one grating prior to the presentation of both gratings. Reflexive eye movements tracked the vector average of both gratings (pattern motion) even though perceptual responses followed one motion direction exclusively (component motion). Observers almost never perceived pattern motion. This dissociation implies the existence of visual-motion signals that guide eye movements in the absence of a corresponding conscious percept.     

Laterality and pattern persistence in bistable motion perception

Bistable motion perception refers to two competing perceptions that can result when frames consisting of three elements are displaced laterally by one element. At short inter-frame intervals, the dominant percept is that the end elements in the display are moving; at long inter-frame intervals, perception is of all the elements moving coherently to the right or left. This research shows that coherent motion is more likely to be perceived when presentations are parafoveal than foveal and when they are to the right visual field than the left visual field. These results support the idea that visual pattern persistence is shorter in the parafovea than in the fovea, and shorter in the right than in the left visual field.     

The stream/bounce effect occurs for luminance- and disparity-defined motion targets

We generalised the stream/bounce effect to dynamic random element displays containing luminance- or disparity-defined targets. Previous studies investigating audio-visual interactions in this context have exclusively employed motion sequences with luminance-defined disks or squares and have focused on properties of the accompanying brief stimuli rather than the visual properties of the motion targets. We found that the presence of a brief sound temporally close to coincidence, or a visual flash at coincidence significantly promote bounce perception for motion targets defined by either luminance contrast or disparity contrast. A brief tone significantly promoted bouncing of luminance-defined targets above a no-sound baseline when it was presented at least 250 ms before coincidence and 100 ms after coincidence. A similar pattern was observed for disparity-defined targets, though the tone promoted bouncing above the no-sound baseline when presented at least 350 ms before and 300 ms after coincidence. We further explored the temporal properties of audio-visual interactions for these two display types and found that bounce perception saturated at similar durations after coincidence. The stream/bounce illusion manifests itself in dynamic random-element displays and is similar for luminance- and disparity-defined motion targets.     

A unique visual rhythm does not pop out

We investigated attentional demands in visual rhythm perception of periodically moving stimuli using a visual search paradigm. A dynamic search display consisted of vertically “bouncing dots” with regular rhythms. The search target was defined by a unique visual rhythm (i.e., a shorter or longer period) among rhythmic distractors with identical periods. We found that search efficiency for a faster or a slower periodically moving target decreased as the number of distractors increased, although searching for a faster target was about one second faster than searching for a slower target. We conclude that perception of a visual rhythm defined by a unique period is not a “pop-out” process, but a serial one that demands considerable attention.     

First-order and second-order spectral 'motion' mechanisms in the human auditory system

Light energy displaced along the retinal photoreceptor array leads to a perception of visual motion. In audition, displacement of mechanical energy along the cochlear hair cell array is conceptually similar but leads to a perception of 'movement' in frequency space (spectral motion)--a rising or falling pitch. In vision there are other types of stimuli that also evoke a percept of motion but do not involve a displacement of energy across the photoreceptors (second-order stimuli). In this study, we used psychophysical methods to determine if such second-order stimuli also exist in audition, and if the resulting percept would rival that of first-order spectral motion. First-order auditory stimuli consisted of a frequency sweep of sixteen non-harmonic tones between 297 and 12123 Hz. Second-order stimuli consisted of the same tones, but with a random subset turned on at the beginning of a trial. During the trial, each tone in sequence randomly changed state (ON-to-OFF, or OFF-to-ON). Thus, state transitions created a 'sweep' having no net energy displacement correlated to the sweep direction. At relatively slow sweep speeds, subjects readily identified the sweep direction for both first-order and second-order stimuli, though accuracy decreased for second-order stimuli as the sweep speed increased. This latter characteristic is also true of some second-order visual stimuli. These results suggest a stronger parallelism between auditory and visual processing than previously appreciated.     

Visual attention modulates audiovisual speech perception

Speech perception is audiovisual, as demonstrated by the McGurk effect in which discrepant visual speech alters the auditory speech percept. We studied the role of visual attention in audiovisual speech perception by measuring the McGurk effect in two conditions. In the baseline condition, attention was focused on the talking face. In the distracted attention condition, subjects ignored the face and attended to a visual distractor, which was a leaf moving across the face. The McGurk effect was weaker in the latter condition, indicating that visual attention modulated audiovisual speech perception. This modulation may occur at an early, unisensory processing stage, or it may be due to changes at the stage where auditory and visual information is integrated. We investigated this issue by conventional statistical testing, and by fitting the Fuzzy Logical Model of Perception (Massaro, 1998) to the results. The two methods suggested different interpretations, revealing a paradox in the current methods of analysis.     

Perceiving object motion using vision and touch

In a previous experiment, we showed that bistable visual object motion was partially disambiguated by tactile input. Here, we investigated this effect further by employing a more potent visuotactile stimulus. Monocular viewing of a tangible wire-frame sphere (TS) rotating about its vertical axis produced bistable alternations of direction. Touching the TS biased simultaneous and subsequent visual perception of motion. Both of these biases were in the direction of the tactile stimulation and, therefore, constituted facilitation or priming, as opposed to interference or adaptation. Although touching the TS biased visual perception, tactile stimulation was not able to override the ambiguous visual percept. This led to periods of sensory conflict, during which visual and tactile motion percepts were incongruent. Visual and tactile inputs can sometimes be fused to form a coherent percept of object motion but, when they are in extreme conflict, can also remain independent.     

Stimulus eccentricity and spatial frequency interact to determine circular vection.

While early research suggested that peripheral vision dominates the perception of self-motion, subsequent studies found little or no effect of stimulus eccentricity. In contradiction to these broad notions of 'peripheral dominance' and 'eccentricity independence', the present experiments showed that the spatial frequency of optic flow interacts with its eccentricity to determine circular vection magnitude--central stimulation producing the most compelling vection for high-spatial-frequency stimuli and peripheral stimulation producing the most compelling vection for lower-spatial-frequency stimuli. This interaction appeared to be due, in part at least, to the effect that the higher-spatial-frequency moving pattern had on subjects' ability to organise optic flow into related motion about a single axis. For example, far-peripheral exposure to this high-spatial-frequency pattern caused many subjects to organise the optic flow into independent local regions of motion (a situation which clearly favoured the perception of object motion not self-motion). It is concluded that both high-spatial-frequency and low-spatial-frequency mechanisms are involved in the visual perception of self-motion--with their activities depending on the nature and eccentricity of the motion stimulation.     

The relationship between object files and conscious perception

Object files (OFs) are hypothesized mid-level representations which mediate our conscious perception of persisting objects-e.g. telling us 'which went where'. Despite the appeal of the OF framework, not previous research has directly explored whether OFs do indeed correspond to conscious percepts. Here we present at least one case wherein conscious percepts of 'which went where' in dynamic ambiguous displays diverge from the analogous correspondence computed by the OF system. Observers viewed a 'bouncing/streaming' display in which two identical objects moved such that they could have either bounced off or streamed past each other. We measured two dependent variables: (1) an explicit report of perceived bouncing or streaming; and (2) an implicit 'object-specific preview benefit' (OSPB), wherein a 'preview' of information on a specific object speeds the recognition of that information at a later point when it appears again on the same object (compared to when it reappears on a different object), beyond display-wide priming. When the displays were manipulated such that observers had a strong bias to perceive streaming (on over 95% of the trials), there was nevertheless a strong OSPB in the opposite direction-such that the object files appeared to have 'bounced' even though the percept 'streamed'. Given that OSPBs have been taken as a hallmark of the operation of object files, the five experiments reported here suggest that in at least some specialized (and perhaps ecologically invalid) cases, conscious percepts of 'which went where' in dynamic ambiguous displays can diverge from the mapping computed by the object-file system.