Audiovisual bounce-inducing effect: When sound congruence affects grouping in vision

Two disks moving from opposite points in space, overlapping, and stopping at one another’s starting point can be seen as either bouncing off one another or streaming through one another. With silent displays, observers report streaming, whereas, if a sound is played when the disks are in the overlap region, observers report bouncing. The change in perception is thought to be modulated by a lack of attention that inhibits the integration of the motion signal when disks overlap and by the sound that increases the congruence of the display, in comparison with a real elastic bounce. Here, we accompanied the disks’ motion with either a bounce-congruent sound (a billiard ball) or with bounce-incongruent sounds (a water drop, a firework). When the sound was switched on 200 msec before the disks’ overlap, (1) all the audiovisual displays induced more bounce responses than did the silent display, but (2) the bounce-congruent sound induced more bounce responses than did the bounceincongruent sounds. However, when the sound was switched on at the disks’ overlap, only the first result was observed. These results highlight both the role of attention and that of sound congruence.     

Sound enhances visual perception: cross-modal effects of auditory organization on vision

Six experiments demonstrated cross-modal influences from the auditory modality on the visual modality at an early level of perceptual organization. Participants had to detect a visual target in a rapidly changing sequence of visual distractors. A high tone embedded in a sequence of low tones improved detection of a synchronously presented visual target (Experiment 1), but the effect disappeared when the high tone was presented before the target (Experiment 2). Rhythmically based or order-based anticipation was unlikely to account for the effect because the improvement was unaffected by whether there was jitter (Experiment 3) or a random number of distractors between successive targets (Experiment 4). The facilitatory effect was greatly reduced when the tone was less abrupt and part of a melody (Experiments 5 and 6). These results show that perceptual organization in the auditory modality can have an effect on perceptibility in the visual modality.     

Motion detection and motion verbs: language affects low-level visual perception

Recent theories propose that semantic representation and sensorimotor processing have a common substrate via simulation. We tested the prediction that comprehension interacts with perception, using a standard psychophysics methodology. While passively listening to verbs that referred to upward or downward motion, and to control verbs that did not refer to motion, 20 subjects performed a motion-detection task, indicating whether or not they saw motion in visual stimuli containing threshold levels of coherent vertical motion. A signal detection analysis revealed that when verbs were directionally incongruent with the motion signal, perceptual sensitivity was impaired. Word comprehension also affected decision criteria and reaction times, but in different ways. The results are discussed with reference to existing explanations of embodied processing and the potential of psychophysical methods for assessing interactions between language and perception.     

Reading action word affects the visual perception of biological motion

In the present study, we investigate whether reading an action-word can influence subsequent visual perception of biological motion. The participant's task was to perceptually judge whether a human action identifiable in the biological motion of a point-light display embedded in a high density mask was present or not in the visual sequence, which lasted for 633 ms on average. Prior to the judgement task, participants were exposed to an abstract verb or an action verb for 500 ms, which was related to the human action according to a congruent or incongruent semantic relation. Data analysis showed that correct judgements were not affected by action verbs, whereas a facilitation effect on response time (49 ms on average) was observed when a congruent action verb primed the judgement of biological movements. In relation with the existing literature, this finding suggests that the perception, the planning and the linguistic coding of motor action are subtended by common motor representations.     

Audiovisual influences on the perception of visual apparent motion: exploring the effect of a single sound

Previous research has shown that irrelevant sounds can facilitate the perception of visual apparent motion. Here the effectiveness of a single sound to facilitate motion perception was investigated in three experiments. Observers were presented with two discrete lights temporally separated by stimulus onset asynchronies from 0 to 350 ms. After each trial, observers classified their impression of the stimuli using a categorisation system. A short sound presented temporally (and spatially) midway between the lights facilitated the impression of motion relative to baseline (lights without sound), whereas a sound presented either before the first or after the second light or simultaneously with the lights did not affect motion impression. The facilitation effect also occurred with sound presented far from the visual display, as well as with continuous-sound that was started with the first light and terminated with the second light. No facilitation of visual motion perception occurred if the sound was part of a tone sequence that allowed for intramodal perceptual grouping of the auditory stimuli prior to the critical audiovisual stimuli. Taken together, the findings are consistent with a low-level audiovisual integration approach in which the perceptual system merges temporally proximate sound and light stimuli, thereby provoking the impression of a single multimodal moving object.     

Relativistic effects in visual perception of real and apparent motion

Timing in the visual field is regarded as an additive conjoint measurement structure, psychophysical extension of stimulus path as an extensive measurement structure. These two sets of postulates lead to the derivation of an essential maximum for velocity perception. Apparent phi motion perception under strictly stationary stimulus conditions is described as relative motion of the first stimulus' psychophysical mapping with respect to a moving perceptual subsystem. The essential maximum of velocity modifies the relative motion postulate: relativistic dilatation of seen length of path is predicted. Testable properties of the model, comparison with experimental data from "real" motion and apparent phi motion perception are discussed.     

Perception of auditory motion affects language processing

Previous reports have demonstrated that the comprehension of sentences describing motion in a particular direction (toward, away, up, or down) is affected by concurrently viewing a stimulus that depicts motion in the same or opposite direction. We report 3 experiments that extend our understanding of the relation between perception and language processing in 2 ways. First, whereas most previous studies of the relation between perception and language processing have focused on visual perception, our data show that sentence processing can be affected by the concurrent processing of auditory stimuli. Second, it is shown that the relation between the processing of auditory stimuli and the processing of sentences depends on whether the sentences are presented in the auditory or visual modality.     

Scent and sound of vision: expressing scent or sound as visual forms

This article describes three experiments on the possibility of expressing scent or sound into visual forms made by design engineering students. The hypothesis tested was that people are able to pick up patterns in the energy flow that the students transposed from one perceptual sense to another. In Exp. 1 subjects were given different scents and were asked to choose a sculpture designed according these scents. In Exp. 2 subjects were given different musical pieces and asked to match them with portable cassette players designed according to this music. Exp. 3 was identical to Exp. 2 but different music selections, similar to the ones in Exp. 2, were used. In all three experiments subjects were indeed able to perform the tasks above chance level. Results are discussed within the framework of the theory of direct perception of Gibson.     

The role of vision in auditory distance perception

In humans, multisensory interaction is an important strategy for improving the detection of stimuli of different nature and reducing the variability of response. It is known that the presence of visual information affects the auditory perception in the horizontal plane (azimuth), but there are few researches that study the influence of vision in the auditory distance perception. In general, the data obtained from these studies are contradictory and do not completely define the way in which visual cues affect the apparent distance of a sound source. Here psychophysical experiments on auditory distance perception in humans are performed, including and excluding visual cues. The results show that the apparent distance from the source is affected by the presence of visual information and that subjects can store in their memory a representation of the environment that later improves the perception of distance.     

Attentional modulation of visual motion perception

How is the perception and processing of visual motion affected by attention? This review examines recent research in cognition, perception and neurophysiology that explores how ongoing behavioural tasks (and the attentional states they impose) modulate the processing of visual motion. Although traditional views hold that motion is processed in an obligatory, 'pre-attentive' manner, evidence for processing in a task-independent manner is scant. Recent studies of human perception that have measured motion priming, motion aftereffects, uncertainty effects, and motion-interaction effects indicate instead that even simple aspects of motion processing may be substantially affected by whether motion information in a task is used or ignored by the perceiver. Single-unit studies in brain areas sensitive to visual motion in monkeys, and functional imaging studies on humans, also indicate that task and attentional state affect activity levels in brain regions thought to be important in motion perception. This review brings together these converging findings of attentional modulation of motion perception and considers them in light of object-oriented theories of attention.     

Postcoincidence trajectory duration affects motion event perception

In a two-dimensional display, identical visual targets moving toward and across each other with equal, constant speed can be perceived either to reverse their motion directions at the coincidence point (bouncing percept) or to stream through one another (streaming percept). Although there is a strong tendency to perceive the streaming percept, various factors have been reported to induce the bouncing percept, such as a sound or a visual flash at the moment of the visual target coincidence. By changing duration of the postcoincidence trajectory (PCT), we investigated how long it would take for such bounce-inducing factors to be maximally effective after the visual coincidence. With bounce-inducing factors, the percentage of the bouncing percept did not reach its maximal level immediately after the coincidence but increased as a function of PCT duration up to 150-200 msec. The results clearly reject the possibility of the cognitive-bias hypothesis about the bounce-inducing effect and suggest rather that the bounce-inducing factors have to interact with the PCT for some period after the coincidence to be maximally effective.     

Phasic auditory alerting improves visual conscious perception

Attention is often conceived as a gateway to consciousness (Posner, 1994). Although endogenous spatial attention may be independent of conscious perception (CP) (Koch Tsuchiya, 2007), exogenous spatial orienting seems instead to be an important modulator of CP ( [Chica et al., 2010] and Chica, Lasaponara, et al., 2011). Here, we investigate the role of auditory alerting in CP in normal observers. We used a behavioral task in which phasic alerting tones were presented either at unpredictable or at predictable time intervals prior to the occurrence of a near-threshold visual target. We find, for the first time in neurologically intact observers, that phasic alertness increases CP, both objectively and subjectively. This result is consistent with evidence showing that phasic alerting can ameliorate the spatial bias exhibited by visual neglect patients (Robertson, Mattingley, Rorden, & Driver, 1998). The alerting network may increase the activity of fronto-parietal networks involved in top-down amplification required to bring a stimulus into consciousness (Dehaene, Changeux, Naccache, Sackur, & Sergent, 2006).     

Dividing attention between auditory and visual perception

To what extent is simultaneous visual and auditory perception subject to capacity limitations and attentional control? Two experiments addressed this question by asking observers to recognize test tones and test letters under selective and divided attention. In Experiment 1, both stimuli occurred on each trial, but subjects were cued in advance to process just one or both of the stimuli. In Experiment 2, subjects processed one stimulus and then the other or processed both stimuli simultaneously. Processing time was controlled using a backward recognition masking task. A significant, but small, attention effect was found in both experiments. The present positive results weaken the interpretation that previous attentional effects were due to the particular duration judgment task that was employed. The answer to the question addressed by the experiments appears to be that the degree of capacity limitations and attentional control during visual and auditory perception is small but significant.     

Unimodal and crossmodal effects of endogenous attention to visual and auditory motion

Three experiments were conducted examining unimodal and crossmodal effects of attention to motion. Horizontally moving sounds and dot patterns were presented and participants’ task was to discriminate their motion speed or whether they were presented with a brief gap. In Experiments 1 and 2, stimuli of one modality and of one direction were presented with a higher probability ( p = .7) than other stimuli. Sounds and dot patterns moving in the expected direction were discriminated faster than stimuli moving in the unexpected direction. In Experiment 3, participants had to respond only to stimuli moving in one direction within the primary modality, but to all stimuli regardless of their direction within the rarer secondary modality. Stimuli of the secondary modality moving in the attended direction were discriminated faster than were oppositely moving stimuli. Results suggest that attending to the direction of motion affects perception within vision and audition, but also across modalities.     

Top-down control of visual perception: attention in natural vision

Top-down perceptual influences can bias (or pre-empt) perception. In natural scenes, the receptive fields of neurons in the inferior temporal visual cortex (IT) shrink to become close to the size of objects. This facilitates the read-out of information from the ventral visual system, because the information is primarily about the object at the fovea. Top-down attentional influences are much less evident in natural scenes than when objects are shown against blank backgrounds, though are still present. It is suggested that the reduced receptive-field size in natural scenes, and the effects of top-down attention contribute to change blindness. The receptive fields of IT neurons in complex scenes, though including the fovea, are frequently asymmetric around the fovea, and it is proposed that this is the solution the IT uses to represent multiple objects and their relative spatial positions in a scene. Networks that implement probabilistic decision-making are described, and it is suggested that, when in perceptual systems they take decisions (or 'test hypotheses'), they influence lower-level networks to bias visual perception. Finally, it is shown that similar processes extend to systems involved in the processing of emotion-provoking sensory stimuli, in that word-level cognitive states provide top-down biasing that reaches as far down as the orbitofrontal cortex, where, at the first stage of affective representations, olfactory, taste, flavour, and touch processing is biased (or pre-empted) in humans.     

An experiment on the theory of visual motion perception

Two classes of theories of motion perception were studied: correlation and gradient models (in the sense of D. Marr). Random-dot kinematograms with a shifted square were presented to subjects, and the 80% threshold for detection of correct direction of movement was determined. Correlation models predict scale invariance, that is, a constant shift measured in units of texture elements of the translated pattern in spite of a geometric magnification. This was refuted for all subjects. The increase of the recognizable translation with the area of the translated form was verified except for the patterns with the largest texture elements (8 minutes of arc). This prediction, however, is not very specific for competing theories. Gradient models are not yet fully specified, and they contain some free parameters. They cannot be tested strictly, but there exist reasonable numerical parameter values by which our data can be satisfactorily explained.     

Seeing facial motion affects auditory processing in noise

Speech perception, especially in noise, may be maximized if the perceiver observes the naturally occurring visual-plus-auditory cues inherent in the production of spoken language. Evidence is conflicting, however, about which aspects of visual information mediate enhanced speech perception in noise. For this reason, we investigated the relative contributions of audibility and the type of visual cue in three experiments in young adults with normal hearing and vision. Relative to static visual cues, access to the talker??s phonetic gestures in speech production, especially in noise, was associated with (a) faster response times and sensitivity for speech understanding in noise, and (b) shorter latencies and reduced amplitudes of auditory N1 event-related potentials. Dynamic chewing facial motion also decreased the N1 latency, but only meaningful linguistic motions reduced the N1 amplitude. The hypothesis that auditory?Cvisual facilitation is distinct to properties of natural, dynamic speech gestures was partially supported.     

The effects of global grouping laws on surface lightness perception

Previous studies of lightness perception have shown that local surface grouping laws such as proximity and T junction were powerful determinants of target surface lightness. Recent lightness theories also emphasize the importance of local grouping of surfaces. In this study, we further examined the effects of three global grouping laws--symmetry, repetition, and alternation--on lightness perception. Local surface grouping laws such as proximity and good continuation were controlled across all of our stimulus displays. Participants' lightness perception consistently depended on a given surface's belongingness as determined by these laws--that is, global grouping laws affected a target surface's lightness perception. Our results indicate that global grouping laws determine a target surface's lightness when local surface grouping does not produce any distinct surface belongingness. Implications of our basic results are discussed in terms of a recent lightness theory.     

Cortical dynamics of visual motion perception: short-range and long-range apparent motion.

This article describes further evidence for a new neural network theory of biological motion perception. The theory clarifies why parallel streams V1----V2, V1----MT, and V1----V2----MT exist for static form and motion form processing among the areas V1, V2, and MT of visual cortex. The theory suggests that the static form system (Static BCS) generates emergent boundary segmentations whose outputs are insensitive to direction-of-contrast and to direction-of-motion, whereas the motion form system (Motion BCS) generates emergent boundary segmentations whose outputs are insensitive to direction-of-contrast but sensitive to direction-of-motion. The theory is used to explain classical and recent data about short-range and long-range apparent motion percepts that have not yet been explained by alternative models. These data include beta motion, split motion, gamma motion and reverse-contrast gamma motion, delta motion, and visual inertia. Also included are the transition from group motion to element motion in response to a Ternus display as the interstimulus interval (ISI) decreases; group motion in response to a reverse-contrast Ternus display even at short ISIs; speed-up of motion velocity as interflash distance increases or flash duration decreases; dependence of the transition from element motion to group motion on stimulus duration and size, various classical dependencies between flash duration, spatial separation, ISI, and motion threshold known as Korte's laws; dependence of motion strength on stimulus orientation and spatial frequency; short-range and long-range form-color interactions; and binocular interactions of flashes to different eyes.