Matching the rate of concurrent tone bursts and light flashes as a function of flash surround luminance

Subjects adjusted the rate of a repeating toneburst to match that of a concurrently flashing light, or vice versa. Flashes were viewed in luminous or wholly dark surrounds. Matches usually departed from veridical rate matches, and always were affected in the same direction by changes in surround luminance. Matches were a function of whether subjects controlled the visual or auditory stimulus; also, subjects usually reported “driving” of flash rates by auditory rates when they controlled tone rate, but not when they controlled flash rate.     

Automatic visual bias of perceived auditory location

Studies of reactions to audiovisual spatial conflict (alias “ventriloquism”) are generally presented as informing on the processes of intermodal coordination. However, most of the literature has failed to isolate genuine perceptual effects from voluntary postperceptual adjustments. A new approach, based on psychophysical staircases, is applied to the case of the immediate visual bias of auditory localization. Subjects have to judge the apparent origin of stereophonically controlled sound bursts as left or right of a median reference line. Successive trials belong to one of two staircases, starting respectively at extreme left and right locations, and are moved progressively toward the median on the basis of the subjects’ responses. Response reversals occur for locations farther away from center when a central lamp is flashed in synchrony with the bursts than without flashes (Experiment 1), revealing an attraction of the sounds toward the flashes. The effect cannot originate in voluntary postperceptual decision, since the occurrence of response reversal implies that the subject is uncertain concerning the direction of the target sound. The attraction is contingent on sound-flash synchronization, for early response reversals did no longer occur when the inputs from the two modalities were desynchronized (Experiment 2). Taken together, the results show that the visual bias of auditory localization observed repeatedly in less controlled conditions is due partly at least to an automatic attraction of the apparent location of sound by spatially discordant but temporally correlated visual inputs.     

Visual facilitation of auditory localization in schoolchildren: A signal detection analysis

Warren (1970) has claimed that there are visual facilitation effects on auditory localization in adults but not in children. He suggests that a “visual map” organizes spatial information and that considerable experience of correlated auditory and visual events is necessary before normal spatial perception is developed. In the present experiment, children in Grades 1, 4, and 7 had to identify the position, right or left, of a single tone either blindfolded or with their eyes open. Analysis of the proportion of area under the ROC curve (obtained using reaction times) in the respective conditions showed that Ss were more sensitive to auditory position when vision was available. Reaction time was also generally faster in the light. I argue that the increase in sensitivity in the light represents updating of auditory position memory by voluntary eye movement. In the dark, eye movements are subject to involuntary and unperceived drift, which would introduce noise into the eye control mechanism and hence into auditory spatial memory.     

An examination of the relationship between visual capture and prism adaptation

The phenomena of prismatically induced “visual capture” and adaptation of the hand were compared. In Experiment 1, it was demonstrated that when the subject’s hand was transported for him by the experimenter (passive movement) immediately preceding the measure of visual capture, the magnitude of the immediate shift in felt limb position (visual capture) was enhanced relative to when the subject moved the hand himself (active movement). In Experiment 2, where the dependent measure was adaptation of the prism-exposed hand, the opposite effect was produced by the active/passive manipulation. It appears, then, that different processes operate to produce visual capture and adaptation. It was speculated that visual capture represents an immediate weighting of visual over proprioceptive input as a result of the greater precision of vision and/or the subject’s tendency to direct his attention more heavily to this modality. In contrast, prism adaptation is probably a recalibration of felt limb position in the direction of vision, induced by the presence of a registered discordance between visual and proprioceptive inputs.     

Need for Stimulation as a Factor in Preferences for Different Stimuli

Recent investigations have indicated that “reduction” on the kinesthetic aftereffects (KAE) procedure may indicate high levels of a putative need for stimulation (nStim), while “augmentation” on this procedure may indicate low levels of this motive. The present study supports the interpretation of the KAE task as a measure of nStim by indicating that “reducers” on this measure (a) react favorably to complex auditory and visual stimuli and (b) expose themselves particularly frequently to complex and intense stimulation in their ongoing social activities.     

Cross-modal bias and perceptual fusion with auditory-visual spatial discordance

Investigations of situations involving spatial discordance between auditory and visual data which can otherwise be attributed to a common origin have revealed two main phenomena:cross-modal bias andperceptual fusion (or ventriloquism). The focus of the present study is the relationship between these two. The question asked was whether bias occurred only with fusion, as is predicted by some accounts of reactions to discordance, among them those based on cuesubstitution. The approach consisted of having subjects, on each trial, both point to signals in one modality in the presence of conflicting signals in the other modality and produce same-different origin judgments. To avoid the confounding of immediate effects with cumulative adaptation, which was allowed in most previous studies, the direction and amplitude of discordance was varied randomly from trial to trial. Experiment 1, which was a pilot study, showed that both visual bias of auditory localization and auditory bias of visual localization can be observed under such conditions. Experiment 2, which addressed the main question, used a method which controls for the selection involved in separating fusion from no-fusion trials and showed that the attraction of auditory localization by conflicting visual inputs occurs even when fusion is not reported. This result is inconsistent with purely postperceptual views of cross-modal interactions. The question could not be answered for auditory bias of visual localization, which, although significant, was very small in Experiment 1 and fell below significance under the conditions of Experiment 2.     

Do locally defined feature discontinuities capture attention?

Results from previous visual search studies have suggested that abrupt onsets produce involuntary shifts of attention (i.e., attentional capture), but discontinuities in simple features such as color and brightness do not (Jonides & Yantis, 1988). In the present study we tested whether feature discontinuities (i.e., “singletons”) can produce attentional capture in a visual search task if defined “locally” or over a small spatial range. On each trial, a variable number of letters appeared, one of which differed from the others in color or intensity. The location of this singleton was uncorrelated with target location. Local discontinuities were created by embedding the letters in a dot texture. In Experiment 1, display size effects for singleton targets were not reduced with the addition of a background dot texture. Similar results were obtained in Experiment 2, regardless of variations in texture density. Experiment 3 confirmed that when targets are defined by a color or intensity singleton, they are detected preattentively, and that increasing texture density yields faster detection. We conclude that the spatial range over which feature discontinuities are defined may influence the guidance of spatial attention, but it has no influence on their ability to capture attention.     

Recognition memory and awareness: A large effect of study-test modalities on “know” responses following a highly perceptual orienting task

Abstract

Two experiments investigated the effects of varying the correspondence between presentation and test modes on states of conscious awareness in recognition memory. Experiment 1 used visual test items and showed that auditory and visual study modes differentially affected recollective experience and feelings of familiarity assessed by “remember” and “know” responses accompanying recognition: “know” responses were slightly enhanced following visual presentation, but this was largely offset by a similarly small increase in “remember” responses. Experiment 2 employed conditions designed to maximise any effect on “know” responses of the correspondence between presentation and test modes by using a highly perceptual orienting task at study. Under these conditions, there was a large mode correspondence effect on “know” responses. The results support the idea that “know” responses are particularly sensitive to perceptual factors.     

The aftereffects of ventriloquism: generalization across sound-frequencies

Exposure to synchronous but spatially discordant auditory and visual inputs produces, beyond immediate cross-modal biases, adaptive recalibrations of the respective localization processes that manifest themselves in aftereffects. Such recalibrations probably play an important role in maintaining the coherence of spatial representations across the various spatial senses. The present study is part of a research program focused on the way recalibrations generalize to stimulus values different from those used for adaptation. Considering the case of sound frequency, we recently found that, in contradiction with an earlier report, auditory aftereffects generalize nearly entirely across two octaves. In this new experiment, participants were adapted to an 18 degrees auditory-visual discordance with either 400 or 6400 Hz tones, and their subsequent sound localization was tested across this whole four-octave frequency range. Substantial aftereffects, decreasing significantly with increasing difference between test and adapter frequency, were obtained at all combinations of adapter and test frequency. Implications of these results concerning the functional site at which visual recalibration of auditory localization might take place are discussed.     

Not letting the left leg know what the right leg is doing: limb-specific locomotor adaptation to sensory-cue conflict

We investigated the phenomenon of limb-specific locomotor adaptation in order to adjudicate between sensory-cue-conflict theory and motor-adaptation theory. The results were consistent with cue-conflict theory in demonstrating that two different leg-specific hopping aftereffects are modulated by the presence of conflicting estimates of self-motion from visual and nonvisual sources. Experiment 1 shows that leg-specific increases in forward drift during attempts to hop in place on one leg while blindfolded vary according to the relationship between visual information and motor activity during an adaptation to outdoor forward hopping. Experiment 2 shows that leg-specific changes in performance on a blindfolded hopping-to-target task are similarly modulated by the presence of cue conflict during adaptation to hopping on a treadmill. Experiment 3 shows that leg-specific aftereffects from hopping additionally produce inadvertent turning during running in place while blindfolded. The results of these experiments suggest that these leg-specific locomotor aftereffects are produced by sensory-cue conflict rather than simple motor adaptation.     

Behavioral evidence for task-dependent “what” versus “where” processing within and across modalities

Task-dependent information processing for the purpose of recognition or spatial perception is considered a principle common to all the main sensory modalities. Using a dual-task interference paradigm, we investigated the behavioral effects of independent information processing for shape identification and localization of object features within and across vision and touch. In Experiment 1, we established that color and texture processing (i.e., a “what” task) interfered with both visual and haptic shape-matching tasks and that mirror image and rotation matching (i.e., a “where” task) interfered with a feature-location-matching task in both modalities. In contrast, interference was reduced when a “where” interference task was embedded in a “what” primary task and vice versa. In Experiment 2, we replicated this finding within each modality, using the same interference and primary tasks throughout. In Experiment 3, the interference tasks were always conducted in a modality other than the primary task modality. Here, we found that resources for identification and spatial localization are independent of modality. Our findings further suggest that multisensory resources for shape recognition also involve resources for spatial localization. These results extend recent neuropsychological and neuroimaging findings and have important implications for our understanding of high-level information processing across the human sensory systems.     

Cross-modal links in exogenous covert spatial orienting between touch, audition, and vision

Three experiments investigated cross-modal links between touch, audition, and vision in the control of covert exogenous orienting. In the first two experiments, participants made speeded discrimination responses (continuous vs. pulsed) for tactile targets presented randomly to the index finger of either hand. Targets were preceded at a variable stimulus onset asynchrony (150,200, or 300 msec) by a spatially uninformative cue that was either auditory (Experiment 1) or visual (Experiment 2) on the same or opposite side as the tactile target. Tactile discriminations were more rapid and accurate when cue and target occurred on the same side, revealing cross-modal covert orienting. In Experiment 3, spatially uninformative tactile cues were presented prior to randomly intermingled auditory and visual targets requiring an elevation discrimination response (up vs. down). Responses were significantly faster for targets in both modalities when presented ipsilateral to the tactile cue. These findings demonstrate that the peripheral presentation of spatially uninforrnative auditory and visual cues produces cross-modal orienting that affects touch, and that tactile cues can also produce cross-modal covert orienting that affects audition and vision.     

Interocular transfer of orientation-contingent color aftereffects with external and internal adaptation

The issue of whether McCollough effects transfer interocularly was examined in two experiments. In Experiment 1, as in previous experiments, no interocular transfer of McCollough effects was obtained when subjects adapted monocularly to externally present patterns with their unused eyes fully occluded. However, when subjects adapted monocularly (with the unused eye fully occluded) to visual images of those patterns superimposed on physically present chromatic backgrounds, McCollough effects transferred interocularly. In Experiment 2, subjects adapted to either physically present patterns or to images of those patterns (as in Experiment 1), but the unused eye was exposed to unpatterned diffuse white light. In contrast to Experiment 1, the externally adapted subjects showed interocular transfer of McCollough effects. In Experiment 2, the magnitude of the interocular transfer effects produced by imagery was significantly larger than the magnitude of the effects produced by external adaptation, but the magnitude of the imagery-induced aftereffects did not differ from Experiment 1 to Experiment 2. These results extend earlier findings by Kunen and May (1980) and show that McCollough effects can be produced through adaptation to imagery, even though the direction of the imagery-induced aftereffects indicates adaptation to higher spatial frequencies whereas externally derived aftereffects indicate adaptation to lower spatial frequencies. It is concluded that failure of previous studies to obtain interocular transfer of McCollough effects may have resulted from complete occlusion of the contralateral eye. These data point up some interesting similarities and some important differences between imagery and actual vision. The implications for analogical models of visual imagery are discussed.     

Perceptual load influences auditory space perception in the ventriloquist aftereffect

A period of exposure to trains of simultaneous but spatially offset auditory and visual stimuli can induce a temporary shift in the perception of sound location. This phenomenon, known as the ‘ventriloquist aftereffect’, reflects a realignment of auditory and visual spatial representations such that they approach perceptual alignment despite their physical spatial discordance. Such dynamic changes to sensory representations are likely to underlie the brain’s ability to accommodate inter-sensory discordance produced by sensory errors (particularly in sound localization) and variability in sensory transduction. It is currently unknown, however, whether these plastic changes induced by adaptation to spatially disparate inputs occurs automatically or whether they are dependent on selectively attending to the visual or auditory stimuli. Here, we demonstrate that robust auditory spatial aftereffects can be induced even in the presence of a competing visual stimulus. Importantly, we found that when attention is directed to the competing stimuli, the pattern of aftereffects is altered. These results indicate that attention can modulate the ventriloquist aftereffect.     

The cognitive locus of distraction by acoustic novelty in the cross-modal oddball task

Unexpected stimuli are often able to distract us away from a task at hand. The present study seeks to explore some of the mechanisms underpinning this phenomenon. Studies of involuntary attention capture using the oddball task have repeatedly shown that infrequent auditory changes in a series of otherwise repeating sounds trigger an automatic response to the novel or deviant stimulus. This attention capture has been shown to disrupt participants' behavioral performance in a primary task, even when distractors and targets are asynchronous and presented in distinct sensory modalities. This distraction effect is generally considered as a by-product of the capture of attention by the novel or deviant stimulus, but the exact cognitive locus of this effect and the interplay between attention capture and target processing has remained relatively ignored. The present study reports three behavioral experiments using a cross-modal oddball task to examine whether the distraction triggered by auditory novelty affects the processing of the target stimuli. Our results showed that variations in the demands placed on the visual analysis (Experiment 1) or categorical processing of the target (Experiment 2) did not impact on distraction. Instead, the cancellation of distraction by the presentation of an irrelevant visual stimulus presented immediately before the visual target (Experiment 3) suggested that distraction originated in the shifts of attention occurring between attention capture and the onset of the target processing. Possible accounts of these shifts are discussed.     

Interactions Between Exogenous Auditory and Visual Spatial Attention

Six experiments were carried out to investigate the issue of cross-modality between exogenous auditory and visual spatial attention employing Posner's cueing paradigm in detection, localization, and discrimination tasks. Results indicated cueing in detection tasks with visual or auditory cues and visual targets but not with auditory targets (Experiment 1). In the localization tasks, cueing was found with both visual and auditory targets. Inhibition of return was apparent only in the within-modality conditions (Experiment 2). This suggests that it is important whether the attention system is activated directly (within a modality) or indirectly (between modalities). Increasing the cue validity from 50% to 80% influenced performance only in the localization task (Experiment 4). These findings are interpreted as being indicative for modality-specific but interacting attention mechanisms. The results of Experiments 5 and 6 (up/down discrimination tasks) also show cross-modal cueing but not with visual cues and auditory targets. Furthermore, there was no inhibition of return in any condition. This suggests that some cueing effects might be task dependent.     

Auditory adaptation to gradual rise or fall in intensity of a tone

The auditory system adapts to gradual changes of intensity. After listening to a gradually loudening (or softening) tone, a steady test tone appeared to be growing gradually softer (or louder). “Tuning curves” for these aftereffects were measured by adapting to one audio frequency and testing at another. Softening tones produced larger aftereffects, which were more sharply tuned for audiofrequency, than did loudening tones.     

跨感觉通道任务中刺激驱动注意捕获的不对称性

通过两个实验探讨了来自不同感觉通道或不同空间方位的奇异刺激在视听搜索范式中的注意捕获。实验一使用定位任务探讨刺激驱动注意捕获中跨通道效应且及其不对称性, 结果发现, 当靶子在听觉通道时, 只有来自听觉同侧空间方位的奇异刺激成功的捕获了注意; 而来自视觉同侧空间和异侧空间的奇异刺激并没有捕获注意。实验二使用检测任务进一步探讨刺激驱动的注意捕获中的跨通道效应且具有不对称性, 结果发现, 来自同侧和异侧的听觉奇异刺激均捕获了注意, 但是视觉奇异刺激却不能捕获注意。研究表明, 视听搜索范式中的奇异刺激的注意捕获具有不对称性, 且存在着一种超感觉通道的注意资源。     

The ventriloquist effect does not depend on the direction of deliberate visual attention

It is well known that discrepancies in the location of synchronized auditory and visual events can lead to mislocalizations of the auditory source, so-called ventriloquism. In two experiments, we tested whether such cross-modal influences on auditory localization depend on deliberate visual attention to the biasing visual event. In Experiment 1, subjects pointed to the apparent source of sounds in the presence or absence of a synchronous peripheral flash. They also monitored for target visual events, either at the location of the peripheral flash or in a central location. Auditory localization was attracted toward the synchronous peripheral flash, but this was unaffected by where deliberate visual attention was directed in the monitoring task. In Experiment 2, bilateral flashes were presented in synchrony with each sound, to provide competing visual attractors. When these visual events were equally salient on the two sides, auditory localization was unaffected by which side subjects monitored for visual targets. When one flash was larger than the other, auditory localization was slightly but reliably attracted toward it, but again regardless of where visual monitoring was required. We conclude that ventriloquism largely reflects automatic sensory interactions, with little or no role for deliberate spatial attention.     

Establishing equivalence–equivalence analogical relations via tact and listener training

We investigated the role of verbal behavior on the emergence of analogy-type responding as measured via equivalence–equivalence relations. In Experiment 1, 8 college students learned to label arbitrary stimuli as, “vek,” “zog,” and “paf”, and in Experiment 2, 8 additional participants learned to select these stimuli when hearing their names in an auditory–visual matching-to-sample (MTS) task. Experimenters tested for the emergence of relational tacts (i.e., “same” and “different”) and equivalence–equivalence relations (analogy tests) via visual–visual MTS. Half of the participants were exposed to a think-aloud procedure. Even though they all passed analogy tests while tacting stimuli relationally, only participants exposed to tact training (Experiment 1) did so without the need for remediation. The results of these experiments confirm that individual discriminative and relational control of stimuli established through verbal behavior training is sufficient to produce equivalence–equivalence analogical responding, advancing the analysis of complex cognitive (problem-solving) phenomena.