Teaching children with autism spectrum disorder to tact auditory stimuli

Studies on teaching tacts to individuals with autism spectrum disorder (ASD) have primarily focused on visual stimuli, despite published clinical recommendations to teach tacts of stimuli in other sensory domains as well. In the current study, two children with ASD were taught to tact auditory stimuli under two stimulus‐presentation arrangements: isolated (auditory stimuli presented without visual cues) and compound (auditory stimuli presented with visual cues). Results indicate that compound stimulus presentation was a more effective teaching procedure, but that it interfered with prior object‐name tacts. A modified compound arrangement in which object‐name tact trials were interspersed with auditory‐stimulus trials mitigated this interference.     

Modulation of auditory spatial attention by visual emotional cues: differential effects of attentional engagement and disengagement for pleasant and unpleasant cues

Previous research has demonstrated that threatening, compared to neutral pictures, can bias attention towards non-emotional auditory targets. Here we investigated which subcomponents of attention contributed to the influence of emotional visual stimuli on auditory spatial attention. Participants indicated the location of an auditory target, after brief (250 ms) presentation of a spatially non-predictive peripheral visual cue. Responses to targets were faster at the location of the preceding visual cue, compared to at the opposite location (cue validity effect). The cue validity effect was larger for targets following pleasant and unpleasant cues compared to neutral cues, for right-sided targets. For unpleasant cues, the crossmodal cue validity effect was driven by delayed attentional disengagement, and for pleasant cues, it was driven by enhanced engagement. We conclude that both pleasant and unpleasant visual cues influence the distribution of attention across modalities and that the associated attentional mechanisms depend on the valence of the visual cue.     

Multisensory-mediated auditory localization

Multisensory integration is a powerful mechanism for maximizing sensitivity to sensory events. We examined its effects on auditory localization in healthy human subjects. The specific objective was to test whether the relative intensity and location of a seemingly irrelevant visual stimulus would influence auditory localization in accordance with the inverse effectiveness and spatial rules of multisensory integration that have been developed from neurophysiological studies with animals [Stein and Meredith, 1993 The Merging of the Senses (Cambridge, MA: MIT Press)]. Subjects were asked to localize a sound in one condition in which a neutral visual stimulus was either above threshold (supra-threshold) or at threshold. In both cases the spatial disparity of the visual and auditory stimuli was systematically varied. The results reveal that stimulus salience is a critical factor in determining the effect of a neutral visual cue on auditory localization. Visual bias and, hence, perceptual translocation of the auditory stimulus appeared when the visual stimulus was supra-threshold, regardless of its location. However, this was not the case when the visual stimulus was at threshold. In this case, the influence of the visual cue was apparent only when the two cues were spatially coincident and resulted in an enhancement of stimulus localization. These data suggest that the brain uses multiple strategies to integrate multisensory information.     

Localisation of auditory targets during optokinetic nystagmus

Previous studies have shown that the perceived location of visual stimuli briefly flashed during smooth pursuit, saccades, or optokinetic nystagmus (OKN) is not veridical. We investigated whether these mislocalisations can also be observed for brief auditory stimuli presented during OKN. Experiments were carried out in a lightproof sound-attenuated chamber. Participants performed eye movements elicited by visual stimuli. An auditory target (white noise) was presented for 5 ms. Our data clearly indicate that auditory targets are mislocalised during reflexive eye movements. OKN induces a shift of perceived location in the direction of the slow eye movement and is modulated in the temporal vicinity of the fast phase. The mislocalisation is stronger for look- as compared to stare-nystagmus. The size and temporal pattern of the observed mislocalisation are different from that found for visual targets. This suggests that different neural mechanisms are at play to integrate oculomotor signals and information on the spatial location of visual as well as auditory stimuli.     

Switching in the cocktail party: exploring intentional control of auditory selective attention

Using a novel variant of dichotic selective listening, we examined the control of auditory selective attention. In our task, subjects had to respond selectively to one of two simultaneously presented auditory stimuli (number words), always spoken by a female and a male speaker, by performing a numerical size categorization. The gender of the task-relevant speaker could change, as indicated by a visual cue prior to auditory stimulus onset. Three experiments show clear performance costs with instructed attention switches. Experiment 2 varied the cuing interval to examine advance preparation for an attention switch. Experiment 3 additionally isolated auditory switch costs from visual cue priming by using two cues for each gender, so that gender repetition could be indicated by a changed cue. Experiment 2 showed that switch costs decreased with prolonged cuing intervals, but Experiment 3 revealed that preparation did not affect auditory switch costs but only visual cue priming. Moreover, incongruent numerical categories in competing auditory stimuli produced interference and substantially increased error rates, suggesting continued processing of task-relevant information that often leads to responding to the incorrect auditory source. Together, the data show clear limitations in advance preparation of auditory attention switches and suggest a considerable degree of inertia in intentional control of auditory selection criteria.     

Capuchin monkeys (Cebus apella) use positive, but not negative, auditory cues to infer food location

Nonhuman primates appear to capitalize more effectively on visual cues than corresponding auditory versions. For example, studies of inferential reasoning have shown that monkeys and apes readily respond to seeing that food is present (“positive” cuing) or absent (“negative” cuing). Performance is markedly less effective with auditory cues, with many subjects failing to use this input. Extending recent work, we tested eight captive tufted capuchins (Cebus apella) in locating food using positive and negative cues in visual and auditory domains. The monkeys chose between two opaque cups to receive food contained in one of them. Cup contents were either shown or shaken, providing location cues from both cups, positive cues only from the baited cup, or negative cues from the empty cup. As in previous work, subjects readily used both positive and negative visual cues to secure reward. However, auditory outcomes were both similar to and different from those of earlier studies. Specifically, all subjects came to exploit positive auditory cues, but none responded to negative versions. The animals were also clearly different in visual versus auditory performance. Results indicate that a significant proportion of capuchins may be able to use positive auditory cues, with experience and learning likely playing a critical role. These findings raise the possibility that experience may be significant in visually based performance in this task as well, and highlight that coming to grips with evident differences between visual versus auditory processing may be important for understanding primate cognition more generally.     

The role of vision in auditory distance perception

In humans, multisensory interaction is an important strategy for improving the detection of stimuli of different nature and reducing the variability of response. It is known that the presence of visual information affects the auditory perception in the horizontal plane (azimuth), but there are few researches that study the influence of vision in the auditory distance perception. In general, the data obtained from these studies are contradictory and do not completely define the way in which visual cues affect the apparent distance of a sound source. Here psychophysical experiments on auditory distance perception in humans are performed, including and excluding visual cues. The results show that the apparent distance from the source is affected by the presence of visual information and that subjects can store in their memory a representation of the environment that later improves the perception of distance.     

A comparison of visual and auditory representational momentum in spatial tasks

Similarities have been observed in the localization of the final position of moving visual and moving auditory stimuli: Perceived endpoints that are judged to be farther in the direction of motion in both modalities likely reflect extrapolation of the trajectory, mediated by predictive mechanisms at higher cognitive levels. However, actual comparisons of the magnitudes of displacement between visual tasks and auditory tasks using the same experimental setup are rare. As such, the purpose of the present free-field study was to investigate the influences of the spatial location of motion offset, stimulus velocity, and motion direction on the localization of the final positions of moving auditory stimuli (Experiment 1 and 2) and moving visual stimuli (Experiment 3). To assess whether auditory performance is affected by dynamically changing binaural cues that are used for the localization of moving auditory stimuli (interaural time differences for low-frequency sounds and interaural intensity differences for high-frequency sounds), two distinct noise bands were employed in Experiments 1 and 2. In all three experiments, less precise encoding of spatial coordinates in paralateral space resulted in larger forward displacements, but this effect was drowned out by the underestimation of target eccentricity in the extreme periphery. Furthermore, our results revealed clear differences between visual and auditory tasks. Displacements in the visual task were dependent on velocity and the spatial location of the final position, but an additional influence of motion direction was observed in the auditory tasks. Together, these findings indicate that the modality-specific processing of motion parameters affects the extrapolation of the trajectory.     

Perception of the duration of auditory and visual stimuli in children and adults

This experiment investigated the effect of modality on temporal discrimination in children aged 5 and 8 years and adults using a bisection task with visual and auditory stimuli ranging from 200 to 800 ms. In the first session, participants were required to compare stimulus durations with standard durations presented in the same modality (within-modality session), and in the second session in different modalities (cross-modal session). Psychophysical functions were orderly in all age groups, with the proportion of long responses (judgement that a duration was more similar to the long than to the short standard) increasing with the stimulus duration, although functions were flatter in the 5-year-olds than in the 8-year-olds and adults. Auditory stimuli were judged to be longer than visual stimuli in all age groups. The statistical results and a theoretical model suggested that this modality effect was due to differences in the pacemaker speed of the internal clock. The 5-year-olds also judged visual stimuli as more variable than auditory ones, indicating that their temporal sensitivity was lower in the visual than in the auditory modality.     

Estimating the relative weights of visual and auditory tau versus heuristic-based cues for time-to-contact judgments in realistic,familiar scenes by older and younger adults

Estimating time to contact (TTC) involves multiple sensory systems, including vision and audition. Previous findings suggested that the ratio of an object’s instantaneous optical size/sound intensity to its instantaneous rate of change in optical size/sound intensity (τ) drives TTC judgments. Other evidence has shown that heuristic-based cues are used, including final optical size or final sound pressure level. Most previous studies have used decontextualized and unfamiliar stimuli (e.g., geometric shapes on a blank background). Here we evaluated TTC estimates by using a traffic scene with an approaching vehicle to evaluate the weights of visual and auditory TTC cues under more realistic conditions. Younger (18–39 years) and older (65+ years) participants made TTC estimates in three sensory conditions: visual-only, auditory-only, and audio–visual. Stimuli were presented within an immersive virtual-reality environment, and cue weights were calculated for both visual cues (e.g., visual τ, final optical size) and auditory cues (e.g., auditory τ, final sound pressure level). The results demonstrated the use of visual τ as well as heuristic cues in the visual-only condition. TTC estimates in the auditory-only condition, however, were primarily based on an auditory heuristic cue (final sound pressure level), rather than on auditory τ. In the audio–visual condition, the visual cues dominated overall, with the highest weight being assigned to visual τ by younger adults, and a more equal weighting of visual τ and heuristic cues in older adults. Overall, better characterizing the effects of combined sensory inputs, stimulus characteristics, and age on the cues used to estimate TTC will provide important insights into how these factors may affect everyday behavior.     

Transfer of noncorresponding spatial associations to the auditory Simon task

Four experiments examined transfer of noncorresponding spatial stimulus-response associations to an auditory Simon task for which stimulus location was irrelevant. Experiment 1 established that, for a horizontal auditory Simon task, transfer of spatial associations occurs after 300 trials of practice with an incompatible mapping of auditory stimuli to keypress responses. Experiments 2-4 examined transfer effects within the auditory modality when the stimuli and responses were varied along vertical and horizontal dimensions. Transfer occurred when the stimuli and responses were arrayed along the same dimension in practice and transfer but not when they were arrayed along orthogonal dimensions. These findings indicate that prior task-defined associations have less influence on the auditory Simon effect than on the visual Simon effect, possibly because of the stronger tendency for an auditory stimulus to activate its corresponding response.     

Auditory–visual intermodal matching based on individual recognition in a chimpanzee (Pan troglodytes)

The ability to recognize familiar individuals with different sensory modalities plays an important role in animals living in complex physical and social environments. Individual recognition of familiar individuals was studied in a female chimpanzee named Pan. In previous studies, Pan learned an auditory–visual intermodal matching task (AVIM) consisting of matching vocal samples with the facial pictures of corresponding vocalizers (humans and chimpanzees). The goal of this study was to test whether Pan was able to generalize her AVIM ability to new sets of voice and face stimuli, including those of three infant chimpanzees. Experiment 1 showed that Pan performed intermodal individual recognition of familiar adult chimpanzees and humans very well. However, individual recognition of infant chimpanzees was poorer relative to recognition of adults. A transfer test with new auditory samples (Experiment 2) confirmed the difficulty in recognizing infants. A remaining question was what kind of cues were crucial for the intermodal matching. We tested the effect of visual cues (Experiment 3) by introducing new photographs representing the same chimpanzees in different visual perspectives. Results showed that only the back view was difficult to recognize, suggesting that facial cues can be critical. We also tested the effect of auditory cues (Experiment 4) by shortening the length of auditory stimuli, and results showed that 200 ms vocal segments were the limit for correct recognition. Together, these data demonstrate that auditory–visual intermodal recognition in chimpanzees might be constrained by the degree of exposure to different modalities and limited to specific visual cues and thresholds of auditory cues.     

High-intensity sound increases the size of visually perceived objects

The effect of audiovisual interactions on size perception has yet to be examined, despite its fundamental importance in daily life. Previous studies have reported that object length can be estimated solely on the basis of the sounds produced when an object is dropped. Moreover, it has been shown that people typically and easily perceive the correspondence between object sizes and sound intensities. It is therefore possible that auditory stimuli may act as cues for object size, thereby altering the visual perception of size. Thus, in the present study we examined the effects of auditory stimuli on the visual perception of size. Specifically, we investigated the effects of the sound intensity of auditory stimuli, the temporal window of audiovisual interactions, and the effects of the retinal eccentricity of visual stimuli. The results indicated that high-intensity auditory stimuli increased visually perceived object size, and that this effect was especially strong in the peripheral visual field. Additional consideration indicated that this effect on the visual perception of size is induced when the cue reliability is relatively higher for the auditory than for the visual stimuli. In addition, we further suggest that the cue reliabilities of visual and auditory stimuli relate to retinal eccentricity and sound intensity, respectively.     

Simultaneous auditory discrimination.

Stimuli in many visual stimulus control studies typically are presented simultaneously; in contrast the stimuli in auditory discrimination studies are presented successively. Many everyday auditory stimuli that control responding occur simultaneously. This suggests that simultaneous auditory discriminations should be readily acquired. The purpose of the present experiment was to train rats in a simultaneous auditory discrimination. The apparatus consisted of a cage with two response levers mounted on one wall and a speaker mounted adjacent to each lever. A feeder was mounted on the opposite wall. In a go-right/go-left procedure, two stimuli were presented on each trial, a wide-band noise burst through one speaker and a 2-kHz complex signal through the other. The stimuli alternated randomly from side to side across trials, and the stimulus correlated with reinforcement for presses varied across subjects. The rats acquired the discrimination in 400 to 700 trials, and no response position preference developed during acquisition. The ease with which the simultaneous discrimination was acquired suggests that procedures, such as matching to sample, that require simultaneous presentation of stimuli can be used with auditory stimuli in animals having poor vision.     

Spatial stimulus cue information supplying auditory saltation

Auditory saltation is a misperception of the spatial location of repetitive, transient stimuli. It arises when clicks at one location are followed in perfect temporal cadence by identical clicks at a second location. This report describes two psychophysical experiments designed to examine the sensitivity of auditory saltation to different stimulus cues for auditory spatial perception. Experiment 1 was a dichotic study in which six different six-click train stimuli were used to generate the saltation effect. Clicks lateralised by using interaural time differences and clicks lateralised by using interaural level differences produced equivalent saltation effects, confirming an earlier finding. Switching the stimulus cue from an interaural time difference to an interaural level difference (or the reverse) in mid train was inconsequential to the saltation illusion. Experiment 2 was a free-field study in which subjects rated the illusory motion generated by clicks emitted from two sound sources symmetrically disposed around the interaural axis, ie on the same cone of confusion in the auditory hemifield opposite one ear. Stimuli in such positions produce spatial location judgments that are based more heavily on monaural spectral information than on binaural computations. The free-field stimuli produced robust saltation. The data from both experiments are consistent with the view that auditory saltation can emerge from spatial processing, irrespective of the stimulus cue information used to determine click laterality or location.     

The effect of auditory input on cerebral laterality

Cerebral laterality was examined for third-, fourth-, and fifth-grade deaf and hearing subjects. The experimental task involved the processing of word and picture stimuli presented singly to the right and left visual hemifields. The analyses indicated the deaf children were faster than the hearing children in overall processing efficiency, and that they performed differently in regard to hemispheric lateralization. The deaf children processed the stimuli more efficiently in the right hemisphere, while the hearing children demonstrated a left-hemisphere proficiency. This finding is discussed in terms of the hypothesis that cerebral lateralization is influenced by auditory processing.     

Selective attention to multidimensional auditory stimuli

Event-related brain potentials (ERPs) elicited by multidimensional auditory stimuli were recorded from the scalp in a selective-attention task. Subjects listened to tone pips varying orthogonally between two levels each of pitch, location, and duration and responded to longer duration target stimuli having specific values of pitch and location. The discriminability of the pitch and location attributes was varied between groups. By examining the ERPs to tones that shared pitch and/or locational cues with the designated target, we inferred interrelationships among the processing of these attributes. In all groups, stimuli that failed to match the target tone in an easily discriminable cue elicited only transitory ERP signs of selective processing. Tones sharing the "easy" but not the "hard" cue with the target elicited ERPs that indicated more extensive processing, but not as extensive as stimuli sharing both cues. In addition, reaction times and ERP latencies to the designated targets were not influenced by variations in the discriminability of pitch and location. This pattern of results is consistent with parallel, self-terminating models and holistic models of processing and contradicts models specifying either serial or exhaustive parallel processing of these dimensions. Both the parallel, self-terminating models and the holistic models provide a generalized mechanism for hierarchical stimulus selections that achieve an economy of processing, an underlying goal of classic, multiple-stage theories of selective attention.     

Improving visual spatial working memory in younger and older adults: effects of cross-modal cues

Spatially informative auditory and vibrotactile (cross-modal) cues can facilitate attention but little is known about how similar cues influence visual spatial working memory (WM) across the adult lifespan. We investigated the effects of cues (spatially informative or alerting pre-cues vs. no cues), cue modality (auditory vs. vibrotactile vs. visual), memory array size (four vs. six items), and maintenance delay (900 vs. 1800 ms) on visual spatial location WM recognition accuracy in younger adults (YA) and older adults (OA). We observed a significant interaction between spatially informative pre-cue type, array size, and delay. OA and YA benefitted equally from spatially informative pre-cues, suggesting that attentional orienting prior to WM encoding, regardless of cue modality, is preserved with age. Contrary to predictions, alerting pre-cues generally impaired performance in both age groups, suggesting that maintaining a vigilant state of arousal by facilitating the alerting attention system does not help visual spatial location WM.     

The other modality: Auditory stimuli in language switching

Language switching studies typically implement visual stimuli and visual language cues to trigger a concept and a language response, respectively. In the present study we set out to generalise this to another stimulus modality by investigating language switching with auditory stimuli next to visual stimuli. The results showed that switch costs can be obtained with both auditory and visual stimuli. Yet, switch costs were relatively larger with visual stimuli than with auditory stimuli. Both methodological and theoretical implications of these findings are discussed.