Localizing Localization: The Role of Working Memory in Auditory Localization

It is unclear from current accounts of working memory which, if any, of its components might be involved in our ability to specify the location of a sound source. A series of studies were performed to assess the degree of interference in localization of broadband noise, by a concurrent articulatory suppression (articulatory loop—Experiment 1), serial recall (phonological store and articulatory loop—Experiment 2), and Paced Visual Serial Addition Test (central executive—Experiment 3). No significant disruption of auditory localization was revealed by the first two experiments, ruling out a role for the phonological loop in auditory localization. In Experiment 3, a large degree of error was exhibited in localization, when performed concurrently with the addition task, indicating a requirement for central resources. This suggestion is confirmed by comparison of localization performance across all three studies, which demonstrates a clear deterioration in performance as the demand of concurrent tasks on central resources increases. Finally, concurrent localization was shown to disrupt the primacy portion of the serial position curve, as well as performance on the Paced Visual Serial Addition Test.     

The impact of working memory load on task execution and online plan adjustment during multitasking in a virtual environment

Three experiments investigated the impact of working memory load on online plan adjustment during a test of multitasking in young, nonexpert, adult participants. Multitasking was assessed using the Edinburgh Virtual Errands Test (EVET). Participants were asked to memorize either good or poor plans for performing multiple errands and were assessed both on task completion and on the extent to which they modified their plans during EVET performance. EVET was performed twice, with and without a secondary task loading a component of working memory. In Experiment 1, articulatory suppression was used to load the phonological loop. In Experiment 2, oral random generation was used to load executive functions. In Experiment 3, spatial working memory was loaded with an auditory spatial localization task. EVET performance for both good- and poor-planning groups was disrupted by random generation and sound localization, but not by articulatory suppression. Additionally, people given a poor plan were able to overcome this initial disadvantage by modifying their plans online. It was concluded that, in addition to executive functions, multiple errands performance draws heavily on spatial, but not verbal, working memory resources but can be successfully completed on the basis of modifying plans online, despite a secondary task load.     

The ventriloquist effect does not depend on the direction of deliberate visual attention

It is well known that discrepancies in the location of synchronized auditory and visual events can lead to mislocalizations of the auditory source, so-called ventriloquism. In two experiments, we tested whether such cross-modal influences on auditory localization depend on deliberate visual attention to the biasing visual event. In Experiment 1, subjects pointed to the apparent source of sounds in the presence or absence of a synchronous peripheral flash. They also monitored for target visual events, either at the location of the peripheral flash or in a central location. Auditory localization was attracted toward the synchronous peripheral flash, but this was unaffected by where deliberate visual attention was directed in the monitoring task. In Experiment 2, bilateral flashes were presented in synchrony with each sound, to provide competing visual attractors. When these visual events were equally salient on the two sides, auditory localization was unaffected by which side subjects monitored for visual targets. When one flash was larger than the other, auditory localization was slightly but reliably attracted toward it, but again regardless of where visual monitoring was required. We conclude that ventriloquism largely reflects automatic sensory interactions, with little or no role for deliberate spatial attention.     

How eye movements affect unpleasant memories: support for a working-memory account

Eye movement desensitization and reprocessing can reduce ratings of the vividness and emotionality of unpleasant memories-hence it is commonly used to treat posttraumatic stress disorder. The present experiments compared three accounts of how eye movements produce these benefits. Participants rated unpleasant autobiographical memories before and after eye movements or an eyes stationary control condition. In Experiment 1, eye movements produced benefits only when memories were held in mind during the movements, and eye movements increased arousal, contrary to an investigatory-reflex account. In Experiment 2, horizontal and vertical eye movements produced equivalent benefits, contrary to an interhemispheric-communication account. In Experiment 3, two other distractor tasks (auditory shadowing, drawing) produced benefits that were negatively correlated with working-memory capacity. These findings support a working-memory account of the eye movement benefits in which the central executive is taxed when a person performs a distractor task while attempting to hold a memory in mind.     

Behavioral evidence for task-dependent “what” versus “where” processing within and across modalities

Task-dependent information processing for the purpose of recognition or spatial perception is considered a principle common to all the main sensory modalities. Using a dual-task interference paradigm, we investigated the behavioral effects of independent information processing for shape identification and localization of object features within and across vision and touch. In Experiment 1, we established that color and texture processing (i.e., a “what” task) interfered with both visual and haptic shape-matching tasks and that mirror image and rotation matching (i.e., a “where” task) interfered with a feature-location-matching task in both modalities. In contrast, interference was reduced when a “where” interference task was embedded in a “what” primary task and vice versa. In Experiment 2, we replicated this finding within each modality, using the same interference and primary tasks throughout. In Experiment 3, the interference tasks were always conducted in a modality other than the primary task modality. Here, we found that resources for identification and spatial localization are independent of modality. Our findings further suggest that multisensory resources for shape recognition also involve resources for spatial localization. These results extend recent neuropsychological and neuroimaging findings and have important implications for our understanding of high-level information processing across the human sensory systems.     

Behavioral evidence for task-dependent "what" versus "where" processing within and across modalities

Task-dependent information processing for the purpose of recognition or spatial perception is considered a principle common to all the main sensory modalities. Using a dual-task interference paradigm, we investigated the behavioral effects of independent information processing for shape identificationand localization ofobject features within and across vision and touch. In Experiment 1, we established that color and texture processing (i.e., a "what" task) interfered with both visual and haptic shape-matching tasks and that mirror image and rotation matching (i.e., a "where" task) interfered with a feature-location-matching task in both modalities. In contrast, interference was reduced when a "where" interference task was embedded in a "what" primary task and vice versa. In Experiment 2, we replicated this finding within each modality, using the same interference and primary tasks throughout. In Experiment 3, the interference tasks were always conducted in a modality other than the primary task modality. Here, we found that resources for identification and spatial localization are independent of modality. Our findings further suggest that multisensory resources for shape recognition also involve resources for spatial localization. These results extend recent neuropsychological and neuroimaging findings and have important implications for our understanding of high-level information processing across the human sensory systems.     

Verbal working memory and on-line syntactic processing: Evidence from self-paced listening

Two experiments investigated the relationship between performance on standard tests of verbal working-memory and the on-line construction of syntactic form. In Experiment 1, working-memory was measured in 100 college students on a version of the Daneman and Carpenter (1980) reading-span task, and online syntactic processing was assessed using a self-paced listening task with four sentence types. In Experiment 2, working-memory was measured in 48 college students on two versions of the reading-span task and two other tests of verbal working-memory, and on-line syntactic processing was assessed using the self-paced listening task with an additional sentence type. In both experiments, there was no relationship between working-memory capacity and the increase in processing time seen for the on-line construction of syntactic form for either syntactically more complex or syntactically simpler sentences. The results indicate that the capacity of the working-memory system that is measured by standard working-memory tests does not determine the efficiency of on-line syntactic processing. They are consistent with the view that the working-memory system used for parsing is at least partially separate from that measured by traditional measures of working-memory capacity.     

Interactions Between Exogenous Auditory and Visual Spatial Attention

Six experiments were carried out to investigate the issue of cross-modality between exogenous auditory and visual spatial attention employing Posner's cueing paradigm in detection, localization, and discrimination tasks. Results indicated cueing in detection tasks with visual or auditory cues and visual targets but not with auditory targets (Experiment 1). In the localization tasks, cueing was found with both visual and auditory targets. Inhibition of return was apparent only in the within-modality conditions (Experiment 2). This suggests that it is important whether the attention system is activated directly (within a modality) or indirectly (between modalities). Increasing the cue validity from 50% to 80% influenced performance only in the localization task (Experiment 4). These findings are interpreted as being indicative for modality-specific but interacting attention mechanisms. The results of Experiments 5 and 6 (up/down discrimination tasks) also show cross-modal cueing but not with visual cues and auditory targets. Furthermore, there was no inhibition of return in any condition. This suggests that some cueing effects might be task dependent.     

Exploring the visuo-spatial scratch pad

A series of four studies examines the relationship between the visuo-spatial scratch pad, the central executive and the articulatory loop. For this purpose a visuo-spatial memory task that does not have a large verbal component was developed. In Experiment 1 this task was used to demonstrate that the scratch pad, although functionally independent of the articulatory loop, is subject to interference from a tracking task. Experiment 2 examined the locus of interference of the tracking task with spatial memory. It was shown that interference is confined to the encoding phase; post presentation tracking does not disrupt visuo-spatial representation. Experiment 3 demonstrated that the tracking task employed requires some central executive resources. The final experiment examined the effect of a near-span consonant memory load on the spatial memory task. Disruption only occurred when the verbal load preceded, or was concurrent with, the spatial task. The results suggest that central executive resources are probably required to operate the scratch pad in most circumstances. However, minimal central capacity is required for maintenance rehersal.     

Executive processing in learning-disabled readers

Three experiments determine whether memory difficulties experienced by learning-disabled readers may be attributable, in part, to executive processing. In Experiment 1, learning-disabled readers, matched to skilled readers on chronological age and reading-comprehension ability, were compared on the Concurrent Digit Span measure presented under high (6-digit) and low (3-digit) memory-load conditions as they concurrently sorted cards into verbal or nonverbal categories. Experiment 2 utilized the same task as Experiment 1, except that learning-disabled (LD) readers of low and average working-memory capacity were compared with skilled readers of average and high working-memory capacity. Both experiments clearly showed that LD readers' performance is depressed under high memory-load conditions. Experiment 2 also indicated that overall performance of non-learning-disabled (NLD) readers was predictable from LD readers, suggesting that ability-group differences are related to global processing efficiency rather than to isolated verbal and nonverbal systems. In Experiment 3, LD and NLD readers were compared on their ability to recall central and secondary information from base and elaborative sentences tasks under high- and low-effort encoding conditions. LD readers had significantly poorer secondary recall during high-effort encoding conditions when compared to skilled readers. Taken together, the results are discussed within a framework that views individual differences in reading comprehension as reflecting, in part, executive processing (i.e., resource monitoring) activities.     

Binding facilitates attention switching within working memory

The authors investigated the units of selective attention within working memory. In Experiment 1, a group of participants kept 1 count and 1 location in working memory and updated them repeatedly in random order. Another group of participants were instructed to achieve the same goal by memorizing the verbal and spatial information in an integrative way as a moving digit. The behavioral data showed that switching attention between properties of an integrated working-memory item was faster than switching between respective properties of different items. Experiment 2 demonstrated that this switching facilitation cannot be simply ascribed to the different amount of working-memory items maintained by the two groups of participants. Finally, by adopting a pure verbal task in Experiment 3, the authors observed the same binding facilitation, with the possibility of "location-based selection" excluded. They summarize the observations of all 3 experiments in the study and suggest both a location- and object-based mechanism for attention selection in working memory.     

A comparison of visual and auditory representational momentum in spatial tasks

Similarities have been observed in the localization of the final position of moving visual and moving auditory stimuli: Perceived endpoints that are judged to be farther in the direction of motion in both modalities likely reflect extrapolation of the trajectory, mediated by predictive mechanisms at higher cognitive levels. However, actual comparisons of the magnitudes of displacement between visual tasks and auditory tasks using the same experimental setup are rare. As such, the purpose of the present free-field study was to investigate the influences of the spatial location of motion offset, stimulus velocity, and motion direction on the localization of the final positions of moving auditory stimuli (Experiment 1 and 2) and moving visual stimuli (Experiment 3). To assess whether auditory performance is affected by dynamically changing binaural cues that are used for the localization of moving auditory stimuli (interaural time differences for low-frequency sounds and interaural intensity differences for high-frequency sounds), two distinct noise bands were employed in Experiments 1 and 2. In all three experiments, less precise encoding of spatial coordinates in paralateral space resulted in larger forward displacements, but this effect was drowned out by the underestimation of target eccentricity in the extreme periphery. Furthermore, our results revealed clear differences between visual and auditory tasks. Displacements in the visual task were dependent on velocity and the spatial location of the final position, but an additional influence of motion direction was observed in the auditory tasks. Together, these findings indicate that the modality-specific processing of motion parameters affects the extrapolation of the trajectory.     

The impact of anxiety on analogical reasoning

The effect of state anxiety on analogical reasoning was investigated by examining qualitative differences in mapping performance between anxious and non-anxious individuals reasoning about pictorial analogies. The working-memory restriction theory of anxiety, coupled with theories of analogy that link complexity of mapping with working-memory capacity, predicts that high anxiety will impair the ability to find correspondences based on relations between multiple objects relative to correspondences based on overlap of attributes between individual objects. Anxiety was induced in one condition by a stressful speeded subtraction task administered prior to the analogy task. Anxious participants produced fewer relational responses and more attribute responses than did non-anxious participants, both in the absence of explicit instructions to find relational mappings (Experiment 1) and after receiving such instructions (Experiment 2). The findings support the postulated links among anxiety, working memory, and the ability to perform complex analogical mapping.     

An investigation of spatial representation of pitch in individuals with congenital amusia

Spatial representation of pitch plays a central role in auditory processing. However, it is unknown whether impaired auditory processing is associated with impaired pitch–space mapping. Experiment 1 examined spatial representation of pitch in individuals with congenital amusia using a stimulus–response compatibility (SRC) task. For amusic and non-amusic participants, pitch classification was faster and more accurate when correct responses involved a physical action that was spatially congruent with the pitch height of the stimulus than when it was incongruent. However, this spatial representation of pitch was not as stable in amusic individuals, revealed by slower response times when compared with control individuals. One explanation is that the SRC effect in amusics reflects a linguistic association, requiring additional time to link pitch height and spatial location. To test this possibility, Experiment 2 employed a colour-classification task. Participants judged colour while ignoring a concurrent pitch by pressing one of two response keys positioned vertically to be congruent or incongruent with the pitch. The association between pitch and space was found in both groups, with comparable response times in the two groups, suggesting that amusic individuals are only slower to respond to tasks involving explicit judgments of pitch.     

Children's working-memory processes: a response-timing analysis

Recall response durations were used to clarify processing in working-memory tasks. Experiment 1 examined children's performance in reading span, a task in which sentences were processed and the final word of each sentence was retained for subsequent recall. Experiment 2 examined the development of listening-, counting-, and digit-span task performance. Responses were much longer in the reading-and listening-span tasks than in the other span tasks, suggesting that participants in sentence-based span tasks take time to retrieve the semantic or linguistic structure as cues to recall of the sentence-final words. Response durations in working-memory tasks helped to predict academic skill and achievement, largely separate from the contributions of the memory spans themselves. Response durations thus are important in the interpretation of span task performance.     

Effects of spatial and selective attention on basic multisensory integration

When participants respond to auditory and visual stimuli, responses to audiovisual stimuli are substantially faster than to unimodal stimuli (redundant signals effect, RSE). In such tasks, the RSE is usually higher than probability summation predicts, suggestive of specific integration mechanisms underlying the RSE. We investigated the role of spatial and selective attention on the RSE in audiovisual redundant signals tasks. In Experiment 1, stimuli were presented either centrally (narrow attentional focus) or at 1 of 3 unpredictable locations (wide focus). The RSE was accurately described by a coactivation model assuming linear superposition of modality-specific activation. Effects of spatial attention were explained by a shift of the evidence criterion. In Experiment 2, stimuli were presented at 3 locations; participants had to respond either to all signals regardless of location (simple response task) or to central stimuli only (selective attention task). The RSE was consistent with task-specific coactivation models; accumulation of evidence, however, differed between the 2 tasks.     

Head-centred meridian effect on auditory spatial attention orienting

Six experiments examined the issue of whether one single system or separate systems underlie visual and auditory orienting of spatial attention. When auditory targets were used, reaction times were slower on trials in which cued and target locations were at opposite sides of the vertical head-centred meridian than on trials in which cued and target locations were at opposite sides of the vertical visual meridian or were not separated by any meridian. The head-centred meridian effect for auditory stimuli was apparent when targets were cued by either visual (Experiments 2, 3, and 6) or auditory cues (Experiment 5). Also, the head-centred meridian effect was found when targets were delivered either through headphones (Experiments 2, 3, and 5) or external loudspeakers (Experiment 6). Conversely, participants showed a visual meridian effect when they were required to respond to visual targets (Experiment 4). These results strongly suggest that auditory and visual spatial attention systems are indeed separate, as far as endogenous orienting is concerned.     

Effects of visual attentional load on low-level auditory scene analysis

The sharing of processing resources between the senses was investigated by examining the effects of visual task load on auditory event-related brain potentials (ERPs). In Experiment 1, participants completed both a zero-back and a one-back visual task while a tone pattern or a harmonic series was presented. N1 and P2 waves were modulated by visual task difficulty, but neither mismatch negativity (MMN) elicited by deviant stimuli from the tone pattern nor object-related negativity (ORN) elicited by mistuning from the harmonic series was affected. In Experiment 2, participants responded to identity (what) or location (where) in vision, while ignoring sounds alternating in either pitch (what) or location (where). Auditory ERP modulations were consistent with task difficulty, rather than with task specificity. In Experiment 3, we investigated auditory ERP generation under conditions of no visual task. The results are discussed with respect to a distinction between process-general (N1 and P2) and processspecific (MMN and ORN) auditory ERPs.     

Is cross-modal integration of emotional expressions independent of attentional resources?

In this study, we examined whether integration of visual and auditory information about emotions requires limited attentional resources. Subjects judged whether a voice expressed happiness or fear, while trying to ignore a concurrently presented static facial expression. As an additional task, the subjects had to add two numbers together rapidly (Experiment 1), count the occurrences of a target digit in a rapid serial visual presentation (Experiment 2), or judge the pitch of a tone as high or low (Experiment 3). The visible face had an impact on judgments of the emotion of the heard voice in all the experiments. This cross-modal effect was independent of whether or not the subjects performed a demanding additional task. This suggests that integration of visual and auditory information about emotions may be a mandatory process, unconstrained by attentional resources.     

Cross-modal bias and perceptual fusion with auditory-visual spatial discordance

Investigations of situations involving spatial discordance between auditory and visual data which can otherwise be attributed to a common origin have revealed two main phenomena:cross-modal bias andperceptual fusion (or ventriloquism). The focus of the present study is the relationship between these two. The question asked was whether bias occurred only with fusion, as is predicted by some accounts of reactions to discordance, among them those based on cuesubstitution. The approach consisted of having subjects, on each trial, both point to signals in one modality in the presence of conflicting signals in the other modality and produce same-different origin judgments. To avoid the confounding of immediate effects with cumulative adaptation, which was allowed in most previous studies, the direction and amplitude of discordance was varied randomly from trial to trial. Experiment 1, which was a pilot study, showed that both visual bias of auditory localization and auditory bias of visual localization can be observed under such conditions. Experiment 2, which addressed the main question, used a method which controls for the selection involved in separating fusion from no-fusion trials and showed that the attraction of auditory localization by conflicting visual inputs occurs even when fusion is not reported. This result is inconsistent with purely postperceptual views of cross-modal interactions. The question could not be answered for auditory bias of visual localization, which, although significant, was very small in Experiment 1 and fell below significance under the conditions of Experiment 2.