The costs of crossing paths and switching tasks between audition and vision

Switching from one functional or cognitive operation to another is thought to rely on executive/control processes. The efficacy of these processes may depend on the extent of overlap between neural circuitry mediating the different tasks; more effective task preparation (and by extension smaller switch costs) is achieved when this overlap is small. We investigated the performance costs associated with switching tasks and/or switching sensory modalities. Participants discriminated either the identity or spatial location of objects that were presented either visually or acoustically. Switch costs between tasks were significantly smaller when the sensory modality of the task switched versus when it repeated. This was the case irrespective of whether the pre-trial cue informed participants only of the upcoming task, but not sensory modality (Experiment 1) or whether the pre-trial cue was informative about both the upcoming task and sensory modality (Experiment 2). In addition, in both experiments switch costs between the senses were positively correlated when the sensory modality of the task repeated across trials and not when it switched. The collective evidence supports the independence of control processes mediating task switching and modality switching and also the hypothesis that switch costs reflect competitive interference between neural circuits.     

The cost of a voluntary task switch

Task-switching paradigms are widely used to study executive control. However, standard paradigms may not require active control to switch tasks. We examined voluntary task switching by having subjects choose which task to perform on a series of bivalent stimuli. Subjects performed parity or magnitude judgments on single digits. Instructions were to perform the two tasks equally often and in a random order. The response-to-stimulus interval (RSI) was either 100 or 1,000 ms, manipulated between blocks. Task alternations were slower than task repetitions, and this switch cost was greater at the short RSI than at the long RSI (310 and 94 ms, respectively). Additionally, subjects produced more task repetitions than expected if the tasks were performed in a random sequence. These results show costs associated with a voluntary task switch, when subjects must actively control the choice of the task to be performed.     

Task coordination and aging: explorations of executive control processes in the task switching paradigm.

A number of models of cognitive aging suggest that older adults exhibit disproportionate performance decrements on tasks which require executive control processes. In a series of three studies we examined age-related differences in executive control processes and more specifically in the executive control processes which underlie performance in the task switching paradigm. Young and old adults were presented with rows of digits and were required to indicate whether the number of digits (element number task) or the value of the digits (digit value task) were greater than or less than five. Switch costs were assessed by subtracting the reaction times obtained on non-switch trials from trials following a task switch. Several theoretically interesting results were obtained. First, large age-related differences in switch costs were found early in practice. Second, and most surprising, after relatively modest amounts of practice old and young adults switch costs were equivalent. Older adults showed large practice effects on switch trials. Third, age-equivalent switch costs were maintained across a two month retention period. Finally, the main constraint on whether age equivalence was observed in task switching performance was memory load. Older adults were unable to capitalize on practice under high memory loads. These data are discussed in terms of their implications for both general and process specific cognitive aging models.     

Crossmodal attention switching: Auditory dominance in temporal discrimination tasks

Visual stimuli are often processed more efficiently than accompanying stimuli in another modality. In line with this “visual dominance”, earlier studies on attentional switching showed a clear benefit for visual stimuli in a bimodal visual–auditory modality-switch paradigm that required spatial stimulus localization in the relevant modality. The present study aimed to examine the generality of this visual dominance effect. The modality appropriateness hypothesis proposes that stimuli in different modalities are differentially effectively processed depending on the task dimension, so that processing of visual stimuli is favored in the dimension of space, whereas processing auditory stimuli is favored in the dimension of time. In the present study, we examined this proposition by using a temporal duration judgment in a bimodal visual–auditory switching paradigm. Two experiments demonstrated that crossmodal interference (i.e., temporal stimulus congruence) was larger for visual stimuli than for auditory stimuli, suggesting auditory dominance when performing temporal judgment tasks. However, attention switch costs were larger for the auditory modality than for visual modality, indicating a dissociation of the mechanisms underlying crossmodal competition in stimulus processing and modality-specific biasing of attentional set.     

预知任务转换的内源性准备和外源性调节

采用ERP探究在预知条件下任务转换的神经机制。使用“任务转换”实验范式,以汉字（名词/动词）或数字（奇数/偶数）的归类为任务。被试执行任务重复（AA,BB,…）或在两个任务间转换（AB,BA,…）,每个任务序列为连续的两个任务（任务1,任务2）。结果发现,任务1中,在刺激呈现后300ms左右,数、词的转换序列与重复序列出现显著差异;在中央区,320ms左右,任务1中转换序列比重复序列、任务2中转换任务比重复任务都产生一个更为负走向的波。这些结果表明,在预知条件下,简单任务转换的内源性准备可能开始于任务1呈现后300ms左右;内源性准备和外源性调节可能都反映了认知冲突过程,此认知冲突可能是转换代价的主要来源     

Individual differences in multiple types of shifting attention

Many researchers consider costs in shifting attention and mental set to reflect a basic ability to use top-down goal information to guide action. Although switch costs have been used as measures of individuals' executive function, whether common abilities underlie task set switching across different types of shifting tasks has not been well studied. In 249 participants, we studied whether switch costs in a novel two-choice reaction time task were correlated across variations in two variables: thelocus of representation (stimuli were either perceptually available or stored in working memory [WM]) and which of two judgment tasks was performed. Switch costs were asymmetrical, in that it was easier to switch to the easier judgment, and were related to overall and relative processing speed: Switch costs were higher when the task was more difficult. These factors should be accounted for when one is measuring individual differences in switch costs. After controlling for these effects, we found evidence for a common ability underlying switch costs that involved both task set preparation and response selection; however, residual shift costs, which involve only response selection, were uncorrelated across tasks. Correlations among switch costs were substantially higher within task type (e.g., correlations of WM shifting tasks with other WM shifting tasks and of perceptual tasks with perceptual ones), suggesting that there are also processes unique to switching within WM and switching among visible stimuli.     

Tasks of a feather flock together: Similarity effects in task switching

Recent research on task switching has paid little attention to how tasks are represented and how the relations between task representations might affect the executive processes engaged to achieve a task switch. Two experiments investigated the effect of task similarity on task switching. Similarity was defined in terms of shared component operations--attentional control settings in Experiment 1 and response modality in Experiment 2--with tasks sharing more component operations said to be more similar to each other than tasks sharing fewer component operations. Across both experiments, task similarity facilitated task switching, seen in reduced switch costs for switching between similar tasks as opposed to dissimilar tasks. These results indicate that task similarity defined in terms of component operations can be used to define a multidimensional task space in which the executive processes of task selection and implementation are active.     

The role of input–output modality compatibility in task switching

Input–output (I–O) modality compatibility refers to the similarity of stimulus modality and modality of response-related sensory consequences. A previous study found higher switch costs in task switching in I–O modality incompatible tasks (auditory-manual and visual-vocal) than in I–O modality compatible tasks (auditory-vocal and visual-manual). However, these tasks had spatially compatible S–R mappings, which implied dimensional overlap (DO). DO may have led to automatic activation of the corresponding compatible responses in the incorrect response modality, thus increasing interference effects. The present study was aimed to examine the influence of DO on I–O modality compatibility effects. In two experiments, we found that I–O modality compatibility affects task switching even in tasks without DO, which even tended to result in further increased modality influences. This finding suggests that I–O modality mappings affect response selection by affecting between-task cross-talk not on the level of specific response codes but on the level of modality-specific processing pathways.     

Selection of objects and tasks in working memory

When people hold several objects (such as digits or words) in working memory and select one for processing, switching to a new object takes longer than selecting the same object as that on the preceding processing step. Similarly, selecting a new task incurs task- switching costs. This work investigates the selection of objects and of tasks in working memory using a combination of object-switching and task-switching paradigms. Participants used spatial cues to select one digit held in working memory and colour cues to select one task (addition or subtraction) to apply to it. Across four experiments the mapping between objects and their cues and the mapping between tasks and their cues were varied orthogonally. When mappings varied from trial to trial for both objects and tasks, switch costs for objects and tasks were additive, as predicted by sequential selection or resource sharing. When at least one mapping was constant across trials, allowing learning of long-term associations, switch costs were underadditive, as predicted by partially parallel selection. The number of objects in working memory affected object-switch costs but not task-switch costs, counter to the notion of a general resource of executive attention.     

Action video game experience reduces the cost of switching tasks

Video game expertise has been shown to have beneficial effects for visual attention processes, but the effects of action video game playing on executive functions, such as task switching and filtering out distracting information, are less well understood. In the main experiment presented here, video game players (VGPs) and nonplayers (nVGPs) switched between two tasks of unequal familiarity: a familiar task of responding in the direction indicated by an arrow, and a novel task of responding in the opposite direction. nVGPs had large response time costs for switching from the novel task to the familiar task, and small costs for switching from the familiar task to the novel task, replicating prior findings. However, as compared to the nVGPs, VGPs were more facile in switching between tasks, producing overall smaller and more symmetric switching costs, suggesting that experience with action video games produces improvements in executive functioning. In contrast, VGPs and nVGPs did not differ in filtering out the irrelevant flanking stimuli or in remembering details of aurally presented stories. The lack of global differences between the groups suggests that the improved task-switching performance seen in VGPs was not due to differences in global factors, such as VGPs being more motivated than nVGPs.     

Modality switching costs emerge in concept creation as well as retrieval

Theories of embodied cognition hold that the conceptual system uses perceptual simulations for the purposes of representation. A strong prediction is that perceptual phenomena should emerge in conceptual processing, and, in support, previous research has shown that switching modalities from one trial to the next incurs a processing cost during conceptual tasks. However, to date, such research has been limited by its reliance on the retrieval of familiar concepts. We therefore examined concept creation by asking participants to interpret modality-specific compound phrases (i.e., conceptual combinations). Results show that modality switching costs emerge during the creation of new conceptual entities: People are slower to simulate a novel concept (e.g., auditory jingling onion) when their attention has already been engaged by a different modality in simulating a familiar concept (e.g., visual shiny penny). Furthermore, these costs cannot be accounted for by linguistic factors alone. Rather, our findings support the embodied view that concept creation, as well as retrieval, requires situated perceptual simulation.     

Task- and location-switching effects on visual attention

In two experiments, we examined the effects of task and location switching on the accuracy of reporting target characters in an attentional blink (AB) paradigm. Single-character streams were presented at a rate of 100 msec per character in Experiment 1, and successive pairs of characters on either side of fixation were presented in Experiment 2. On each trial, two targets appeared that were either white letters or black digits embedded in a stream of black letter distractors, and they were separated by between zero and five items in the stream (lags 1-6). Experiment 1 showed that report of the first target was least accurate if it immediately preceded the second target and if the two targets were either both letters or both digits (task repetition cost). Report of the second target was least accurate if one or two distractors intervened between the two targets (the U-shaped AB lag effect) and if one target was a letter and the other a digit (task switch cost). Experiment 2 added location uncertainty as a factor and showed similar effects as Experiment 1, with one exception. Lag 1 sparing (the preserved accuracy in reporting the second of two targets if the second immediately follows the first) was completely eliminated when the task required attention switching across locations. Two-way additive effects were found between task switching and location switching in the AB paradigm. These results suggests separate loci for their attentional effects. It is likely that the AB deficit is due mainly to central memory limitations, whereas location-switching costs occur at early visual levels. Task-switching costs occur at an intermediate visual level, since the present task switch involved encoding differences without changes in stimulus-response mapping rules (i.e., the task was character identification for both letters and digits).     

Asymmetrical switch costs in children

Switching between tasks produces decreases in performance as compared to repeating the same task. Asymmetrical switch costs occur when switching between two tasks of unequal difficulty. This asymmetry occurs because the cost is greater when switching to the less difficult task than when switching to the more difficult task. Various theories about the origins of these asymmetrical switch costs have emerged from numerous and detailed experiments with adults. There is no documented evidence of asymmetrical switch costs in children. We conducted a series of studies that examined age-related changes in asymmetrical switch costs, within the same paradigm. Similarities in the patterns of asymmetrical switch costs between children and adults suggested that theoretical explanations of the cognitive mechanisms driving asymmetrical switch costs in adults could be applied to children. Age-related differences indicate that these theoretical explanations need to incorporate the relative contributions and interactions of developmental processes and task mastery.     

Effects of repetition and foreknowledge in task-set reconfiguration

To examine the roles of executive control and automatic activation in task switching, we manipulated foreknowledge as well as task transitions. In Experiments 1 and 2, performance with foreknowledge was faster than performance with no foreknowledge, but the amount of switch cost did not depend on foreknowledge. This result suggests that switch costs primarily reflect persisting activation rather than inadequate preparation. In Experiment 3, switch cost was greater with foreknowledge about task transition alone than with foreknowledge about both task transition and identity, suggesting that foreknowledge about specific task identity did allow preparation for a switched task. We argue that task repetition and foreknowledge effects are independent. Although foreknowledge allows preparation for both repeated and switched tasks, repeating the same task has benefits over task switching regardless of foreknowledge.     

Executive control in set switching: residual switch cost and task-set inhibition.

Executive processes necessary for flexibly switching between different tasks were studied using a set switching paradigm that requires participants to rapidly switch between different tasks across consecutive trials. Switch cost reflects poorer performance for task-switch trials than for consecutive same-task trials. Significant switch cost was observed even with considerable preparation time before a task-switch, an effect known as residual switch cost. This study tested the hypothesis that one process underlying residual switch cost is inhibition of the previous task-set. We used semantic categorization tasks to compare switch cost between alternating task series (ABA) and nonalternating series (ABC) in order to test the generality of a task-set inhibition effect previously observed with perceptual judgment tasks (Mayr & Keele, in press). The results yielded significant switch cost only for alternating tasks, in both response times and errors resulting from performance of the wrong task. Thus, resolving inhibition associated with previously abandoned task-sets may be the main process underlying residual switch costs, suggesting that task-set inhibition is an important executive control process.     

Interaction between stimulus-driven orienting and top-down modulation in attentional capture

The present study investigated whether visual and kinesthetic stimuli are stored as multisensory or modality-specific representations in unimodal and crossmodal working memory tasks. To this end, angle-shaped movement trajectories were presented to 16 subjects in delayed matching-to-sample tasks either visually or kinesthetically during encoding and recognition. During the retention interval, a secondary visual or kinesthetic interference task was inserted either immediately or with a delay after encoding. The modality of the interference task interacted significantly with the encoding modality. After visual encoding, memory was more impaired by a visual than by a kinesthetic secondary task, while after kinesthetic encoding the pattern was reversed. The time when the secondary task had to be performed interacted with the encoding modality as well. For visual encoding, memory was more impaired, when the secondary task had to be performed at the beginning of the retention interval. In contrast, memory after kinesthetic encoding was more affected, when the secondary task was introduced later in the retention interval. The findings suggest that working memory traces are maintained in a modality-specific format characterized by distinct consolidation processes that take longer after kinesthetic than after visual encoding.     

多感觉整合范式中潜在的跨通道转换效应

大脑可以对来自不同感觉通道的信息进行处理与整合。与单一感觉通道相比, 个体对同时呈现在不同感觉通道的目标信号的响应会更快。对于这种现象的一种主要理论解释是共同激活模型, 该模型认为来自不同通道的刺激在特定的脑区汇聚整合, 比如顶叶内沟、颞上沟和前额叶皮层区域。整合后的信号强度更大, 可以更快地触发反应, 但是信号的整合发生在认知加工的哪一阶段目前尚未有明确结论。当个体对出现在不同感觉通道之间的任务转换进行加工时, 产生与感觉通道相关的任务转换的损失小于跨感觉通道转换损失与任务转换损失的总和, 这为与感觉通道相关的转换代价来源于任务设置的惯性和干扰提供了证据。而在单通道和多通道之间发生转换时, 跨通道转换代价会减小甚至消失, 这是由于同时发生的多感觉整合抵消了一部分损失, 这种现象支持了共同激活模型理论。然而, 多感觉信号整合对任务转换的神经加工过程产生怎样的影响并不清楚, 在未来的研究中可以把多感觉整合范式同经典的任务转换范式结合改进, 进而确定跨通道转换的加工机制和多感觉信号整合的发生阶段。     

Modality independence of order coding in working memory: Evidence from cross-modal order interference at recall

Working memory researchers do not agree on whether order in serial recall is encoded by dedicated modality-specific systems or by a more general modality-independent system. Although previous research supports the existence of autonomous modality-specific systems, it has been shown that serial recognition memory is prone to cross-modal order interference by concurrent tasks. The present study used a serial recall task, which was performed in a single-task condition and in a dual-task condition with an embedded memory task in the retention interval. The modality of the serial task was either verbal or visuospatial, and the embedded tasks were in the other modality and required either serial or item recall. Care was taken to avoid modality overlaps during presentation and recall. In Experiment 1, visuospatial but not verbal serial recall was more impaired when the embedded task was an order than when it was an item task. Using a more difficult verbal serial recall task, verbal serial recall was also more impaired by another order recall task in Experiment 2. These findings are consistent with the hypothesis of modality-independent order coding. The implications for views on short-term recall and the multicomponent view of working memory are discussed.     

Spatial working memory for locations specified by vision and audition: Testing the amodality hypothesis

Spatial working memory can maintain representations from vision, hearing, and touch, representations referred to here as spatial images. The present experiment addressed whether spatial images from vision and hearing that are simultaneously present within working memory retain modality-specific tags or are amodal. Observers were presented with short sequences of targets varying in angular direction, with the targets in a given sequence being all auditory, all visual, or a sequential mixture of the two. On two thirds of the trials, one of the locations was repeated, and observers had to respond as quickly as possible when detecting this repetition. Ancillary detection and localization tasks confirmed that the visual and auditory targets were perceptually comparable. Response latencies in the working memory task showed small but reliable costs in performance on trials involving a sequential mixture of auditory and visual targets, as compared with trials of pure vision or pure audition. These deficits were statistically reliable only for trials on which the modalities of the matching location switched from the penultimate to the final target in the sequence, indicating a switching cost. The switching cost for the pair in immediate succession means that the spatial images representing the target locations retain features of the visual or auditory representations from which they were derived. However, there was no reliable evidence of a performance cost for mixed modalities in the matching pair when the second of the two did not immediately follow the first, suggesting that more enduring spatial images in working memory may be amodal.     

Positive affect facilitates task switching in the dimensional change card sort task: Implications for the shifting aspect of executive function

Using the modified Dimensional Change Card Sort task, we examined the influence of positive affect on task switching by inspecting various markers for the costs, including restart cost, switch cost and mixing cost. Given that the executive-control processes that underlie switching performance—i.e., inhibition or shifting—are distinct from the component processes that underlie non-switching performance—i.e., stimulus evaluation, resource allocation or response execution—we hypothesised that if positive affect facilitates task switching via executive-control processes, rather than via component processes, positive affect would reduce both switch and restart costs, but not mixing cost, because both switch and restart costs rely on executive processes, while mixing cost imposes only minimal demands on executive processes. We found beneficial effects of positive affect on both restart and switch costs, but not on mixing costs. These results suggest that positive affect improves switching abilities via executive processes rather than via component processes.