Working memory interference during processing texts and pictures: Implications for the explanation of the modality effect

Auditory text presentation improves learning with pictures and texts. With sequential text–picture presentation, cognitive models of multimedia learning explain this modality effect in terms of greater visuo‐spatial working memory load with visual as compared to auditory texts. Visual texts are assumed to demand the same working memory subsystem as pictures, while auditory texts make use of an additional cognitive resource. We provide two alternative assumptions that relate to more basic processes: First, acoustic‐sensory information causes a retention advantage for auditory over visual texts which occurs no matter if a picture is presented or not. Second, eye movements during reading hamper visuo‐spatial rehearsal. Two experiments applying elementary procedures provide first evidence for these assumptions. Experiment 1 demonstrates that, regarding text recall, the auditory advantage is independent of visuo‐spatial working memory load. Experiment 2 reveals worse matrix recognition performance after reading text requiring eye movements than after listening or reading without eye movements. Copyright © 2008 John Wiley & Sons, Ltd.     

When auditory presentations should and should not be a component of multimedia instruction

Based on cognitive load theory, two experiments investigated the conditions under which audiovisual‐based instruction may be an effective or an ineffective instructional technique. Results from Experiment 1 indicated that visual with audio presentations were superior to equivalent visual‐only presentations. In this experiment, neither the auditory nor the visual material could be understood in isolation. Both sources of information were interrelated and were essential to render the material intelligible. In contrast, Experiment 2 demonstrated that a non‐essential explanatory text, presented aurally with similar written text contained in a diagram, hindered learning. This result was obtained because when compared to a diagram only format, the aural material was unnecessary and therefore created a redundancy effect. Differences between groups were stronger when information was high in complexity. It was concluded that the effectiveness of multimedia instruction depends very much on how and when auditory information is used. Copyright © 2003 John Wiley & Sons, Ltd.     

Cognitive load and the imagination effect

When presented with a procedure or concept to learn, imagining the procedure or concept may be an effective instructional technique compared to conventional studying, thus generating an imagination effect. However, it was hypothesized that the importance of learning through imagining as an instructional technique depends on modes of presentation. Experiment 1 tested adults studying or imagining contour maps as participants and was designed to verify the generality of the imagination effect. Imagination instructions were superior to study instructions on subsequent test questions. Experiment 2 further investigated the effect by comparing much younger students (Grade 4), studying or imagining temperature/time graphs presented in either a split‐attention (spatially separated diagram and text) or an integrated (spatially combined diagram and text) format. Results on a subsequent test indicated that the Grade 4 students found imagining beneficial to their learning, compared with studying the material but the effect was only obtained using an integrated rather than a split‐attention format. Experiment 3 was conducted to obtain verbal protocols from Grade 4 imagination and study groups using the same instructional materials to throw light on the cognitive mechanisms behind the imagination effect. Copyright © 2004 John Wiley & Sons, Ltd.     

The perceptual basis of the modality effect in multimedia learning

Various studies have demonstrated an advantage of auditory over visual text modality when learning with texts and pictures. To explain this modality effect, two complementary assumptions are proposed by cognitive theories of multimedia learning: first, the visuospatial load hypothesis, which explains the modality effect in terms of visuospatial working memory overload in the visual text condition; and second, the temporal contiguity assumption, according to which the modality effect occurs because solely auditory texts and pictures can be attended to simultaneously. The latter explanation applies only to simultaneous presentation, the former to both simultaneous and sequential presentation. This paper introduces a third explanation, according to which parts of the modality effect are due to early, sensory processes. This account predicts that-for texts longer than one sentence-the modality effect with sequential presentation is restricted to the information presented most recently. Two multimedia experiments tested the influence of text modality across three different conditions: simultaneous presentation of texts and pictures versus sequential presentation versus presentation of text only. Text comprehension and picture recognition served as dependent variables. An advantage for auditory texts was restricted to the most recent text information and occurred under all presentation conditions. With picture recognition, the modality effect was restricted to the simultaneous condition. These findings clearly support the idea that the modality effect can be attributed to early processes in perception and sensory memory rather than to a working memory bottleneck.     

Cognitive load theory,modality of presentation and the transient information effect

The modality effect occurs when audio/visual instructions are superior to visual only instructions. The effect was explored in two experiments conducted within a cognitive load theory framework. In Experiment 1, two groups of primary school students (N = 24) were presented with either audio/visual or visual only instructions on how to read a temperature graph. The group presented with visual text and a diagram rather than audio text and a diagram was superior, reversing most previous data on the modality effect. It was hypothesized that the reason for the reversal was that the transitory auditory text component was too long to be processed easily in working memory compared to more permanent written information. Experiment 2 (N = 64) replicated the experiment with the variation of a reduced length of both auditory and visual text instructions. Results indicated a reinstatement of the modality effect with audio/visual instructions proving superior to visual only instructions. Copyright © 2011 John Wiley & Sons, Ltd.     

Effects of working memory load on long‐term word priming

Three experiments assessed the role of verbal and visuo‐spatial working memory in supporting long‐term repetition priming for written words. In Experiment 1, two priming tasks (word stem completion and category‐exemplar production) were included with three levels of load on working memory: (1) without memory load, (2) memory load that involved storing a string of six digits, and (3) memory load that involved storing a graphic shape. Experiments 2 and 3 compared the effects of a verbal (Experiment 2) or a visual (Experiment 3) working memory load at encoding on both an implicit (word stem completion) and an explicit test (cued recall). The results show no effect of memory load in any of the implicit memory tests, suggesting that priming does not rely on working memory resources. By contrast, loading working memory at encoding causes a significant disruptive effect on the explicit memory test for words when the load is verbal but not visual.     

Contextual cueing under working memory load: Selective interference of visuospatial load with expression of learning

In a series of experiments, we investigated the dependence of contextual cueing on working memory resources. A visual search task with 50 % repeated displays was run in order to elicit the implicit learning of contextual cues. The search task was combined with a concurrent visual working memory task either during an initial learning phase or a later test phase. The visual working memory load was either spatial or nonspatial. Articulatory suppression was used to prevent verbalization. We found that nonspatial working memory load had no effect, independent of presentation in the learning or test phase. In contrast, visuospatial load diminished search facilitation in the test phase, but not during learning. We concluded that visuospatial working memory resources are needed for the expression of previously learned spatial contexts, whereas the learning of contextual cues does not depend on visuospatial working memory.     

On the nature of the span of apprehension

Summary Typically, people can only report about four or five items from a briefly presented array of alphanumeric items. A new span task was used to explore the basis of this limitation. In Experiment l, performance suffered when very brief display durations were combined with a verbal-load task, but no significant effects of display duration were found when there was no verbal load. In Experiment 2, a similar interaction was observed between verbal load and the presence of a visual suffix; performance was worse in the verbal-load condition with a visual suffix, but no such effect was observed without verbal load. In both experiments, poorer performance was associated with enhanced serial-position effects. The results can be explained on the assumption that the verbal-load task required some processing resources, and that the quality of information in visual working memory depends on available resources. Thus, both brief-array presentation and the visual suffix degrade the information in visual working memory, but span performance is impaired only when processing resources are relatively scarce.     

The efficiency of multimedia learning into old age

Background: On the basis of a multimodal model of working memory, cognitive load theory predicts that a multimedia‐based instructional format leads to a better acquisition of complex subject matter than a purely visual instructional format. Aims: This study investigated the extent to which age and instructional format had an impact on training efficiency among both young and old adults. It was hypothesised that studying worked examples that are presented as a narrated animation (multimedia condition) is a more efficient means of complex skill training than studying visually presented worked examples (unimodal condition) and solving conventional problems. Furthermore, it was hypothesised that multimedia‐based worked examples are especially helpful for elderly learners, who have to deal with a general decline of working‐memory resources, because they address both mode‐specific working‐memory stores. Sample: The sample consisted of 60 young (mean age = 15.98 years) and 60 old adults (mean age = 64.48 years). Methods: Participants of both age groups were trained in either a conventional, a unimodal, or a multimedia condition. Subsequently, they had to solve a series of test problems. Dependent variables were perceived cognitive load during the training, performance on the test, and efficiency in terms of the ratio between these two variables. Results: Results showed that for both age groups multimedia‐based worked examples were more efficient than the other training formats in that less cognitive load led to at least an equal performance level. Conclusion: Although no difference in the beneficial effect of multimedia learning was found between the age groups, multimedia‐based instructions seem promising for the elderly.     

Split‐Attention and Coherence Principles in Multimedia Instruction Can Rescue Performance for Learners with Lower Working Memory Capacity

This study examined the relation between working memory capacity (WMC) and the principles of Split‐Attention (Experiment 1) and Coherence (Experiment 2) in multimedia learning. Split‐Attention refers to reduced comprehension when learners must divide their attention between images and text, and Coherence refers to reduced comprehension when learners must process irrelevant information. In Experiment 1, those with lower WMC performed worse compared with those with higher WMC when learning from the Split‐Attention condition (audio + on‐screen text + images), but not when learning from the Complementary condition (audio + images). In Experiment 2, those with lower WMC performed worse compared with those with higher WMC when learning from the Incongruent condition (audio + irrelevant images), but not when learning from the Congruent condition (audio + relevant images). Findings reinforce the importance of pedagogically sound instructional design, as it may especially benefit those with lower WMC and equate learning across working memory abilities. Copyright © 2016 John Wiley & Sons, Ltd.     

Attention is required for maintenance of feature binding in visual working memory

Working memory and attention are intimately connected. However, understanding the relationship between the two is challenging. Currently, there is an important controversy about whether objects in working memory are maintained automatically or require resources that are also deployed for visual or auditory attention. Here we investigated the effects of loading attention resources on precision of visual working memory, specifically on correct maintenance of feature-bound objects, using a dual-task paradigm. Participants were presented with a memory array and were asked to remember either direction of motion of random dot kinematograms of different colour, or orientation of coloured bars. During the maintenance period, they performed a secondary visual or auditory task, with varying levels of load. Following a retention period, they adjusted a coloured probe to match either the motion direction or orientation of stimuli with the same colour in the memory array. This allowed us to examine the effects of an attention-demanding task performed during maintenance on precision of recall on the concurrent working memory task. Systematic increase in attention load during maintenance resulted in a significant decrease in overall working memory performance. Changes in overall performance were specifically accompanied by an increase in feature misbinding errors: erroneous reporting of nontarget motion or orientation. Thus in trials where attention resources were taxed, participants were more likely to respond with nontarget values rather than simply making random responses. Our findings suggest that resources used during attention-demanding visual or auditory tasks also contribute to maintaining feature-bound representations in visual working memory—but not necessarily other aspects of working memory.     

The role of attention in binding visual features in working memory: Evidence from cognitive ageing

Two experiments were conducted to assess the costs of attentional load during a feature (colour–shape) binding task in younger and older adults. Experiment 1 showed that a demanding backwards counting task, which draws upon central executive/general attentional resources, reduced binding to a greater extent than individual feature memory, but the effect was no greater in older than in younger adults. Experiment 2 showed that presenting memory items sequentially rather than simultaneously, such that items are required to be maintained while new representations are created, selectively affects binding performance in both age groups. Although this experiment exhibited an age-related binding deficit overall, both age groups were affected by the attention manipulation to an equal extent. While a role for attentional processes in colour–shape binding was apparent across both experiments, manipulations of attention exerted equal effects in both age groups. We therefore conclude that age-related binding deficits neither emerge nor are exacerbated under conditions of high attentional load. Implications for theories of visual working memory and cognitive ageing are discussed.     

视觉工作记忆负载对听觉偏差干扰效应的调控：来自不同外周提示线索的证据

采用线索提示和视听Oddball结合的实验设计, 通过四个实验探讨了外周视线索下视觉工作记忆负载对听觉偏差干扰效应的调控作用。结果表明：(1)操作外周视线索有效性为50%的条件下, 视觉工作记忆负载的增加使得原本抑制无关声音的过程失败, 因此出现了显著的听觉偏差干扰效应; (2)操作外周视线索有效性为80%的条件下, 发现有策略性的注意定向消耗了注意资源, 无视觉工作记忆负载的情况下同样出现听觉偏差干扰, 并且随着工作记忆负载增加偏差干扰效应增强。研究说明视觉工作记忆负载对听觉偏差干扰的调控是有条件性的, 当来自不同感觉通道的无关刺激与目标的加工过程存在重叠时, 视觉工作记忆负载的增加使得听觉偏差干扰效应增强而不是减弱。     

Improved effectiveness of cueing by self‐explanations when learning from a complex animation

A major problem in learning from instructional animations is that the complex perceptual and cognitive processing exceeds the learner's limited processing capacities. Although attention cueing might help learners in focusing on essential parts of an animation, previous studies have shown that it does not necessarily improve learning performance. This study investigated whether generating self‐explanations while studying a cued or an uncued animation might engage learners in cognitive activities necessary for learning. It was hypothesized that learning from a cued animation that reduces working memory load associated with searching for specific elements might be improved by generating self‐explanations, whereas self‐explaining with an uncued animation would have no positive effect on learning. The results confirmed the hypothesized interaction between cueing and self‐explaining. They suggest that self‐explanation enhances learning if visual cues are used to structure and highlight the essential parts of an animation. Copyright © 2010 John Wiley & Sons, Ltd.     

听觉警报失聪的认知因素

无意失聪是指当人们集中精神做事时,会忽略非预期的听觉刺激。这种现象在现实生活中普遍存在,它对航空和交通安全构成巨大威胁。研究者们发现高知觉和高认知负荷条件下因认知资源有限导致无意失聪。且随着负荷增加,被忽略的听觉刺激诱发的早期（N100）和晚期（P300）神经活动有所减弱。这表明非预期听觉刺激可以进入晚期加工阶段,但其能否被感知可能取决于与工作记忆相关的额顶网络的抑制控制程度。而工作记忆容量及其负荷与无意失聪关系的现有研究结果之间还存在不一致性。未来研究可以将其范式和理论拓展到其他安全生产领域,并运用认知训练等方式规避无意失聪,从而减少人因失误。     

Cognitive Load Theory and the Format of Instruction

Cognitive load theory suggests that effective instructional material facilitates learning by directing cognitive resources toward activities that are relevant to learning rather than toward preliminaries to learning. One example of ineffective instruction occurs if learners unnecessarily are required to mentally integrate disparate sources of mutually referring information such as separate text and diagrams. Such split-source information may generate a heavy cognitive load, because material must be mentally integrated before learning can commence. This article reports findings from six experiments testing the consequences of split-source and integrated information using electrical engineering and biology instructional materials. Experiment 1 was designed to compare conventional instructions with integrated instructions over a period of several months in an industrial training setting. The materials chosen were unintelligible without mental integration. Results favored integrated instructions throughout the 3-month study. Experiment 2 was designed to investigate the possible differences between conventional and integrated instructions in areas in which it was not essential for sources of information to be integrated to be understood. The results suggest that integrated instructions were no better than split-source information in such areas. Experiments 3, 4, and 5 indicate that the introduction of seemingly useful but nonessential explanatory material (e.g., a commentary on a diagram) could have deleterious effects even when presented in integrated format. Experiment 6 found that the need for physical integration was restored if the material was organized in such a manner that individual units could not be understood alone. In light of these results and previous findings, suggestions are made for cognitively guided instructional packages.     

Selective impairment of auditory selective attention under concurrent cognitive load

Load theory predicts that concurrent cognitive load impairs selective attention. For visual stimuli, it has been shown that this impairment can be selective: Distraction was specifically increased when the stimulus material used in the cognitive load task matches that of the selective attention task. Here, we report four experiments that demonstrate such selective load effects for auditory selective attention. The effect of two different cognitive load tasks on two different auditory Stroop tasks was examined, and selective load effects were observed: Interference in a nonverbal-auditory Stroop task was increased under concurrent nonverbal-auditory cognitive load (compared with a no-load condition), but not under concurrent verbal-auditory cognitive load. By contrast, interference in a verbal-auditory Stroop task was increased under concurrent verbal-auditory cognitive load but not under nonverbal-auditory cognitive load. This double-dissociation pattern suggests the existence of different and separable verbal and nonverbal processing resources in the auditory domain.     

Effects of working memory load on long-term word priming

Three experiments assessed the role of verbal and visuo-spatial working memory in supporting long-term repetition priming for written words. In Experiment 1, two priming tasks (word stem completion and category-exemplar production) were included with three levels of load on working memory: (1) without memory load, (2) memory load that involved storing a string of six digits, and (3) memory load that involved storing a graphic shape. Experiments 2 and 3 compared the effects of a verbal (Experiment 2) or a visual (Experiment 3) working memory load at encoding on both an implicit (word stem completion) and an explicit test (cued recall). The results show no effect of memory load in any of the implicit memory tests, suggesting that priming does not rely on working memory resources. By contrast, loading working memory at encoding causes a significant disruptive effect on the explicit memory test for words when the load is verbal but not visual.     

Temporal Grouping Effects in Immediate Recall: A Working Memory Analysis

The presence of temporal pauses during list presentation can markedly improve immediate memory for a sequence of verbal items. A series of experiments analysed this effect using Baddeley's (1986) model of working memory. Experiment 1 showed that the effect of temporal grouping on memory for visual sequences was removed by either articulatory suppression or reciting random digits. Experiment 2 indicated that effects of temporal grouping were insensitive to the word length of the items. Experiment 3 showed that articulatory suppression did not remove the temporal grouping effect for auditory lists. Experiment 4 showed that the temporal grouping effect was insensitive to the phonemic similarity of the items. The effects of concurrent articulation suggest that grouping influences the phonological loop component of working memory. However, the working memory model is insufficiently well specified to account for the insensitivity of grouping effects to word length and phonemic similarity. The main findings could be simulated by a connectionist model of the phonological loop, which invokes a context timing signal (Burgess & Hitch, 1992, in press), This assumed that pauses during list presentation affect the timing signal in a similar way to the pause before list presentation and made some novel predictions.     

Cognitive Load While Learning to Use a Computer Program

There is a growing body of evidence suggesting that many traditional instructional techniques may unnecessarily overload limited working memory and impede learning. Based on cognitive load theory, it was hypothesised that instructional design only takes on a crucial role when there is a high level of interaction between learning elements resulting in those elements having to be simultaneously held in working memory. When there is little, if any, interaction between individual learning elements, then the format of presentation should be inconsequential. These hypotheses were tested using a computer-aided design/computer-aided manufacture (CAD/CAM) package with trainees from a Sydney company. Results showed that when instructions involved high element interactivity, a self-contained manual that physically integrated disparate information and did not require the use of the computer hardware was vastly superior to instructional formats that involved continual interaction with the computer. No differences were found between instructional formats when the learning material entailed low element interactivity. Evidence that these findings were due to cognitive load rather than other factors came from secondary task analysis. In light of these and previous results, suggestions are made for cognitively guided instructional packages.