Procedural Multimedia Presentations: The Effects of Working Memory and Task Complexity on Instruction Time and Assembly Accuracy

Procedural text conveys information of a series of steps to be performed. This study examined the role of verbal and visuo‐spatial WM in comprehension and execution of assembly instructions, as a function of format (text, images, multimedia) and task complexity (three or five steps). One hundred and eight participants read and executed 27 instructions to assemble a LEGO^TM object, in single and dual task conditions. Study times and errors during assembly were measured. Participants processed faster pictorial and multimedia instructions than text instructions, and made fewer errors in the execution of multimedia instructions. Dual task affected more text or picture‐only, than multimedia presentation. A verbal secondary task caused more errors in text or picture‐only presentations, and spatial secondary task also caused interference in text‐only instructions. Overall, these results support the multimedia advantage, and the role of both verbal and visuo‐spatial WM, when understanding instructions. Copyright © 2016 John Wiley & Sons, Ltd.     

Individual differences and task format in sentence verification

This study examined the effects of task format and individual differences in cognitive ability on performance of the sentence-picture verification task. Four groups of subjects were selected on the basis of their scores on psychometric tests of verbal and spatial ability: low verbal/low spatial, high verbal/high spatial, low verbal/high spatial, and high verbal/low spatial. Subjects completed four different versions of the sentence verification task, each version differing in terms of the interval between sentence and picture presentation. The results confirmed the importance of both task format and spatial ability in determining the cognitive strategy employed by subjects in carrying out the task. The finding of principal interest, however, was that spatial ability was the most significant determinant of strategy choice, and that low spatial subjects made substantial use of linguistic strategies under task formats designed to encourage imaginal strategies.     

Using selective interference to investigate spatial memory representations

Two experiments used a selective interference procedure in an attempt to determine whether nonverbal visual stimuli were represented in memory in a verbal or spatial format. A spatial representation was clearly implicated. In both experiments, Ss were required to remember either the positions or the identities of seven target items in a 25-item array. During the retention interval for that information, Ss attempted to recognize schematic face or airplane photograph stimuli in a same-different memory task. Memory performance on one or both tasks was greatly impaired when the recall task involved position or spatial information, but was either much less or not at all affected by an identity or verbal information recall task. Because of the selective nature of the interference and on the basis of certain correlational evidence, the experimental results were also interpreted as providing support for the notion that verbal and spatial information are stored and processed in separate information-processing systems.     

视觉与言语工作记忆表征对视觉注意的引导

通过三个实验,探讨不同时间进程下视觉和言语工作记忆表征对视觉注意的引导。实验1采用注意引导与注意探测任务同时呈现的范式,探讨不同记忆项与注意项时间间隔(SOA)条件下言语工作记忆表征对视觉注意的引导。结果发现,当SOA较短时言语工作记忆表征对视觉注意有引导效应,而SOA较长时未观察到引导效应;实验2、3采用注意引导任务与注意探测任务分离范式,分别探讨在不同SOA条件下言语和视觉工作记忆表征对视觉注意的引导。结果发现,不同SOA条件下言语和视觉工作记忆表征均能引导视觉注意。研究表明,不同实验范式对言语工作记忆表征对视觉注意的引导有明显影响,视觉比言语工作记忆表征能在更广泛条件下引导视觉注意。     

The role of information presentation formats in belief updating

The effects of the format by which information is presented on the cognitive processes of belief updating were investigated in the present research. Because of the differences in the affordance of verbal vs. numerical information, it is predicted that the belief updating processes involved in processing verbal and numerical information would be different. Specifically, the additive rule is used to combine information using verbal formats, while the averaging rule is used to combine information using numerical formats. Two experiments were conducted to test these hypotheses. Experiment 1 tested the belief updating process in the positive direction, and Experiment 2 tested the process in the negative direction. Two independent variables were manipulated: information presentation format (verbal vs numerical) and presentation order (strong–weak vs weak–strong). The participants were asked to adjust their purchase likelihood of a consumer product based on the sequential presentations of two experts' opinions. These two opinions varied in their formats (verbal vs. numerical) and strengths (strong vs weak). The two opinions were presented in either the strong–weak order or the weak–strong order. Participants were instructed to first anchor their purchase likelihood at 50%, and then adjust the purchase likelihood, first based on the first expert's opinion, and second based on both experts' opinions. In both experiments the hypotheses that participants employed an additive rule to integrate verbal information and an averaging rule to integrate numerical information were supported.     

The role of working memory components and visuospatial abilities in route learning within a virtual environment

An experiment was run to complete our understanding of the involvement of working memory (WM) components in the construction of a spatial model from visual input, considering some of the visuospatial abilities known to modulate performance. In addition, to allow for consideration of the flexibility of the spatial representation, routes in a virtual environment were presented with a route perspective, and tests were presented with route and survey perspectives. The results indicate that the verbal and spatial WM are only involved in the memorisation of certain types of information, and that their involvement depends on the change of perspective necessitated by the task. Thus, even when the learning material and the tests used to assess performance are only visual, a verbal recoding of some information is necessary. Moreover, individual differences modulate the involvement of WM; an individual with higher visuospatial capacities uses more spatial WM than an individual with lower spatial capacities.     

Visuospatial memory and phonological loop in learning from multimedia

The dual‐task paradigm was used to show how visuospatial working memory and the phonological loop are involved in processing scientific texts and illustrations presented via computer. In experiment 1, two presentation formats were compared: text‐only and text‐with‐illustrations. With a concurrent tapping task, the beneficial effect of illustrations disappeared, while a concurrent articulatory task impaired performance similarly in both presentation formats. An analysis of individual differences revealed that this pattern of results was present in high, but not low spatial span subjects. These results support the selective involvement of visuospatial working memory in processing illustrated texts. In Experiment 2, the text‐only presentation format was compared to an illustrations‐only format. The concurrent articulatory task selectively impaired text‐only processing, compared with processing illustrations‐only. In addition, this pattern of results was found for high, but not low digit span subjects. These results suggest that individual differences define the extent to which the two subsystems of working memory are involved in learning from multimedia. These two subsystems would be mainly involved in the maintenance of a visual trace of illustrations and of a verbatim representation of linguistic information respectively, these representations being the basis for higher‐level comprehension processes. Copyright © 2002 John Wiley & Sons, Ltd.     

Development of memory for pattern and path: Further evidence for the fractionation of visuo-spatial memory

Evidence from a number of sources now suggests that the visuo-spatial sketchpad (VSSP) of working memory may be composed of two subsystems: one for maintaining visual information and the other for spatial information. In this paper we present three experiments that examine this fractionation using a developmental approach. In Experiment 1, 5-, 8-, and 10-year old children were presented with a visuo-spatial working memory task (the matrices task) with two presentation formats (static and dynamic). A developmental dissociation in performance was found for the static and dynamic conditions of both tasks, suggesting that the activation of separable subsystems of the VSSP is dependent upon a static/dynamic distinction in information content rather than a visual/spatial one. A highly similar pattern of performance was found for a mazes task with static and dynamic formats. However, one strategic activity, the use of simple verbal recoding, may also have been responsible for the observed pattern of performance in the matrices task. In Experiments 2 and 3 this was investigated using concurrent articulatory suppression. No evidence to support this notion was found, and it is therefore proposed that static and dynamic visuo-spatial information is maintained in working memory by separable subcomponents of the VSSP.     

Lateralization of spatial categories: A comparison of verbal and visuospatial categorical relations

Many reports show that spatial relations between and within objects show differences in hemispheric lateralization. Coordinate, metric relations concerning distances are processed with a right-hemisphere advantage, whereas a left-hemisphere advantage is thought to be related to categorical, abstract relations (Kosslyn, 1987). Kemmerer and Tranel (2000) argued that the left-hemisphere advantage for categorical processing might apply only for verbal spatial categories, however, whereas a right-hemisphere advantage is related to visuospatial categories. To test this idea, we examined categorical processing for stimuli in both verbal and visuospatial formats, with a visual half-field, match-to-sample design. In Experiment 1, we manipulated the format of the second stimulus to compare response patterns for both verbal and visuospatial stimuli. In Experiment 2, we varied the expectancy of the format of the second stimulus, allowing for an assessment of strategy use. The results showed that a left-hemisphere advantage was related to verbal stimulus format only, but not in all conditions. A right-hemisphere advantage was found only with a visuospatial expectancy, visuospatial format, and brief interval. The theory we present to explain these results proposes that the lateralization related to basic categorical processing can be strongly influenced by verbal characteristics and, to some extent, by additional coordinate processing. The lateralization measured in such cases does not represent lateralization related purely to categorical processing, but to these additional effects as well. This stresses the importance of careful task and stimulus design when examining categorical processing in order to reduce the influence of those additional processes.     

Understanding ‘Unlikely (20% Likelihood)’ or ‘20% Likelihood (Unlikely)’ Outcomes: The Robustness of the Extremity Effect

Calls to communicate uncertainty using mixed, verbal‐numerical formats (‘unlikely [0–33%]’) have stemmed from research comparing mixed with solely verbal communications. Research using the new ‘which outcome’ approach to investigate understanding of verbal probability expressions suggests, however, that mixed formats might convey disadvantages compared with purely numerical communications. When asked to indicate an outcome that is ‘unlikely’, participants have been shown to often indicate outcomes with a value exceeding the maximum value shown, equivalent to a 0% probability —an ‘extremity effect’. Recognising the potential consequences of communication recipients expecting an ‘unlikely’ event to never occur, we extend the ‘which outcome’ work across four experiments, using verbal, numerical, and verbal‐numerical communication formats, as well as a previously unconsidered numerical‐verbal format. We examine how robust the effect is in the context of consequential outcomes and over non‐normal distributions. We also investigate whether participants are aware of the inconsistency in their responses from a traditional ‘how likely’ and ‘which outcome’ task. We replicate and extend previous findings, with preference for extreme outcomes (including above maximum values) observed in both verbal and verbal‐numerical formats. Our results suggest caution in blanket usage of recently recommended verbal‐numerical formats for the communication of uncertainty. Copyright © 2018 John Wiley & Sons, Ltd.     

Cross-modal priming and explicit memory in patients with verbal production deficits

Implicit memory is often thought to reflect an influence of past experience on perceptual processes, yet priming effects are found when the perceptual format of stimuli changes between study and test episodes. Such cross-modal priming effects have been hypothesized to depend upon stimulus recoding processes whereby a stimulus presented in one modality is converted to other perceptual formats. The present research examined recoding accounts of cross-modal priming by testing patients with verbal production deficits that presumably impair the conversion of visual words into auditory/phonological forms. The patients showed normal priming in a visual stem completion task following visual study (Experiment 1), but showed impairments following auditory study in both implicit (Experiment 2) and explicit (Experiment 3) stem completion. The results are consistent with the hypothesis that verbal production processes contribute to the recoding of visual stimuli and support cross-modal priming. The results also indicate that shared processes contribute to both explicit memory and cross-modal implicit memory.     

Verbal, visual, and spatial working memory in written language production

College students wrote definitions of either abstract or concrete nouns in longhand while performing a concurrent working memory (WM) task. They detected either a verbal (syllable), visual (shape), or spatial (location) stimulus and decided whether it matched the last one presented 15-45s earlier. Writing definitions of both noun types elevated the response time to verbal targets above baseline. Such interference was observed for visual targets only when defining concrete nouns and was eliminated entirely with spatial targets. The interference effect for verbal targets was the same whether they were read or heard, implicating phonological storage. The findings suggest that language production requires phonological or verbal WM. Visual WM is selectively engaged when imaging the referents of concrete nouns.     

Evidence of separable spatial representations in a virtual navigation task

Three experiments investigated spatial orientation in a virtual navigation task. Subjects had to adjust a homing vector indicating their end position relative to the origin of the path. It was demonstrated that sparse visual flow was sufficient for accurate path integration. Moreover, subjects were found to prefer a distinct egocentric or allocentric reference frame to solve the task. "Turners" reacted as if they had taken on the new orientation during turns of the path by mentally rotating their sagittal axis (egocentric frame). "Nonturners," by contrast, tracked the new orientation without adopting it (allocentric frame). When instructed to use their nonpreferred reference frame, both groups displayed no decline in response accuracy relative to their preferred frame; even when presented with reaction formats based on either ego or allocentric coordinates, with format unpredictable on a trial, both groups responded highly accurately. These findings support the assumption of coexisting spatial representations during navigation.     

A secondary task is not always costly: Context‐based guidance of visual search survives interference from a demanding working memory task

Repeatedly encountering a visual search display with the target located at a fixed position relative to the distractors facilitates target detection, relative to novel displays – which is attributed to search guidance by (acquired) long‐term memory (LTM) of the distractor ‘context’ of the target. Previous research has shown that this ‘contextual cueing’ effect is severely impeded during learning when participants have to perform a demanding spatial working memory (WM) task concurrently with the search task, though it does become manifest when the WM task is removed. This has led to the proposal that search guidance by LT context memories critically depends on spatial WM to become ‘expressed’ in behaviour. On this background, this study, of two experiments, asked: (1) Would contextual cueing eventually emerge under dual‐task learning conditions if the practice on the task(s) is extended beyond the short training implemented in previous studies? and given sufficient practice, (2) Would performing the search under dual‐task conditions actually lead to an increased cueing effect compared to performing the visual search task alone? The answer is affirmative to both questions. In particular, Experiment 1 showed that a robust contextual cueing effect emerges within 360–720 dual‐task trials as compared to some 240 single‐task trials. Further, Experiment 2 showed that when dual‐ and single‐task conditions are performed in alternating trials blocks, the cueing effect for the very same set of repeated displays is significantly larger in dual‐task blocks than in single‐task blocks. This pattern of effects suggests that dual‐task practice eventually leads to direct, or ‘automatic’, guidance of visual search by learnt spatial LTM representations, bypassing WM processes. These processes are normally engaged in single‐task performance might actually interfere with direct LTM‐based search guidance.     

Recollecting positive and negative autobiographical memories disrupts working memory

The present article reports two experiments examining the impact of recollecting emotionally valenced autobiographical memories on subsequent working memory (WM) task performance. Experiment 1 found that negatively valenced recollection significantly disrupted performance on a supra-span spatial WM task. Experiment 2 replicated and extended these findings to a verbal WM task (digit recall), and found that both negative and positive autobiographical recollections had a detrimental effect on verbal WM. In addition, we observed that these disruptive effects were more apparent on early trials, immediately following autobiographical recollection. Overall, these findings show that both positive and negative affect can disrupt WM when the mood-eliciting context is based on autobiographical memories. Furthermore, these results indicate that the emotional disruption of WM can take place across different modalities of WM (verbal and visuo-spatial).     

The influence of working memory load on the Simon effect

The contribution of spatial and verbal working memory (WM) to the processing of stimulus location in the Simon task was examined in two experiments. Subjects performed the Simon task in single- and dual-task conditions. In the dual-task conditions, individuals first encoded the locations of four boxes/pseudocharacters or the identity of seven characters in memory, then responded to the color of a Simon stimulus, and finally received a test for the WM set. The Simon effect was insensitive to spatial WM load and was abolished by verbal WM load. These data reveal a role of verbal WM in the processing of spatial location in the Simon task. In addition, the data are inconsistent with dual-route models of the Simon effect and may support response discrimination accounts of the mechanisms underlying the processing of spatial location in the Simon task.     

Attending to visual or auditory verbal information while performing other concurrent tasks

Investigation of the effect that a word recognition task has on concurrent nonverbal tasks showed (a) auditory verbal messages affected visual tracking performance but not the detection of brief light flashes in the visual periphery, (b) greater impairment, both of tracking and light detections, when verbal messages were visual rather than auditory. With a kinaesthetic tracking task, errors increased significantly during auditory messages but were even greater during visual messages. There was no interaction between the modality of tracking error feedback (auditory or visual) and the modality of the verbal message. Nor was the decrement from visual messages reduced by changing the presentation format. It is suggested that different temporal characteristics of visual and auditory information affect the attentional demands of verbal messages.