Presentation format effects in working memory: The role of attention

Four experiments are reported in which participants attempted to remember three or six concrete nouns, presented as pictures, spoken words, or printed words, while also verifying the accuracy of sentences. Hypotheses meant to explain the higher recall of pictures and spoken words over printed words were tested. Increasing the difficulty and changing the type of processing task from arithmetic to a visual/spatial reasoning task did not influence recall. An examination of long-term modality effects showed that those effects were not sufficient to explain the superior performance with spoken words and pictures. Only when we manipulated the allocation of attention to the items in the storage task by requiring the participants to articulate the items and by presenting the stimulus items under a degraded condition were we able to reduce or remove the effect of presentation format. The findings suggest that the better recall of pictures and spoken words over printed words result from the fact that under normal presentation conditions, printed words receive less processing attention than pictures and spoken words do.     

Reduction and elimination of format effects on recall

Two experiments investigated whether the recall advantage of pictures and spoken words over printed words in working memory (Foos & Goolkasian, 2005; Goolkasian & Foos, 2002) could be reduced by manipulating letter case and sequential versus simultaneous presentation. Participants were required to remember 3 or 6 items presented in varied presentation formats while verifying the accuracy of a sentence. Presenting words in alternating uppercase and lowercase improved recall, and presenting words simultaneously rather than successively removed the effect of presentation format. The findings suggest that when forcing participants to pay attention to printed words you can make them more memorable and thereby diminish or remove any disadvantage in the recall of printed words in comparison with pictures and spoken words.     

Presentation format effects in a levels-of-processing task

Three experiments were conducted to examine better performance in long-term memory when stimulus items are pictures or spoken words compared to printed words. Hypotheses regarding the allocation of attention to printed words, the semantic link between pictures and processing, and a rich long-term representation for pictures were tested. Using levels-of-processing tasks eliminated format effects when no memory test was expected and processing was deep (El), and when study and test formats did not match (E3). Pictures produced superior performance when a memory test was expected (El & 2) and when study and test formats were the same (E3). Results of all experiments support the attenuation of attention model and that picture superiority is due to a more direct access to semantic processing and a richer visual code. General principles to guide the processing of stimulus information are discussed.     

Presentation format and its effect on working memory

In three experiments, we examined the separate cognitive demands of processing and storage in working memory and looked at how effective the coordination was when items for storage varied in format/modality. A sentence verification task involving arithmetic facts was combined with a span task involving two to six items presented in picture, printed word, or spoken word format. The first two experiments were the same, except for the added requirement of articulation of the math sentence in Experiment 2. Experiment 3 varied the length of the span item and compared recall with recognition performance. The results showed that both spoken words and picture produced superior recall and recognition, as compared with printed words, and are consistent with Baddeley and Logie's (1999) and Mayer's (2001) models of working memory. Also, the differences in processing performance across spans varied with the difficulty of the task but showed the strongest support for the resource allocation model (Foos 1995).     

Pictures, words, and sounds: from which format are we best able to reason?

The effect of presentation format on reasoning was studied with a sentence verification task. Background information was presented in single-format and combined conditions that included pictured, printed, or spoken versions of the stimulus items. In Experiment 1, a test sentence appeared together with the background at varied stimulus onset asynchronies, to study how format influences the acquisition of the stimulus information. In Experiments 2 and 3, however, the test sentence followed the presentation of the background, to test the effect of format on memory. Reaction time responses to the test sentences showed a consistent picture advantage. However, when participants responded to materials stored in memory, both pictured and spoken formats provided quicker responses in comparison to printed words, and the format difference was smaller than when materials were readily available on the screen. Multimedia presentations, when compared with single-format conditions, did not provide additional benefits.     

Pictures,Words, and Sounds: From Which Format Are We Best Able to Reason?

The effect of presentation format on reasoning was studied with a sentence verification task. Background information was presented in single-format and combined conditions that included pictured, printed, or spoken versions of the stimulus items. In Experiment 1, a test sentence appeared together with the background at varied stimulus onset asynchronies, to study how format influences the acquisition of the stimulus information. In Experiments 2 and 3, however, the test sentence followed the presentation of the background, to test the effect of format on memory. Reaction time responses to the test sentences showed a consistent picture advantage. However, when participants responded to materials stored in memory, both pictured and spoken formats provided quicker responses in comparison to printed words, and the format difference was smaller than when materials were readily available on the screen. Multimedia presentations, when compared with single-format conditions, did not provide additional benefits.     

Selective disruption of hypermnesia for pictures and words

In this study, we investigated the effects of various interpolated tasks on hypermnesia (improved recall across repeated tests) for pictures and words. In five experiments, subjects studied either pictures or words and then completed two free-recall tests, with varying activities interpolated between the tests. The tasks performed between tests were varied to test several hypotheses concerning the possible factor(s) responsible for disruption of the hypermnesic effect. In each experiment, hypermnesia was obtained in a control condition in which there was no interpolated task between tests. The remaining conditions showed that the effect of the interpolated tasks was related to the overlap of the cognitive processes involved in encoding the target items and performing the interpolated tasks. When pictures were used as the target items, no hypermnesia was obtained when subjects engaged in imaginal processing interpolated tasks, even when these tasks involved materials that were very distinct from the target items. When words were used as the target items, no hypermnesia was obtained when the interpolated tasks required verbal/linguistic processing, even when the items used in these tasks were auditorily presented. The results are discussed in terms of a strength-based model of associative memory.     

Sex differences in memory estimates for pictures and words

Memory performance estimates of men and women before and after a recall test were investigated. College students (17 men and 20 women), all juniors, participated in a memory task involving the recall of 80 stimuli (40 pictures and 40 words). Before and after the task they were asked to provide estimates of their pre- and postrecall performance. Although no sex differences were found for total correct recall, recall for pictures, and recall for words, or in the estimates of memory performance before the recall task, there were significant differences after the test: women underestimated their performance on the words and men underestimated their performance on the picture items.     

Pictures and words in visual search

Ss in three experiments searched through an array of pictures or words for a target item that had been presented as a picture or a word. In Experiments I and II, the pictures were line drawings of familiar objects and the words were their printed labels; in Experiment III, the stimuli were photographs of the faces of famous people and their corresponding printed names. Search times in Experiments I and II were consistently faster when the array items were pictures than when they were words, regardless of the mode of the target items. Search was also faster with pictures than with words as targets when the search array also consisted of pictures, but target mode had no consistent effect with words as array items. Experiment III yielded a completely different pattern of results: Search time with names as targets and faces as search array items was significantly slower than in the other three conditions, which did not differ from each other. Considered in relation to several theories, the results are most consistent with a dual-coding interpretation. That is, items that are cognitively represented both verbally and as nonverbal images can be searched and compared in either mode, depending on the demands of the task. The mode actually used depends on whether the search must be conducted through an array of pictures or words.     

Objects,pictures and words. Effects of representational format on four-year-olds’ quantity knowledge / Objetos,imágenes y palabras. Efectos del formato representacional en el conocimiento de la cantidad a los cuatro años

Abstract

This study explores whether and how different external representations of quantity influence the performance of four-year-old children when constructing sets of a given cardinal number. Three representations of quantity were presented (objects, pictures and number words) in an adaptation of the ‘Give-a-number’ task, in sets that contained 2–6 items. The results show that the use of pictures facilitated children’s performance with quantities of three and four. These results are discussed by analysing the cognitive demands of representational formats, in terms of dual representation and iconicity, and according to the magnitude of the quantities involved.     

The low-frequency encoding disadvantage: Word frequency affects processing demands

Low-frequency words produce more hits and fewer false alarms than high-frequency words in a recognition task. The low-frequency hit rate advantage has sometimes been attributed to processes that operate during the recognition test (e.g., L. M. Reder et al., 2000). When tasks other than recognition, such as recall, cued recall, or associative recognition, are used, the effects seem to contradict a low-frequency advantage in memory. Four experiments are presented to support the claim that in addition to the advantage of low-frequency words at retrieval, there is a low-frequency disadvantage during encoding. That is, low-frequency words require more processing resources to be encoded episodically than high-frequency words. Under encoding conditions in which processing resources are limited, low-frequency words show a larger decrement in recognition than high-frequency words. Also, studying items (pictures and words of varying frequencies) along with low-frequency words reduces performance for those stimuli.     

Recombinative reading derived from pseudoword instruction in a miniature linguistic system

A miniature linguistic system was used to study acquisition of recombinative symbolic behavior. Three studies evaluated the teaching conditions of conditional discriminations with printed and spoken pseudowords that could potentially generate recombinative reading. Fifty-four college students across all studies learned to match 12 printed pseudowords to 12 spoken pseudowords. Some also matched pictures to the same spoken words. Each two-syllable pseudoword was formed by symbols from an arbitrarily created alphabet composed of four vowels and four consonants. Letters had univocal correspondence with phonemes. Recombinative receptive reading, comprehensive reading, and textual responding to pseudowords were periodically assessed. Experiment 1 (n = 20) showed that recombinative reading increased as the number of trained words composed of the same symbols increased. Experiment 2 (n = 14) showed that overtraining the same two words did not produce recombinative reading for most participants. Experiment 3 (n = 20), in which training with pictures was omitted, showed that elemental control by within-syllable units can develop even when the trained pseudowords are meaningless (not related to pictures). The present results support the utility of the miniature linguistic system methodology for identifying and controlling environmental determinants of rudimentary reading skills.     

Sex differences in memory estimates for pictures and words with multiple recall trials

Undergraduate students (23 men and 23 women) provided memory performance estimates before and after each of three recall trials involving 80 stimuli (40 pictures and 40 words). No sex differences were found across trials for the total recall of items or for the recall of pictures and words separately. A significant increase in recall for pictures (not words) was found for both sexes across trials. The previous results of Ionescu were replicated on the first and second recall trials: men underestimated their performance on the pictures and women underestimated their performance on the word items. These differences in postrecall estimates were not found after the third recall trial: men and women alike underestimated their performance on both the picture and word items. The disappearance of item-specific sex differences in postrecall estimates for the third recall trial does not imply that men and women become more accurate at estimating their actual performance with multiple recall trials.     

Effects of talker variability on recall of spoken word lists

Three experiments were conducted to investigate recall of lists of words containing items spoken by either a single talker or by different talkers. In each experiment, recall of early list items was better for lists spoken by a single talker than for lists of the same words spoken by different talkers. The use of a memory preload procedure demonstrated that recall of visually presented preload digits was superior when the words in a subsequent list were spoken by a single talker than by different talkers. In addition, a retroactive interference task demonstrated that the effects of talker variability on the recall of early list items were not due to use of talker-specific acoustic cues in working memory at the time of recall. Taken together, the results suggest that word lists produced by different talkers require more processing resources in working memory than do lists produced by a single talker. The findings are discussed in terms of the role that active rehearsal plays in the transfer of spoken items into long-term memory and the factors that may affect the efficiency of rehearsal.     

Hypermnesia for pictures: Incremental memory for pictures but not words in multiple recall trials

Two studies are reported on multiple forced recall following a single visual presentation of a sequence of pictures or words. In both experiments, a hypermnesic memory function (in which performance improved with repeated recall) was obtained for pictures, while a flat, nonincremental function was obtained for words. Interpolation of intervals of thinking between recall trials further enhanced hypermnesia for pictorial items. Retrieval, whether overt (recall trials) or covert (thinking), apparently produces increased net recovery of pictures but not words.     

Sex differences in memory estimates,revisited

207 undergraduate students (95 men and 112 women) representing all four years in college provided estimates of memory performance before and after a recall task involving 80 stimuli (41) pictures and 40 words). The study was intended as a replication of the work of Ionescu in 2000 wherein men underestimated their performance on the picture items and women underestimated their performance on the words. No sex differences were found for correct recall totals, recall for pictures, recall for words, and total prerecall performance estimates. Although both men and women underestimated their pre- and postrecall performance, women underestimated their postrecall performance more than men. More importantly, men underestimated their performance on recall of pictures, whereas women underestimated their performance on the word items, thereby validating prior results with a larger sample. The possible bases for this phenomenon are still not clear.     

Interference of spoken word recognition through phonological priming from visual objects and printed words

Three cross-modal priming experiments examined the influence of preexposure to pictures and printed words on the speed of spoken word recognition. Targets for auditory lexical decision were spoken Dutch words and nonwords, presented in isolation (Experiments 1 and 2) or after a short phrase (Experiment 3). Auditory stimuli were preceded by primes, which were pictures (Experiments 1 and 3) or those pictures’ printed names (Experiment 2). Prime–target pairs were phonologically onset related (e.g., pijl–pijn, arrow–pain), were from the same semantic category (e.g., pijl–zwaard, arrow–sword), or were unrelated on both dimensions. Phonological interference and semantic facilitation were observed in all experiments. Priming magnitude was similar for pictures and printed words and did not vary with picture viewing time or number of pictures in the display (either one or four). These effects arose even though participants were not explicitly instructed to name the pictures and where strategic naming would interfere with lexical decision making. This suggests that, by default, processing of related pictures and printed words influences how quickly we recognize spoken words.     

Tactile Short-term Memory Revisited

In spite of a large body of empirical research demonstrating the importance of multisensory integration in cognition, there is still little research about multimodal encoding and maintenance effects in working memory. In this study we investigated multimodal encoding in working memory by means of an immediate serial recall task with different modality and format conditions. In a first non-verbal condition participants were presented with sequences of non-verbal inputs representing familiar (concrete) objects, either in visual, auditory or audio-visual formats. In a second verbal condition participants were presented with written, spoken, or bimodally presented words denoting the same objects represented by pictures or sounds in the non-verbal condition. The effects of articulatory suppression were assessed in both conditions. We found a bimodal superiority effect on memory span with non-verbal material, and a larger span with auditory (or bimodal) versus visual presentation with verbal material, with a significant effect of articulatory suppression in the two conditions.     

Learning lyrics: To sing or not to sing?

According to common practice and oral tradition, learning verbal materials through song should facilitate word recall. In the present study, we provide evidence against this belief. In Experiment 1, 36 university students, half of them musicians, learned an unfamiliar song in three conditions. In the sung-sung condition, the song to be learned was sung, and the response was sung too. In the sung-spoken condition, the response was spoken. In the divided-spoken condition, the presented lyrics (accompanied by music) and the response were both spoken. Superior word recall in the sung-sung condition was predicted. However, fewer words were recalled when singing than when speaking. Furthermore, the mode of presentation, whether sung or spoken, had no influence on lyric recall, in either short- or long-term recall. In Experiment 2, singing was assessed with and without words. Altogether, the results indicate that the text and the melody of a song have separate representations in memory, making singing a dual task to perform, at least in the first steps of learning. Interestingly, musical training had little impact on performance, suggesting that vocal learning is a basic and widespread skill.     

Short-term retention of pictures and words: evidence for dual coding systems.

The recall of picture and word triads was examined in three experiments that manipulated the type of distraction in a Brown-Peterson short-term retention task. In all three experiments recall of pictures was superior to words under auditory distraction conditions. Visual distraction produced high performance levels with both types of stimuli, whereas combined auditory and visual distraction significantly reduced picture recall without further affecting word recall. The results were interpreted in terms of the dual coding hypothesis and indicated that pictures are encoded into separate visual and acoustic processing systems while words are primarily acoustically encoded.