Short-term memory as a function of input modality

The two experiments reported are concerned with short-term memory for digit lists simultaneously presented both auditorily and visually. Results showed (1) that interpolated written and verbal recall differentially affect retention depending on whether the to-be-recalled list was presented auditorily or visually. (2) That input modality appears to be far more important for recall than was directing subjects' attention to a list during input, when that list might or might not have been subsequently required for recall. The results suggest that short-term storage is modality specific. In this case, Broadbent's P and S mechanisms do not adequately describe what happens during simultaneous visual and auditory presentation. Nor would Sperling's suggestion of a final auditory store appear to be supported.     

Abstract versus modality-specific memory representations in processing auditory and visual speech

Serial recall of lip-read, auditory, and audiovisual memory lists with and without a verbal suffix was examined. Recency effects were the same in the three presentation modalities. The disrupting effect of a suffix was largest when it was presented in the same modality as the list items. The results suggest that abstract linguistic as well as modality-specific codes play a role in memory for auditory and visual speech.     

Stimulus modality and working memory performance in Greek children with reading disabilities: Additional evidence for the pictorial superiority hypothesis

This study investigated the effects of stimulus presentation modality on working memory performance in children with reading disabilities (RD) and in typically developing children (TDC), all native speakers of Greek. It was hypothesized that the visual presentation of common objects would result in improved learning and recall performance as compared to the auditory presentation of stimuli. Twenty children, ages 10–12, diagnosed with RD were matched to 20 TDC age peers. The experimental tasks implemented a multitrial verbal learning paradigm incorporating three modalities: auditory, visual, and auditory plus visual. Significant group differences were noted on language, verbal and nonverbal memory, and measures of executive abilities. A mixed-model MANOVA indicated that children with RD had a slower learning curve and recalled fewer words than TDC across experimental modalities. Both groups of participants benefited from the visual presentation of objects; however, children with RD showed the greatest gains during this condition. In conclusion, working memory for common verbal items is impaired in children with RD; however, performance can be facilitated, and learning efficiency maximized, when information is presented visually. The results provide further evidence for the pictorial superiority hypothesis and the theory that pictorial presentation of verbal stimuli is adequate for dual coding.     

Neuropsychological evidence of visual storage in short-term memory tasks

Auditory and visual presentation of verbal material were compared in a single patient having an auditory verbal S.T.M. deficit. A Peterson short-term forgetting experiment and an immediate memory span task are reported. Striking differences in performance related to modality of input were obtained. Auditory short-term forgetting was more rapid, whereas with visual presentation short-term decay functions were relatively normal. With visual presentation there was no evidence of acoustic confusion errors but there was some evidence of visual confusion errors. The findings are interpreted in terms of a separate post-perceptual visual S.T.M. system.     

Modality effects in retrieval of information from short-term memory

Retrieval from short-term memory was studied in a reaction time task, Ss decided whether or not a target item was present in a memorized list. When modality of presentation was manipulated, fastest reaction times resulted if the target item and all memory items were presented auditorily. All conditions involving pure visual presentation or mixed auditory-visual presentation resulted in reaction times which were longer than those for the pure auditory conditions out which did not differ from each other. This suggests that retrieval can be trom either a precategorical auditory memory or a postcategorical memory which does not preserve the sensory features of the memorized material.     

跨视听通道的相继错误记忆效应

本研究以12名大学生为被试采用DRM范式,考察了视觉与听觉通道在编码阶段的错误记忆ERP效应,以从更深层面认识错误记忆的内在加工机制。研究结果表明,在编码阶段,在300-500ms及500-700ms的时间窗视觉通道均存在相继错误记忆效应（DIM）,比听觉通道表现出较强的DIM效应,这不仅说明视觉通道与听觉通道在编码阶段具有不同的脑机制,而且表明DIM效应与学习项目的语义加工有着密切的关系。     

Modality effects in delayed free recall and recognition: Visual is better than auditory

During presentation of auditory and visual lists of words, different groups of subjects generated words that either rhymed with the presented words or that were associates. Immediately after list presentation, subjects recalled either the presented or the generated words. After presentation and test of all lists, a final free recall test and a recognition test were given. Visual presentation generally produced higher recall and recognition than did auditory presentation for both encoding conditions. The results are not consistent with explanations of modality effects in terms of echoic memory or greater temporal distinctiveness of auditory items. The results are more in line with the separate-streams hypothesis, which argues for different kinds of input processing for auditory and visual items.     

Paradoxical effects of presentation modality on false memory.

Two experiments are reported in which the effects of presentation modality on false memory in recall and recognition are studied. False recall and recognition of critical targets are lower for non-presented items related to a study list when that study list is presented visually than when presented auditorily. This pattern of low levels of false memory for critical targets holds even when participants read the visually presented study items aloud. These results suggest that recollection of visual detail plays a role in the prevention of false memory. However, both the hit rates (true memory) and the false-alarm rates to weakly related distractors (non-critical targets) were higher for visual presentation than for auditory presentation, suggesting that more than one mechanism may underlie false recognition.     

Auditory and visual spatial working memory

A series of experiments compared short-term memory for object locations in the auditory and visual modalities. The stimulus materials consisted of sounds and pictures presented at different locations in space. Items were presented in pure- or mixed-modality lists of increasing length. At test, participants responded to renewed presentation of the objects by indicating their original position. If two independent modality-specific and resource-limited short-term memories support the remembering of locations, memory performance should be higher in the mixed-modality than in the pure-modality condition. Yet, memory performance was the same for items in both types of list. In addition, responses to the memory load manipulation in both modalities showed very similar declines in performance. The results are interpreted in terms of object files binding object and location information in episodic working memory, independently of the input modality.     

Attention and the order of items in short-term visual memory

Summary Subjects first detected a target presented at the left of fixation, and then attempted to report, in order, the first four items (numerals or shapes) in a stream of items presented to the right of fixation. At comparably difficult presentation rates, 10/s for numerals and 5/s for shapes, reports showed a mixture of correctly ordered items with items reported in a direction opposite to their order of presentation. Reports fit a three-parameter attention-gating model (AGM), which assumes that (1) after target detection, an attention gate opens briefly to allow items to enter visual short-term memory (VSTM), and (2) report order is determined by the attention each item receives in VSTM. Items presented either early or late in the stream tend to receive less attention and are thus reported as later than more central items. The fit to the AGM for both numerals and unlabelled shapes provides evidence that reports reflect order in short-term visual (rather than verbal) memory.This research was supported by USAF grant no. 84-ML-044     

Memory for modality of presentation: Within-modality discrimination

Two experiments demonstrated that Ss are capable of making within-modality memory discriminations in both visual and auditory modalities. In Experiment I Ss studied mixed lists of pictures and labels representing common objects and were subsequently required to judge whether the original presentation was pictorial or verbal The high level of performance achieved on this task was unaffected by degree of categorical relatedness of items within method of presentation or by instructions to produce visual images when items were presented verbally. In Experiment II Ss demonstrated the ability to remember whether a sentence was originally presented by a male or a female speaker. Some strategies by which within-modality discrimination in memory might be accomplished are discussed.     

"The W and M are mixing me up": use of a visual code in verbal short-term memory tasks

When normal participants are presented with written verbal short-term memory tasks (e.g., remembering a set of letters for immediate spoken recall) there is evidence to suggest that the information is re-coded into phonological form. This paper presents a single case study of MJK whose reading follows the pattern of phonological dyslexia. In short-term memory tasks MJK does not phonologically re-code written materials but, it is argued, uses a visual code. There are three main lines of evidence for this, (a) MJK tends to substitute visually similar items for one another, (b) her performance is better with visual than auditory stimuli, and (c) she is able to remember numbers (e.g., 8) better than written words (e.g., eight). Investigation of short-term memory in others with phonological dyslexia is warranted.     

Accuracy of temporal coding: Auditory-visual comparisons

Three experiments were designed to decide whether temporal information is coded more accurately for intervals defined by auditory events or for those defined by visual events. In the first experiment, the irregular-list technique was used, in which a short list of items was presented, the items all separated by different interstimulus intervals. Following presentation, the subject was given three items from the list, in their correct serial order, and was asked to judge the relative interstimulus intervals. Performance was indistinguishable whether the items were presented auditorily or visually. In the second experiment, two unfilled intervals were defined by three nonverbal signals in either the auditory or the visual modality. After delays of 0, 9, or 18 sec (the latter two filled with distractor activity), the subjects were directed to make a verbal estimate of the length of one of the two intervals, which ranged from 1 to 4 sec and from 10 to 13 sec. Again, performance was not dependent on the modality of the time markers. The results of Experiment 3, which was procedurally similar to Experiment 2 but with filled rather than empty intervals, showed significant modality differences in one measure only. Within the range of intervals employed in the present study, our results provide, at best, only modest support for theories that predict more accurate temporal coding in memory for auditory, rather than visual, stimulus presentation.     

A new model of verbal short-term memory

Two experiments tested a neo-Piagetian model of verbal short-term memory and compared it with the articulatory loop model. Experiment 1 (n = 113, age range 9-11) tested word span for 2-, 3-, and 4-syllable words, with both visual and auditory presentation. Experiment 2 (with the same participants) tested recall of visually presented supraspan lists. Measures of M capacity (as conceived in Pascual-Leone's neo-Piagetian theory) and articulation rate were also used. The proposed model can account for the effects of M capacity, word length, and presentation modality. The fit of this model to the data was acceptable, and parameter estimates were consistent across experiments. Furthermore, a correlation was found between M capacity and word span which resisted partialling out of age and articulation rate.     

A spatial modality effect in serial memory

In 2 experiments, the authors tested whether the classical modality effect-that is, the stronger recency effect for auditory items relative to visual items-can be extended to the spatial domain. An order reconstruction task was undertaken with four types of material: visual-spatial, auditory-spatial, visual-verbal, and auditory-verbal. Similar serial position curves were obtained regardless of the nature of the to-be-remembered sequences, with the exception that a modality effect was found with spatial as well as with verbal materials. The results are discussed with regard to a number of models of short-term memory.     

Modality differences in short-term memory for rhythms

Prior research has established that performance in short-term memory tasks using auditory rhythmic stimuli is frequently superior to that in tasks using visual stimuli. In five experiments, the reasons for this were explored further. In a same-different task, pairs of brief rhythms were presented in which each rhythm was visual or auditory, resulting in two same-modality conditions and two cross-modality conditions. Three different rates of presentation were used. The results supported the temporal advantage of the auditory modality in short-term memory, which was quite robust at the quickest presentation rates. This advantage tended to decay as the presentation rate was slowed down, consistent with the view that, with time, the temporal patterns were being recoded into a more generic form.     

Inverting the modality effect in serial recall

Differences in recall ability between immediate serial recall of auditorily and visually presented verbal material have traditionally been considered restricted to the end of to-be-recalled lists, the recency section of the serial position curve (e.g., Crowder & Morton, 1969). Later studies showed that--under certain circumstances--differences in recall between the two modalities can be observed across the whole of the list (Frankish, 1985). However in all these studies the advantage observed is for recall of material presented in the auditorily modality. Six separate conditions across four experiments demonstrate that a visual advantage can be obtained with serial recall if participants are required to recall the list in two distinct sections using serial recall. Judged on a list-wide basis, the visual advantage is of equivalent size to the auditory advantage of the classical modality effect. The results demonstrate that differences in representation of auditory and visual verbal material in short-term memory persist beyond lexical and phonological categorization and are problematic for current theories of the modality effect.     

Phonological short-term store and the nature of the recency effect: Evidence from neuropsychology

The recency effect in immediate recall of lists of unrelated words was investigated in P.V., a left hemisphere-damaged patient, who had a grossly reduced auditory verbal span, attributed to a selective impairment of a phonological short-term store. No recency effect was evident in free recall of auditorily presented material. When the patient was instructed to recall the final items of the list first, the recency performance remained defective, even though P.V. was able to adopt a recall from end order. In the case of visual presentation, P.V.'s free recall performance was within the normal range and a clear recency occurred in the recall from end condition. These results are consistent with the view that the standard recency effect in immediate free recall of auditorily presented material represents the output of a phonological short-term store to which ordinal retrieval strategies, in P.V.'s case unimpaired, are applied. Finally, the contributions of the phonological short-term store and the process of rehearsal to the recency effect and to immediate memory span performance are discussed.     

Modality effect in false recognition: evidence from Chinese characters

Using the Deese/Roediger‐McDermott (DRM) false memory method, Smith and Hunt ( 1998 ) first reported the modality effect on false memory and showed that false recall from DRM lists was lower following visual study than following auditory study, which led to numerous studies on the mechanism of modality effect on false memory and provided many competing explanations. In the present experiment, the authors tested the modality effect in false recognition by using a blocked presentation condition and a random presentation condition. The present experiment found a modality effect different from the results of the previous research; namely, false recognition was shown to be greater following visual study than following auditory study, especially in the blocked presentation condition rather than in the random presentation condition. The authors argued that this reversed modality effect may be due to different encoding and processing characteristics between Chinese characters and English words. Compared with English words, visual graphemes of critical lures in Chinese lists are likely to be activated and encoded in participants' minds, thus it is more difficult for participants to discriminate later inner graphemes from those items presented in visual modality. Hence visual presentation could lead to more false recognition than auditory presentation in Chinese lists. The results in the present experiment demonstrated that semantic activation occurring during the encoding and retrieve phases played an important role in modality effect in false recognition, and our findings might be explained by the activation‐monitoring account.     

Similarity as an organising principle in short-term memory

Abstract

The role of stimulus similarity as an organising principle in short-term memory was explored in a series of seven experiments. Each experiment involved the presentation of a short sequence of items that were drawn from two distinct physical classes and arranged such that item class changed after every second item. Following presentation, one item was re-presented as a probe for the ‘target’ item that had directly followed it in the sequence. Memory for the sequence was considered organised by class if probability of recall was higher when the probe and target were from the same class than when they were from different classes. Such organisation was found when one class was auditory and the other was visual (spoken vs. written words, and sounds vs. pictures). It was also found when both classes were auditory (words spoken in a male voice vs. words spoken in a female voice) and when both classes were visual (digits shown in one location vs. digits shown in another). It is concluded that short-term memory can be organised on the basis of sensory modality and on the basis of certain features within both the auditory and visual modalities.