Two modality effects in verbal short-term memory: Evidence from sentence recall

This paper investigates the mechanisms underlying the standard modality effect (i.e., better recall performance for auditorily presented than for visually presented materials), and the modality congruency effect (i.e., better memory performance if the mode of recall and presentation are congruent rather than incongruent). We tested the assumption that the standard modality effect is restricted to the most recent word(s) of the sentences but occurs in both verbatim and gist recall (Experiments 1 and 2), whereas the modality congruency effect should be evident for the rest of the sentence when using verbatim recall (Experiment 3) but not when using gist recall (Experiment 4). All experiments used the Potter-Lombardi intrusion paradigm. When the target word was the most recent word of the sentence, a standard modality effect was found with both verbatim recall and gist recall. When the target word was included in the middle of the sentences, a modality congruency effect was found with verbatim recall but not with gist recall.     

Impact of auditory selective attention on verbal short-term memory and vocabulary development

This study investigated the role of auditory selective attention capacities as a possible mediator of the well-established association between verbal short-term memory (STM) and vocabulary development. A total of 47 6- and 7-year-olds were administered verbal immediate serial recall and auditory attention tasks. Both task types probed processing of item and serial order information because recent studies have shown this distinction to be critical when exploring relations between STM and lexical development. Multiple regression and variance partitioning analyses highlighted two variables as determinants of vocabulary development: (a) a serial order processing variable shared by STM order recall and a selective attention task for sequence information and (b) an attentional variable shared by selective attention measures targeting item or sequence information. The current study highlights the need for integrative STM models, accounting for conjoined influences of attentional capacities and serial order processing capacities on STM performance and the establishment of the lexical language network.     

Context in verbal short-term memory

We tested the hypothesis that stimulus-related contextual information that is incidental to task demands-an episodic code-is automatically, obligatorily encoded and stored as a part of short-term memory (STM) representations. Four experiments employed a running span task to investigate the effects of manipulating two types of contextual information: stimulus grouping and color. Three experiments established that grouping context effects are sensitive neither to volitional control nor to task difficulty and that they generalize across testing procedures (yes/no recognition and immediate serial recall). A fourth experiment demonstrated an effect of manipulating the congruity of the color of stimuli between study and test. These demonstrations of the robustness and generality of context effects in STM are consistent with the predictions of the episodic coding model of STM.     

Transfer across modality in perceptual implicit memory

We examined whether words studied in one modality (visual or auditory) would prime performance in the opposite modality in five different perceptual implicit memory tests: auditory perceptual identification, auditory stem completion, visual perceptual identification, visual stem completion, and visual fragment completion. Significant transfer across modality was observed in all five tasks. However, a large proportion of the subjects reported using explicit retrieval strategies during the implicit tests. Those subjects who claimed not to have used explicit retrieval processes during the test phase demonstrated transfer across modalities in the stem completion tests and the perceptual identification tests, but not in the fragment completion test. The results indicate that implicit visual word-fragment completion is unique, in the sense that it relies exclusively on perceptual memory processes, whereas the other tasks rely, in part, on nonperceptual memory processes.     

Presentation modality,rehearsal-prevention conditions,and auditory confusions in tests of short-term memory

Ss were presented four-letter sequences either auditorily or visually and asked for ordered recall after 0, 2.1, 4.2, 8.4, or 12.6 sec of digit categorization. Three different rehearsal-prevention conditions were required during presentation of the memory set: categorizing, suppressing (saying "dah"), or pronouncing each letter. Recall was worst after categorizing, best after pronouncing. Auditory presentation led to better recall after no delay but more rapid forgetting than visual presentation, regardless of the rehearsal-prevention condition. These results, and analyses of auditory confusions, are inconsistent with a view of memory which asserts that sensory information is encoded auditorily regardless of presentation modality or vocalization behavior during presentation.     

Interference in short-term auditory memory

Although interference is a well-established forgetting function in short-term auditory memory, an adequate understanding of its underlying mechanisms and time course has yet to be attained. The present study therefore aimed to explore these issues in memory for timbre. Listeners compared standard and comparison complex tones, having distinct timbres (four components varying in frequency), over a 4.7-s retention interval and made a same–different response. This interval either was silent or included one of 15 distractor tones occurring 0 ms, 100 ms, or 1,200 ms after the standard. These distractors varied in the extent to which the frequencies of their component tones were shared with the standard. Performance in comparing the two tones was significantly impaired by distractors composed of novel frequencies, regardless of the temporal position at which the distractor occurred. These results were fully compatible with the recent timbre memory model (McKeown & Wellsted, 2009 McKeown, D. and Wellsted, D. 2009. Auditory memory for timbre. Journal of Experimental Psychology: Human Perception and Performance, 35: 855–875. [Crossref], [PubMed], [Web of Science ®] , [Google Scholar]) and suggested that interference in auditory memory operates via a feature-overwriting mechanism.     

A new model of verbal short-term memory

Two experiments tested a neo-Piagetian model of verbal short-term memory and compared it with the articulatory loop model. Experiment 1 (n = 113, age range 9-11) tested word span for 2-, 3-, and 4-syllable words, with both visual and auditory presentation. Experiment 2 (with the same participants) tested recall of visually presented supraspan lists. Measures of M capacity (as conceived in Pascual-Leone's neo-Piagetian theory) and articulation rate were also used. The proposed model can account for the effects of M capacity, word length, and presentation modality. The fit of this model to the data was acceptable, and parameter estimates were consistent across experiments. Furthermore, a correlation was found between M capacity and word span which resisted partialling out of age and articulation rate.     

Effects of perceptual modality on verbatim and gist memory

In two experiments, predictions of the fuzzy-trace theory of memory were tested. Perceptual information may play a role in retrieval and recognition processes for verbatim, but not for gist, memory. Perceptual modality effects were assessed in the present study by presenting three-sentence stories (e.g.,The bird is in the cage. The cage is over the table. The bird is yellow) and then testing recognition of probes that varied on three dimensions: (1) semantic accuracy (true vs. false), (2) wording (all original words vs. one novel word included), and (3) sentence type (premise vs. inference). In Experiment 1, study modality (auditory vs. visual) was manipulated, and in Experiment 2, both study and test modalities were manipulated. Despite replicating a number of findings consistent with fuzzy-trace theory (e.g., instruction and probe type effects), the results of both experiments failed to support the idea that perceptual information plays a role in performance on verbatim memory tests.     

Group aggregates and individual reliability: The case of verbal short-term memory

Two experiments examined the generalizability of the effects of word length and phonological similarity with visual and auditory presentation in immediate verbal serial ordered recall. In Experiment 1, data were collected from 251 adult volunteers drawn from a broad cross-section of the normal population. Word length and phonological similarity in both presentation modes significantly influenced the group means. However, 43% of the subjects failed to show at least one of the effects, and the likelihood that effects appeared was highly correlated with verbal memory span. In Experiment 2, 40 subjects of the original sample were retested, 20 of whom had failed to show one or more effects in Experiment 1. Whether or not an effect had appeared for individual subjects on the first test session was a poor predictor of whether the effect would appear on retest. Finally, an analysis of subject reports demonstrated that the patterns of experimental data could be accounted for in part by the strategies that subjects reported using, and the effect of strategy was independent of the effect of span. The implications of these findings for theories of verbal short-term memory are discussed.     

Modality effects and the structure of short-term verbal memory

The effects of auditory and visual presentation upon short-term retention of verbal stimuli are reviewed, and a model of the structure of short-term memory is presented. The main assumption of the model is that verbal information presented to the auditory and visual modalities is processed in separate streams that have different properties and capabilities. Auditory items are automatically encoded in both the A (acoustic) code, which, in the absence of subsequent input, can be maintained for some time without deliberate allocation of attention, and a P (phonological) code. Visual items are retained in both the P code and a visual code. Within the auditory stream, successive items are strongly associated; in contrast, in the visual modality, it is simultaneously presented items that are strongly associated. These assumptions about the structure of short-term verbal memory are shown to account for many of the observed effect of presentation modality.     

Heterogeneity of auditory verbal working memory in schizophrenia

The heterogeneity of schizophrenia remains an obstacle for understanding its pathophysiology. Studies using a tone discrimination screening test to classify patients have found evidence for 2 subgroups having either a specific deficit in verbal working memory (WM) or deficits in both verbal and nonverbal memory. This study aimed to (a) replicate in larger samples differences between these subgroups in auditory verbal WM; (b) evaluate their performance on tests of explicit memory and sustained attention; (c) determine the relation of verbal WM deficits to auditory hallucinations and other symptoms; and (d) examine medication effects. The verbal WM and tone discrimination performance did not differ between medicated (n = 45) and unmedicated (n = 38) patients. Patients with schizophrenia who passed the tone screening test (discriminators; n = 60) were compared with those who did not (nondiscriminators; n = 23) and healthy controls (n = 47). The discriminator subgroup showed poorer verbal WM than did controls and a deficit in verbal but not visual memory on the Wechsler Memory Scale-Revised (Wechsler, 1987), whereas the nondiscriminator subgroup showed overall poorer performance on both verbal and nonverbal tests and a marked deficit in sustained attention. Verbal WM deficits in discriminators were correlated with auditory hallucinations but not with negative symptoms. The results are consistent with a verbal memory deficit in a subgroup of schizophrenia having intact auditory perception, which may stem from dysfunction of language-related cortical regions, and a more generalized cognitive deficit in a subgroup having auditory perceptual and attentional dysfunction.     

Presentation modality and mode of recall in verbal false memory

False memories were investigated for aurally and visually presented lists of semantically associated words. In Experiment 1, false written recall of critical intrusions was reliably lower following visual presentation compared with aural presentation. This presentation modality effect was attributed to the use of orthographic features during written recall to edit critical intrusions from visually presented lists. As predicted by this hypothesis, the modality effect was eliminated when the mode of recall was spoken rather than written. In Experiment 2, the modality effect in written recall was again replicated and then eliminated with an orienting task that ensured orthographic encoding even of aurally presented words. Thus, the modality effect appears to depend on using orthographic information to distinguish true from false verbal memories.     

The role of visuospatial and verbal working memory in perceptual category learning

The role of verbal and visuospatial working memory in rule-based and information-integration category learning was examined. Previously, Maddox, Ashby, Ing, and Pickering found that a sequentially presented verbal working memory task did not affect information-integration learning, but disrupted rule-based learning when the rule was on the spatial frequency of a Gabor stimulus. This pattern was replicated in Experiment 1, in which the same category structures were used, but in which the verbal working memory task was replaced with a visuospatial analog. Experiment 2A examined rule-based learning on an oblique orientation and also found both verbal and visuospatial working memory tasks disrupting learning. Experiment 2B examined rule-based learning on a cardinal orientation and found a minimal effect of the verbal working memory task, but a large effect of the visuospatial working memory task. The conceptual significance of cardinal orientations and the role of visuospatial and verbal working memory in category learning are discussed.     

Relations between vocabulary development and verbal short-term memory: The relative importance of short-term memory for serial order and item information

Although many studies have shown an association between verbal short-term memory (STM) and vocabulary development, the precise nature of this association is not yet clear. The current study reexamined this relation in 4- to 6-year-olds by designing verbal STM tasks that maximized memory for either item or serial order information. Although empirical data suggest that distinct STM processes determine item and serial order recall, these were generally confounded in previous developmental studies. We observed that item and order memory tasks were independently related to vocabulary development. Furthermore, vocabulary development was more strongly associated with STM for order information in 4- and 6-year-olds and with STM for item information in 5-year-olds. These data highlight the specificity of verbal STM for serial order and item information and suggest a causal association between order STM processes and vocabulary development, at least in 4- and 6-year-olds.     

Long-term associative learning predicts verbal short-term memory performance

Studies using tests such as digit span and nonword repetition have implicated short-term memory across a range of developmental domains. Such tests ostensibly assess specialized processes for the short-term manipulation and maintenance of information that are often argued to enable long-term learning. However, there is considerable evidence for an influence of long-term linguistic learning on performance in short-term memory tasks that brings into question the role of a specialized short-term memory system separate from long-term knowledge. Using natural language corpora, we show experimentally and computationally that performance on three widely used measures of short-term memory (digit span, nonword repetition, and sentence recall) can be predicted from simple associative learning operating on the linguistic environment to which a typical child may have been exposed. The findings support the broad view that short-term verbal memory performance reflects the application of long-term language knowledge to the experimental setting.     

Word-frequency and phonological-neighborhood effects on verbal short-term memory

Immediate memory span and maximal articulation rate were assessed for word sets differing in frequency, word-neighborhood size, and average word-neighborhood frequency. Memory span was greater for high- than low-frequency words, greater for words from large than small phonological neighborhoods, and greater for words from high- than low-frequency phonological neighborhoods. Maximal articulation rate was also facilitated by word frequency, phonological-neighborhood size, and neighborhood frequency. In a final study all 3 lexical variables were found to influence the recall outcome for individual words. These effects of phonological-word neighborhood on memory performance suggest that phonological information in long-term memory plays an active role in recall in short-term-memory tasks, and they present a challenge to current theories of short-term memory.     

Visual and auditory stores in short-term memory

If retrieval in short-term memory can be either from a pre-perceptual sensory store or from a post-perceptual memory then recall should vary as a function of input into sensory store. To test this possibility two experiments with paired associates compared visual and auditory presentation under conditions as comparable as possible. In both experiments modality interacted with retention interval: more recency with auditory but, in Experiment I, more primacy with visual. The interaction was taken as support for the hypothesis. An alternative hypothesis (that storage is post-perceptual but not a-historical) was discussed and weak negative evidence presented.