Influence of recall procedures on the modality effect with numbers and enumerated stimuli

Experiment 1 extended J. S. Nairne and W. L. McNabb's (1985) counting procedure for presenting numerical stimuli to examine the modality effect. The present authors presented participants with dots and beeps and instructed participants to count the items to derive to-be-remembered numbers. In addition, the authors presented numbers as visual and auditory symbols, and participants recalled items by using free-serial written recall. Experiment 1 demonstrated primacy effects, recency effects, and modality effects for visual and auditory symbols and for counts of dots and beeps. Experiment 2 replicated the procedure in Experiment 1 using strict-serial written recall instead of free-serial written recall. The authors demonstrated primacy and recency effects across all 4 presentation conditions and found a modality effect for numbers that the authors presented as symbols. However, the authors found no modality effect when they presented numbers as counts of beeps and dots. The authors discuss the implications of the results in terms of methods for testing modality effects.     

The separate but related origins of the recency effect and the modality effect in free recall

The recency effect found in free recall can be accounted for almost entirely in terms of the recall of ordered sequences of items. It is such sequences, presented at the end of the stimulus list but recalled at the very beginning of the response protocol, which produce a recency effect. Such sequences are recalled at the beginning of the response protocol equally often following auditory and visual presentation. These same stimulus sequences are also frequently recalled other than initially in the response protocol following auditory presentation. However, such responses are rarely found following visual presentation. The modality effect in free recall, the advantage of auditory over visual presentation, can be substantially accounted for in these terms. Theoretical and procedural implications of these data are discussed.     

Inverting the modality effect in serial recall

Differences in recall ability between immediate serial recall of auditorily and visually presented verbal material have traditionally been considered restricted to the end of to-be-recalled lists, the recency section of the serial position curve (e.g., Crowder & Morton, 1969). Later studies showed that--under certain circumstances--differences in recall between the two modalities can be observed across the whole of the list (Frankish, 1985). However in all these studies the advantage observed is for recall of material presented in the auditorily modality. Six separate conditions across four experiments demonstrate that a visual advantage can be obtained with serial recall if participants are required to recall the list in two distinct sections using serial recall. Judged on a list-wide basis, the visual advantage is of equivalent size to the auditory advantage of the classical modality effect. The results demonstrate that differences in representation of auditory and visual verbal material in short-term memory persist beyond lexical and phonological categorization and are problematic for current theories of the modality effect.     

Paradoxical effects of presentation modality on false memory.

Two experiments are reported in which the effects of presentation modality on false memory in recall and recognition are studied. False recall and recognition of critical targets are lower for non-presented items related to a study list when that study list is presented visually than when presented auditorily. This pattern of low levels of false memory for critical targets holds even when participants read the visually presented study items aloud. These results suggest that recollection of visual detail plays a role in the prevention of false memory. However, both the hit rates (true memory) and the false-alarm rates to weakly related distractors (non-critical targets) were higher for visual presentation than for auditory presentation, suggesting that more than one mechanism may underlie false recognition.     

Recency and suffix effects in serial recall of musical stimuli

Auditory presentation of verbal items leads to larger recency effects in recall than visual presentation. This enhanced recency can be eliminated if a stimulus suffix (an irrelevant sound) follows the last item. Four experiments tested the hypothesis that recency and suffix effects in serial recall result from a speech-specific process. It was demonstrated that serial recall of musical notes played on a piano exhibited substantial recency effects. These recency effects were reduced when the list items were followed by either a piano chord or the word start. However, a white-noise suffix had no effect on recency. This pattern of data is consistent with current work on auditory perception and places constraints on theories of recency and suffix effects.     

Modalities of memory: Is reading lips like hearing voices?

Functional similarities in verbal memory performance across presentation modalities (written, heard, lipread) are often taken to point to a common underlying representational form upon which the modalities converge. We show here instead that the pattern of performance depends critically on presentation modality and different mechanisms give rise to superficially similar effects across modalities. Lipread recency is underpinned by different mechanisms to auditory recency, and while the effect of an auditory suffix on an auditory list is due to the perceptual grouping of the suffix with the list, the corresponding effect with lipread speech is due to misidentification of the lexical content of the lipread suffix. Further, while a lipread suffix does not disrupt auditory recency, an auditory suffix does disrupt recency for lipread lists. However, this effect is due to attentional capture ensuing from the presentation of an unexpected auditory event, and is evident both with verbal and nonverbal auditory suffixes. These findings add to a growing body of evidence that short-term verbal memory performance is determined by modality-specific perceptual and motor processes, rather than by the storage and manipulation of phonological representations.     

Abstract versus modality-specific memory representations in processing auditory and visual speech

Serial recall of lip-read, auditory, and audiovisual memory lists with and without a verbal suffix was examined. Recency effects were the same in the three presentation modalities. The disrupting effect of a suffix was largest when it was presented in the same modality as the list items. The results suggest that abstract linguistic as well as modality-specific codes play a role in memory for auditory and visual speech.     

Schedules of presentation and temporal distinctiveness in human memory

Recency, in remembering a series of events, reflects the simple fact that memory is vivid for what has just happened but deteriorates over time. Theories based on distinctiveness, an alternative to the multistore model, assert that the last few events in a series are well remembered because their times of occurrence are more highly distinctive than those of earlier items. Three experiments examined the role of temporal and ordinal factors in auditorily and visually presented lists that were temporally organized by distractor materials interpolated between memory items. With uniform distractor periods, the results were consistent with Glenberg's (1987) temporal distinctiveness theory. When the procedure was altered so that distractor periods became progressively shorter from the beginning to the end of the list, the results were consistent for only the visual modality; the auditory modality produced a different and unpredicted (by the theory) pattern of results, thus falsifying the claim that the auditory modality derives more benefit from temporal information than the visual modality. We distinguish serial order information from specifically temporal information, arguing that the former may be enhanced by auditory presentation but that the two modalities are more nearly equal with respect to the latter.     

Ineffectiveness of visual distinctiveness in enhancing immediate recall

Glenberg (1984) and others have theorized that greater recency effects are obtained with auditory as opposed to visual presentation because of greater temporal distinctiveness of items in auditory sequences. We tested a number of ways of enhancing visual distinctiveness, including the use of color, spatial location, and minimized visual interference. None of the seven experiments provided any evidence of improved recall from enhanced visual distinctiveness. In particular, no increase in recency effects was obtained with increased distinctiveness. Additional analyses of pairwise dependency in recall across serial positions also failed to show any evidence of the near-independence of recall of the terminal item that characterizes recall of auditory sequences. Visual-perceptual distinctiveness does not get mapped in any simple way onto memorial distinctiveness in an immediate-serial-recall task.     

Recency effects in delayed recall of mouthed stimuli

Auditory presentation leads to greater recency effects in recall than does visual presentation. This phenomenon (the modality effect) is found in both free and serial recall and in both immediate and delayed recall. Silent mouthing of visually presented stimuli also leads to enhanced recency effects in immediate serial recall. Two experiments reported here extend the generality of the mouthing effect by demonstrating that enhanced recency effects of mouthed stimuli occur in delayed serial and free recall. These results are inconsistent with theories that attribute the modality effect to a purely auditory sensory memory.     

Boundary conditions for the modality effect: Type of material and time of test

Recall of auditory and visually presented information was studied using three different types of material. Although auditory presentation was superior for all three types of material, the size and pattern of the effect depended upon whether the visual presentation was successive or simultaneous. Successive visual presentation reduced recall performance compared to auditory presentation considerably for real sentences, while smaller effects were obtained for noun sequences and scrambled sentences. The overall supremacy in recall of auditory presented items was reduced further when compared to a simultaneous visual presentation. In the last two experiments, all effects of modality of presentation obtained in immediate recall were replicated, but were eliminated in delayed recall. The finding that this modality effect is eliminated in delayed recall is taken to indicate that semantic aspects of the material to be remembered are by and large unaffected by mode of presentation, while more superficial aspects are the ones influenced.     

Phonological short-term store and the nature of the recency effect: Evidence from neuropsychology

The recency effect in immediate recall of lists of unrelated words was investigated in P.V., a left hemisphere-damaged patient, who had a grossly reduced auditory verbal span, attributed to a selective impairment of a phonological short-term store. No recency effect was evident in free recall of auditorily presented material. When the patient was instructed to recall the final items of the list first, the recency performance remained defective, even though P.V. was able to adopt a recall from end order. In the case of visual presentation, P.V.'s free recall performance was within the normal range and a clear recency occurred in the recall from end condition. These results are consistent with the view that the standard recency effect in immediate free recall of auditorily presented material represents the output of a phonological short-term store to which ordinal retrieval strategies, in P.V.'s case unimpaired, are applied. Finally, the contributions of the phonological short-term store and the process of rehearsal to the recency effect and to immediate memory span performance are discussed.     

The word-length effect in probed and serial recall

The word-length effect in immediate serial recall has been explained as the possible consequence of rehearsal processes or of output processes. In the first experiment adult subjects heard lists of five long or short words while engaging in articulatory suppression during presentation. Full serial recall or probed recall for a single item followed the list either immediately or after a 5-second delay to encourage rehearsal. The word-length effect was not influenced by recall delay, but was much smaller in probed than in serial recall. Examination of the serial position curves suggested that this might be due to a recency component operating in probed recall. Experiment 2 confirmed a word-length-insensitive recency effect in probed recall and showed that this was resistant to an auditory suffix, unlike the small recency effect found in serial recall. Experiment 3 used visual presentation without concurrent articulation. Under these conditions there was no recency effect for either recall method, but the word-length effect was again much smaller in probed than in serial recall. This was confirmed in Experiment 4, in which the presentation of serial and probed recall was randomized across trials, showing that the differences between recall methods could not be due to encoding strategies. We conclude that for visual presentation, at least part of the word-length effect originates in output processes. For auditory presentation the position is less clear, as serial and probed recall appear to draw on different resources. The nature of the output processes that may give rise to word-length effects is discussed.     

Timing is everything: Changes in presentation rate have opposite effects on auditory and visual implicit statistical learning

Implicit statistical learning (ISL) is exclusive to neither a particular sensory modality nor a single domain of processing. Even so, differences in perceptual processing may substantially affect learning across modalities. In three experiments, statistically equivalent auditory and visual familiarizations were presented under different timing conditions that either facilitated or disrupted temporal processing (fast or slow presentation rates). We find an interaction of rate and modality of presentation: At fast rates, auditory ISL was superior to visual. However, at slow presentation rates, the opposite pattern of results was found: Visual ISL was superior to auditory. Thus, we find that changes to presentation rate differentially affect ISL across sensory modalities. Additional experiments confirmed that this modality-specific effect was not due to cross-modal interference or attentional manipulations. These findings suggest that ISL is rooted in modality-specific, perceptually based processes.     

Timing is everything: changes in presentation rate have opposite effects on auditory and visual implicit statistical learning

Implicit statistical learning (ISL) is exclusive to neither a particular sensory modality nor a single domain of processing. Even so, differences in perceptual processing may substantially affect learning across modalities. In three experiments, statistically equivalent auditory and visual familiarizations were presented under different timing conditions that either facilitated or disrupted temporal processing (fast or slow presentation rates). We find an interaction of rate and modality of presentation: At fast rates, auditory ISL was superior to visual. However, at slow presentation rates, the opposite pattern of results was found: Visual ISL was superior to auditory. Thus, we find that changes to presentation rate differentially affect ISL across sensory modalities. Additional experiments confirmed that this modality-specific effect was not due to cross-modal interference or attentional manipulations. These findings suggest that ISL is rooted in modality-specific, perceptually based processes.     

Modality effects in delayed free recall and recognition: Visual is better than auditory

During presentation of auditory and visual lists of words, different groups of subjects generated words that either rhymed with the presented words or that were associates. Immediately after list presentation, subjects recalled either the presented or the generated words. After presentation and test of all lists, a final free recall test and a recognition test were given. Visual presentation generally produced higher recall and recognition than did auditory presentation for both encoding conditions. The results are not consistent with explanations of modality effects in terms of echoic memory or greater temporal distinctiveness of auditory items. The results are more in line with the separate-streams hypothesis, which argues for different kinds of input processing for auditory and visual items.     

Suffix effects and probe modality in probed recall: Implications for readout from sensory memory

The effect of a redundant stimulus suffix was investigated in three experiments using a probed recall task. The probe modality was auditory in the first experiment and visual in the second. In the third experiment, both probe modalities were tested. The presentation rate was varied in all three experiments and was confounded with the delay of the suffix. The suffix was found to have a small and approximately equal effect on the last six serial positions. In contrast to the predictions from Crowder and Morton's (1969) PAS model, presentation rate and suffix delay did not interact with the suffix effect. The results indicate that at least six items can be simultaneously represented in PAS, that there is no appreciable decay of information from PAS during presentation of the last list items, and that readout of PAS information does not occur either during list presentation or between the end of list presentation and the beginning of overt report.     

Recency and suffix effects with immediate recall of olfactory stimuli

In contrast to our understanding of the immediate recall of auditory and visual material, little is known about the corresponding characteristics of short-term olfactory memory. The current study investigated the pattern of immediate serial recall and the associated suffix effect using olfactory stimuli. Subjects were trained initially to identify and name correctly nine different odours. Experiment 1 established an immediate correct recall span of approximately six items. In Experiment 2 participants recalled serially span equivalent lists which were followed by a visual, auditory, or olfactory suffix. Primacy was evident in the recall curves for all three suffix conditions. Recency, in contrast, was evident in the auditory and visual suffix conditions only; there was a strong suffix effect in the olfactory suffix condition. Experiment 3 replicated this pattern of effects using seven-item lists, and demonstrated that the magnitude of the recency and suffix effects obtained in the olfactory modality can equate to that obtained in the auditory modality. It is concluded that the pattern of recency and suffix effects in the olfactory modality is reliable, and poses difficulties for those theories that rely on the presence of a primary linguistic code, sound, or changing state as determinants of these effects in serial recall.     

Tactile “capture” of audition

Previous research has demonstrated that the localization of auditory or tactile stimuli can be biased by the simultaneous presentation of a visual stimulus from a different spatial position. We investigated whether auditory localization judgments could also be affected by the presentation of spatially displaced tactile stimuli, using a procedure designed to reveal perceptual interactions across modalities. Participants made left—right discrimination responses regarding the perceived location of sounds, which were presented either in isolation or together with tactile stimulation to the fingertips. The results demonstrate that the apparent location of a sound can be biased toward tactile stimulation when it is synchronous, but not when it is asynchronous, with the auditory event. Directing attention to the tactile modality did not increase the bias of sound localization toward synchronous tactile stimulation. These results provide the first demonstration of the tactilecapture of audition.     

Recency and suffix effects with immediate recall of olfactory stimuli

In contrast to our understanding of the immediate recall of auditory and visual material, little is known about the corresponding characteristics of short-term olfactory memory. The current study investigated the pattern of immediate serial recall and the associated suffix effect using olfactory stimuli. Subjects were trained initially to identify and name correctly nine different odours. Experiment 1 established an immediate correct recall span of approximately six items. In Experiment 2 participants recalled serially span equivalent lists which were followed by a visual, auditory, or olfactory suffix. Primacy was evident in the recall curves for all three suffix conditions. Recency, in contrast, was evident in the auditory and visual suffix conditions only; there was a strong suffix effect in the olfactory suffix condition. Experiment 3 replicated this pattern of effects using seven-item lists, and demonstrated that the magnitude of the recency and suffix effects obtained in the olfactory modality can equate to that obtained in the auditory modality. It is concluded that the pattern of recency and suffix effects in the olfactory modality is reliable, and poses difficulties for those theories that rely on the presence of a primary linguistic code, sound, or changing state as determinants of these effects in serial recall.