Word length,phonemic, and visual similarity effects in poor and normal readers

Serial order recall for visually and auditorily presented stimuli was examined in a group of 12-year-old poor readers and 7-year-old reading-age controls. With pictorial presentation, the poor readers showed a visual similarity effect, no word length effect, and a smaller phonemic similarity effect than that of controls. However, with visual presentation of printed words and with auditory presentation, poor readers showed word length and phonemic similarity effects of similar magnitude to that of controls. It is concluded that poor readers rely on visual information in tasks where the presented images are highly codable, and where verbal recoding is not obligatory, but that they will make use of phonological coding when the stimuli are not as easily codable visually in memory.     

Boundary conditions for the modality effect: Type of material and time of test

Recall of auditory and visually presented information was studied using three different types of material. Although auditory presentation was superior for all three types of material, the size and pattern of the effect depended upon whether the visual presentation was successive or simultaneous. Successive visual presentation reduced recall performance compared to auditory presentation considerably for real sentences, while smaller effects were obtained for noun sequences and scrambled sentences. The overall supremacy in recall of auditory presented items was reduced further when compared to a simultaneous visual presentation. In the last two experiments, all effects of modality of presentation obtained in immediate recall were replicated, but were eliminated in delayed recall. The finding that this modality effect is eliminated in delayed recall is taken to indicate that semantic aspects of the material to be remembered are by and large unaffected by mode of presentation, while more superficial aspects are the ones influenced.     

Paradoxical effects of presentation modality on false memory.

Two experiments are reported in which the effects of presentation modality on false memory in recall and recognition are studied. False recall and recognition of critical targets are lower for non-presented items related to a study list when that study list is presented visually than when presented auditorily. This pattern of low levels of false memory for critical targets holds even when participants read the visually presented study items aloud. These results suggest that recollection of visual detail plays a role in the prevention of false memory. However, both the hit rates (true memory) and the false-alarm rates to weakly related distractors (non-critical targets) were higher for visual presentation than for auditory presentation, suggesting that more than one mechanism may underlie false recognition.     

Prelexical facilitation and lexical interference in auditory word recognition.

Phonological priming effects were examined in an auditory single-word shadowing task. In 6 experiments, target items were preceded by auditorily or visually presented, phonologically similar, word or nonword primes. Results revealed facilitation in response time when a target was preceded by a word or nonword prime having the same initial phoneme when the prime was auditorily presented but not when it was visually presented. Second, modality-independent interference was observed when the phonological overlap between the prime and target increased from 1 to 3 phonemes for word primes but not for nonword primes. Taken together, these studies suggest that phonological information facilitates word recognition as a result of excitation at a prelexical level and increases response time as a result of competition at a lexical level. These processes are best characterized by connectionist models of word recognition.     

Short-term memory as a function of input modality

The two experiments reported are concerned with short-term memory for digit lists simultaneously presented both auditorily and visually. Results showed (1) that interpolated written and verbal recall differentially affect retention depending on whether the to-be-recalled list was presented auditorily or visually. (2) That input modality appears to be far more important for recall than was directing subjects' attention to a list during input, when that list might or might not have been subsequently required for recall. The results suggest that short-term storage is modality specific. In this case, Broadbent's P and S mechanisms do not adequately describe what happens during simultaneous visual and auditory presentation. Nor would Sperling's suggestion of a final auditory store appear to be supported.     

Inverting the modality effect in serial recall

Differences in recall ability between immediate serial recall of auditorily and visually presented verbal material have traditionally been considered restricted to the end of to-be-recalled lists, the recency section of the serial position curve (e.g., Crowder & Morton, 1969). Later studies showed that--under certain circumstances--differences in recall between the two modalities can be observed across the whole of the list (Frankish, 1985). However in all these studies the advantage observed is for recall of material presented in the auditorily modality. Six separate conditions across four experiments demonstrate that a visual advantage can be obtained with serial recall if participants are required to recall the list in two distinct sections using serial recall. Judged on a list-wide basis, the visual advantage is of equivalent size to the auditory advantage of the classical modality effect. The results demonstrate that differences in representation of auditory and visual verbal material in short-term memory persist beyond lexical and phonological categorization and are problematic for current theories of the modality effect.     

Interaction between mode of learning and subjective experience: Translation effects in long-term memory

It has been suggested that writing auditorily presented words at encoding involves distinctive translation processes between visual and auditory domains, leading to the formation of distinctive memory traces at retrieval. This translation effect leads to higher levels of recognition than the writing of visually presented words, a non-translation effect. The present research investigated whether writing and the other translation effect of vocalisation (vocalising visually presented words) would be present in tests of recall, recognition memory and whether these effects are based on the subjective experience of remembering or knowing. Experiment 1 found a translation effect in the auditory domain in recall, as the translation effect of writing yielded higher recall than both non-translation effects of vocalisation and silently hearing. Experiment 2 found a translation effect in the visual domain in recognition, as the translation effect of vocalisation yielded higher recognition than both non-translation effects of writing and silently reading. This translation effect was attributable to the subjective experience of remembering rather than knowing. The present research therefore demonstrates the beneficial effect of translation in both recall and recognition, with the effect of vocalisation in recognition being based on rich episodic remembering.     

Order of recall,output interference,and the modality effect

Summary In free recall, the order of recall following auditory and visual presentation differs; it tends to be forward for auditory but backward for visual. The first two experiments examined to what extent this difference in output order could account for the modality effect (i.e., a superior retention of auditorily as opposed to visually presented words). Order of recall was manipulated using postcued (Experiment 1) and precued (Experiment 2) procedures. Whereas the modality effect was unaffected with postcueing it was reduced to approximately half its size with precueing. It was concluded from these two studies that although output order cannot explain the whole modality effect, it does seem to play an important role for part of the effect in some situations. Experiments 3 and 4 used a mixed-mode and a probed recall procedure, respectively, to examine the role of output interference in modality experiments. The data suggested that output interference effects were non-monotonic; they were greater for visual than for auditory early in recall, but apparently no different later in recall. The two-store hypotheses (Murdock and Walker, 1969) was elaborated slightly to account for these results.This research was supported by Research Grants APA 146 from the National Research Council of Canada and OMHF 164 from the Ontario Mental Health Foundation. We would like to thank Doris Glavnov and Janet Metcalfe for help with the data analyses.     

Short-term memory and the nature of interference from concurrent shadowing

Trigrams were presented visually or auditorily and followed by a 12 s retention interval filled with shadowing numbers or letters. Auditory memory letters followed by letter shadowing were recalled less than auditory memory letters followed by number shadowing or visual memory letters followed by either type of shadowing. The latter three conditions did not differ among themselves. An analysis of the recall intrusions suggested that forgetting of auditory memory letters followed by letter shadowing was caused mainly by a confusion between covert rehearsals and shadowing activity, while forgetting in the other three conditions was caused primarily by proactive interference from earlier memory trials.     

Parallel scanning of auditory and visual information

Two experiments involving memory retrieval of auditorilv and visually presented materials were performed. In Experiment I, subjects were presented with memory sets of 1, 2, or 4 stimuli and then with a test item to be classified as belonging or not belonging to the memory set. In Condition 1, each memory stimulus was a single, auditorily presented letter. In Condition 2, each memory stimulus was a visually presented letter. In Conditions 3 and 4, each memory stimulus was a pair of letters, one presented visually and the other auditorily. Mean reaction time (RT) for the classification task increased as a function of number of memory stimuli at equal rates for all four conditions. This was interpreted as evidence for a parallel scanning process in Conditions 3 and 4 where the auditory item and visual item of each memory stimulus pair can be scanned simultaneously. Experiment II compared memory retrieval for a simultaneous condition in which auditory and visual memory items were presented as pairs with a sequential condition in which mixed auditory-visual memory sets were presented one item at a time. RTs were shorter for the simultaneous condition. This was interpreted as evidence that parallel scanning may depend upon memory input parameters.     

The role of visual interference in producing the long-term modality effect

Typically, recall of the last of a list of auditory items greatly exceeds recall of the last of a list of visual items. This modality effect has been found in serial recall, free recall, and recall using the distractor paradigm in which each to-be-remembered item is preceded and followed by distractor activity. One source of the auditory advantage may be visual interference that reduces recall of visual stimuli. In three experiments, sources of visual interference were minimized. Although this manipulation reduced the modality effect, it did not eliminate the effect.     

Suppression effects on musical and verbal memory

Three experiments contrasted the effects of articulatory suppression on recognition memory for musical and verbal sequences. In Experiment 1, a standard/comparison task was employed, with digit or note sequences presented visually or auditorily while participants remained silent or produced intermittent verbal suppression (saying "the") or musical suppression (singing "la"). Both suppression types decreased performance by equivalent amounts, as compared with no suppression. Recognition accuracy was lower during suppression for visually presented digits than during that for auditorily presented digits (consistent with phonological loop predictions), whereas accuracy was equivalent for visually presented notes and auditory tones. When visual interference filled the retention interval in Experiment 2, performance with visually presented notes but not digits was impaired. Experiment 3 forced participants to translate visually presented music sequences by presenting comparison sequences auditorily. Suppression effects for visually presented music resembled those for digits only when the recognition task required sensory translation of cues.     

Lexical influences in audiovisual speech perception

Phoneme identification with audiovisually discrepant stimuli is influenced hy information in the visual signal (the McGurk effect). Additionally, lexical status affects identification of auditorily presented phonemes. The present study tested for lexical influences on the McGurk effect. Participants identified phonemes in audiovisually discrepant stimuli in which lexical status of the auditory component and of a visually influenced percept was independently varied. Visually influenced (McGurk) responses were more frequent when they formed a word and when the auditory signal was a nonword (Experiment 1). Lexical effects were larger for slow than for fast responses (Experiment 2), as with auditory speech, and were replicated with stimuli matched on physical properties (Experiment 3). These results are consistent with models in which lexical processing of speech is modality independent.     

Effects of Auditory/Visual and Lexical/Non-lexical Comparisons on Event-related Potentials in a Memory-scanning Task

A late parietal positivity (P3) and behavioural measures were studied during performance of a two-item memory-scanning task. Stimuli were digits presented as memorised items in one modality (auditory or visual) while the following probe, also a digit, was presented in the same or the other modality. In a separate set of experiments, P3 and behaviour were similarly studied using only visual stimuli that were either lexical (digits) or non-lexical (novel fonts with the same contours as the digits) to which subjects assigned numerical values. Reaction times (RTs) and P3 latencies were prolonged to non-lexical compared to lexical stimuli. Although RTs were longer to auditory than to visual stimuli, P3 latencies to memorised items were prolonged in response to visually compared to auditorily presented memorised items, and were further prolonged when preceding visual probes. P3 amplitudes were smaller to auditory than to visual stimuli, and were smaller for the second memorised item when lexical/non-lexical comparisons were involved. The most striking finding was scalp distribution variations indicating changes in relative contributions of brain structures involved in processing memorised items, according to the probes that followed. These findings are compatible, in general, with a phonological memorisation, but they suggest that the process is modified by memorising the item in the same terms as the expected probe that follows.     

Learning and retrieval rate of words presented auditorily and visually

Mode of presentation (visual or auditory) of a multitrial free recall test is stressed as an important factor in improving the diagnosis of certain neurological patients. For further use in neuropsychological research, an experiment was carried out using normal subjects, in which the effects of presentation mode and order of modality were investigated. There were no differential effects of these variables on several parameters, such as the number of words recalled and the learning curve. The time needed for the responses in immediate recall was the same in both auditory and visual conditions. In delayed recall, however, the interresponse times were significantly shorter when words had been presented auditorily than when presented visually. The results are discussed in light of further application in the field of neuropsychology.     

The effect of study modality on false recognition

A number of previous studies have shown that false recognition of critical items in the Deese/Roediger-McDermott paradigm is reduced when study items are presented visually rather than auditorily; however, this effect has not been uniformly demonstrated. We investigated three potential boundary conditions of the effect of study modality in false recognition. Experiments 1 and 2 showed no reduction in false recognition following visual study presentation when the yes-no recognition test was not preceded by a recall test. Experiment 3 showed that visual study presentation can reduce false recognition without a preceding recall test, if the recognition test uses remember-know instructions. The order of the recognition test items did not influence the effect of visual study presentation on false recognition in Experiment 1. In general, the data imply that distinctive processing at study can reduce false memory in recognition if the test demands draw attention to the dimension of distinctive processing.     

Presentation modality,rehearsal-prevention conditions,and auditory confusions in tests of short-term memory

Ss were presented four-letter sequences either auditorily or visually and asked for ordered recall after 0, 2.1, 4.2, 8.4, or 12.6 sec of digit categorization. Three different rehearsal-prevention conditions were required during presentation of the memory set: categorizing, suppressing (saying "dah"), or pronouncing each letter. Recall was worst after categorizing, best after pronouncing. Auditory presentation led to better recall after no delay but more rapid forgetting than visual presentation, regardless of the rehearsal-prevention condition. These results, and analyses of auditory confusions, are inconsistent with a view of memory which asserts that sensory information is encoded auditorily regardless of presentation modality or vocalization behavior during presentation.     

The effects of recoding and presentation format on recency and suffix effects

The primary linguistic theory of Shand and Klima (1981) hypothesizes that stimuli that cannot be directly processed without recoding are not in the primary linguistic mode of the subject and thus should lead to lesser recency and associated suffix effects. In three experiments, different normal hearing subjects learned to pair American Sign Language (ASL) stimuli, visual "quasivocables" (QVs), word-like letter strings, and auditory QVs with common English words. In the first experiment, the subjects were given sequences of ASL or QV stimuli and required to recall the associated words in strict serial order. In two other experiments involving auditory and visual presentation, respectively, subjects who had never been given paired associate training were required to recall the English words that had previously been associated with the ASL and QV stimuli, in a standard suffix paradigm. The results showed recency and suffix effects to be present only with auditorily presented QVs and words. Contrary to the predictions of the primary linguistic hypothesis, greater recency and larger suffix effects were present with the auditory QVs than with the auditory words, although the QVs were not primary linguistic and the task involved forced recoding. Previous results showing recency with ASL stimuli in normal subjects were not replicated. It is concluded that recency and suffix effects are not related either to the primary linguistic mode of the subject or to stimulus recoding, as we and Shand and Klima have defined them.     

Irrelevant speech effects and statistical learning

Immediate serial recall of visually presented verbal stimuli is impaired by the presence of irrelevant auditory background speech, the so-called irrelevant speech effect. Two of the three main accounts of this effect place restrictions on when it will be observed, limiting its occurrence either to items processed by the phonological loop (the phonological loop hypothesis) or to items that are not too dissimilar from the irrelevant speech (the feature model). A third, the object-oriented episodic record (O-OER) model, requires only that the memory task involves seriation. The present studies test these three accounts by examining whether irrelevant auditory speech will interfere with a task that does not involve the phonological loop, does not use stimuli that are compatible with those to be remembered, but does require seriation. Two experiments found that irrelevant speech led to lower levels of performance in a visual statistical learning task, offering more support for the O-OER model and posing a challenge for the other two accounts.     

Does the testing effect depend on presentation modality?

We examined whether the testing effect generalizes to an auditory presentation modality. Five lists of unrelated words (Experiment 1) and related words (Experiment 2) were presented to participants, half of whom studied them visually and half studied them auditorily. Participants in the study-only condition performed a short distractor task following lists 1–4, whereas those in the testing condition completed a short distractor task and then attempted to recall each list. Both groups were subsequently tested on List 5 and on all five lists 30 min later. In both experiments, we found a testing effect for both List 5 and for the final cumulative recall test. However, the effect did not interact with study modality, despite the fact that proactive interference was greater following auditory study. These results have important implications for educational practice, suggesting that initial testing is important for materials presented in auditory as well as visual formats.