Distinctiveness revisited: Unpredictable temporal isolation does not benefit short-term serial recall of heard or seen events

The notion of a link between time and memory is intuitively appealing and forms the core assumption of temporal distinctiveness models. Distinctiveness models predict that items that are temporally isolated from their neighbors at presentation should be recalled better than items that are temporally crowded. By contrast, event-based theories consider time to be incidental to the processes that govern memory, and such theories would not imply a temporal isolation advantage unless participants engaged in a consolidation process (e.g., rehearsal or selective encoding) that exploited the temporal structure of the list. In this report, we examine two studies that assessed the effect of temporal distinctiveness on memory, using auditory (Experiment 1) and auditory and visual (Experiment 2) presentation with unpredictably varying interitem intervals. The results show that with unpredictable intervals temporal isolation does not benefit memory, regardless of presentation modality.     

Duration discrimination of empty and filled intervals marked by auditory and visual signals

Experiments 1 and 2 compared, with a single-stimulus procedure, the discrimination of filled and empty intervals in both auditory and visual modalities. In Experiment 1, in which intervals were about 250 msec, the discrimination was superior with empty intervals in both modalities. In Experiment 2, with intervals lasting about 50 msec, empty intervals showed superior performance with visual signals only. In Experiment 3, for the auditory modality at 250 msec, the discrimination was easier with empty intervals than with filled intervals with both the forced-choice (FC) and the single stimulus (SS) modes of presentation, and the discrimination was easier with the FC than with the SS method. Experiment 4, however, showed that at 50 and 250 msec, with a FC-adaptive procedure, there were no differences between filled and empty intervals in the auditory mode; the differences observed with the visual mode in Experiments 1 and 2 remained significant. Finally, Experiment 5 compared differential thresholds for four marker-type conditions, filled and empty intervals in the auditory and visual modes, for durations ranging from .125 to 4 sec. The results showed (1) that the differential threshold differences among marker types are important for short durations but decrease with longer durations, and (2) that a generalized Weber’s law generally holds for these conditions. The results as a whole are discussed in terms of timing mechanisms.     

Interactions between presentation modality and encoding task in frequency judgements

In two experiments, presentation modality of a list of items and encoding task were varied, and subjects judged the frequency with which certain words had been presented in the list. In Experiment 1, auditory presentation led to higher judgements of frequency than did visual presentation when subjects counted the consonants in the words but not when they rated imageability or when they kept a running count of the number of presentations of each word. In Experiment 2, encoding questions about the rhyme or spelling patterns of target words produced opposite effects for auditory and visual items. The results are interpreted as indicating that cross-modal translation during encoding produces a bias towards higher-frequency judgements and may also produce better frequency discrimination.     

Schedules of presentation and temporal distinctiveness in human memory

Recency, in remembering a series of events, reflects the simple fact that memory is vivid for what has just happened but deteriorates over time. Theories based on distinctiveness, an alternative to the multistore model, assert that the last few events in a series are well remembered because their times of occurrence are more highly distinctive than those of earlier items. Three experiments examined the role of temporal and ordinal factors in auditorily and visually presented lists that were temporally organized by distractor materials interpolated between memory items. With uniform distractor periods, the results were consistent with Glenberg's (1987) temporal distinctiveness theory. When the procedure was altered so that distractor periods became progressively shorter from the beginning to the end of the list, the results were consistent for only the visual modality; the auditory modality produced a different and unpredicted (by the theory) pattern of results, thus falsifying the claim that the auditory modality derives more benefit from temporal information than the visual modality. We distinguish serial order information from specifically temporal information, arguing that the former may be enhanced by auditory presentation but that the two modalities are more nearly equal with respect to the latter.     

Echoic and retrieval accounts of the long-term modality effect tested using the suffix procedure

The long-term modality effect is the advantage in recall of the last of a list of auditory to-be-remembered (TBR) items compared with the last of a list of visual TBR items when the list is followed by a filled retention interval. If the auditory advantage is due to echoic sensory memory mechanisms, then recall of the last auditory TBR item should be substantially reduced when it is followed by a redundant, not-to-be-recalled auditory suffix. Contrary to this prediction, Experiment 1 demonstrated that a redundant auditory suffix does not significantly reduce recall of the last auditory TBR item. In Experiment 2 a nonredundant auditory suffix produced a large reduction in the last auditory item. Redundancy is not the only factor controlling the effectiveness of a suffix, however. Experiment 3 demonstrated that a nonredundant visual suffix does not reduce recall of the last auditory TBR item. These results are discussed in reference to a retrieval account of the long-term modality effect.     

Enumeration produces poor primacy for tactile presentation relative to visual and auditory presentation as the only modality effect

Methodological biases may help explain the modality effect, which is superior recall of auditory recency (end of list) items relative to visual recency items. In 1985 Nairne and McNabb used a counting procedure to reduce methodological biases, and they produced modality-like effects, such that recall of tactile recency items was superior to recall of visual recency items. The present study extended Nairne and McNabb's counting procedure and controlled several variables which may have enhanced recall of tactile end items or disrupted recall of visual end items in their study. Although the results of the present study indicated general serial position effects across tactile, visual, and auditory presentation modalities, the tactile condition showed lower recall for the initial items in the presentation list than the other two conditions. Moreover, recall of the final list item did not differ across the three presentation modalities; modality effects were not found. These results did not replicate the findings of Nairne and McNabb, or much of the past research showing superior recall of auditory recency items. Implications of these findings are discussed.     

Modality effects in delayed free recall and recognition: Visual is better than auditory

During presentation of auditory and visual lists of words, different groups of subjects generated words that either rhymed with the presented words or that were associates. Immediately after list presentation, subjects recalled either the presented or the generated words. After presentation and test of all lists, a final free recall test and a recognition test were given. Visual presentation generally produced higher recall and recognition than did auditory presentation for both encoding conditions. The results are not consistent with explanations of modality effects in terms of echoic memory or greater temporal distinctiveness of auditory items. The results are more in line with the separate-streams hypothesis, which argues for different kinds of input processing for auditory and visual items.     

Auditory and visual differences in time perception

The present experiment assessed intersensory differences in temporal judgments, that is, auditory stimuli are perceived as longer than physically equivalent visual stimuli. The results confirmed the intersensory difference. Auditorially defined intervals were experienced as longer than visually defined intervals. Auditory boundaries were perceived as longer than visual ones. An interaction of boundary modality and interval modality was obtained which suggested that auditorially defined intervals provided more temporal information about events occurring in close temporal proximity than visually defined intervals. It was hypothesized that cognitive factors, specifically stimulus complexity, would affect the auditory and visual systems differentially. This hypothesis was not substantiated, although highly complex stimuli were experienced as longer than those of low complexity.     

Temporal Grouping Effects in Immediate Recall: A Working Memory Analysis

The presence of temporal pauses during list presentation can markedly improve immediate memory for a sequence of verbal items. A series of experiments analysed this effect using Baddeley's (1986) model of working memory. Experiment 1 showed that the effect of temporal grouping on memory for visual sequences was removed by either articulatory suppression or reciting random digits. Experiment 2 indicated that effects of temporal grouping were insensitive to the word length of the items. Experiment 3 showed that articulatory suppression did not remove the temporal grouping effect for auditory lists. Experiment 4 showed that the temporal grouping effect was insensitive to the phonemic similarity of the items. The effects of concurrent articulation suggest that grouping influences the phonological loop component of working memory. However, the working memory model is insufficiently well specified to account for the insensitivity of grouping effects to word length and phonemic similarity. The main findings could be simulated by a connectionist model of the phonological loop, which invokes a context timing signal (Burgess & Hitch, 1992, in press), This assumed that pauses during list presentation affect the timing signal in a similar way to the pause before list presentation and made some novel predictions.     

跨视听通道的相继错误记忆效应

本研究以12名大学生为被试采用DRM范式,考察了视觉与听觉通道在编码阶段的错误记忆ERP效应,以从更深层面认识错误记忆的内在加工机制。研究结果表明,在编码阶段,在300-500ms及500-700ms的时间窗视觉通道均存在相继错误记忆效应（DIM）,比听觉通道表现出较强的DIM效应,这不仅说明视觉通道与听觉通道在编码阶段具有不同的脑机制,而且表明DIM效应与学习项目的语义加工有着密切的关系。     

The crossmodal facilitation of visual object representations by sound: evidence from the backward masking paradigm

We report a series of experiments designed to demonstrate that the presentation of a sound can facilitate the identification of a concomitantly presented visual target letter in the backward masking paradigm. Two visual letters, serving as the target and its mask, were presented successively at various interstimulus intervals (ISIs). The results demonstrate that the crossmodal facilitation of participants' visual identification performance elicited by the presentation of a simultaneous sound occurs over a very narrow range of ISIs. This critical time-window lies just beyond the interval needed for participants to differentiate the target and mask as constituting two distinct perceptual events (Experiment 1) and can be dissociated from any facilitation elicited by making the visual target physically brighter (Experiment 2). When the sound is presented at the same time as the mask, a facilitatory, rather than an inhibitory effect on visual target identification performance is still observed (Experiment 3). We further demonstrate that the crossmodal facilitation of the visual target by the sound depends on the establishment of a reliable temporally coincident relationship between the two stimuli (Experiment 4); however, by contrast, spatial coincidence is not necessary (Experiment 5). We suggest that when visual and auditory stimuli are always presented synchronously, a better-consolidated object representation is likely to be constructed (than that resulting from unimodal visual stimulation).     

Is there a modality effect? Evidence for visual recency and suffix effects

In Experiment 1, four groups of 16 subjects performed ordered recall of six-syllable lists in both suffix and nonsuffix conditions. Sequential presentation of the lists varied for each group. In the auditory presentation, the syllables were delivered from one location only and were read aloud by the subjects. For the visual, spatially nondistributed presentation, the syllables appeared in one location only and were read silently. For visual, spatially distributed presentations, the syllables were spread out either vertically or horizontally and were read silently. Very robust recency and suffix effects were found in the auditory presentation, as well as in visual, spatially distributed presentations. In Experiment 2, 16 subjects performed ordered recall of visually presented lists with the items spread out vertically and conflicting spatial and temporal orders. A reliable recency effect was found for the final block of trials. In Experiment 3, 16 subjects performed ordered recall in the same conditions as in Experiment 2, except that they were instructed to recall the temporal order in which the spatial positions would be filled in. A bow-shaped curve and a strong recency effect were obtained.     

Perceptual learning in auditory temporal discrimination: No evidence for a cross-modal transfer to the visual modality

Perceptual learning was used to study potential transfer effects in a duration discrimination task. Subjects were trained to discriminate between two empty temporal intervals marked with auditory beeps, using a twoalternative forced choice paradigm. The major goal was to examine whether perceptual learning would generalize to empty intervals that have the same duration but are marked by visual flashes. The experiment also included longer intervals marked with auditory beeps and filled auditory intervals of the same duration as the trained interval, in order to examine whether perceptual learning would generalize to these conditions within the same sensory modality. In contrast to previous findings showing a transfer from the haptic to the auditory modality, the present results do not indicate a transfer from the auditory to the visual modality; but they do show transfers within the auditory modality.     

Using several modalities to perceive one temporal pattern

In two experiments, subjects identified temporal patterns. The patterns consisted of eight dichotomous (left-right) elements, e.g. LLRRLRLR, continuously repeated until the subject was able to identify the pattern. In Experiment 1, one pattern was presented either separately in the auditory, tactual, or visual modalities or one pattern was presented simultaneously in two modalities (compatible presentation). In Experiment 2, one pattern was simultaneously presented in two modalities, but the pattern was presented in one modality and the complement of the pattern (the complement of LLRRLRLR is RRLLRLRL) was presented in the second modality. Therefore, opposite spatial elements appeared in each modality (incompatible presentation).

The results indicated that the rate of pattern identification was the same for compatible and incompatible presentation. These methods produce better performance than individual modality presentation at fast presentation rates (2 elements/sec. and faster) although individual modality presentation was better at slower rates. This suggests that when a pattern is presented in two modalities, the pattern in each modality is integrated, not the particular spatial elements in each modality. Furthermore, the rate of pattern identification using individual modalities did not predict the difficulty using pairs of modalities. These results demonstrate the Gestalt nature of pattern perception; the pattern is perceptually salient and the performance of pairs of modalities depends on the inherent properties of the individual modalities.     

Duration discrimination of filled and empty auditory intervals: Cognitive and perceptual factors

Adult subjects were presented with two auditory stimuli per trial, and their task was to decide which of the two was longer in duration. An adaptive psychophysical procedure was used. In Experiments 1, 2, and 4, the base duration was 50 msec, whereas in Experiment 3, the base duration was 1 sec. In Experiments 1, 2, and 4, it was found that filled intervals (continuous tones) were discriminated more accurately than empty intervals (with onset and offset marked by clicks). It was concluded that this difference was perceptual rather than cognitive in nature, since performance on filled and empty intervals was not affected by increasing cognitive load in a dual-task procedure (Experiment 2) but was affected by backward masking (Experiment 4). In contrast, the results of Experiment 3 showed that duration discrimination of filled auditory intervals of longer duration was cognitively influenced, since performance was impaired by increasing cognitive load. Implications for notions of perceptual processing and timing mechanisms tanderlying differences in duration discrimination with filled and empty intervals are discussed.     

Duration discrimination of filled and empty auditory intervals: cognitive and perceptual factors.

Adult subjects were presented with two auditory stimuli per trial, and their task was to decide which of the two was longer in duration. An adaptive psychophysical procedure was used. In Experiments 1, 2, and 4, the base duration was 50 msec, whereas in Experiment 3, the base duration was 1 sec. In Experiments 1, 2, and 4, it was found that filled intervals (continuous tones) were discriminated more accurately than empty intervals (with onset and offset marked by clicks). It was concluded that this difference was perceptual rather than cognitive in nature, since performance on filled and empty intervals was not affected by increasing cognitive load in a dual-task procedure (Experiment 2) but was affected by backward masking (Experiment 4). In contrast, the results of Experiment 3 showed that duration discrimination of filled auditory intervals of longer duration was cognitively influenced, since performance was impaired by increasing cognitive load. Implications for notions of perceptual processing and timing mechanism underlying differences in duration discrimination with filled and empty intervals are discussed.     

A temporal distinctiveness theory of recency and modality effects

A temporal distinctiveness theory of contextually cued retrieval from memory is presented and applied to recency and modality effects. According to this theory, one part of the mnemonic trace of an item is a representation of the item's time of presentation. Time of presentation may be encoded with a coarse grain (so that it is consistent with a wide range of times) or with a fine grain (so that it is consistent with a narrow range of times). Retrieval proceeds by constructing temporally defined search sets that include representations of items consistent with the temporal bounds of the search set. The temporal width of the search set increases as the retention interval increases. Recency effects arise from retrieval of recently presented items from narrow search sets that include representations of few items; within the context of the search set, these items are distinctive and recalled well. Superiority in recall of recently presented auditory information in comparison with recently presented visual information is attributed to differences in the grain of time of presentation representations for aurally (fine grain) and visually (coarse grain) presented information. Four experiments confirm qualitative and quantitative predictions of the theory, including the prediction of auditory superiority at the beginning of the list when the initial items are temporally distinct.     

A systematic investigation of same and cross modality priming using written and spoken responses

Effects of presentation modality and response format were investigated using visual and auditory versions of the word stem completion task. Study presentation conditions (visual, auditory, non-studied) were manipulated within participants, while test conditions (visual/written, visual/spoken, auditory/written, auditory/spoken, recall-only) were manipulated between participants. Results showed evidence for same modality and cross modality priming on all four word stem completion tasks. Words from the visual study list led to comparable levels of priming across all test conditions. In contrast, words from the auditory study list led to relatively low levels of priming in the visual/written test condition and high levels of priming in the auditory/spoken test condition. Response format was found to influence priming performance following auditory study in particular. The findings confirm and extend previous research and suggest that, for implicit memory studies that require auditory presentation, it may be especially beneficial to use spoken rather than written responses.     

Modality effects in the coding reproduction of rhythms

The temporal coding assumption is that time of presentation is coded more accurately for auditory events than for visual events. This assumption has been used to explain the modality effect, in which recall of recent auditory events is superior to recall of recent visual events. We tested the temporal coding assumption by examining the coding and reproduction of quintessentially temporal stimuli-rhythms. The rhythms were produced by sequences of short and long auditory stimuli or short and long visual stimuli; in either case, the task was to reproduce the temporal sequence. The results from four experiments demonstrated reproduction of auditory rhythms superior to that of visual rhythms. We conclude that speech-based explanations of modality effects cannot accommodate these findings, whereas the findings are consistent with explanations based on the temporal coding assumption.     

A comparison of visual and auditory short-term memory in monkeys (Cebus apella)

Using a visual and an acoustic sample set that appeared to favour the auditory modality of the monkey subjects, in Experiment 1 retention gradients generated in closely comparable visual and auditory matching (go/no-go) tasks revealed a more durable short-term memory (STM) for the visual modality. In Experiment 2, potentially interfering visual and acoustic stimuli were introduced during the retention intervals of the auditory matching task. Unlike the case of visual STM, delay-interval visual stimulation did not affect auditory STM. On the other hand, delay-interval music decreased auditory STM, confirming that the monkeys maintained an auditory trace during the retention intervals. Surprisingly, monkey vocalizations injected during the retention intervals caused much less interference than music. This finding, which was confirmed by the results of Experiments 3 and 4, may be due to differential processing of “arbitrary” (the acoustic samples) and species-specific (monkey vocalizations) sounds by the subjects. Although less robust than visual STM, auditory STM was nevertheless substantial, even with retention intervals as long as 32 sec.