Accuracy of temporal coding: Auditory-visual comparisons

Three experiments were designed to decide whether temporal information is coded more accurately for intervals defined by auditory events or for those defined by visual events. In the first experiment, the irregular-list technique was used, in which a short list of items was presented, the items all separated by different interstimulus intervals. Following presentation, the subject was given three items from the list, in their correct serial order, and was asked to judge the relative interstimulus intervals. Performance was indistinguishable whether the items were presented auditorily or visually. In the second experiment, two unfilled intervals were defined by three nonverbal signals in either the auditory or the visual modality. After delays of 0, 9, or 18 sec (the latter two filled with distractor activity), the subjects were directed to make a verbal estimate of the length of one of the two intervals, which ranged from 1 to 4 sec and from 10 to 13 sec. Again, performance was not dependent on the modality of the time markers. The results of Experiment 3, which was procedurally similar to Experiment 2 but with filled rather than empty intervals, showed significant modality differences in one measure only. Within the range of intervals employed in the present study, our results provide, at best, only modest support for theories that predict more accurate temporal coding in memory for auditory, rather than visual, stimulus presentation.     

A temporal distinctiveness theory of recency and modality effects

A temporal distinctiveness theory of contextually cued retrieval from memory is presented and applied to recency and modality effects. According to this theory, one part of the mnemonic trace of an item is a representation of the item's time of presentation. Time of presentation may be encoded with a coarse grain (so that it is consistent with a wide range of times) or with a fine grain (so that it is consistent with a narrow range of times). Retrieval proceeds by constructing temporally defined search sets that include representations of items consistent with the temporal bounds of the search set. The temporal width of the search set increases as the retention interval increases. Recency effects arise from retrieval of recently presented items from narrow search sets that include representations of few items; within the context of the search set, these items are distinctive and recalled well. Superiority in recall of recently presented auditory information in comparison with recently presented visual information is attributed to differences in the grain of time of presentation representations for aurally (fine grain) and visually (coarse grain) presented information. Four experiments confirm qualitative and quantitative predictions of the theory, including the prediction of auditory superiority at the beginning of the list when the initial items are temporally distinct.     

Modality effects in delayed free recall and recognition: Visual is better than auditory

During presentation of auditory and visual lists of words, different groups of subjects generated words that either rhymed with the presented words or that were associates. Immediately after list presentation, subjects recalled either the presented or the generated words. After presentation and test of all lists, a final free recall test and a recognition test were given. Visual presentation generally produced higher recall and recognition than did auditory presentation for both encoding conditions. The results are not consistent with explanations of modality effects in terms of echoic memory or greater temporal distinctiveness of auditory items. The results are more in line with the separate-streams hypothesis, which argues for different kinds of input processing for auditory and visual items.     

Distinctiveness revisited: Unpredictable temporal isolation does not benefit short-term serial recall of heard or seen events

The notion of a link between time and memory is intuitively appealing and forms the core assumption of temporal distinctiveness models. Distinctiveness models predict that items that are temporally isolated from their neighbors at presentation should be recalled better than items that are temporally crowded. By contrast, event-based theories consider time to be incidental to the processes that govern memory, and such theories would not imply a temporal isolation advantage unless participants engaged in a consolidation process (e.g., rehearsal or selective encoding) that exploited the temporal structure of the list. In this report, we examine two studies that assessed the effect of temporal distinctiveness on memory, using auditory (Experiment 1) and auditory and visual (Experiment 2) presentation with unpredictably varying interitem intervals. The results show that with unpredictable intervals temporal isolation does not benefit memory, regardless of presentation modality.     

Modality effects in the coding reproduction of rhythms

The temporal coding assumption is that time of presentation is coded more accurately for auditory events than for visual events. This assumption has been used to explain the modality effect, in which recall of recent auditory events is superior to recall of recent visual events. We tested the temporal coding assumption by examining the coding and reproduction of quintessentially temporal stimuli-rhythms. The rhythms were produced by sequences of short and long auditory stimuli or short and long visual stimuli; in either case, the task was to reproduce the temporal sequence. The results from four experiments demonstrated reproduction of auditory rhythms superior to that of visual rhythms. We conclude that speech-based explanations of modality effects cannot accommodate these findings, whereas the findings are consistent with explanations based on the temporal coding assumption.     

时序信息提取特点的实验研究

时距区分性理论认为提取时序信息是在包含一定项目的检索系中检索抽样的过程,通道因素应当对正确率和速度都产生影响.本文分别以英文字母和汉字为材料,对时序信息的提取特点作了两个实验研究.结果表明,只有材料的呈现顺序对时序信息提取的正确率和速度都有影响,通道因素只影响提取的正确率,而速度则未受其影响。显然,用时距区分性理论来说明时序信息提取特点是缺乏足够证据的.本文提出,对时序信息提取机制尚值得进一步研究.     

The Attentional Boost Effect: Transient increases in attention to one task enhance performance in a second task

Recent work on event perception suggests that perceptual processing increases when events change. An important question is how such changes influence the way other information is processed, particularly during dual-task performance. In this study, participants monitored a long series of distractor items for an occasional target as they simultaneously encoded unrelated background scenes. The appearance of an occasional target could have two opposite effects on the secondary task: It could draw attention away from the second task, or, as a change in the ongoing event, it could improve secondary task performance. Results were consistent with the second possibility. Memory for scenes presented simultaneously with the targets was better than memory for scenes that preceded or followed the targets. This effect was observed when the primary detection task involved visual feature oddball detection, auditory oddball detection, and visual color-shape conjunction detection. It was eliminated when the detection task was omitted, and when it required an arbitrary response mapping. The appearance of occasional, task-relevant events appears to trigger a temporal orienting response that facilitates processing of concurrently attended information (Attentional Boost Effect).     

The effects of divided attention on auditory priming

Traditional theorizing stresses the importance of attentional state during encoding for later memory, based primarily on research with explicit memory. Recent research has begun to investigate the role of attention in implicit memory but has focused almost exclusively on priming in the visual modality. The present experiments examined the effect of divided attention on auditory implicit memory, using auditory perceptual identification, word-stem completion and word-fragment completion. Participants heard study words under full attention conditions or while simultaneously carrying out a distractor task (the divided attention condition). In Experiment 1, a distractor task with low response frequency failed to disrupt later auditory priming (but diminished explicit memory as assessed with auditory recognition). In Experiment 2, a distractor task with greater response frequency disrupted priming on all three of the auditory priming tasks as well as the explicit test. These results imply that although auditory priming is less reliant on attention than explicit memory, it is still greatly affected by at least some divided-attention manipulations. These results are consistent with research using visual priming tasks and have relevance for hypotheses regarding attention and auditory priming.     

Recency effects in memory,as a function of modality of intervening events

Summary Word lists of fifteen items were presented to eye or to ear, with recall either immediately, or after a visual task, or after an auditory one. Instructions were to recall the last items first. An intervening task using the same modality greatly reduced recall of the last items presented; whereas a visual task did not do so for acoustically presented items. An auditory task reduced visual memory. These results suggest a specific auditory memory for recent events, over-written by subsequent auditory events.This research was supported by the Medical Research Council. The experimental work was performed at the Applied Psychology Unit, Cambridge.     

Evidence for auditory temporal distinctiveness: modality effects in order and frequency judgments

Two new, long-lasting phenomena involving modality of stimulus presentation are documented. In one series of experiments we investigated effects of modality of presentation on order judgments. Order judgments for auditory words were more accurate than order judgments for visual words at both the beginning and the end of lists, and the auditory advantage increased with the temporal separation of the successive items. A second series of experiments investigated effects of modality on estimates of presentation frequency. Frequency estimates of repeated auditory words exceeded frequency estimates of repeated visual words. The auditory advantage increased with frequency of presentation, and this advantage was not affected by the retention interval. These various effects were taken as support for a temporal coding assumption, that auditory presentation produces a more accurate encoding of time of presentation than does visual presentation.     

The role of visual interference in producing the long-term modality effect

Typically, recall of the last of a list of auditory items greatly exceeds recall of the last of a list of visual items. This modality effect has been found in serial recall, free recall, and recall using the distractor paradigm in which each to-be-remembered item is preceded and followed by distractor activity. One source of the auditory advantage may be visual interference that reduces recall of visual stimuli. In three experiments, sources of visual interference were minimized. Although this manipulation reduced the modality effect, it did not eliminate the effect.     

Cross-modality translations improve recognition by reducing false alarms

Conway and Gathercole [(1990). Writing and long-term memory: Evidence for a “translation” hypothesis. The Quarterly Journal of Experimental Psychology, 42, 513–527] proposed a translation account to explain why certain types of encoding produce benefits in memory: Switching modalities from what is presented to what is encoded enhances item distinctiveness. We investigated this hypothesis in a recognition experiment in which the presentation modality of a study list (visual vs. auditory) and the encoding activity (speaking vs. typing vs. passive encoding) were manipulated between-subjects. Manipulating encoding activity between-subjects ruled out any potential influence of the relationally distinct processing that can occur in a within-subject manipulation (in which all previous translation effects have been demonstrated). We found no overall difference in memory for words presented auditorily vs. visually nor for visual vs. auditory encoding, but critically presentation modality and encoding activity did interact. Translating from one modality to another – particularly from auditory presentation to visual encoding (typing) – led to the best memory discrimination. This was largely because of reduced false alarms, not increased hits, consistent with the distinctiveness heuristic.     

Auditory and visual differences in time perception? An investigation from a developmental perspective with neuropsychological tests

Adults and children (5- and 8-year-olds) performed a temporal bisection task with either auditory or visual signals and either a short (0.5-1.0s) or long (4.0-8.0s) duration range. Their working memory and attentional capacities were assessed by a series of neuropsychological tests administered in both the auditory and visual modalities. Results showed an age-related improvement in the ability to discriminate time regardless of the sensory modality and duration. However, this improvement was seen to occur more quickly for auditory signals than for visual signals and for short durations rather than for long durations. The younger children exhibited the poorest ability to discriminate time for long durations presented in the visual modality. Statistical analyses of the neuropsychological scores revealed that an increase in working memory and attentional capacities in the visuospatial modality was the best predictor of age-related changes in temporal bisection performance for both visual and auditory stimuli. In addition, the poorer time sensitivity for visual stimuli than for auditory stimuli, especially in the younger children, was explained by the fact that the temporal processing of visual stimuli requires more executive attention than that of auditory stimuli.     

Flexible target templates improve visual search accuracy for faces depicting emotion

Theories of visual attention hypothesize that target selection depends upon matching visual inputs to a memory representation of the target – i.e., the target or attentional template. Most theories assume that the template contains a veridical copy of target features, but recent studies suggest that target representations may shift "off veridical" from actual target features to increase target-to-distractor distinctiveness. However, these studies have been limited to simple visual features (e.g., orientation, color), which leaves open the question of whether similar principles apply to complex stimuli, such as a face depicting an emotion, the perception of which is known to be shaped by conceptual knowledge. In three studies, we find confirmatory evidence for the hypothesis that attention modulates the representation of an emotional face to increase target-to-distractor distinctiveness. This occurs over-and-above strong pre-existing conceptual and perceptual biases in the representation of individual faces. The results are consistent with the view that visual search accuracy is determined by the representational distance between the target template in memory and distractor information in the environment, not the veridical target and distractor features.

     

Disruption of short-term memory by distractor speech: Does content matter?

Four experiments replicate the finding that auditory distractors that are lexically identical to the visual target items dramatically increase the irrelevant-speech effect on serial recall. This effect was previously attributed to interference of incompatible order cues. The present results suggest that a different interpretation of this effect is required. Experiment 2 replicates the order congruence effect observed by Hughes and Jones (2005), but shows that this effect is most likely due to an attenuation of interference that is caused by strategic attention shifts to the nominally irrelevant material. Experiments 3 and 4 show that the between-stream similarity effect generalizes to a condition in which the distractor items were drawn from the same category as the targets, but were not identical to them. By showing that nonacoustic distractor features can increase interference in serial recall of lists of supposedly “meaningless” items such as digits or consonants, the results are most consistent with models that postulate an integration of short-term and long-term memory such as the embedded-processes model and the feature model and are inconsistent with classical structural accounts of memory.     

跨视听通道的相继错误记忆效应

本研究以12名大学生为被试采用DRM范式,考察了视觉与听觉通道在编码阶段的错误记忆ERP效应,以从更深层面认识错误记忆的内在加工机制。研究结果表明,在编码阶段,在300-500ms及500-700ms的时间窗视觉通道均存在相继错误记忆效应（DIM）,比听觉通道表现出较强的DIM效应,这不仅说明视觉通道与听觉通道在编码阶段具有不同的脑机制,而且表明DIM效应与学习项目的语义加工有着密切的关系。     

Memory Without Consolidation: Temporal Distinctiveness Explains Retroactive Interference

Is consolidation needed to account for retroactive interference in free recall? Interfering mental activity during the retention interval of a memory task impairs performance, in particular if the interference occurs in temporal proximity to the encoding of the to‐be‐remembered (TBR) information. There are at least two rival theoretical accounts of this temporal gradient of retroactive interference. The cognitive neuroscience literature has suggested neural consolidation is a pivotal factor determining item recall. According to this account, interfering activity interrupts consolidation processes that would otherwise stabilize the memory representations of TBR items post‐encoding. Temporal distinctiveness theory, by contrast, proposes that the retrievability of items depends on their isolation in psychological time. According to this theory, information processed after the encoding of TBR material will reduce the temporal distinctiveness of the TBR information. To test between these accounts, implementations of consolidation were added to the SIMPLE model of memory and learning. We report data from two experiments utilizing a two‐list free recall paradigm. Modeling results imply that SIMPLE was able to model the data and did not benefit from the addition of consolidation. It is concluded that the temporal gradient of retroactive interference cannot be taken as evidence for memory consolidation.     

Modality differences in short-term memory for rhythms

Prior research has established that performance in short-term memory tasks using auditory rhythmic stimuli is frequently superior to that in tasks using visual stimuli. In five experiments, the reasons for this were explored further. In a same-different task, pairs of brief rhythms were presented in which each rhythm was visual or auditory, resulting in two same-modality conditions and two cross-modality conditions. Three different rates of presentation were used. The results supported the temporal advantage of the auditory modality in short-term memory, which was quite robust at the quickest presentation rates. This advantage tended to decay as the presentation rate was slowed down, consistent with the view that, with time, the temporal patterns were being recoded into a more generic form.     

Ineffectiveness of visual distinctiveness in enhancing immediate recall

Glenberg (1984) and others have theorized that greater recency effects are obtained with auditory as opposed to visual presentation because of greater temporal distinctiveness of items in auditory sequences. We tested a number of ways of enhancing visual distinctiveness, including the use of color, spatial location, and minimized visual interference. None of the seven experiments provided any evidence of improved recall from enhanced visual distinctiveness. In particular, no increase in recency effects was obtained with increased distinctiveness. Additional analyses of pairwise dependency in recall across serial positions also failed to show any evidence of the near-independence of recall of the terminal item that characterizes recall of auditory sequences. Visual-perceptual distinctiveness does not get mapped in any simple way onto memorial distinctiveness in an immediate-serial-recall task.     

Modality effects and the structure of short-term verbal memory

The effects of auditory and visual presentation upon short-term retention of verbal stimuli are reviewed, and a model of the structure of short-term memory is presented. The main assumption of the model is that verbal information presented to the auditory and visual modalities is processed in separate streams that have different properties and capabilities. Auditory items are automatically encoded in both the A (acoustic) code, which, in the absence of subsequent input, can be maintained for some time without deliberate allocation of attention, and a P (phonological) code. Visual items are retained in both the P code and a visual code. Within the auditory stream, successive items are strongly associated; in contrast, in the visual modality, it is simultaneously presented items that are strongly associated. These assumptions about the structure of short-term verbal memory are shown to account for many of the observed effect of presentation modality.