Orienting attention to locations in mental representations

Many cognitive processes depend on our ability to hold information in mind, often well beyond the offset of the original sensory input. The capacity of this visual short-term memory (VSTM) is limited to around three to four items. Recent research has demonstrated that the content of VSTM can be modulated by top-down attentional biases. This has been demonstrated using retrodictive spatial cues, termed "retro-cues," which orient subjects' attention to spatial locations within VSTM. In the present article, we tested whether the use of these cues is modulated by memory load and cue delay. There are a number of important conclusions: (1) Top-down biases can operate on very brief iconic traces as well as on older VSTM representations (Exp. 1). (2) When operating within capacity, subjects use the cue to prioritise where they initiate their memory search, rather than to discard uncued items (Exps. 2 and 3). (3) When capacity is exceeded, there is little benefit to top-down biasing relative to a neutral condition; however, unattended items are lost, with there being a substantial cost of invalid spatial cueing (Exp. 3). (4) These costs and benefits of orienting spatial attention differ across iconic memory and VSTM representations when VSTM capacity is exceeded (Exp. 4).     

Recall cues interfere with retrieval from visuospatial working memory

Visuospatial working memory allows us to hold multiple visual objects over short delays. It is typically tested by presenting an array of objects, then after a delay showing a ‘probe’ indicating which memory item to recall or reproduce by adjusting a target feature. However, recent studies demonstrate that information at the time of probe can disrupt recall. Here, in three experiments we test whether traditional memory probes, which contain features that compete with the feature to be recalled, may themselves interfere with performance. We asked participants to report the direction of one of the several coloured arrows in memory, based on its colour. First, we demonstrate that recall is better when the probe is initially just a coloured dot, rather than a coloured arrow which has to be adjusted to match orientation memory, consistent with interference from features of the probe itself. Second, this interference is present even when a mask follows the memory array, suggesting that the interference does not work by degrading immediate or iconic memory. Finally, when items are shown sequentially, the first and last items are invulnerable to probe interference. Our findings support recent theories of associative recall, in which probes reactivate features in WM, retrieving information by pattern completion.     

Read-out of emotional information from iconic memory: the longevity of threatening stimuli

Previous research has shown that emotional stimuli are more likely than neutral stimuli to be selected by attention, indicating that the processing of emotional information is prioritized. In this study, we examined whether the emotional significance of stimuli influences visual processing already at the level of transient storage of incoming information in iconic memory, before attentional selection takes place. We used a typical iconic memory task in which the delay of a poststimulus cue, indicating which of several visual stimuli has to be reported, was varied. Performance decreased rapidly with increasing cue delay, reflecting the fast decay of information stored in iconic memory. However, although neutral stimulus information and emotional stimulus information were initially equally likely to enter iconic memory, the subsequent decay of the initially stored information was slowed for threatening stimuli, a result indicating that fear-relevant information has prolonged availability for read-out from iconic memory. This finding provides the first evidence that emotional significance already facilitates stimulus processing at the stage of iconic memory.     

Iconic memory and visible persistence

There are three senses in which a visual stimulus may be said to persist psychologically for some time after its physical offset. First, neural activity in the visual system evoked by the stimulus may continue after stimulus offset (“neural persistence”). Second, the stimulus may continue to be visible for some time after its offset (“visible persistence”). Finally, information about visual properties of the stimulus may continue to be available to an observer for some time after stimulus offset (“informational persistence”). These three forms of visual persistence are widely assumed to reflect a single underlying process: a decaying visual trace that (1) consists of afteractivity in the visual system, (2) is visible, and (3) is the source of visual information in experiments on decaying visual memory. It is argued here that this assumption is incorrect. Studies of visible persistence are reviewed; seven different techniques that have been used for investigating visible persistence are identified, and it is pointed out that numerous studies using a variety of techniques have demonstrated two fundamental properties of visible persistence: theinverse duration effect (the longer a stimulus lasts, the shorter is its persistence after stimulus offset) and theinverse intensity effect (the more intense the stimulus, the briefer its persistence). Only when stimuli are so intense as to produce afterimages do these two effects fail to occur. Work on neural persistences is briefly reviewed; such persistences exist at the photoreceptor level and at various stages in the visual pathways. It is proposed that visible persistence depends upon both of these types of neural persistence; furthermore, there must be an additional neural locus, since a purely stereoscopic (and hence cortical) form of visible persistence exists. It is argued that informational persistence is defined by the use of the partial report methods introduced by Averbach and Coriell (1961) and Sperling (1960), and the term “iconic memory” is used to describe this form of persistence. Several studies of the effects of stimulus duration and stimulus intensity upon the duration of iconic memory have been carried out. Their results demonstrate that the duration of iconic memory is not inversely related to stimulus duration or stimulus intensity. It follows that informational persistence or iconic memory cannot be identified with visible persistence, since they have fundamentally different properties. One implication of this claim that one cannot investigate iconic memory by tasks that require the subject to make phenomenological judgments about the duration of a visual display. In other words, the so-called “direct methods” for studying iconic memory do not provide information about iconic memory. Another implication is that iconic memory is not intimately tied to processes going on in the visual system (as visible persistence is); provided a stimulus is adequately legible, its physical parameters have little influence upon its iconic memory. The paper concludes by pointing out that there exists an alternative to the usual view of iconic memory as a precategorical sensory buffer. According to this alternative, iconic memory is post-categorical, occurring subsequent to stimulus identification. Here, stimulus identification is considered to be a rapid automatic process which does not require buffer storage, but which provides no information about episodic properties of a visual stimulus. Information about these physical stimulus properties must, in some way, be temporarily attached to a representation of the stimulus in semantic memory; and it is this temporarily attached physical information which constitutes iconic memory.     

The level of subjective visibility at different stages of memory processing

Recent research suggests that the content of iconic memory (IM) and fragile visual short-term memory could be associated with a similar level of conscious accessibility as working memory (WM). The results of our studies, in which we used a subjective visibility scale in a partial-report change detection paradigm, indicate that it is possible to distinguish separate stages of memory based on both discriminative accuracy and conscious accessibility. The highest scores were associated with IM and the lowest with WM, while somewhere in the middle there was fragile memory. Based on classical assumptions, WM accessibility should be greater than the other two types of memory; however, our study showed that this might not always be the case. We discuss the potential sources of this outcome, of which one may be the task construction, as we only tested items that were directly in the focus of attention.     

Visual attribute modulates the time course of iconic memory decay

Studies on iconic memory demonstrate that rich information from a visual scene quickly becomes unavailable with the passage of time. The decay rate of iconic memory refers to the dynamics of memory availability. The present study investigated the iconic memory decay of different stimulus attributes that comprised an object. Specifically, in Experiment 1, participants were presented with eight coloured numbers (e.g., red 4) and required to remember only one attribute, either colour or number, over different blocks of trials. The participants then reported the cued attribute in which the cue Stimulus Onset Asynchrony (SOA) from the memory array onset was varied (0, 100, 200, 300, 500, and 1000?ms). We found that numerical information became unavailable more quickly than colour information, despite the fact that the memory accuracies at 0 and 1000?ms SOAs were comparable between the two attributes. In Experiment 2, we replicated the finding that a numerical representation was lost more quickly than a colour representation when visual masks followed the target stimulus. These results suggest that the various visual attributes comprising an object are lost over time at different rates in iconic memory. We discuss this finding in relation to how perceptual representation is transferred to the capacity-limited visual working memory.     

Some functional properties of iconic storage in retarded and nonretarded subjects

Four experiments, which examine some functional properties of iconic storage in mildly retarded subjects, are reported. Experiments I and III demonstrated that retardates report about three items, or one item less than normal subjects, after a single brief tachistoscopic exposure and that this span of attention was independent of size of array. Both normal and retarded groups reported more items correctly when arrays were arranged on two lines. Experiment II determined that exposure durations up to 250 msec did not influence the number of items reported by either group. Experiment IV compared the form of decay for both groups by cuing report of each of seven positions at five poststimulus delays. The presence of the typical W-shaped curve for both groups at all delays permitted the inference that items in iconic storage decay together rather than individually. Although quantitative differences existed between groups, in no case did intelligence interact with the manipulated variables. The results were discussed in relation to control processes and structural features within an information processing model of memory.     

A two-stage model of visual search

A two-stage visual search model is described. Four experiments show that in a VM paradigm sequential presentation of the display leads to increases in accuracy and decreases in latency. Slower presentation rates allow for faster transfer of the target from iconic to short-term memory. When target and background items are dissimilar, transfer to short-term memory need not take place and decisions can be made on the basis of the Stage 1 analysis. These results are taken as confirming evidence for the proposed model.     

Is Iconic Memory Iconic?

Short-term memory in vision is typically thought to divide into at least two memory stores: a short, fragile, high-capacity store known as iconic memory, and a longer, durable, capacity-limited store known as visual working memory (VWM). This paper argues that iconic memory stores icons, i.e., image-like perceptual representations. The iconicity of iconic memory has significant consequences for understanding consciousness, nonconceptual content, and the perception–cognition border. Steven Gross and Jonathan Flombaum have recently challenged the division between iconic memory and VWM by arguing against the idea of capacity limits in favor of a flexible resource-based model of short-term memory. I argue that, while VWM capacity is probably governed by flexible resources rather than a sharp limit, the two memory stores should still be distinguished by their representational formats. Iconic memory stores icons, while VWM stores discursive (i.e., language-like) representations. I conclude by arguing that this format-based distinction between memory stores entails that prominent views about consciousness and the perception–cognition border will likely have to be revised.     

Iconic memory for natural scenes: Evidence using a modified change-detection procedure

ABSTRACT

Change blindness for the contents of natural scenes suggests that only items that are attended while the scene is still visible are stored, leading some to characterize our visual experiences as sparse. Experiments on iconic memory for arrays of discrete symbols or objects, however, indicate observers have access to more visual information for at least several hundred milliseconds at offset of a display. In the experiment presented here, we demonstrate an iconic memory for complex natural or real-world scenes. Using a modified change detection task in which to-be changed objects are cued at offset of the scene, we show that more information from a natural scene is briefly stored than change blindness predicts and more than is contained in visual short-term memory. In our experiment, a cue appearing 0, 300, or 1000?msec after offset of the pre-change scene or at onset of the second scene presentation (a Post Cue) directed attention to the location of a possible change. Compared to a no-cue condition, subjects were significantly better at detecting changes and identifying what changed in the cue condition, with the cue having a diminishing effect as a function of time and no effect when its onset coincided with that of the second scene presentation. The results suggest that an iconic memory of a natural scene exists for at least 1000?msec after scene offset, from which subjects can access the identity of items in the pre-change scene. This implies that change blindness underestimates the amount of information available to the visual system from a brief glance of a natural scene.     

Always look on the broad side of life: happiness increases the breadth of sensory memory

Research has shown that positive affect increases the breadth of information processing at several higher stages of information processing, such as attentional selection or knowledge activation. In the present study, we examined whether these affective influences are already present at the level of transiently storing incoming information in sensory memory, before attentional selection takes place. After inducing neutral, happy, or sad affect, participants performed an iconic memory task which measures visual sensory memory. In all conditions, iconic memory performance rapidly decreased with increasing delay between stimulus presentation and test, indicating that affect did not influence the decay of iconic memory. However, positive affect increased the amount of incoming information stored in iconic memory. In particular, our results showed that this occurs due to an elimination of the spatial bias typically observed in iconic memory. Whereas performance did not differ at positions where observers in the neutral and negative conditions showed the highest performance, positive affect enhanced performance at all positions where observers in the neutral and negative conditions were relatively "blind." These findings demonstrate that affect influences the breadth of information processing already at earliest processing stages, suggesting that affect may produce an even more fundamental shift in information processing than previously believed.     

Absence of a cross-modal “suffix effect” in short-term memory

Three experiments are reported involving the presentation of lists of either letters or digits for immediate serial recall. The main variable was the presence or absence of a suffix-prefix, an item (tick or cross) occurring at the end of the list which had to be copied before recall of the stimulus list. With auditory stimuli and an auditory suffix-prefix there was a large and selective increase in the number of errors on the last few serial positions—the typical “suffix effect”. The suffix effect was not found with auditory stimuli and a visual suffix-prefix nor with a visual stimulus and an auditory suffix-prefix. These results are interpreted as supporting a model for short-term memory proposed by Crowder and Morton (1969) in which it is suggested that with serial recall information concerning the final items following auditory presentation has a different, precategorical, origin from that concerning other items.     

Short-term memory for repeated and non-repeated items

Digit sequences containing repeated items are retained differently in short-term memory from sequences containing no repeated items. The repeated items are remembered better or worse than items in the corresponding positions of “all-different” sequences depending on the number of times the item is repeated, the number of items repeated, the number of items intervening between the occurences of a repeated item, and the position of the repeated items in relation to the beginning and end of the sequence. In every type of repetition studied, except one, memory for the non-repeated items in sequences with repeated items is better than for the corresponding items of all-different sequences. This is true in some cases despite significant specific interference between the (non-repeated) items following the separated occurrences of repeated items. The negative effects in memory for repeated items and the positive effects in memory for non-repeated items are greater when the items are presented at the rate of five per sec. than at one per sec., contrary to the hypothesis that differential rehearsal is responsible for these effects. The results are interpreted as supporting an “associative,” as opposed to a “non-associative,” theory of short-term memory, as this distinction is defined in the paper.     

An expectancy model for memory search

A model for memory search is proposed in which S first forms an “expectancy” regarding the item to be tested on the next trial, then carries out a memory search. It is proposed that an expected item is encoded faster (or perhaps responded to more quickly), but the memory scanning process for expected and nonexpected items is otherwise identical. Assuming a serial exhaustive scanning process, we were able to fit much of the data in the literature. In addition, we tested the model by having S give his expectancy aloud before each trial. The data showed about a 100-msec advantage for expected items that did not interact with memory load. The model fit this data reasonably well.     

Reward boosts working memory encoding over a brief temporal window

Selection mechanisms for WM are ordinarily studied by explicitly cueing a subset of memory items. However, we might also expect the reward associations of stimuli we encounter to modulate their probability of being represented in working memory (WM). Theoretical and computational models explicitly predict that reward value should determine which items will be gated into WM. For example, a model by Braver and colleagues in which phasic dopamine signalling gates WM updating predicts a temporally-specific but not item-specific reward-driven boost to encoding. In contrast, Hazy and colleagues invoke reinforcement learning in cortico-striatal loops and predict an item-wise reward-driven encoding bias. Furthermore, a body of prior work has demonstrated that reward-associated items can capture attention, and it has been shown that attentional capture biases WM encoding. We directly investigated the relationship between reward history and WM encoding. In our first experiment, we found an encoding benefit associated with reward-associated items, but the benefit generalized to all items in the memory array. In a second experiment this effect was shown to be highly temporally specific. We speculate that in real-world contexts in which the environment is sampled sequentially with saccades/shifts in attention, this mechanism could effectively mediate an item-wise encoding bias, because encoding boosts would occur when rewarded items were fixated.     

The basis of the primacy effect: Some experiments with running memory

Five experiments using the “running memory span” (RMS) technique are reported, in which subjects attempt to recall a specified number of items from the end of long sequences of digits, presented at a rate of 2/s. In Experiments I-III critical lists are included in the series which are exactly equal in length to the specified recall series. Despite the RMS set, these critical lists exhibit (I) marked primacy effects, and (2) an impairment in recall of terminal items (a “rebound effect”), compared to the baseline RMS performance. The rebound effect occurs (Experiments IV and V) even when recall of earlier items is not required. These two phenomena are robust: they occur in different experiments in which, rehearsal patterns, report order, expectancies and retrieval load are controlled. The results suggest an origin for primacy which is of a perceptual (i.e. pre-storage) nature, and that selective rehearsal is not a necessary condition for the effect to occur. A possible role of habituation of the orienting response in this phenomenon is discussed.     

Integration of information in visual search

Two experiments studied how information about the nontarget items in a visual search task is used for the control of the search. The first experiment used the detection of “hurdle” stimuli to demonstrate that efficient memory representations of the context items can be established within each particular trial. This finding is explained by a model for the short-term integration of context information. The second experiment which varied the complexity of the local but not the global context provided some information about the nature of the integration operations involved. In its final version the model postulates two stages of processing with independent mechanisms of integration. Spatial integration at the first stage deletes repetitions within small samples. Temporal integration at the second stage stores and primes the memory representations of the context items over larger intervals. It is assumed that transient temporal integration within trials is mediated by the same mechanism that underlies permanent temporal integration between trials.     

Why the spontaneous images created by the hands during talk can help make TV advertisements more effective

The design of effective communications depends upon an adequate model of the communication process. The traditional model is that speech conveys semantic information and bodily movement conveys information about emotion and interpersonal attitudes. But McNeill (2000) argues that this model is fundamentally wrong and that some bodily movements, namely spontaneous hand movements generated during talk (iconic gestures), are integral to semantic communication. But can we increase the effectiveness of communication using this new theory? Focusing on advertising we found that advertisements in which the message was split between speech and iconic gesture (possible on TV) were significantly more effective than advertisements in which meaning resided purely in speech or language (radio/ newspaper). We also found that the significant differences in communicative effectiveness were maintained across five consecutive trials. We compared the communicative power of professionally made TV advertisements in which a spoken message was accompanied either by iconic gestures or by pictorial images, and found the iconic gestures to be more effective. We hypothesized that iconic gestures are so effective because they illustrate and isolate just the core semantic properties of a product. This research suggests that TV advertisements can be made more effective by incorporating iconic gestures with exactly the right temporal and semantic properties.     

Distinctiveness models of memory and absolute identification: evidence for local, not global, effects

Many models of memory assume that the probability of remembering an item is related to how distinctive that item is relative to all the other items in the set, with no distinction made between the contributions of near or far items. These “global” distinctiveness models do well in accounting for the ubiquitous serial position effects observed in numerous memory paradigms, including absolute identification. Here, we provide experimental confirmation of Bower's (1971) suggestion that, contrary to a fundamental prediction of global distinctiveness models, midseries items can be more discriminable than their immediate neighbours. We show that such data are consistent with a revised distinctiveness account in which the factor affecting discrimination performance is primarily the distinctiveness of an item relative to its close neighbours.     

Synaptic tagging, evaluation of memories, and the distal reward problem

Long-term synaptic plasticity exhibits distinct phases. The synaptic tagging hypothesis suggests an early phase in which synapses are prepared, or "tagged," for protein capture, and a late phase in which those proteins are integrated into the synapses to achieve memory consolidation. The synapse specificity of the tags is consistent with conventional neural network models of associative memory. Memory consolidation through protein synthesis, however, is neuron specific, and its functional role in those models has not been assessed. Here, using a theoretical network model, we test the tagging hypothesis on its potential to prolong memory lifetimes in an online-learning paradigm. We find that protein synthesis, though not synapse specific, prolongs memory lifetimes if it is used to evaluate memory items on a cellular level. In our model we assume that only "important" memory items evoke protein synthesis such that these become more stable than "unimportant" items, which do not evoke protein synthesis. The network model comprises an equilibrium distribution of synaptic states that is very susceptible to the storage of new items: Most synapses are in a state in which they are plastic and can be changed easily, whereas only those synapses that are essential for the retrieval of the important memory items are in the stable late phase. The model can solve the distal reward problem, where the initial exposure of a memory item and its evaluation are temporally separated. Synaptic tagging hence provides a viable mechanism to consolidate and evaluate memories on a synaptic basis.