The contributions of visual and central attention to visual working memory

We investigated the role of two kinds of attention—visual and central attention—for the maintenance of visual representations in working memory (WM). In Experiment 1 we directed attention to individual items in WM by presenting cues during the retention interval of a continuous delayed-estimation task, and instructing participants to think of the cued items. Attending to items improved recall commensurate with the frequency with which items were attended (0, 1, or 2 times). Experiments 1 and 3 further tested which kind of attention—visual or central—was involved in WM maintenance. We assessed the dual-task costs of two types of distractor tasks, one tapping sustained visual attention and one tapping central attention. Only the central attention task yielded substantial dual-task costs, implying that central attention substantially contributes to maintenance of visual information in WM. Experiment 2 confirmed that the visual-attention distractor task was demanding enough to disrupt performance in a task relying on visual attention. We combined the visual-attention and the central-attention distractor tasks with a multiple object tracking (MOT) task. Distracting visual attention, but not central attention, impaired MOT performance. Jointly, the three experiments provide a double dissociation between visual and central attention, and between visual WM and visual object tracking: Whereas tracking multiple targets across the visual filed depends on visual attention, visual WM depends mostly on central attention.     

Bindings in working memory: The role of object-based attention

Over the past decade, it has been debated whether retaining bindings in working memory (WM) requires more attention than retaining constituent features, focusing on domain-general attention and space-based attention. Recently, we proposed that retaining bindings in WM needs more object-based attention than retaining constituent features (Shen, Huang, & Gao, 2015, Journal of Experimental Psychology: Human Perception and Performance, doi: 10.1037/xhp0000018). However, only unitized visual bindings were examined; to establish the role of object-based attention in retaining bindings in WM, more emperical evidence is required. We tested 4 new bindings that had been suggested requiring no more attention than the constituent features in the WM maintenance phase: The two constituent features of binding were stored in different WM modules (cross-module binding, Experiment 1), from auditory and visual modalities (cross-modal binding, Experiment 2), or temporally (cross-time binding, Experiments 3) or spatially (cross-space binding, Experiments 4–6) separated. In the critical condition, we added a secondary object feature-report task during the delay interval of the change-detection task, such that the secondary task competed for object-based attention with the to-be-memorized stimuli. If more object-based attention is required for retaining bindings than for retaining constituent features, the secondary task should impair the binding performance to a larger degree relative to the performance of constituent features. Indeed, Experiments 1–6 consistently revealed a significantly larger impairment for bindings than for the constituent features, suggesting that object-based attention plays a pivotal role in retaining bindings in WM.     

The concreteness effect on judgments of learning: Evaluating the contributions of fluency and beliefs

Researchers have often determined how cues influence judgments of learning (JOLs; e.g., concrete words are assigned higher JOLs than are abstract words), and recently there has been an emphasis in understanding why cues influence JOLs (i.e., the mechanisms that underlie cue effects on JOLs). The analytic-processing (AP) theory posits that JOLs are constructed in accordance with participants’ beliefs of how a cue will influence memory. Even so, some evidence suggests that fluency is also important to cue effects on JOLs. In the present experiments, we investigated the contributions of participants’ beliefs and processing fluency to the concreteness effect on JOLs. To evaluate beliefs, participants estimated memory performance in a hypothetical experiment (Experiment 1), and studied concrete and abstract words and made a pre-study JOL for each (Experiments 2 and 3). Participants’ predictions demonstrated the belief that concrete words are more likely to be remembered than are abstract words, consistent with the AP theory. To evaluate fluency, response latencies were measured during lexical decision (Experiment 4), self-paced study (Experiment 5), and mental imagery (Experiment 7). Number of trials to acquisition was also evaluated (Experiment 6). Fluency did not differ between concrete and abstract words in Experiments 5 and 6, and it did not mediate the concreteness effect on JOLs in Experiments 4 and 7. Taken together, these results demonstrate that beliefs are a primary mechanism driving the concreteness effect on JOLs.     

Revisiting von Restorff’s early isolation effect

An item that is conceptually or physically different from other items in a series is often remembered well. This isolation effect has been found independent of the position of the isolated item in the list, suggesting that special attention to or processing of the isolated item is not a necessary precondition of the effect. Three experiments are reported that challenge this conclusion. In Experiment 1a, we compared memory for conceptually isolated items to memory for the same items in unrelated and homogeneous lists. Under moderately distracting conditions, isolation effects were observed with midlist but not with early isolates. In fact, early isolation impaired memory for the conceptually distinct items relative to the same items in homogeneous lists. Experiment 1b replicated this memory impairment for early conceptual isolates and extended it to nondistracting conditions. In Experiment 2, we focused on early isolation, manipulating the type of isolation and whether or not participants performed judgments of learning (JOLs). An early isolation effect was observed for numbers isolated in lists of words (and vice versa), but not for conceptual isolates. Performing the JOL task reduced the size of the early isolation effect. These results suggest that number/word stimulus contrasts are coded automatically and support an isolation effect independent of list position. However, conceptual contrasts require relational processing and will only support an early isolation effect when such processing occurs. The results of Experiments 1a, 1b, and 2 suggest that attentional resources during list presentation and a favorable retrieval environment combine to support good memory for distinctive events.     

Relationship between vection and motion perception in depth

This study examined the effects of cues to motion in depth – namely, stereoscopic (i.e., changing-disparity cues and interocular velocity differences) and changing-size cues on forward and backward vection. We conducted four experiments in which participants viewed expanding or contracting optical flows with the addition of either or both cues. In Experiment 1, participants reported vection by pressing a button whenever they felt it. After each trial, they also rated the magnitude of the vection (from 0 to 100). In Experiments 2 and 3, the participants rated the perceived velocity and motion-in-depth impression of the flows relative to standard stimuli, respectively. In Experiment 4, the participants rated the perceived depth and distance of the display. We observed enhancements in vection, motion-in-depth impression, and perceived depth and distance when either or both types of cues indicated motion-in-depth, as compared to those when the cues did not (Experiments 1, 3, and 4). The perceived velocity changed with cue conditions only for the high velocity condition (Experiment 2). Correlational analyses showed that the vection can be best explained by the motion-in-depth impression. This was partially supported by the multiple regression analyses. These results indicate that the enhancement of vection caused by cues is related to the impression of motion-in-depth rather than the perceived velocity and perceived three-dimensionality.     

Explaining the forgetting bias effect on value judgments: The influence of memory for a past test

People often feel that information that was forgotten is less important than remembered information. Prior work has shown that participants assign higher importance to remembered information while undervaluing forgotten information. The current study examined two possible accounts of this finding. In three experiments, participants studied lists of words in which each word was randomly assigned a point value denoting the value of remembering the word. Following the presentation of each list participants engaged in a free recall test. After the presentation of all lists participants were shown each of the words they had studied and asked to recall the point value that was initially paired with each word. Experiment 1 tested a fluency-based account by presenting items for value judgments in a low-fluency or high-fluency format. Experiment 2 examined whether value judgments reflect attributions based on the familiarity of an item when value judgments are made. Finally, in Experiment 3, we evaluated whether participants believe that forgotten words are less important by having them judge whether an item was initially recalled or forgotten prior to making a value judgment. Manipulating the fluency of an item presented for judgment had no influence on value ratings (Experiment 1) and familiarity exerted a limited influence on value judgments (Experiment 2). More importantly, participants’ value judgments appeared to reflect a theory that remembered information is more valuable than forgotten information (Experiment 3). Overall, the present work suggests that individuals may apply a theory about remembering and forgetting to retrospectively assess the value of information.     

The influence of attention on value integration

People often have to make decisions based on many pieces of information. Previous work has found that people are able to integrate values presented in a rapid serial visual presentation (RSVP) stream to make informed judgements on the overall stream value (Tsetsos et al. Proceedings of the National Academy of Sciences of the United States of America, 109(24), 9659–9664, 2012). It is also well known that attentional mechanisms influence how people process information. However, it is unknown how attentional factors impact value judgements of integrated material. The current study is the first of its kind to investigate whether value judgements are influenced by attentional processes when assimilating information. Experiments 1–3 examined whether the attentional salience of an item within an RSVP stream affected judgements of overall stream value. The results showed that the presence of an irrelevant high or low value salient item biased people to judge the stream as having a higher or lower overall mean value, respectively. Experiments 4–7 directly tested Tsetsos et al.’s (Proceedings of the National Academy of Sciences of the United States of America, 109(24), 9659–9664, 2012) theory examining whether extreme values in an RSVP stream become over-weighted, thereby capturing attention more than other values in the stream. The results showed that the presence of both a high (Experiments 4, 6 and 7) and a low (Experiment 5) value outlier captures attention leading to less accurate report of subsequent items in the stream. Taken together, the results showed that valuations can be influenced by attentional processes, and can lead to less accurate subjective judgements.     

Simultaneous utilization of multiple cues in judgments of learning

There is much evidence that metacognitive judgments, such as people’s predictions of their future memory performance (judgments of learning, JOLs), are inferences based on cues and heuristics. However, relatively little is known about whether and when people integrate multiple cues in one metacognitive judgment or focus on a single cue without integrating further information. The current set of experiments systematically addressed whether and to what degree people integrate multiple extrinsic and intrinsic cues in JOLs. Experiment 1 varied two cues: number of study presentations (1 vs. 2) and font size (18 point vs. 48 point). Results revealed that people integrated both cues in their JOLs. Experiment 2 demonstrated that the two word characteristics concreteness (abstract vs. concrete) and emotionality (neutral vs. emotional) were integrated in JOLs. Experiment 3 showed that people integrated all four cues in their JOLs when manipulated simultaneously. Finally, Experiment 4 confirmed integration of three cues that varied on a continuum rather than in two easily distinguishable levels. These results demonstrate that people have a remarkable capacity to integrate multiple cues in metacognitive judgments. In addition, our findings render an explanation of cue effects on JOLs in terms of demand characteristics implausible.     

Cue quality and criterion setting in recognition memory

Previous studies on how people set and modify decision criteria in old-new recognition tasks (in which they have to decide whether or not a stimulus was seen in a study phase) have almost exclusively focused on properties of the study items, such as presentation frequency or study list length. In contrast, in the three studies reported here, we manipulated the quality of the test cues in a scene-recognition task, either by degrading through Gaussian blurring (Experiment 1) or by limiting presentation duration (Experiment 2 and 3). In Experiments 1 and 2, degradation of the test cue led to worse old-new discrimination. Most importantly, however, participants were more liberal in their responses to degraded cues (i.e., more likely to call the cue “old”), demonstrating strong within-list, item-by-item, criterion shifts. This liberal response bias toward degraded stimuli came at the cost of increasing the false alarm rate while maintaining a constant hit rate. Experiment 3 replicated Experiment 2 with additional stimulus types (words and faces) but did not provide accuracy feedback to participants. The criterion shifts in Experiment 3 were smaller in magnitude than Experiments 1 and 2 and varied in consistency across stimulus type, suggesting, in line with previous studies, that feedback is important for participants to shift their criteria.     

Predictive visual search: Role of environmental regularities in the learning of context cues

Repeatedly searching through invariant spatial arrangements in visual search displays leads to the buildup of memory about these displays (contextual-cueing effect). In the present study, we investigate (1) whether contextual cueing is influenced by global statistical properties of the task and, if so, (2) whether these properties increase the overall strength (asymptotic level) or the temporal development (speed) of learning. Experiment 1a served as baseline against which we tested the effects of increased or decreased proportions of repeated relative to nonrepeated displays (Experiments 1b and 1c, respectively), thus manipulating the global statistical properties of search environments. Importantly, probability variations were achieved by manipulating the number of nonrepeated (baseline) displays so as to equate the total number of repeated displays across experiments. In Experiment 1d, repeated and nonrepeated displays were presented in longer streaks of trials, thus establishing a stable environment of sequences of repeated displays. Our results showed that the buildup of contextual cueing was expedited in the statistically rich Experiments 1b and 1d, relative to the baseline Experiment 1a. Further, contextual cueing was entirely absent when repeated displays occurred in the minority of trials (Experiment 1c). Together, these findings suggest that contextual cueing is modulated by observers’ assumptions about the reliability of search environments.     

Dissociations of the number and precision of visual short-term memory representations in change detection

The present study dissociated the number (i.e., quantity) and precision (i.e., quality) of visual short-term memory (STM) representations in change detection using receiver operating characteristic (ROC) and experimental manipulations. Across three experiments, participants performed both recognition and recall tests of visual STM using the change-detection task and the continuous color-wheel recall task, respectively. Experiment 1 demonstrated that the estimates of the number and precision of visual STM representations based on the ROC model of change-detection performance were robustly correlated with the corresponding estimates based on the mixture model of continuous-recall performance. Experiments 2 and 3 showed that the experimental manipulation of mnemonic precision using white-noise masking and the experimental manipulation of the number of encoded STM representations using consolidation masking produced selective effects on the corresponding measures of mnemonic precision and the number of encoded STM representations, respectively, in both change-detection and continuous-recall tasks. Altogether, using the individual-differences (Experiment 1) and experimental dissociation (Experiment 2 and 3) approaches, the present study demonstrated the some-or-none nature of visual STM representations across recall and recognition.     

The roles of order,distance, and interstitial items in temporal visual statistical learning

Humans are adept at learning regularities in a visual environment, even without explicit cues to structure and in the absence of instruction—this has been termed “visual statistical learning” (VSL). The nature of the representations resulting from VSL are still poorly understood. In five experiments, we examined the specificity of temporal VSL representations. In Experiments 1A, 1B, and 2, we compared recognition rates of triplets and all embedded pairs to chance. Robust learning of all structures was evident, and even pairs of non-adjacent items in a sequentially presented triplet (AC extracted from a triplet composed of ABC) were recognized at above-chance levels. In Experiment 3, we asked whether people could recognize rearranged pairs to examine the flexibility of learned representations. Recognition of all possible orders of target triplets and pairs was significantly higher than chance, and there were no differences between canonical orderings and their corresponding randomized orderings, suggesting that learners were not dependent upon originally experienced stimulus orderings to recognize co-occurrence. Experiment 4 demonstrates the essential role of an interstitial item in VSL representations. By comparing the learning of quadruplet sets (e.g., ABCD) and triplet sets (e.g., ABC), we found learning of AC and BD in ABCD (quadruplet) sets were better than the learning of AC in ABC (triplet) sets. This pattern of results might result from the critical role of interstitial items in statistical learning. In short, our work supports the idea of generalized representation in VSL and provides evidence about how this representation is structured.     

Cognitive load effects on early visual perceptual processing

Contrast-based early visual processing has largely been considered to involve autonomous processes that do not need the support of cognitive resources. However, as spatial attention is known to modulate early visual perceptual processing, we explored whether cognitive load could similarly impact contrast-based perception. We used a dual-task paradigm to assess the impact of a concurrent working memory task on the performance of three different early visual tasks. The results from Experiment 1 suggest that cognitive load can modulate early visual processing. No effects of cognitive load were seen in Experiments 2 or 3. Together, the findings provide evidence that under some circumstances cognitive load effects can penetrate the early stages of visual processing and that higher cognitive function and early perceptual processing may not be as independent as was once thought.     

The Perceptual Basis of Vast Space

“Vast” is a word often applied to environmental terrain that is perceived to have large spatial extent. This judgment is made even at viewing distances where traditional metric depth cues are not useful. This paper explores the perceptual basis of vast experience, including reliability and visual precursors. Experiment 1 demonstrated strong agreement in ratings of the spatial extent of two-dimensional (2D) scene images by participants in two countries under very different viewing conditions. Image categories labeled “vast” often exemplified scene attributes of ruggedness and openness (Oliva & Torralba, 2001). Experiment 2 quantitatively assessed whether these properties predict vastness. High vastness ratings were associated with highly open, or moderately open but rugged, scenes. Experiment 3 provided evidence, consistent with theory, that metric distance perception does not directly mediate the observed vastness ratings. The question remains as to how people perceive vast space when information about environmental scale is unavailable from metric depth cues or associated scene properties. We consider possible answers, including contribution from strong cues to relative depth.     

The mere exposure effect for visual image

Mere exposure effect refers to a phenomenon in which repeated stimuli are evaluated more positively than novel stimuli. We investigated whether this effect occurs for internally generated visual representations (i.e., visual images). In an exposure phase, a 5 × 5 dot array was presented, and a pair of dots corresponding to the neighboring vertices of an invisible polygon was sequentially flashed (in red), creating an invisible polygon. In Experiments 1, 2, and 4, participants visualized and memorized the shapes of invisible polygons based on different sequences of flashed dots, whereas in Experiment 3, participants only memorized positions of these dots. In a subsequent rating phase, participants visualized the shape of the invisible polygon from allocations of numerical characters on its vertices, and then rated their preference for invisible polygons (Experiments 1, 2, and 3). In contrast, in Experiment 4, participants rated the preference for visible polygons. Results showed that the mere exposure effect appeared only when participants visualized the shape of invisible polygons in both the exposure and rating phases (Experiments 1 and 2), suggesting that the mere exposure effect occurred for internalized visual images. This implies that the sensory inputs from repeated stimuli play a minor role in the mere exposure effect. Absence of the mere exposure effect in Experiment 4 suggests that the consistency of processing between exposure and rating phases plays an important role in the mere exposure effect.     

Training specificity and transfer in time and distance estimation

Learning is often specific to the conditions of training, making it important to identify which aspects of the testing environment are crucial to be matched in the training environment. In the present study, we examined training specificity in time and distance estimation tasks that differed only in the focus of processing (FOP). External spatial cues were provided for the distance estimation task and for the time estimation task in one condition, but not in another. The presence of a concurrent alphabet secondary task was manipulated during training and testing in all estimation conditions in Experiment 1. For distance as well as for time estimation in both conditions, training of the primary estimation task was found to be specific to the presence of the secondary task. In Experiments 2 and 3, we examined transfer between one estimation task and another, with no secondary task in either case. When all conditions were equal aside from the FOP instructions, including the presence of external spatial cues, Experiment 2 showed “transfer” between tasks, suggesting that training might not be specific to the FOP. When the external spatial cues were removed from the time estimation task, Experiment 3 showed no transfer between time and distance estimations, suggesting that external task cues influenced the procedures used in the estimation tasks.     

Age of avatar modulates the altercentric bias in a visual perspective-taking task: ERP and behavioral evidence

Despite being able to rapidly and accurately infer their own and other peoples’ visual perspectives, healthy adults experience difficulty ignoring the irrelevant perspective when the two perspectives are in conflict; they experience egocentric and altercentric interference. We examine for the first time how the age of an observed person (adult vs. child avatar) influences adults’ visual perspective-taking, particularly the degree to which they experience interference from their own or the other person’s perspective. Participants completed the avatar visual perspective-taking task, in which they verified the number of discs in a visual scene according to either their own or an on-screen avatar’s perspective (Experiments 1 and 2) or only from their own perspective (Experiment 3), where the two perspectives could be consistent or in conflict. Age of avatar was manipulated between (Experiment 1) or within (Experiments 2 and 3) participants, and interference was assessed using behavioral (Experiments 1–3) and ERP (Experiment 1) measures. Results revealed that altercentric interference is reduced or eliminated when a child avatar was present, suggesting that adults do not automatically compute a child avatar’s perspective. We attribute this pattern to either enhanced visual processing for own-age others or an inference on reduced mental awareness in younger children. The findings argue against a purely attentional basis for the altercentric effect, and instead support an account where both mentalising and directional processes modulate automatic visual perspective-taking, and perspective-taking effects are strongly influenced by experimental context.     

Numerosity estimates for attended and unattended items in visual search

The goal of this research was to examine memories created for the number of items during a visual search task. Participants performed a visual search task for a target defined by a single feature (Experiment 1A), by a conjunction of features (Experiment 1B), or by a specific spatial configuration of features (Experiment 1C). On some trials following the search task, subjects were asked to recall the total number of items in the previous display. In all search types, participants underestimated the total number of items, but the severity of the underestimation varied depending on the efficiency of the search. In three follow-up studies (Experiments 2A, 2B, and 2C) using the same visual stimuli, the participants’ only task was to estimate the number of items on each screen. Participants still underestimated the numerosity of the items, although the degree of underestimation was smaller than in the search tasks and did not depend on the type of visual stimuli. In Experiment 3, participants were asked to recall the number of items in a display only once. Subjects still displayed a tendency to underestimate, indicating that the underestimation effects seen in Experiments 1A-1C were not attributable to knowledge of the estimation task. The degree of underestimation depends on the efficiency of the search task, with more severe underestimation in efficient search tasks. This suggests that the lower attentional demands of very efficient searches leads to less encoding of numerosity of the distractor set.     

Memory guidance in distractor suppression is governed by the availability of cognitive control

Information stored in the memory systems can affect visual search. Previous studies have shown that holding the to-be-ignored features of distractors in working memory (WM) could accelerate target selection. However, such facilitation effect was only observed when the cued to-be-ignored features remained unchanged within an experimental block (i.e., the fixed cue condition). No search benefit was obtained if the to-be-ignored features varied from trial to trial (i.e., the varied cue condition). In the present study, we conducted three behavioral experiments to investigate whether the WM and long-term memory (LTM) representations of the to-be-ignored features could facilitate visual search in the fixed cue (Experiment 1) and varied cue (Experiments 2 and 3) conditions. Given the importance of the processing time of cognitive control in distractor suppression, we divided visual search trials into five quintiles based on their reaction times (RTs) and examined the temporal characteristics of the suppression effect. Results showed that both the WM and LTM representations of the to-be-ignored features could facilitate distractor suppression in the fixed cue condition, and the facilitation effects were evident across the quintiles in the RT distribution. However, in the varied cue condition, the RT benefits of the WM-matched distractors occurred only in the trials with the longest RTs, whereas no advantage of the LTM-matched distractors was observed. These results suggest that the effective WM-guided distractor suppression depends on the availability of cognitive control and the LTM-guided suppression occurs only if sufficient WM resource is accessible by LTM reactivation.     

Visual search for changes in scenes creates long-term,incidental memory traces

Humans are very good at remembering large numbers of scenes over substantial periods of time. But how good are they at remembering changes to scenes? In this study, we tested scene memory and change detection two weeks after initial scene learning. In Experiments 1–3, scenes were learned incidentally during visual search for change. In Experiment 4, observers explicitly memorized scenes. At test, after two weeks observers were asked to discriminate old from new scenes, to recall a change that they had detected in the study phase, or to detect a newly introduced change in the memorization experiment. Next, they performed a change detection task, usually looking for the same change as in the study period. Scene recognition memory was found to be similar in all experiments, regardless of the study task. In Experiment 1, more difficult change detection produced better scene memory. Experiments 2 and 3 supported a “depth-of-processing” account for the effects of initial search and change detection on incidental memory for scenes. Of most interest, change detection was faster during the test phase than during the study phase, even when the observer had no explicit memory of having found that change previously. This result was replicated in two of our three change detection experiments. We conclude that scenes can be encoded incidentally as well as explicitly and that changes in those scenes can leave measurable traces even if they are not explicitly recalled.