Memory for where, but not what, is used during visual search

Although the role of memory in visual search is debatable, most researchers agree with a limited-capacity model of memory in visual search. The authors demonstrate the role of memory by replicating previous findings showing that visual search is biased away from old items (previously examined items) and toward new items (nonexamined items). Furthermore, the authors examined the type of memory representations used to bias search by changing an item's individuating feature or location during search. Changing the individuating feature of an item did not disrupt normal search biases. However, when the location of an item changed, normal search biases were disrupted. These results suggest that memory used in visual search is based on items' locations rather than their identity.     

Bilateral saccades increase intrahemispheric processing but not interhemispheric interaction: Implications for saccade-induced retrieval enhancement

Retrieval of memories is enhanced when bilateral saccades are made immediately before attempting retrieval. One hypothesis is that saccades enhance retrieval by increasing interaction of the brain hemispheres. To test this, subjects viewed arrays of lateralized letters and indicated whether target letters matched either of two probe letters. Matching targets and probes were presented to either the same hemisphere (within-hemisphere trials) or separate hemispheres (across-hemisphere trials). Match detection requires interhemispheric interaction on across-hemisphere trials but primarily intrahemispheric processing on within-hemisphere trials. Subjects performed letter matching following saccades and a fixation control condition. Saccades increased match-detection accuracy on within-hemisphere trials only, suggesting that, counter to the hypothesis, saccades enhance intrahemispheric processing but not interhemispheric interaction. Across-hemisphere accuracy was higher, however, for subjects who were not strongly right-handed, versus those who were, and the absence of strong right-handedness may reflect greater interhemispheric interaction. We discuss implications for accounts of saccade-induced retrieval enhancement.     

The effect of saccades on number processing

Recent research has shown that saccadic eye movements interfere with dorsal-stream tasks such as judgments of object orientation, but not with ventral-stream tasks such as object recognition. Because saccade programming and execution also rely on the dorsal stream, it has been hypothesized that cognitive saccadic suppression occurs as a result of dual-task interference within the dorsal stream. Judging whether one number is larger or smaller than another (magnitude comparison) is a dorsal-stream task that relies especially on the right parietal cortex. In contrast, judging whether a number is odd or even (parity judgment) does not involve the dorsal stream. In the present study, one group of subjects judged whether two-digit numbers were greater than or less than 65, whereas another group judged whether two-digit numbers were odd or even. Subjects in both groups made these judgments while making no, short, or long saccades. Saccade distance had no effect on parity judgments, but reaction times to make magnitude comparison judgments increased with saccade distance when the eyes moved from right to left. Because the right parietal cortex is instrumental in generating leftward saccades, these results provide further evidence for the hypothesis that cognitive suppression during saccades occurs as a result of dual-task interference within the dorsal stream.     

The control of stimulus-driven saccades is subject not to central, but to visual attention limitations

In three experiments, we investigated whether the control of reflexive saccades is subject to central attention limitations. In a dual-task procedure, Task 1 required either unspeeded reporting or ignoring of briefly presented masked stimuli, whereas Task 2 required a speeded saccade toward a visual target. The stimulus onset asynchrony (SOA) between the two tasks was varied. In Experiments 1 and 2, the Task 1 stimulus was one or three letters, and we asked how saccade target selection is influenced by the number of items. We found (1) longer saccade latencies at short than at long SOAs in the report condition, (2) a substantially larger latency increase for three letters than for one letter, and (3) a latency difference between SOAs in the ignore condition. Broadly, these results match the central interference theory. However, in Experiment 3, an auditory stimulus was used as the Task 1 stimulus, to test whether the interference effects in Experiments 1 and 2 were due to visual instead of central interference. Although there was a small saccade latency increase from short to long SOAs, this difference did not increase from the ignore to the report condition. To explain visual interference effects between letter encoding and stimulus-driven saccade control, we propose an extended theory of visual attention.     

The recovery of identity and relative position from visual input: Further evidence for the independence of processing of what and where

Four experiments examined the manner in which item identity and relative position are recovered from visual input. A successivesame/different matching paradigm was designed in which each trial contained a prime and a target display. Each display contained a reference object (i.e., a “+”) and a located object (i.e., a letter, which fell to either the right or the left of the “+”). In Experiment 1, subjects carried out identity judgments on the letters. Experiment 2 examined relative position judgments; in Experiment 3, subjects had to judge both item identity and relative position information. Overall, these initial data suggested that identity and positional information are recovered via independent mechanisms and that these operate concurrently. This suggestion was supported by the results of Experiment 4, which in turn disconfirmed an alternative response account of performance.     

Transient stimulation does not aid visual search: Implications for the role of saccades

Previous demonstrations that performance on visual search tasks with briefly flashed presentations declines over time after the initial onset imply that saccades might be necessary for efficient acquisition of visual information. We imposed an additional abrupt onset and a displacement on character arrays containing letters and one numeral while subjects searched for the numeral. Presentations were always followed by a visual noise field. Presentation time varied (50–800 msec). We found that performance with presentations containing one onset and remaining continuously visible was better than performance with presentations containing two onsets and containing displacements. Furthermore, information acquired near the onset of a continuous presentation was as effective as information acquired later. Our results demonstrate that abrupt onsets and displacements do not improve information acquisition with displays of alphanumeric characters. This finding is consistent with earlier reports that saccade-like retinal image motion does not contribute either to the maintenance of target visibility or to visual acuity.     

Suppressing visual feedback in written composition: Effects on processing demands and coordination of the writing processes

The goal of this experiment was to investigate the role of visual feedback during written composition. Effects of suppression of visual feedback were analyzed both on processing demands and on on‐line coordination of low‐level execution processes and of high‐level conceptual and linguistic processes. Writers composed a text and copied it either with or without visual feedback. Processing demands of the writing processes were evaluated with reaction times to secondary auditory probes, which were analyzed according to whether participants were handwriting (in a composing and a copying task) or engaged in high‐level processes (when pausing in a composing task). Suppression of visual feedback increased reaction time interference (secondary reaction time minus baseline reaction time) during handwriting in the copying task and not during pauses in the composing task. This suggests that suppression of visual feedback only affected processing demands of execution processes and not those of high‐level conceptual and linguistic processes. This is confirmed by analysis of the quality of the texts produced by participants, which were little, if at all, affected by the suppression of visual feedback. Results also indicate that the increase in processing demands of execution related to suppression of visual feedback affected on‐line coordination of the writing processes. Indeed, when visual feedback was suppressed, reaction time interferences associated with handwriting were not reliably different in the copying task or the composing task but were significantly different when visual feedback was not suppressed: They were lower in the copying task than in the composition task. When visual feedback was suppressed, writers activated step‐by‐step execution processes and high‐level writing processes, whereas they concurrently activated these writing processes when composing with visual feedback.     

Defining the cortical visual systems: "what", "where", and "how"

The visual system historically has been defined as consisting of at least two broad subsystems subserving object and spatial vision. These visual processing streams have been organized both structurally as two distinct pathways in the brain, and functionally for the types of tasks that they mediate. The classic definition by Ungerleider and Mishkin labeled a ventral "what" stream to process object information and a dorsal "where" stream to process spatial information. More recently, Goodale and Milner redefined the two visual systems with a focus on the different ways in which visual information is transformed for different goals. They relabeled the dorsal stream as a "how" system for transforming visual information using an egocentric frame of reference in preparation for direct action. This paper reviews recent research from psychophysics, neurophysiology, neuropsychology and neuroimaging to define the roles of the ventral and dorsal visual processing streams. We discuss a possible solution that allows for both "where" and "how" systems that are functionally and structurally organized within the posterior parietal lobe.     

Impaired initiation but not execution of contralesional saccades in hemispatial neglect

Patients with unilateral neglect are impaired at making saccades to contralesional targets. Whether this problem arises from a deficit in perception, in planning the saccade or in executing the eye movement or some combination thereof remains unclear. We measured several variables related to the initiation and execution of saccades in an experiment which crossed two factors: target side (left, right) and direction of saccade (leftwards, rightwards). Relative to control subjects, patients with left-sided neglect were impaired in planning but not executing the contralesional saccade; while the latency to move their eyes following the onset of the target was increased, the duration and velocity to reach the target were normal. In addition, there were also no directional differences for saccades that were hypometric or inaccurate in the patients, further ruling out an execution impairment. Interestingly, this directional initiation deficit was exaggerated for leftward saccades to left targets, compared with all other conditions. We suggest that the disadvantage for contralesional saccades in neglect patients is attributable to a deficit not only in perceiving contralateral targets but also in planning leftward saccades. Once the saccade is initiated, however, execution apparently proceeds unimpaired.     

Do you see what I see? Affect and visual information processing

Individuals in sad moods process information in a less global and more local manner than do those in happier moods. This experiment investigates whether processing speed is associated with these mood effects, whether task ambiguity moderates these mood effects, and whether making feelings appear irrelevant to the task can eliminate these mood effects. Participants in happy, sad, and neutral moods were lead to experience their feelings as being either relevant or irrelevant to a global/local processing task. As predicted, sad moods decreased global processing relative to happier moods when feelings seemed relevant to the task and when the criteria for responding were ambiguous, but not when feelings seemed irrelevant or when the criteria were unambiguous. Consistent with the idea that mood guides processing, increases in affect intensity were associated with faster reaction times. Overall, the results suggest that mood and processing effects share some core similarities with mood and judgement effects.     

Neural evidence for the contribution of holistic processing but not attention allocation to the other-race effect on face memory

Multiple mechanisms have been suggested to contribute to the other-race effect on face memory, the phenomenon of better memory performance for own-race than other-race faces. Here, two of these mechanisms, increased attention allocation and greater holistic processing during memory encoding for own-race than other-race faces, were tested in two separate experiments. In these experiments event-related potentials were measured during study (the difference due to memory, Dm) and test phase (old/new effects) to examine brain activation related to memory encoding and retrieval, allowing for selective investigations of these memory sub-processes. In Experiment 1, participants studied own-race (Caucasian) and other-race (Chinese) faces under focused or divided attention. In Experiment 2, participants studied own-race (Caucasian) and other-race (African American) faces presented upright or upside down (i.e., inverted). Both experiments showed decreases in memory performance when attention allocation or holistic processing was reduced, but these effects were similar for own-race and other-race faces. Manipulations of holistic processing, but not attention allocation, influenced the neural other-race effects during memory encoding. Inverted own-race faces showed similar neural patterns as upright other-race faces, indicating that when holistic processing of own-race faces was reduced, these faces were encoded similarly as upright other-race faces. No influences of the experimental manipulations on other-race effects during memory retrieval were found. The present study provides the first neural evidence that increased holistic processing during memory encoding contributes to the other-race effect on face memory.     

Rhesus monkeys (Macaca mulatta) demonstrate robust memory for what and where,but not when,in an open-field test of memory

We adapted a paradigm developed by Clayton and Dickinson (1998), who demonstrated memory for what, where, and when in scrub jays, for use with rhesus monkeys. In the study phase of each trial, monkeys found a preferred and a less-preferred food reward in a trial-unique array of three locations in a large room. After 1 h, monkeys returned to the test room, where they found foods placed as during study. Twenty-five hours after the study phase monkeys again searched the room, but now the preferred food was replaced with a distasteful food remnant, while the less-preferred food was still present. Although monkeys remembered the locations of the foods for up to 25 h, they did not learn that the preferred food was available after the short, but not after the long delay. Thus, monkeys demonstrated long-term memory for the type and location of food but failed to demonstrate sensitivity to when they acquired that knowledge.     

The effect of a visual indicator on rate of visual search: Evidence for processing control

Search rates were estirnated from response latencies in a visual search task of the type used by Atkinson, Holmgren, and Juola (1969), in which a S searches a small set of letters to determine the presence or absence of a predesignated target. Half of the visual displays contained a marker above one of the letters. The marked letter was the only one that had to be checked to determine whether or not the display contained the target. The presence of a marker in a display significantly increased the estimated rate of search, but the data clearly indicated that Ss did not restrict processing to the marked item. Letters in the vicinity of the marker were also processed. These results were interpreted as showing that Ss are able to exercise some degree of control over the search process in this type of task.     

It's not what you say but the way you say it: matching faces and voices

Recent studies have shown that the face and voice of an unfamiliar person can be matched for identity. Here the authors compare the relative effects of changing sentence content (what is said) and sentence manner (how it is said) on matching identity between faces and voices. A change between speaking a sentence as a statement and as a question disrupted matching performance, whereas changing the sentence itself did not. This was the case when the faces and voices were from the same race as participants and speaking a familiar language (English; Experiment 1) or from another race and speaking an unfamiliar language (Japanese; Experiment 2). Altering manner between conversational and clear speech (Experiment 3) or between conversational and casual speech (Experiment 4) was also disruptive. However, artificially slowing (Experiment 5) or speeding (Experiment 6) speech did not affect cross-modal matching performance. The results show that bimodal cues to identity are closely linked to manner but that content (what is said) and absolute tempo are not critical. Instead, prosodic variations in rhythmic structure and/or expressiveness may provide a bimodal, dynamic identity signature.     

Naloxone modulates visual judgments of similarity but not dissimilarity

Endogenous opioids have been implicated in mediating (placebo) analgesia, in reward processes, and in the regulation of socially relevant emotions. To explore their potential contributions to higher cognitive functions, we used a novel task with tachistoscopically presented (for 150 ms) pairs of meaningless figures. Healthy right-handed men judged the similarities and dissimilarities between the two figures on a visual analogue scale (VAS) in two separate runs. In a double-blind, between-subjects design, subjects were administered intravenously either 0.2-mg/kg naloxone or placebo 10 min prior to the task, and VAS judgments and response latencies were measured. We found a significant interaction between substance group and type of judgment: The magnitude of the similarity judgments was lower in the naloxone than in the placebo group, while dissimilarity judgments remained uninfluenced by the treatment. Reaction latencies and mood scores, assessed before and after substance administration, did not differ between the two groups, indicating that the findings did not rely on altered motor performance or motivation. We suggest that naloxone decreased the “similarity criterion” in comparative judgments, indicating its potentially modulatory effect on visual cognition. The task introduced here could be used for the implicit study and quantification of subtle affective–cognitive processes beyond the level of mere questionnaire data.     

Behavioral evidence for task-dependent "what" versus "where" processing within and across modalities

Task-dependent information processing for the purpose of recognition or spatial perception is considered a principle common to all the main sensory modalities. Using a dual-task interference paradigm, we investigated the behavioral effects of independent information processing for shape identificationand localization ofobject features within and across vision and touch. In Experiment 1, we established that color and texture processing (i.e., a "what" task) interfered with both visual and haptic shape-matching tasks and that mirror image and rotation matching (i.e., a "where" task) interfered with a feature-location-matching task in both modalities. In contrast, interference was reduced when a "where" interference task was embedded in a "what" primary task and vice versa. In Experiment 2, we replicated this finding within each modality, using the same interference and primary tasks throughout. In Experiment 3, the interference tasks were always conducted in a modality other than the primary task modality. Here, we found that resources for identification and spatial localization are independent of modality. Our findings further suggest that multisensory resources for shape recognition also involve resources for spatial localization. These results extend recent neuropsychological and neuroimaging findings and have important implications for our understanding of high-level information processing across the human sensory systems.     

The effect of early visual deprivation on the development of face processing

We evaluated the importance of early visual input for the later development of expertise in face processing by studying 17 patients, aged 10 to 38 years, treated for bilateral congenital cataracts that deprived them of patterned visual input for the first 7 weeks or more after birth. We administered five computerized tasks that required matching faces on the basis of identity (with changed facial expression or head orientation), facial expression, gaze direction and lip reading. Compared to an age–matched control group, patients’ recognition of facial identity was impaired significantly when there was a change in head orientation (e.g. from frontal to tilted up), and tended to be impaired when there was a change in facial expression (e.g. from happy to surprised). Patients performed normally when matching facial expression and direction of gaze (e.g. looking left or right), and in reading lips (e.g. pronouncing ‘u’ or ‘a’). The results indicate that visual input during early infancy is necessary for the normal development of some aspects of face processing, and are consistent with theories postulating the importance of early visual experience (de Schonen & Mathivet, 1989; Johnson & Morton, 1991) and separate neural mediation of different components of face processing (Bruce & Young, 1986).