Defining the cortical visual systems: "what", "where", and "how"

The visual system historically has been defined as consisting of at least two broad subsystems subserving object and spatial vision. These visual processing streams have been organized both structurally as two distinct pathways in the brain, and functionally for the types of tasks that they mediate. The classic definition by Ungerleider and Mishkin labeled a ventral "what" stream to process object information and a dorsal "where" stream to process spatial information. More recently, Goodale and Milner redefined the two visual systems with a focus on the different ways in which visual information is transformed for different goals. They relabeled the dorsal stream as a "how" system for transforming visual information using an egocentric frame of reference in preparation for direct action. This paper reviews recent research from psychophysics, neurophysiology, neuropsychology and neuroimaging to define the roles of the ventral and dorsal visual processing streams. We discuss a possible solution that allows for both "where" and "how" systems that are functionally and structurally organized within the posterior parietal lobe.     

Distinct patterns of viewpoint-dependent BOLD activity during common-object recognition and mental rotation

A fundamental but unanswered question about the human visual system concerns the way in which misoriented objects are recognized. One hypothesis maintains that representations of incoming stimuli are transformed via parietally based spatial normalization mechanisms (eg mental rotation) to match view-specific representations in long-term memory. Using fMRI, we tested this hypothesis by directly comparing patterns of brain activity evoked during classic mental rotation and misoriented object recognition involving everyday objects. BOLD activity increased systematically with stimulus rotation within the ventral visual stream during object recognition and within the dorsal visual stream during mental rotation. More specifically, viewpoint-dependent activity was significantly greater in the right superior parietal lobule during mental rotation than during object recognition. In contrast, viewpoint-dependent activity was significantly greater in the right fusiform gyrus during object recognition than during mental rotation. In addition to these differences in viewpoint-dependent activity, object recognition and mental rotation produced distinct patterns of brain activity, independent of stimulus rotation: object recognition resulted in greater overall activity within ventral stream visual areas and mental rotation resulted in greater overall activity within dorsal stream visual areas. The present results are inconsistent with the hypothesis that misoriented object recognition is mediated by structures within the parietal lobe that are known to be involved in mental rotation.     

Face and location processing in children with early unilateral brain injury

Human visuospatial functions are commonly divided into those dependent on the ventral visual stream (ventral occipitotemporal regions), which allows for processing the ‘what’ of an object, and the dorsal visual stream (dorsal occipitoparietal regions), which allows for processing ‘where’ an object is in space. Information about the development of each of the two streams has been accumulating, but very little is known about the effects of injury, particularly very early injury, on this developmental process. Using a set of computerized dorsal and ventral stream tasks matched for stimuli, required response, and difficulty (for typically-developing individuals), we sought to compare the differential effects of injury to the two systems by examining performance in individuals with perinatal brain injury (PBI), who present with selective deficits in visuospatial processing from a young age. Thirty participants (mean = 15.1 years) with early unilateral brain injury (15 right hemisphere PBI, 15 left hemisphere PBI) and 16 matched controls participated. On our tasks children with PBI performed more poorly than controls (lower accuracy and longer response times), and this was particularly prominent for the ventral stream task. Lateralization of PBI was also a factor, as the dorsal stream task did not seem to be associated with lateralized deficits, with both PBI groups showing only subtle decrements in performance, while the ventral stream task elicited deficits from RPBI children that do not appear to improve with age. Our findings suggest that early injury results in lesion-specific visuospatial deficits that persist into adolescence. Further, as the stimuli used in our ventral stream task were faces, our findings are consistent with what is known about the neural systems for face processing, namely, that they are established relatively early, follow a comparatively rapid developmental trajectory (conferring a vulnerability to early insult), and are biased toward the right hemisphere.     

Dorsal and ventral visual streams: Typical and atypical development

The literature on visuospatial processing describes two distinct pathways within the brain: a dorsal route extending from the visual cortex into the parietal lobes that is critical for spatial processing and a ventral route extending from the visual cortex into the temporal lobes that is critical for form perception. These visual streams appear to differ in their developmental trajectories and their vulnerabilities to diverse neurodevelopmental conditions. The present work aims to investigate development and vulnerability in two aspects of dorsal and ventral visual-stream function, namely attention to location and attention to identity. In Study 1, we compare typically-developing (TD) youth aged 9 to 16 years with young adults aged 18 to 22 years on computerized location and identity tasks. In Study 2, we compare children and adolescents who have congenital hypothyroidism (CH), a pediatric endocrine disorder, with age-matched TD controls on the same tasks. The results from Study 1 show that the youths were less accurate than the adults at judging identity, whereas both groups were equally accurate at judging location. The results from Study 2 show that the youths with CH were slower but not less accurate than the TD youths in making both identity and location judgments. The results are interpreted as signifying later development of ventral (identity) stream functions compared to dorsal (location) but equal vulnerability of both functions in CH.     

Central visual persistences: II. Effects of hand and head rotations

In an earlier paper, kinesthetic effects on central visual persistences (CPs) were reported, including the ability to move these images by hand following eye closure. While all CPs could be translated anywhere within the frontal field, the present report documents a more selective influence of manual rotations on CPs in the same subjects. When common objects or figures drawn on cards were rotated (while holding one end of the object or one corner of a card between thumb and forefinger), it was found that CPs of larger objects rotated with the hand. By contrast, CPs of smaller objects, parts of objects, and textures remained stable in space as the hand rotated. It is proposed that CPs of smaller stimuli and textures are represented mainly by the ventral stream (temporal cortex) while larger CPs, which rotate, are represented mainly by the dorsal stream (parietal cortex). A second discovery was that CPs of small objects (but not of line segments or textures) could be rotated when the thumb and fingers surrounded the edges of the object. It is proposed that neuronal convergence of visual and tactile information about shape increases parietal responses to small objects, so that their CPs will rotate. Experiments with CPs offer new tools to infer visual coding differences between ventral and dorsal streams in man.     

Grasping with the eyes: The role of elongation in visual recognition of manipulable objects

Processing within the dorsal visual stream subserves object-directed action, whereas visual object recognition is mediated by the ventral visual stream. Recent findings suggest that the computations performed by the dorsal stream can nevertheless influence object recognition. Little is known, however, about the type of dorsal stream information that is available to assist in object recognition. Here, we present a series of experiments that explored different psychophysical manipulations known to bias the processing of a stimulus toward the dorsal visual stream in order to isolate its contribution to object recognition. We show that elongated-shaped stimuli, regardless of their semantic category and familiarity, when processed by the dorsal stream, elicit visuomotor grasp-related information that affects how we categorize manipulable objects. Elongated stimuli may reduce ambiguity during grasp preparation by providing a coarse cue to hand shaping and orientation that is sufficient to support action planning. We propose that this dorsal-stream-based analysis of elongation along a principal axis is the basis for how the dorsal visual object processing stream can affect categorization of manipulable objects.     

Dorsal stream activation during retrieval of object size and shape

We investigated dorsal visual stream involvement in the retrieval of a variety of visual attributes of common objects, using functional magnetic resonance imaging. Seven subjects made binary decisions about the shape, color, and size of named objects during scanning. Bilateral parietal activity was significantly greater during retrieval of shape and size information than during retrieval of color information. Consistent with a domain-specific distributed model of semantic organization, the finding that dorsal stream activity is associated with size and shape retrieval, as compared with color retrieval, may indicate that both size and shape information are learned partly through dorsally mediated processes, such as visually guided grasping. These results demonstrate that both visual-processing streams (i.e., the ventral “what” pathway and the dorsal “where” pathway) are involved in the storage and/or retrieval of knowledge of object appearance but that, just as in vision, these two pathways may play different roles in conceptual processing.     

Domain-dependent activation during spatial and nonspatial auditory working memory

Visual system has been proposed to be divided into two, the ventral and dorsal, processing streams. The ventral pathway is thought to be involved in object identification whereas the dorsal pathway processes information regarding the spatial locations of objects and the spatial relationships among objects. Several studies on working memory (WM) processing have further suggested that there is a dissociable domain-dependent functional organization within the prefrontal cortex for processing of spatial and nonspatial visual information. Also the auditory system is proposed to be organized into two domain-specific processing streams, similar to that seen in the visual system. Recent studies on auditory WM have further suggested that maintenance of nonspatial and spatial auditory information activates a distributed neural network including temporal, parietal, and frontal regions but the magnitude of activation within these activated areas shows a different functional topography depending on the type of information being maintained. The dorsal prefrontal cortex, specifically an area of the superior frontal sulcus (SFS), has been shown to exhibit greater activity for spatial than for nonspatial auditory tasks. Conversely, ventral frontal regions have been shown to be more recruited by nonspatial than by spatial auditory tasks. It has also been shown that the magnitude of this dissociation is dependent on the cognitive operations required during WM processing. Moreover, there is evidence that within the nonspatial domain in the ventral prefrontal cortex, there is an across-modality dissociation during maintenance of visual and auditory information. Taken together, human neuroimaging results on both visual and auditory sensory systems support the idea that the prefrontal cortex is organized according to the type of information being maintained in WM.     

Integrating faces, houses, motion, and action: spontaneous binding across ventral and dorsal processing streams

Perceiving an event requires the integration of its features across numerous brain maps and modules. Visual object perception is thought to be mediated by a ventral processing stream running from occipital to inferotemporal cortex, whereas most spatial processing and action control is attributed to the dorsal stream connecting occipital, parietal, and frontal cortex. Here we show that integration operates not only on ventral features and objects, such as faces and houses, but also across ventral and dorsal pathways, binding faces and houses to motion and manual action. Furthermore, these bindings seem to persist over time, as they influenced performance on future task-relevant visual stimuli. This is reflected by longer reaction times for repeating one, but alternating other features in a sequence, compared to complete repetition or alternation of features. Our findings are inconsistent with the notion that the dorsal stream is operating exclusively online and has no access to memory.     

The dorsal stream and the visual horizon

Today many philosophers of mind accept that the two cortical streams of visual processing in humans can be distinguished in terms of conscious experience. The ventral stream is thought to produce representations that may become conscious, and the dorsal stream is thought to handle unconscious vision for action. Despite a vast literature on the topic of the two streams, there is currently no account of the way in which the relevant empirical evidence could fit with basic Husserlian phenomenology of vision. Here I offer such an account. In this article, I show how the empirical evidence ought to be understood in a way that is informed by phenomenology. The differences in the two streams are better described as differences in spatial and temporal processing. Rather than simply “unconscious,” the dorsal stream can be better described as making a special contribution to what Husserl identified as the visual horizon.     

Implicit biases in visually guided action

For almost half a century dual-stream advocates have vigorously defended the view that there are two functionally specialized cortical streams of visual processing originating in the primary visual cortex: a ventral, perception-related ‘conscious’ stream and a dorsal, action-related ‘unconscious’ stream. They furthermore maintain that the perceptual and memory systems in the ventral stream are relatively shielded from the action system in the dorsal stream. In recent years, this view has come under scrutiny. Evidence points to two overlapping action pathways: a dorso-dorsal pathway that calculates features of the object to be acted on, and a ventro-dorsal pathway that transmits stored information about skilled object use from the ventral stream to the dorso-dorsal pathway. This evidence suggests that stored information may exert significantly more influence on visually guided action than hitherto assumed. I argue that this, in turn, supports the notion of skilled automatic action that is nonetheless agential. My focus here will be on actions influenced by implicit biases (stereotypes/prejudices). Action that is biased in this way, I argue, is in an important sense intentional and agential.

     

Does the brain's ventral visual pathway compute object shape?

A rich behavioral literature has shown that human object recognition is supported by a representation of shape that is tolerant to variations in an object's appearance. Such 'global' shape representations are achieved by describing objects via the spatial arrangement of their local features, or structure, rather than by the appearance of the features themselves. However, accumulating evidence suggests that the ventral visual pathway – the primary substrate underlying object recognition – may not represent global shape. Instead, ventral representations may be better described as a basis set of local image features. We suggest that this evidence forces a reevaluation of the role of the ventral pathway in object perception and posits a broader network for shape perception that encompasses contributions from the dorsal pathway.     

Testing the dorsal stream attention hypothesis: Electrophysiological correlates and the effects of ventral stream damage

The roles of dorsal and ventral processing streams in visual orienting and conscious perception were examined in two experiments. The first employed high density EEG with source localization. The second comprised a neuropsychological case study. Visual orienting was assessed with an attention procedure, where peripheral letters cued participants towards a target location. In the perception procedure participants responded to the same letters by performing an explicit conscious discrimination. In Experiment 1, the peripheral letters elicited rapid dorsal stream activation in the attention procedure, and this activation preceded top-down enhancement of target processing in occipital cortex. In the perception procedure early ventral stream activation was seen. In addition, peripheral letters elicited an “early directing attention negativity” (EDAN) over parietal recording sites in the attention procedure, but not in the perception procedure. In Experiment 2, a patient with a bilateral ventral stream lesion but preserved dorsal stream function showed clear disruption to performance in the perception procedure, whilst exhibiting a normal visual orienting effect in the attention procedure. Taken together these findings (1) highlight the distinct roles of the dorsal and ventral streams in attention and perception, and (2) suggest how these streams might interact, via reentrant effects of attention on perceptual processing.     

Working memory impairment in people with Williams syndrome: Effects of delay, task and stimuli

Williams syndrome (WS) is a neurodevelopmental disorder associated with impaired visuospatial representations subserved by the dorsal stream and relatively strong object recognition abilities subserved by the ventral stream. There is conflicting evidence on whether this uneven pattern in WS extends to working memory (WM). The present studies provide a new perspective, testing WM for a single stimulus using a delayed recognition paradigm in individuals with WS and typically developing children matched for mental age (MA matches). In three experiments, participants judged whether a second stimulus ‘matched’ an initial sample, either in location or identity. We first examined memory for faces, houses and locations using a 5 s delay (Experiment 1) and a 2 s delay (Experiment 2). We then tested memory for human faces, houses, cat faces, and shoes with a 2 s delay using a new set of stimuli that were better controlled for expression, hairline and orientation (Experiment 3). With the 5 s delay (Experiment 1), the WS group was impaired overall compared to MA matches. While participants with WS tended to perform more poorly than MA matches with the 2 s delay, they also exhibited an uneven profile compared to MA matches. Face recognition was relatively preserved in WS with friendly faces (Experiment 2) but not when the faces had a neutral expression and were less natural looking (Experiment 3). Experiment 3 indicated that memory for object identity was relatively stronger than memory for location in WS. These findings reveal an overall WM impairment in WS that can be overcome under some conditions. Abnormalities in the parietal lobe/dorsal stream in WS may damage not only the representation of spatial location but may also impact WM for visual stimuli more generally.     

Interactions between the dorsal and the ventral pathways in mental rotation: An fMRI study

In this fMRI study, we examined the relationship between activations in the inferotemporal region (ventral pathway) and the parietal region (dorsal pathway), as well as in the prefrontal cortex (associated with working memory), in a modified mental rotation task. We manipulated figural complexity (simple vs. complex) to affect the figure recognition process (associated with the ventral pathway) and the amount of rotation (0° vs. 90°), typically associated with the dorsal pathway. The pattern of activation not only showed that both streams are affected by both manipulations, but also showed an overadditive interaction. The effect of figural complexity was greater for 90° rotation than for 0° in multiple regions, including the ventral, dorsal, and prefrontal regions. In addition, functional connectivity analyses on the correlations across the time courses of activation between regions of interest showed increased synchronization among multiple brain areas as task demand increased. The results indicate that both the dorsal and the ventral pathways show interactive effects of object and spatial processing, and they suggest that multiple regions interact to perform mental rotation.     

Trade-off in object versus spatial visualization abilities: Restriction in the development of visual-processing resources

Previous research indicates relative independence between the ventral and dorsal visual pathways, nassociated with object and spatial visual processing, respectively. The present research shows that, at the individualdifferences level, there is a trade-off, rather than independence, between object and spatial visualization abilities. Across five different age groups with different professional specializations, participants with above-average object visualization abilities (artists) had below-average spatial visualization abilities, and the inverse was true for those with above-average spatial visualization abilities (scientists). No groups showed both above-average object and above-average spatial visualization abilities. Furthermore, while total object and spatial visualization resources increase with age and experience, the trade-off relationship between object and spatial visualization abilities does not. These results suggest that the trade-off originates through a bottleneck that restricts the development of overall visualization resources, rather than through preferential experience in one type of visualization.     

Face processing is enhanced in the left and upper visual hemi-fields

We tested whether two known hemi-field asymmetries would affect visual search with face stimuli. Holistic processing of spatial configurations is better in the left hemi-field, reflecting a right hemisphere specialization, and object recognition is better in the upper visual field, reflecting stronger projections into the ventral stream. Faces tap into holistic processing and object recognition at the same time, which predicts better performance in the left and upper hemi-field, respectively. In the first experiment, participants had to detect a face with a gaze direction different from the remaining faces. Participants were faster to respond when targets were presented in the left and upper hemi-field. The same pattern of results was observed when only the eye region was presented. In the second experiment, we turned the faces upside-down, which eliminated the typical spatial configuration of faces. The left hemi-field advantage disappeared, showing that it is related to holistic processing of faces, whereas the upper hemi-field advantage related to object recognition persisted. Finally, we made the search task easier by asking observers to search for a face with open among closed eyes or vice versa. The easy search task eliminated the need for complex object recognition and, accordingly, the advantage of the upper visual field disappeared. Similarly, the left hemi-field advantage was attenuated. In sum, our findings show that both horizontal and vertical asymmetries affect the search for faces and can be selectively suppressed by changing characteristics of the stimuli.     

Sparing of sensitivity to biological motion but not of global motion after early visual deprivation

Patients deprived of visual experience during infancy by dense bilateral congenital cataracts later show marked deficits in the perception of global motion (dorsal visual stream) and global form (ventral visual stream). We expected that they would also show marked deficits in sensitivity to biological motion, which is normally processed in the superior temporal sulcus via input from both the dorsal and ventral streams. When tested on the same day for sensitivity to biological motion and to global motion at two speeds (4 and 18° s(-1)), patients, as expected, displayed a large deficit in processing global motion at both speeds. Surprisingly, they performed normally in discriminating biological motion from scrambled displays, tolerating as much noise as their age-matched controls. Networks bypassing damaged portions of the dorsal and the ventral streams must mediate the spared sensitivity to biological motion after early visual deprivation.     

A multi-stage account of binding in vision: Neuropsychological evidence

I review neuropsychological evidence on the problems patients can have in binding together the attributes of visual stimuli, following brain damage. The evidence indicates that there can be several kinds of binding deficit in patients. Damage to early visual processing within the ventral visual stream can disrupt the binding of contours into shapes, though the binding of form elements into contours can still operate. This suggests that the process of binding elements into contour is distinct from the process of binding contours into shapes. The latter form of binding seems to operate within the ventral visual system. In addition, damage to the parietal lobe can disrupt the binding of shape to surface information about objects, even when the binding of elements into contours, and contours into shapes, seems to be preserved. These findings are consistent with a multi-stage account of binding in vision, which distinguishes between the processes involved in binding shape information (in the ventral visual stream) and the processes involved in binding shape and surface detail (involving interactions between the ventral and dorsal streams). In addition, I present evidence indicating that a further, transient form of binding can take place, based on stimuli having common visual onsets. I discuss the relations between these different forms of binding.     

The relation between subitizable symbolic and non‐symbolic number processing over the kindergarten school year

A long‐standing debate in the field of numerical cognition concerns the degree to which symbolic and non‐symbolic processing are related over the course of development. Of particular interest is the possibility that this link depends on the range of quantities in question. Behavioral and neuroimaging research with adults suggests that symbolic and non‐symbolic quantities may be processed more similarly within, relative to outside of, the subitizing range. However, it remains unclear whether this unique link exists in young children at the outset of formal education. Further, no study has yet taken numerical size into account when investigating the longitudinal influence of these skills. To address these questions, we investigated the relation between symbolic and non‐symbolic processing inside versus outside the subitizing range, both cross‐sectionally and longitudinally, in 540 kindergarteners. Cross‐sectionally, we found a consistently stronger relation between symbolic and non‐symbolic number processing within versus outside the subitizing range at both the beginning and end of kindergarten. We also show evidence for a bidirectional relation over the course of kindergarten between formats within the subitizing range, and a unidirectional relation (symbolic → non‐symbolic) for quantities outside of the subitizing range. These findings extend current theories on symbolic and non‐symbolic magnitude development by suggesting that non‐symbolic processing may in fact play a role in the development of symbolic number abilities, but that this influence may be limited to quantities within the subitizing range.