Location coding by the human visual system: multiple topological adaptations in a case of strabismic amblyopia

Amblyopia, a major cause of vision loss, is a developmental disorder of visual perception commonly associated with strabismus (squint). Although defined by a reduction in visual acuity, severe distortions of perceived visual location are common in strabismic amblyopia. These distortions can help us understand the cortical coding of visual location and its development in normal vision, as well as in amblyopia. The history of retinotopic mapping in the visual cortex highlights the potential impact of amblyopia. Theories of amblyopia include topological disarray of receptors in primary visual cortex, undersampling from the amblyopic eye compared with normal eyes, and the presence of anomalous retinal correspondence or multiple cortical representations of the strabismic fovea. We examined the distortions in a strabismic amblyope, using a pop-out localization task, in which normal observers made errors dependent on the visual context of the stimulus. The localization errors of the strabismic amblyope were abnormal. We found that none of the available theories could fully explain this one patient's localization performance. Instead, the observed behavior suggests that multiple adaptations of the underlying cortical topology are possible simultaneously in different parts of the visual field.     

Evidence for perceptual learning with repeated stimulation after partial and total cortical blindness

Lesions of occipital cortex result in loss of sight in the corresponding regions of visual fields. The traditional view that, apart from some spontaneous recovery in the acute phase, field defects remain permanently and irreversibly blind, has been challenged. In patients with partial field loss, a range of residual visual abilities in the absence of conscious perception (blindsight) has been demonstrated (Weiskrantz, 1986). Recent findings (Sahraie et al., 2006, 2010) have also demonstrated increased visual sensitivity in the field defect following repeated stimulation. We aimed to extend these findings by systematically exploring whether repeated stimulation can also lead to increased visual sensitivity in two cases with total (bilateral) cortical blindness. In addition, for a case of partial blindness, we examined the extent of the recovery as a function of stimulated region of the visual field, over extended periods of visual training. Positive auditory feedback was provided during the training task for correct detection of a spatial grating pattern presented at specific retinotopic locations using a temporal two alternative forced-choice paradigm (Neuro-Eye Therapy). All three cases showed improved visual sensitivity with repeated stimulation. The findings indicate that perceptual learning can occur through systematic visual field stimulation even in cases of bilateral cortical blindness.     

The cortical representation of centrally presented words: A magnetic stimulation study

The right and left visual fields each project to the contralateral cerebral hemispheres. The current study aimed to investigate the extent of the functional overlap of the two hemifields along the vertical meridian. We applied repetitive transcranial magnetic stimulation (rTMS) over the left and right occipital cortex to investigate whether the foveal representation of words is bilaterally represented or is split between the two hemispheres. Employing a lateralized lexical decision task, we first showed a double dissociation between the stimulated cortical site and performance; right visual field (RVF) but not left visual field (LVF) performance was impaired when the left visual cortex was stimulated, and LVF but not RVF performance was impairred when the right visual cortex was stimulated. Unilateral stimulation also significantly impaired lexical decision latencies to centrally presented words. These findings support the suggestion that foveal representation of words is split. We discuss future strategies for the use of TMS in further tests of the split representation account.     

Single units and visual cortical organization

The visual system has a parallel and hierarchical organization, evident at every stage from the retina onwards. Although the general benefits of parallel and hierarchical organization in the visual system are easily understood, it has not been easy to discern the function of the visual cortical modules. I explore the view that striate cortex segregates information about different attributes of the image, and dispatches it for analysis to different extrastriate areas. I argue that visual cortex does not undertake multiple relatively independent analyses of the image from which it assembles a unified representation that can be interrogated about the what and where of the world. Instead, occipital cortex is organized so that perceptually relevant information can be recovered at every level in the hierarchy, that information used in making decisions at one level is not passed on to the next level, and, with one rather special exception (area MT), through all stages of analysis all dimensions of the image remain intimately coupled in a retinotopic map. I then offer some explicit suggestions about the analyses undertaken by visual areas in occipital cortex, and conclude by examining some objections to the proposals.     

Developmental dissociation of visual dorsal stream parvo and magnocellular representations and the functional impact of negative retinotopic BOLD responses

Localized neurodevelopmental defects provide an opportunity to study structure–function correlations in the human nervous system. This unique multimodal case report of epileptogenic dysplasia in the visual cortex allowed exploring visual function across distinct pathways in retinotopic regions and the dorsal stream, in relation to fMRI retinotopic mapping and spike triggered BOLD responses.     

The representation and perception of 3D space: Interactions between 2D location and depth

ABSTRACT

We live in a 3D world, and yet the majority of vision research is restricted to 2D phenomena, with depth research typically treated as a separate field. Here we ask whether 2D spatial information and depth information interact to form neural representations of 3D space, and if so, what are the perceptual implications? Using fMRI and behavioural methods, we reveal that human visual cortex gradually transitions from 2D to 3D spatial representations, with depth information emerging later along the visual hierarchy, and demonstrate that 2D location holds a fundamentally special place in early visual processing.     

Refining the visual-cortical hypothesis in category learning

Participants produce steep typicality gradients and large prototype-enhancement effects in dot-distortion category tasks, showing that in these tasks to-be-categorized items are compared to a prototypical representation that is the central tendency of the participant’s exemplar experience. These prototype-abstraction processes have been ascribed to low-level mechanisms in primary visual cortex. Here we asked whether higher-level mechanisms in visual cortex can also sometimes support prototype abstraction. To do so, we compared dot-distortion performance when the stimuli were size constant (allowing some low-level repetition-familiarity to develop for similar shapes) or size variable (defeating repetition-familiarity effects). If prototype formation is only mediated by low-level mechanisms, stimulus-size variability should lessen prototype effects and flatten typicality gradients. Yet prototype effects and typicality gradients were the same under both conditions, whether participants learned the categories explicitly or implicitly and whether they received trial-by-trial reinforcement during transfer tests. These results broaden out the visual-cortical hypothesis because low-level visual areas, featuring retinotopic perceptual representations, would not support robust category learning or prototype-enhancement effects in an environment of pronounced variability in stimulus size. Therefore, higher-level cortical mechanisms evidently can also support prototype formation during categorization.     

A continuous smooth map of space in the primary visual cortex of the common marmoset

We examined the fine-scale mapping of the visual world within the primary visual cortex of the marmoset monkey (Callithrix jacchus) using differential optical imaging. We stimulated two sets of complementary stripe-like locations in turn, subtracting them to generate the cortical representations of continuous bands of visual space. Rotating this stimulus configuration makes it possible to map different spatial axes within the primary visual cortex. In a similar manner, shifting the stimulated locations between trials makes it possible to map retinotopy at an even finer scale. Using these methods we found no evidence of any local anisotropies or distortions in the cortical representation of visual space. This is despite the fact that orientation preference is mapped in a discontinuous manner across the surface of marmoset V1. Overall, our results indicate that space is mapped in a continuous and smooth manner in the primary visual cortex of the common marmoset.     

Cortical integration of audio–visual speech and non-speech stimuli

Using fMRI we investigated the neural basis of audio–visual processing of speech and non-speech stimuli using physically similar auditory stimuli (speech and sinusoidal tones) and visual stimuli (animated circles and ellipses). Relative to uni-modal stimuli, the different multi-modal stimuli showed increased activation in largely non-overlapping areas. Ellipse-Speech, which most resembles naturalistic audio–visual speech, showed higher activation in the right inferior frontal gyrus, fusiform gyri, left posterior superior temporal sulcus, and lateral occipital cortex. Circle-Tone, an arbitrary audio–visual pairing with no speech association, activated middle temporal gyri and lateral occipital cortex. Circle-Speech showed activation in lateral occipital cortex, and Ellipse-Tone did not show increased activation relative to uni-modal stimuli. Further analysis revealed that middle temporal regions, although identified as multi-modal only in the Circle-Tone condition, were more strongly active to Ellipse-Speech or Circle-Speech, but regions that were identified as multi-modal for Ellipse-Speech were always strongest for Ellipse-Speech. Our results suggest that combinations of auditory and visual stimuli may together be processed by different cortical networks, depending on the extent to which multi-modal speech or non-speech percepts are evoked.     

Temporally unfolding neural representation of pictorial occlusion

The human visual system possesses a remarkable ability to reconstruct the shape of an object that is partly occluded by an interposed surface. Behavioral results suggest that, under some circumstances, this perceptual process (termed amodal completion) progresses from an initial representation of local image features to a completed representation of a shape that may include features that are not explicitly present in the retinal image. Recent functional magnetic resonance imaging (fMRI) studies have shown that the completed surface is represented in early visual cortical areas. We used fMRI adaptation, combined with brief, masked exposures, to track the amodal completion process as it unfolds in early visual cortical regions. We report evidence for an evolution of the neural representation from the image-based feature representation to the completed representation. Our method offers the possibility of measuring changes in cortical activity using fMRI over a time scale of a few hundred milliseconds.     

The role of constant curvature in 2-D contour shape representations

In early cortex, visual information is encoded by retinotopic orientation-selective units. Higher-level representations of abstract properties, such as shape, require encodings that are invariant to changes in size, position, and orientation. Within the domain of open, 2-D contours, we consider how an economical representation that supports viewpoint-invariant shape comparisons can be derived from early encodings. We explore the idea that 2-D contour shapes are encoded as joined segments of constant curvature. We report three experiments in which participants compared sequentially presented 2-D contour shapes comprised of constant curvature (CC) or non-constant curvature (NCC) segments. We show that, when shapes are compared across viewpoint or for a retention interval of 1000 ms, performance is better for CC shapes. Similar recognition performance is observed for both shape types, however, if they are compared at the same viewpoint and the retention interval is reduced to 500 ms. These findings are consistent with a symbolic encoding of 2-D contour shapes into CC parts when the retention intervals over which shapes must be stored exceed the duration of initial, transient, visual representations.     

数量适应后效的皮层映射特征

本研究探讨了观察者与观察目标存在相对运动时视觉系统对目标数量特征的适应后效的皮层映射特征, 并与对比度适应后效的映射规律进行比较。包括两项实验。其中, 实验一要求被试在适应目标后转换注视点, 考察眼跳后相同和不同视网膜区域以及相同和不同空间区域的适应后效, 发现数量适应后效具有部分空间-皮层映射特性, 而对比度适应后效则表现出完全的视网膜-皮层映射特征。实验二采用固定的注视点, 考察目标运动后目标原位置和新位置区域的适应后效, 发现数量适应后效不完全依赖于视网膜-皮层映射, 它可以“追随”客体运动重新映射到新的位置, 表现出基于客体映射的特征, 而对比度适应后效则完全依赖于视网膜-皮层映射, 不能“追随”客体移动在目标新位置重新形成映射。两项实验结果提示, 相对于对比度等低级表面特征而言, 数量特征对目标的描述涉及更高的加工水平, 它可以与观察目标的相对运动信息进行整合, 且这种整合在眼跳和非眼跳的观察条件下都可发生。     

Competition-induced visual field differences in search

Do visual field effects point to differences in cortical representation, or do they reflect differences in the way these representations are used by other brain regions? This study explored three attributes of visual search that provide strong evidence in favor of differences in use. Competition refers to the finding that visual field differences in search efficiency are larger in whole- than in half-field displays (both left-right and upper-lower half-fields). Task specialization refers to the finding that some tasks favor one hemisphere whereas other tasks favor the other hemisphere, even though the same stimulus displays are used in both tasks. Anatomical alignment refers to the finding that competition effects are altered if the quadrants of the visual display are not aligned with the cortical quadrants of the observer. We propose that visual field specialization in search is the result of a competition involving limited access to cortical visual representations by the extended neural networks of attention.     

A framework for local cortical oscillation patterns

Oscillations are a pervasive feature of neuronal activity in the cerebral cortex. Here, we propose a framework for understanding local cortical oscillation patterns in cognition: two classes of network interactions underlying two classes of cognitive functions produce different local oscillation patterns. Local excitatory-inhibitory interactions shape neuronal representations of sensory, motor and cognitive variables, and produce local gamma-band oscillations. By contrast, the linkage of such representations by integrative functions such as decision-making is mediated by long-range cortical interactions, which yield more diverse local oscillation patterns often involving the beta range. This framework reconciles different cortical oscillation patterns observed in recent studies and helps to understand the link between cortical oscillations and the fMRI signal. Our framework highlights the notion that cortical oscillations index the specific circuit-level mechanisms of cognition.     

Electrophysiological evidence for a shared representational medium for visual images and visual percepts

Does mental imagery involve the activation of representations in the visual system? Systematic effects of imagery on visual signal detection performance have been used to argue that imagery and the perceptual processing of stimuli interact at some common locus of activity (Farah, 1985). However, such a result is neutral with respect to the question of whether the interaction occurs during modality-specific visual processing of the stimulus. If imagery affects stimulus processing at early, modality-specific stages of stimulus representation, this implies that the shared stimulus representations are visual, whereas if imagery affects stimulus processing only at later, amodal stages of stimulus representation, this implies that imagery involves more abstract, postvisual stimulus representations. To distinguish between these two possibilities, we repeated the earlier imagery-perception interaction experiment while recording event-related potentials (ERPs) to stimuli from 16 scalp electrodes. By observing the time course and scalp distribution of the effect of imagery on the ERP to stimuli, we can put constraints on the locus of the shared representations for imagery and perception. An effect of imagery was seen within 200 ms following stimulus presentation, at the latency of the first negative component of the visual ERP, localized at the occipital and posterior temporal regions of the scalp, that is, directly over visual cortex. This finding provides support for the claim that mental images interact with percepts in the visual system proper and hence that mental images are themselves visual representations.     

The time course of abstract visual representation

Studies in change blindness re-enforce the suggestion that veridical, pictorial representations that survive multiple relocations of gaze are unlikely to be generated in the visual system. However, more abstract information may well be extracted and represented by the visual system. In this paper we study the types of information that are retained and the time courses over which these representations are constructed when participants view complex natural scenes. We find that such information is retained and that the resultant abstract representations encode a range of information. Different types of information are extracted and represented over different time courses. After several seconds of viewing natural scenes, our visual system is able to construct a complex information-rich representation.     

Tracing trajectories of audio‐visual learning in the infant brain

Although infants begin learning about their environment before they are born, little is known about how the infant brain changes during learning. Here, we take the initial steps in documenting how the neural responses in the brain change as infants learn to associate audio and visual stimuli. Using functional near‐infrared spectroscopy (fNRIS) to record hemodynamic responses in the infant cortex (temporal, occipital, and frontal cortex), we find that across the infant brain, learning is characterized by an increase in activation followed by a decrease. We take this U‐shaped response as evidence of repetition enhancement during early stages of learning and repetition suppression during later stages, a result that mirrors the Hunter and Ames model of infant visual preference. Furthermore, we find that the neural response to violations of the learned associations can be predicted by the shape of the learning curve in temporal and occipital cortex. These data provide the first look at the shape of the neural response during audio‐visual associative learning in infancy establishing that diverse regions of the infant brain exhibit systematic changes across the time‐course of learning.     

视时距知觉适应后效的空间选择性

时距知觉适应后效是指长时间适应于某一特定时距会导致个体对后续时距产生知觉偏差。其中对视时距知觉适应后效空间选择性的探讨存在争议,有研究支持位置不变性,也有研究支持位置特异性。这类研究能有效揭示时距编码的认知神经机制,位置不变性可能意味着时距编码位于较高级的脑区,而位置特异性则可能意味着时距编码位于初级视觉皮层。未来还可以探究时距知觉适应后效的视觉坐标表征方式,开展多通道研究以及相应的神经基础研究。     

Brain network interactions in auditory, visual and linguistic processing

In the paper, we discuss the importance of network interactions between brain regions in mediating performance of sensorimotor and cognitive tasks, including those associated with language processing. Functional neuroimaging, especially PET and fMRI, provide data that are obtained essentially simultaneously from much of the brain, and thus are ideal for enabling one to assess interregional functional interactions. Two ways to use these types of data to assess network interactions are presented. First, using PET, we demonstrate that anterior and posterior perisylvian language areas have stronger functional connectivity during spontaneous narrative production than during other less linguistically demanding production tasks. Second, we show how one can use large-scale neural network modeling to relate neural activity to the hemodynamically-based data generated by fMRI and PET. We review two versions of a model of object processing - one for visual and one for auditory objects. The regions comprising the models include primary and secondary sensory cortex, association cortex in the temporal lobe, and prefrontal cortex. Each model incorporates specific assumptions about how neurons in each of these areas function, and how neurons in the different areas are interconnected with each other. Each model is able to perform a delayed match-to-sample task for simple objects (simple shapes for the visual model; tonal contours for the auditory model). We find that the simulated electrical activities in each region are similar to those observed in nonhuman primates performing analogous tasks, and the absolute values of the simulated integrated synaptic activity in each brain region match human fMRI/PET data. Thus, this type of modeling provides a way to understand the neural bases for the sensorimotor and cognitive tasks of interest.     

Brain activity during the encoding, retention, and retrieval of stimulus representations

Studies of delayed nonmatching-to-sample (DNMS) performance following lesions of the monkey cortex have revealed a critical circuit of brain regions involved in forming memories and retaining and retrieving stimulus representations. Using event-related functional magnetic resonance imaging (fMRI), we measured brain activity in 10 healthy human participants during performance of a trial-unique visual DNMS task using novel barcode stimuli. The event-related design enabled the identification of activity during the different phases of the task (encoding, retention, and retrieval). Several brain regions identified by monkey studies as being important for successful DNMS performance showed selective activity during the different phases, including the mediodorsal thalamic nucleus (encoding), ventrolateral prefrontal cortex (retention), and perirhinal cortex (retrieval). Regions showing sustained activity within trials included the ventromedial and dorsal prefrontal cortices and occipital cortex. The present study shows the utility of investigating performance on tasks derived from animal models to assist in the identification of brain regions involved in human recognition memory.