Flexible cortical gamma-band correlations suggest neural principles of visual processing

We summarize recent studies of our group from the primary visual cortex V1 of behaving monkeys referring to the hypothesis of spatial feature binding by γ-synchronization (30-90 Hz). In agreement with this hypothesis the data demonstrates decoupling of γ-activities among neural groups representing figure and ground. As γ-synchronization in V1 is restricted to cortical ranges of few millimeters, feature binding may equivalently be restricted in visual space. Closer inspection shows that the restriction in synchrony is due to far-reaching travelling γ-waves with changing phase coupling. Based on this observation we extend the initial binding-by-synchronization hypothesis and suggest object continuity to be coded by phase continuity. It is further argued that the spatial phase changes of the V1 γ-waves in general will also limit lateral phase coupling to higher levels of processing. Instead of phase-locked γ-coupling, corticocortical cooperation among γ-processes may be mediated by mutual amplitude modulations that are more reliable than phase synchrony over larger distances. The relevance of this concept of corticocortical binding is demonstrated with subdural recordings from human subjects performing cognitive tasks. The experimental results are discussed on the basis of network models with spiking neurons.     

大脑腹侧视觉通路知觉表征神经机制研究

大脑腹侧视觉通路知觉表征的神经机制是认知神经科学研究面临的基本问题。本文系统介绍了该问题研究中比较有影响力的理论模型,归纳分析了模型之间的分歧与各自的局限。文章分析指出大脑自上而下的调控机制是腹侧视觉通路神经表征机制问题研究的另一重要维度,如何有效整合知觉表征模型与大脑调控机制的相关研究是进一步深化知觉表征神经机制问题研究的关键。     

Temporally unfolding neural representation of pictorial occlusion

The human visual system possesses a remarkable ability to reconstruct the shape of an object that is partly occluded by an interposed surface. Behavioral results suggest that, under some circumstances, this perceptual process (termed amodal completion) progresses from an initial representation of local image features to a completed representation of a shape that may include features that are not explicitly present in the retinal image. Recent functional magnetic resonance imaging (fMRI) studies have shown that the completed surface is represented in early visual cortical areas. We used fMRI adaptation, combined with brief, masked exposures, to track the amodal completion process as it unfolds in early visual cortical regions. We report evidence for an evolution of the neural representation from the image-based feature representation to the completed representation. Our method offers the possibility of measuring changes in cortical activity using fMRI over a time scale of a few hundred milliseconds.     

Frontotemporal neural systems supporting semantic processing in Alzheimer’s disease

We hypothesized that semantic memory for object concepts involves both representations of visual feature knowledge in modality-specific association cortex and heteromodal regions that are important for integrating and organizing this semantic knowledge so that it can be used in a flexible, contextually appropriate manner. We examined this hypothesis in an fMRI study of mild Alzheimer’s disease (AD). Participants were presented with pairs of printed words and asked whether the words matched on a given visual–perceptual feature (e.g., guitar, violin: SHAPE). The stimuli probed natural kinds and manufactured objects, and the judgments involved shape or color. We found activation of bilateral ventral temporal cortex and left dorsolateral prefrontal cortex during semantic judgments, with AD patients showing less activation of these regions than healthy seniors. Moreover, AD patients showed less ventral temporal activation than did healthy seniors for manufactured objects, but not for natural kinds. We also used diffusion-weighted MRI of white matter to examine fractional anisotropy (FA). Patients with AD showed significantly reduced FA in the superior longitudinal fasciculus and inferior frontal-occipital fasciculus, which carry projections linking temporal and frontal regions of this semantic network. Our results are consistent with the hypothesis that semantic memory is supported in part by a large-scale neural network involving modality-specific association cortex, heteromodal association cortex, and projections between these regions. The semantic deficit in AD thus arises from gray matter disease that affects the representation of feature knowledge and processing its content, as well as white matter disease that interrupts the integrated functioning of this large-scale network.     

On the Correlation between Synchronized Oscillatory Activities and Consciousness

Recent experiments have shown that the amplitudes of cortical gamma band oscillatory activities that occur during anesthesia are often greater than amplitudes of similar activities that occur without anesthesia. This result is apparently at odds with the hypothesis that synchronized oscillatory activities constitute the neural correlate of consciousness. We argue that while synchronization and oscillatory patterning are necessary conditions for consciousness, they are not sufficient. Based on the results of a binocular rivalry study of Fries et al. (1997), we propose that the degrees of oscillatory strength and synchronization of neuronal activities determine the degree of awareness those activities produce. On the other hand, the overal firing rates of neurons in cortical sensory areas are not correlated with the degree of awareness the activities of those neurons produce. The results of the experiment of Fries et al. (1997) appear to conflict with the results of another binocular rivalry experiment, in which monkeys were trained to pull a lever in order to report which stimulus object was being perceived (Leopold & Logothetis, 1996). In the latter experiment, it was demonstrated that the firing rates of neurons in striate cortex did not change during perceptual alterations, while 90% of neurons in inferior and superior temporal cortices changed their firing rate when the perceived image changed. This result led to the conclusion that activities in temporal cortex are correlated with visual awareness, but those in striate cortex are not. We argue that activities in temporal cortex contribute little, if anything, to perceptual awareness, and that their primary function is computational. Thus the correlation between the firing rates of neurons in these areas and the responses of the monkeys is due to the recognition of a particular stimulus object, which in turn is due to the computations made there.     

The rhythm aftereffect: support for time sensitive neurons with broad overlapping tuning curves

Ivry [Ivry, R. B. (1996). The representation of temporal information in perception and motor control. Current Opinion in Neurobiology, 6, 851-857.] proposed that explicit coding of brief time intervals is accomplished by neurons that are tuned to a preferred temporal interval and have broad overlapping tuning curves. This proposal is analogous to the orientation selective cells in visual area V1. To test this proposal, we used a temporal analog to the visual tilt aftereffect. After adapting to a fast auditory rhythm, a moderately fast test rhythm (400 ms between beats) seemed slow and vice versa. If the speed of the adapting rhythm was made too disparate from speed of the test rhythm the effect was diminished. The effect occurred whether the adapting and test stimuli were presented to the same or different ears, but did not occur when an auditory adapting rhythm was followed by a visual test rhythm. Results support the proposition that explicit time information is coded by neural units tuned to specific temporal intervals with broad overlapping tuning curves. In addition, it appears that there is a single timing mechanism for each incoming sensory mode, but distinct timers for different modes.     

3-D vision and figure-ground separation by visual cortex

A neural network theory of three-dimensional (3-D) vision, called FACADE theory, is described. The theory proposes a solution of the classical figure-ground problem for biological vision. It does so by suggesting how boundary representations and surface representations are formed within a boundary contour system (BCS) and a feature contour system (FCS). The BCS and FCS interact reciprocally to form 3-D boundary and surface representations that are mutually consistent. Their interactions generate 3-D percepts wherein occluding and occluded object parts are separated, completed, and grouped. The theory clarifies how preattentive processes of 3-D perception and figure-ground separation interact reciprocally with attentive processes of spatial localization, object recognition, and visual search. A new theory of stereopsis is proposed that predicts how cells sensitive to multiple spatial frequencies, disparities, and orientations are combined by context-sensitive filtering, competition, and cooperation to form coherent BCS boundary segmentations. Several factors contribute to figure-ground pop-out, including: boundary contrast between spatially contiguous boundaries, whether due to scenic differences in luminance, color, spatial frequency, or disparity-partially ordered interactions from larger spatial scales and disparities to smaller scales and disparities; and surface filling-in restricted to regions surrounded by a connected boundary. Phenomena such as 3-D pop-out from a 2-D picture, Da Vinci stereopsis, 3-D neon color spreading, completion of partially occluded objects, and figure-ground reversals are analyzed. The BCS and FCS subsystems model aspects of how the two parvocellular cortical processing streams that join the lateral geniculate nucleus to prestriate cortical area V4 interact to generate a multiplexed representation of Form-And-Color-And-DEpth, orfacade, within area V4. Area V4 is suggested to support figure-ground separation and to interact with cortical mechanisms of spatial attention, attentive object learning, and visual search. Adaptive resonance theory (ART) mechanisms model aspects of how prestriate visual cortex interacts reciprocally with a visual object recognition system in inferotemporal (IT) cortex for purposes of attentive object learning and categorization. Object attention mechanisms of the What cortical processing stream through IT cortex are distinguished from spatial attention mechanisms of the Where cortical processing stream through parietal cortex. Parvocellular BCS and FCS signals interact with the model What stream. Parvocellular FCS and magnocellular motion BCS signals interact with the model Where stream. Reciprocal interactions between these visual, What, and Where mechanisms are used to discuss data about visual search and saccadic eye movements, including fast search of conjunctive targets, search of 3-D surfaces, selective search of like-colored targets, attentive tracking of multielement groupings, and recursive search of simultaneously presented targets.     

Bottom-up influences of voice continuity in focusing selective auditory attention

Selective auditory attention causes a relative enhancement of the neural representation of important information and suppression of the neural representation of distracting sound, which enables a listener to analyze and interpret information of interest. Some studies suggest that in both vision and in audition, the “unit” on which attention operates is an object: an estimate of the information coming from a particular external source out in the world. In this view, which object ends up in the attentional foreground depends on the interplay of top-down, volitional attention and stimulus-driven, involuntary attention. Here, we test the idea that auditory attention is object based by exploring whether continuity of a non-spatial feature (talker identity, a feature that helps acoustic elements bind into one perceptual object) also influences selective attention performance. In Experiment 1, we show that perceptual continuity of target talker voice helps listeners report a sequence of spoken target digits embedded in competing reversed digits spoken by different talkers. In Experiment 2, we provide evidence that this benefit of voice continuity is obligatory and automatic, as if voice continuity biases listeners by making it easier to focus on a subsequent target digit when it is perceptually linked to what was already in the attentional foreground. Our results support the idea that feature continuity enhances streaming automatically, thereby influencing the dynamic processes that allow listeners to successfully attend to objects through time in the cacophony that assails our ears in many everyday settings.     

On the neural correlates of object recognition awareness: relationship to computational activities and activities mediating perceptual awareness

Based on theoretical considerations of Aurell (1979) and Block (1995), we argue that object recognition awareness is distinct from purely sensory awareness and that the former is mediated by neuronal activities in areas that are separate and distinct from cortical sensory areas. We propose that two of the principal functions of neuronal activities in sensory cortex, which are to provide sensory awareness and to effect the computations that are necessary for object recognition, are dissociated. We provide examples of how this dissociation might be achieved and argue that the components of the neuronal activities which carry the computations do not directly enter the awareness of the subject. The results of these computations are sparse representations (i.e., vector or distributed codes) which are activated by the presentation of particular sensory objects and are essentially engrams for the recognition of objects. These final representations occur in the highest order areas of sensory cortex; in the visual analyzer, the areas include the anterior part of the inferior temporal cortex and the perirhinal cortex. We propose, based on lesion and connectional data, that the two areas in which activities provide recognition awareness are the temporopolar cortex and the medial orbitofrontal cortex. Activities in the temporopolar cortex provide the recognition awareness of objects learned in the remote past (consolidated object recognition), and those in the medial orbitofrontal cortex provide the recognition awareness of objects learned in the recent past. The activation of the sparse representation for a particular sensory object in turn activates neurons in one or both of these regions of cortex, and it is the activities of these neurons that provide the awareness of recognition of the object in question. The neural circuitry involved in the activation of these representations is discussed.     

Incremental grouping of image elements in vision

One important task for the visual system is to group image elements that belong to an object and to segregate them from other objects and the background. We here present an incremental grouping theory (IGT) that addresses the role of object-based attention in perceptual grouping at a psychological level and, at the same time, outlines the mechanisms for grouping at the neurophysiological level. The IGT proposes that there are two processes for perceptual grouping. The first process is base grouping and relies on neurons that are tuned to feature conjunctions. Base grouping is fast and occurs in parallel across the visual scene, but not all possible feature conjunctions can be coded as base groupings. If there are no neurons tuned to the relevant feature conjunctions, a second process called incremental grouping comes into play. Incremental grouping is a time-consuming and capacity-limited process that requires the gradual spread of enhanced neuronal activity across the representation of an object in the visual cortex. The spread of enhanced neuronal activity corresponds to the labeling of image elements with object-based attention.     

Cortical dynamics of contextually cued attentive visual learning and search: spatial and object evidence accumulation

How do humans use target-predictive contextual information to facilitate visual search? How are consistently paired scenic objects and positions learned and used to more efficiently guide search in familiar scenes? For example, humans can learn that a certain combination of objects may define a context for a kitchen and trigger a more efficient search for a typical object, such as a sink, in that context. The ARTSCENE Search model is developed to illustrate the neural mechanisms of such memory-based context learning and guidance and to explain challenging behavioral data on positive-negative, spatial-object, and local-distant cueing effects during visual search, as well as related neuroanatomical, neurophysiological, and neuroimaging data. The model proposes how global scene layout at a first glance rapidly forms a hypothesis about the target location. This hypothesis is then incrementally refined as a scene is scanned with saccadic eye movements. The model simulates the interactive dynamics of object and spatial contextual cueing and attention in the cortical What and Where streams starting from early visual areas through medial temporal lobe to prefrontal cortex. After learning, model dorsolateral prefrontal cortex (area 46) primes possible target locations in posterior parietal cortex based on goal-modulated percepts of spatial scene gist that are represented in parahippocampal cortex. Model ventral prefrontal cortex (area 47/12) primes possible target identities in inferior temporal cortex based on the history of viewed objects represented in perirhinal cortex.     

Cognitive architecture of perceptual organization: from neurons to gnosons

What, if anything, is cognitive architecture and how is it implemented in neural architecture? Focusing on perceptual organization, this question is addressed by way of a pluralist approach which, supported by metatheoretical considerations, combines complementary insights from representational, connectionist, and dynamic systems approaches to cognition. This pluralist approach starts from a representationally inspired model which implements the intertwined but functionally distinguishable subprocesses of feedforward feature encoding, horizontal feature binding, and recurrent feature selection. As sustained by a review of neuroscientific evidence, these are the subprocesses that are believed to take place in the visual hierarchy in the brain. Furthermore, the model employs a special form of processing, called transparallel processing, whose neural signature is proposed to be gamma-band synchronization in transient horizontal neural assemblies. In neuroscience, such assemblies are believed to mediate binding of similar features. Their formal counterparts in the model are special input-dependent distributed representations, called hyperstrings, which allow many similar features to be processed in a transparallel fashion, that is, simultaneously as if only one feature were concerned. This form of processing does justice to both the high combinatorial capacity and the high speed of the perceptual organization process. A naturally following proposal is that those temporarily synchronized neural assemblies are “gnosons”, that is, constituents of flexible self-organizing cognitive architecture in between the relatively rigid level of neurons and the still elusive level of consciousness.     

Synchronous Information Presented in 40-Hz Flicker Enhances Visual Feature Binding

Recent neurophysiological studies have encouraged speculation that the synchronization of spatially distributed neural assemblies (at around 40 Hz in the neocortex) is responsible for the binding of discrete stimulus components into coherent wholes during visual object perception. Using a novel paradigm, we demonstrated specific figural priming under 40-Hz stimulus modulation conditions. Further, under these conditions, observers were not aware of the prime"s existence, nor did the prime act as a stimulus-driven attentional cue. These findings provide the first psychophysical support for a theory of preattentive coding of visual objects, based on an externally entrained and thereby synchronized 40-Hz feature-binding mechanism.     

PERCEIVED CONTINUITY OF OCCLUDED VISUAL OBJECTS

Abstract— The human visual system does not rigidly preserve the properties of the retinal image as neural signals are transmitted to higher areas of the brain Instead, it generates a representation that captures stable surface properties despite a retinal image that is often fragmented in space and time because of occlusion caused by object and observer motion The recovery of this coherent representation depends at least in part on input from an abstract representation of three-dimensional (3-D) surface layout In the two experiments reported, a stereoscopic apparent motion display was used to investigate the perceived continuity of a briefly interrupted visual object When a surface appeared in front of the object's location during the interruption, the object was more likely to be perceived as persisting through the interruption (behind an occluder) than when the surface appeared behind the object's location under otherwise identical stimulus conditions The results reveal the influence of 3-D surface-based representations even in very simple visual tasks.     

特征变化的连续性影响客体连续表征的维持

本研究采用客体回溯范式考察了特征变化的连续性对维持客体连续表征的作用。实验1和实验2分别探索了形状维度上的变化方式（不变、渐变、突变）和亮度维度上的变化方式（不变、渐变、随机变化）对客体预览利化效应的影响。在特征连续条件下（不变或渐变）,两个实验都获得了客体预览利化效应。而在特征不连续变化条件下（突变或随机变化）,该效应消失。这些结果说明特征变化的连续性同样影响客体连续表征的维持。     

Visual awareness due to neuronal activities in subcortical structures: a proposal

It has been shown that visual awareness in the blind hemifield of hemianopic cats that have undergone unilateral ablations of visual cortex can be restored by sectioning the commissure of the superior colliculus or by destroying a portion of the substantia nigra contralateral to the cortical lesion (the Sprague effect). We propose that the visual awareness that is recovered is due to synchronized oscillatory activities in the superior colliculus ipsilateral to the cortical lesion. These oscillatory activities are normally partially suppressed by the inhibitory, GABAergic contralateral nigrotectal projection, and the destruction of the substantia nigra, or the sectioning of the collicular commissure, disinhibits the collicular neurons, causing an increase in the extent of oscillatory activity and/or synchronization between activities at different sites. This increase in the oscillatory and synchronized character is sufficient for the activities to give rise to visual awareness. We argue that in rodents and lower vertebrates, normal visual awareness is partly due to synchronized oscillatory activities in the optic tectum and partly due to similar activities in visual cortex. It is only in carnivores and primates that visual awareness is wholly due to cortical activities. Based on von Baerian recapitulation theory, we propose that, even in humans, there is a period in early infancy when visual awareness is partially due to activities in the superior colliculus, but that this awareness gradually disappears as the nigrotectal projection matures.     

The topography of training-induced visual field recovery: Perimetric maps and subjective representations

The cognitive representation of blind regions varies considerably between patients with vision loss and may influence compensatory behaviour and treatment motivation. We therefore measured “objective” visual field topography (perimetry) in 19 patients with postgeniculate visual system lesions and related this to the subjective scotoma representation as expressed by patients’ drawings of the defect and monitored changes of these measures during training-induced recovery of function. Blind regions were mostly adequately represented; however, central regions were overestimated and peripheral areas underestimated in size. Perimetric and subjective defect size decreased significantly during training. Again, training-induced visual field border shifts in central regions were larger in subjective than in perimetric maps but vice versa in the peripheral field. Thus, vision restoration therapy improves “objective” visual field size along with its cognitive representation. The subjective topography is shaped by the functional importance of visual field regions and is a function of cortical magnification, thus resembling the neural representation in visual cortex.     

The cortex in multidimensional space: where do cortical areas come from?

The concept of a cortical ‘area’ as a discrete phylogenetic, developmental and computational unit is evaluated. Evidence including the comparative organization of the forebrain in vertebrates, the organization of cortex in different mammals, the scaling of the areas of the isocortex in mammals, and the early molecular differentiation of the cortex all suggest a special status for the primary sensory cortical areas, particularly the visual cortex. Furthermore, the overlapping gradients of early molecular expression and the patterning of cortical structure and connectivity by thalamic input suggest a new view of cortical organization that is different from the traditional view of a developmentally mosaic cortex; this view proposes that distinct cortical areas arise combinatorily from the multiple overlapping processes imposed upon the developing cortex.     

Abnormal retinotopic representations in human visual cortex revealed by fMRI

The representation of the visual field in early visual areas is retinotopic. The point-to-point relationship on the retina is therefore maintained on the convoluted cortical surface. Functional magnetic resonance imaging (fMRI) has been able to demonstrate the retinotopic representation of the visual field in occipital cortex of normal subjects. Furthermore, visual areas that are retinotopic can be identified on computationally flattened cortical maps on the basis of positions of the vertical and horizontal meridians. Here, we investigate abnormal retinotopic representations in human visual cortex with fMRI. We present three case studies in which patients with visual disorders are investigated. We have tested a subject who only possesses operating rod photoreceptors. We find in this case that the cortex undergoes a remapping whereby regions that would normally represent central field locations now map more peripheral positions in the visual field: In a human albino we also find abnormal visual cortical activity. Monocular stimulation of each hemifield resulted in activations in the hemisphere contralateral to the stimulated eye. This is consistent with abnormal decussation at the optic chiasm in albinism. Finally, we report a case where a lesion to white matter has resulted in a lack of measurable activity in occipital cortex. The activity was absent for a small region of the visual field, which was found to correspond to the subject's field defect. The cases selected have been chosen to demonstrate the power of fMRI in identifying abnormalities in the cortical representations of the visual field in patients with visual dysfunction. Furthermore, the experiments are able to show how the cortex is capable of modifying the visual field representation in response to abnormal input.     

The distributed human neural system for face perception

Face perception, perhaps the most highly developed visual skill in humans, is mediated by a distributed neural system in humans that is comprised of multiple, bilateral regions. We propose a model for the organization of this system that emphasizes a distinction between the representation of invariant and changeable aspects of faces. The representation of invariant aspects of faces underlies the recognition of individuals, whereas the representation of changeable aspects of faces, such as eye gaze, expression, and lip movement, underlies the perception of information that facilitates social communication. The model is also hierarchical insofar as it is divided into a core system and an extended system. The core system is comprised of occipitotemporal regions in extrastriate visual cortex that mediate the visual analysis of faces. In the core system, the representation of invariant aspects is mediated more by the face-responsive region in the fusiform gyrus, whereas the representation of changeable aspects is mediated more by the face-responsive region in the superior temporal sulcus. The extended system is comprised of regions from neural systems for other cognitive functions that can be recruited to act in concert with the regions in the core system to extract meaning from faces.