Emergence of perceptual Gestalts in the human visual cortex: the case of the configural-superiority effect

Many Gestalt phenomena have been described in terms of perception of a whole being not equal to the sum of its parts. It is unclear how these phenomena emerge in the brain. We used functional MRI to study the neural basis of the behavioral configural-superiority effect (i.e., visual search is more efficient when an odd element is part of a configuration than when it is presented by itself). We found that searching for the odd element in a display of four line segments (parts) was facilitated by adding two additional line segments to each of them (creating whole shapes). Functional MRI-based decoding of neural responses to the position of the odd element revealed a neural configural-superiority effect in shape-selective regions but not in low-level retinotopic areas, where decoding of parts was more pronounced. These results show how at least some Gestalt phenomena in vision emerge only at the higher stages of visual information processing and suggest that feed-forward processing might be sufficient to produce such phenomena.     

What does the Ternus display tell us about motion processing in human vision?

The Ternus display is a moving visual stimulus which elicits two very different percepts, according to the length of the interstimulus interval (ISI) between each frame of the motion sequence. These two percepts, referred to as element motion and group motion, have previously been analysed in terms of the operation of a low-level, dedicated short-range motion process (in the case of element motion), and of a higher-level, attentional long-range motion process (in the case of group motion). We used a novel Ternus configuration to show that both element and group motion are, in fact, mediated solely by a process sensitive to changes in the spatial appearance of the Ternus elements. In light of this, it appears that Ternus displays tell us nothing about low-level motion processing, implying that previous studies using Ternus displays, for instance those dealing with dyslexia, require reinterpretation. Further manipulations of the Ternus display revealed that the orientation and spatial-frequency discrimination of the process underlying the analysis of Ternus displays is far worse than thresholds for spatial vision. We conclude that Ternus displays are analysed via a long-range motion, or feature-tracking, process, and that this process is distinct from spatial vision.     

Globally perceived directional flow in static images

Visual sensitivity to spatial direction has classically been associated with motion perception. Yet humans are adept at deriving directional information in the absence of motion, as when they read maps, or follow arrows or animal tracks. Experiments are reported on the perception of parallel arrow-like forms in which a specific visual sensitivity to static direction is demonstrated. Global processing is operationally defined in terms of the relative discriminability of sets and subsets of stimulus elements; a set of parallel elements and a set in which one element is antiparallel to the rest are shown to be processed globally. The result of this global processing is a static analog of unidirectional optic flow. Global spatial direction differs fundamentally from other perceptions derived from static image processing. It involves long-range interactions in texture arrays, it does not carry information about stimulus location, and it is not reducible to the perception of component stimulus elements. Its likely function is in the construction of the layout of visual space.     

Dynamics of perceptual grouping: Similarities in the organization of visual and auditory groups

In vision, the Gestalt principles of perceptual organization are generally well understood and remain a subject of detailed analysis. However, the possibility for a unified theory of grouping across visual and auditory modalities remains largely unexplored. Here we present examples of auditory and visual Gestalt grouping, which share important organizational properties. In particular, similarities are revealed between grouping processes in apparent motion, auditory streaming, and static 2-D displays. Given the substantial difference in the context, within which the phenomena in question occur (auditory vs. visual, static vs. dynamic), these similarities suggest that the dynamics of perceptual organization could be associated with a common (possibly central) mechanism. If the relevance of supramodal invariants of grouping is granted, the question arises as to whether they can be studied empirically. We propose that a “force-field” theory, based on a differential-geometric interpretation of perceptual space, could provide a suitable starting point for a systematic exploration of the subjective properties of certain classes of auditory and visual grouping phenomena.     

Context-sensitive binding by the laminar circuits of V1 and V2: A unified model of perceptual grouping,attention, and orientation contrast

A detailed neural model is presented of how the laminar circuits of visual cortical areas V1 and V2 implement context-sensitive binding processes such as perceptual grouping and attention. The model proposes how specific laminar circuits allow the responses of visual cortical neurons to be determined not only by the stimuli within their classical receptive fields, but also to be strongly influenced by stimuli in the extra-classical surround. This context-sensitive visual processing can greatly enhance the analysis of visual scenes, especially those containing targets that are low contrast, partially occluded, or crowded by distractors. We show how interactions of feedforward, feedback, and horizontal circuitry can implement several types of contextual processing simultaneously, using shared laminar circuits. In particular, we present computer simulations that suggest how top-down attention and preattentive perceptual grouping, two processes that are fundamental for visual binding, can interact, with attentional enhancement selectively propagating along groupings of both real and illusory contours, thereby showing how attention can selectively enhance object representations. These simulations also illustrate how attention may have a stronger facilitatory effect on low contrast than on high contrast stimuli, and how pop-out from orientation contrast may occur. The specific functional roles which the model proposes for the cortical layers allow several testable neurophysiological predictions to be made. The results presented here simulate only the boundary grouping system of adult cortical architecture. However, we also discuss how this model contributes to a larger neural theory of vision that suggests how intracortical and intercortical feedback help to stabilize development and learning within these cortical circuits. Although feedback plays a key role, fast feedforward processing is possible in response to unambiguous information. Model circuits are capable of synchronizing quickly, but context-sensitive persistence of previous events can influence how synchrony develops. Although these results focus on how the interblob cortical processing stream controls boundary grouping and attention, related modelling of the blob cortical processing stream suggests how visible surfaces are formed, and modelling of the motion stream suggests how transient responses to scenic changes can control long-range apparent motion and also attract spatial attention.     

Gestalt Issues in Modern Neuroscience

We present select examples of how visual phenomena can serve as tools to uncoverbrain mechanisms. Specifically, receptive field organization is proposed as a Gestalt-like neural mechanism of perceptual organization. Appropriate phenomena, such as brightness and orientation contrast, subjective contours, filling-in, and aperture-viewed motion, allow for a quantitative comparison between receptive fields and their psychophysical counterparts, perceptive fields. Phenomenology might thus be extended from the study of perceptual qualities to their transphenomenal substrates, including memory functions. In conclusion, classic issues of Gestalt psychology can now be related to modern “Gestalt psychophysics” and neuroscience.     

Orientation Constancy in Neurons of Monkey Visual Cortex

We studied the effect of body tilt on the orientation selectivity of single neurons in the visual cortex of an alert monkey. The monkey performed a visual fixation task either in the upright position or with its whole body tilted about the naso-occipital (roll) axis by ±25° or ±30°. We determined the preferred stimulus orientation for 51 of 117 neurons in two or, if possible, three body positions (i.e. with the whole body upright, and tilted either left ear or right ear down). In striate cortex, most of the neurons were of a non-compensatory type, showing a change in the preferred orientation according to the body tilt and the estimated counterrolling of the eye. By contrast, about 40% of the neurons in prestriate cortex were of a compensatory type, preferring similar orientations in all body positions. This suggests that mechanisms which produce orientation constancy with respect to the direction of gravity are implemented at an early stage of cortical processing.     

Cortical dynamics of visual motion perception: short-range and long-range apparent motion.

This article describes further evidence for a new neural network theory of biological motion perception. The theory clarifies why parallel streams V1----V2, V1----MT, and V1----V2----MT exist for static form and motion form processing among the areas V1, V2, and MT of visual cortex. The theory suggests that the static form system (Static BCS) generates emergent boundary segmentations whose outputs are insensitive to direction-of-contrast and to direction-of-motion, whereas the motion form system (Motion BCS) generates emergent boundary segmentations whose outputs are insensitive to direction-of-contrast but sensitive to direction-of-motion. The theory is used to explain classical and recent data about short-range and long-range apparent motion percepts that have not yet been explained by alternative models. These data include beta motion, split motion, gamma motion and reverse-contrast gamma motion, delta motion, and visual inertia. Also included are the transition from group motion to element motion in response to a Ternus display as the interstimulus interval (ISI) decreases; group motion in response to a reverse-contrast Ternus display even at short ISIs; speed-up of motion velocity as interflash distance increases or flash duration decreases; dependence of the transition from element motion to group motion on stimulus duration and size, various classical dependencies between flash duration, spatial separation, ISI, and motion threshold known as Korte's laws; dependence of motion strength on stimulus orientation and spatial frequency; short-range and long-range form-color interactions; and binocular interactions of flashes to different eyes.     

Pattern orientation, not motion, determines the two-dimensional tilt illusion

In the usual tilt illusion (TI) configuration, an inducing stimulus which has a single orientation is used psychophysically to explore orientation analysis in the human visual system. Recently, this approach has been extended to the use of inducing stimuli which have two orientations. Such a two-dimensional (2-D) stimulus permits investigation of the low-level analysis of visual patterns. Prior experimentation has left it unclear whether it is the spatial or the motion properties of a moving crossed-grating plaid which determine two-dimensional tilt illusions (2-D TIs) because these two parameters previously were perfectly correlated. In the present experiments pattern orientation and motion were decoupled. It is shown that 2-D TIs are determined by the spatial properties of an inducing annulus and not by its motion properties. The results also support the existence of a mechanism which extracts axes of symmetry, and which is difficult to account for in terms of local cross-orientation domain inhibition.     

Perceptual adaptation: motion parallels orientation

Adaptation phenomena provide striking examples of perceptual plasticity and offer valuable insight into the mechanisms of visual coding. Within the context of recent progress in neurobiology and computational modelling, I review evidence from studies employing psychophysical adaptation to investigate orientation and motion processing. These studies reveal marked similarities between the orientation and motion domains, raising the possibility that common computational principles underlie the processing of orientation and motion despite apparently distinct cortical substrates.     

The lost direction in binocular vision: the neglected signs posted by Wells, Towne, and LeConte

Studies of vision have informed theories first in philosophy and then in psychology. Over the centuries, an increasing number of phenomena have been enlisted to refute or reinforce particular theories. Nowhere has this been more evident than in binocular vision. How we see a single world with two eyes is one of the oldest and most consistently studied topics in vision research. It has been discussed at least since the time of Aristotle and it has been examined experimentally since the second century, when Ptolemy defined lines of visual correspondence for the two eyes. Prior to Wheatstone's invention of the stereoscope in the 1830s, binocular vision had been studied in terms of visual directions. The stereoscope established distance (or depth) as well as direction as dimensions of binocular vision. Subsequently, depth rather than direction has been the principal concern of students of vision, and texts in English devoted to analyses of direction rather than depth have been neglected. We examine the experiments on binocular visual direction conducted by Wells before Wheatstone, and by Towne and LeConte after him, and discuss the reasons for their neglect.     

On the mental representations originating during the interaction between language and vision

The interaction between vision and language processing is clearly of interest to both cognitive psychologists and psycholinguists. Recent research has begun to create understanding of the interaction between vision and language in terms of the representational issues involved. In this paper, we first review some of the theoretical and methodological issues in the current vision–language interaction debate. Later, we develop a model that attempts to account for effects of affordances and visual context on language-scene interaction as well as the role of sensorimotor simulation. The paper addresses theoretical issues related to the mental representations that arise when visual and linguistic systems interact.     

Visual orientation illusions: global mechanisms involved in hierarchical effects and frames of reference

Five experiments were conducted in order to determine which of two hypotheses, initially proposed by Rock (1990), accounts for interactions between oriented elements in a visual scene. We also explored the suggestion that two hypothetical processes--namely, frame of reference and hierarchical organization--describe phenomena arising from distinct mechanisms (Spinelli, Antonucci, Daini, Martelli, & Zoccolotti, 1999). Double inducing stimulus versions of one-dimensional and two-dimensional tilt illusions, the rod-and-frame illusion, and combinations of these were used. Our data suggest that both hypotheses can predict orientation interactions in conditions in which only one mechanism--namely, the global visual mechanism of symmetry axes extraction (Wenderoth & Beh, 1977)--is activated. Which hypothesis is appropriate to predict the perceived orientation depends on some physical features of the objects.     

Emotion Improves and Impairs Early Vision

ABSTRACT— Recent studies indicate that emotion enhances early vision, but the generality of this finding remains unknown. Do the benefits of emotion extend to all basic aspects of vision, or are they limited in scope? Our results show that the brief presentation of a fearful face, compared with a neutral face, enhances sensitivity for the orientation of subsequently presented low-spatial-frequency stimuli, but diminishes orientation sensitivity for high-spatial-frequency stimuli. This is the first demonstration that emotion not only improves but also impairs low-level vision. The selective low-spatial-frequency benefits are consistent with the idea that emotion enhances magnocellular processing. Additionally, we suggest that the high-spatial-frequency deficits are due to inhibitory interactions between magnocellular and parvocellular pathways. Our results suggest an emotion-induced trade-off in visual processing, rather than a general improvement. This trade-off may benefit perceptual dimensions that are relevant for survival at the expense of those that are less relevant.     

Serial and parallel search in pattern vision?

The nature of the processing of combinations of stimulus dimensions in human vision has recently been investigated. A study is reported in which visual search for suprathreshold positional information--vernier offsets, stereoscopic disparity, lateral separation, and orientation--was examined. The initial results showed that reaction times for visual search for conjunctions of stereoscopic disparity and either vernier offsets or orientation were independent of the number of distracting stimuli displayed, suggesting that disparity was searched in parallel with vernier offsets or orientation. Conversely, reaction times for detection of conjunctions of vernier offsets and orientation, or lateral separation and each of the other positional judgements, were related linearly to the number of distractors, suggesting serial search. However, practice has a significant effect upon the results, indicative of a shift in the mode of search from serial to parallel for all conjunctions tested as well as for single features. This suggests a reinterpretation of these and perhaps other studies that use the Treisman visual search paradigm, in terms of perceptual segregation of the visual field by disparity, motion, color, and pattern features such as colinearity, orientation, lateral separation, or size.     

Grouping and emergent features in vision: toward a theory of basic Gestalts

Gestalt phenomena are often so powerful that mere demonstrations can confirm their existence, but Gestalts have proven hard to define and measure. Here we outline a theory of basic Gestalts (TBG) that defines Gestalts as emergent features (EFs). The logic relies on discovering wholes that are more discriminable than are the parts from which they are built. These wholes contain EFs that can act as basic features in human vision. As context is added to a visual stimulus, a hierarchy of EFs appears. Starting with a single dot and adding a second yields the first two potential EFs: the proximity (distance) and orientation (angle) between the two dots. A third dot introduces two more potential EFs: symmetry and linearity; a fourth dot produces surroundedness. This hierarchy may extend to collinearity, parallelism, closure, and more. We use the magnitude of Configural Superiority Effects to measure the salience of EFs on a common scale, potentially letting us compare the strengths of various grouping principles. TBG appears promising, with our initial experiments establishing and quantifying at least three basic EFs in human vision.     

Why do parallel cortical systems exist for the perception of static form and moving form?

This article analyzes computational properties that clarify why the parallel cortical systems V1----V2, V1----MT, and V1----V2----MT exist for the perceptual processing of static visual forms and moving visual forms. The article describes a symmetry principle, called FM symmetry, that is predicted to govern the development of these parallel cortical systems by computing all possible ways of symmetrically gating sustained cells with transient cells and organizing these sustained-transient cells into opponent pairs of on-cells and off-cells whose output signals are insensitive to direction of contrast. This symmetric organization explains how the static form system (static BCS) generates emergent boundary segmentations whose outputs are insensitive to direction of contrast and insensitive to direction of motion, whereas the motion form system (motion BCS) generates emergent boundary segmentations whose outputs are insensitive to direction of contrast but sensitive to direction of motion. FM symmetry clarifies why the geometries of static and motion form perception differ--for example, why the opposite orientation of vertical is horizontal (90 degrees), but the opposite direction of up is down (180 degrees). Opposite orientations and directions are embedded in gated dipole opponent processes that are capable of antagonistic rebound. Negative afterimages, such as the MacKay and waterfall illusions, are hereby explained as are aftereffects of long-range apparent motion. These antagonistic rebounds help to control a dynamic balance between complementary perceptual states of resonance and reset. Resonance cooperatively links features into emergent boundary segmentations via positive feedback in a CC loop, and reset terminates a resonance when the image changes, thereby preventing massive smearing of percepts. These complementary preattentive states of resonance and reset are related to analogous states that govern attentive feature integration, learning, and memory search in adaptive resonance theory. The mechanism used in the V1----MT system to generate a wave of apparent motion between discrete flashes may also be used in other cortical systems to generate spatial shifts of attention. The theory suggests how the V1----V2----MT cortical stream helps to compute moving form in depth and how long-range apparent motion of illusory contours occurs. These results collectively argue against vision theories that espouse independent processing modules. Instead, specialized subsystems interact to overcome computational uncertainties and complementary deficiencies, to cooperatively bind features into context-sensitive resonances, and to realize symmetry principles that are predicted to govern the development of the visual cortex.     

Effect of prolonged tilt on visual orientation

Visual orientation during lateral tilt is viewed in terms of orientation constancy. The postural systems involved in the maintenance of constancy are considered to be those of the otolith, neck and trunk. The relative contribution of these systems was investigated by obtaining visual verticality judgments immediately upon and several minutes after head, body, and trunk tilts. Due to the apparent non-adaptation of the otolith system any changes in visual orientation resulting from prolonged tilt would be attributed to adaptation of the proprioceptive system stimulated. For 30° head tilt visual orientation over-constancy was reduced by about 2°, reflecting the influence of the neck system. Prolonged body tilts of 30°, 60° and 90° reduced the constancy operating by approximately 1°, 3° and 8°, respectively. This was taken to indicate the contribution of the trunk system, which increased with increasing degrees of body tilt. The above interpretations received strong support from experiments involving trunk tilt, which stimulates only the neck and trunk systems.     

A computational feature binding model of human texture perception

We present a computational model for human texture perception which assigns functional principles to the Gestalt laws of similarity and proximity. Motivated by early vision mechanisms, in the first stage, local texture features are extracted by utilizing multi-scale filtering and nonlinear spatial pooling. In the second stage, features are grouped according to the spatial feature binding model of the competitive layer model (CLM; Wersing et al. 2001). The CLM uses cooperative and competitive interactions in a recurrent network, where binding is expressed by the layer-wise coactivation of feature-representing neurons. The Gestalt law of similarity is expressed by a non-Euclidean distance measure in the abstract feature space with proximity being taken into account by a spatial component. To choose the stimulus dimensions which allow the most salient similarity-based texture segmentation, the feature similarity metrics is reduced to the directions of maximum variance. We show that our combined texture feature extraction and binding model performs segmentation in strong conformity with human perception. The examples range from classical microtextures and Brodatz textures to other classical Gestalt stimuli, which offer a new perspective on the role of texture for more abstract similarity grouping.