Incremental grouping of image elements in vision

One important task for the visual system is to group image elements that belong to an object and to segregate them from other objects and the background. We here present an incremental grouping theory (IGT) that addresses the role of object-based attention in perceptual grouping at a psychological level and, at the same time, outlines the mechanisms for grouping at the neurophysiological level. The IGT proposes that there are two processes for perceptual grouping. The first process is base grouping and relies on neurons that are tuned to feature conjunctions. Base grouping is fast and occurs in parallel across the visual scene, but not all possible feature conjunctions can be coded as base groupings. If there are no neurons tuned to the relevant feature conjunctions, a second process called incremental grouping comes into play. Incremental grouping is a time-consuming and capacity-limited process that requires the gradual spread of enhanced neuronal activity across the representation of an object in the visual cortex. The spread of enhanced neuronal activity corresponds to the labeling of image elements with object-based attention.     

The time course of perceptual grouping: the role of segregation and shape formation

The time course of perceptual grouping was examined in two experiments, using a primed matching task. In different conditions, elements were grouped into columns/rows by common lightness, into a shape (triangle/ arrow or square/cross) by common lightness, and into a shape without segregation of elements. The results showed an early and rapid grouping into columns/rows by common lightness and into a shape when no segregation from other elements was involved. Goodness of shape (i.e., triangle/arrow vs. square/cross) had no influence on how early grouping was evident, but the relatively poorer shapes appeared to consolidate with time. In contrast, grouping into a shape that involved segregation and required resolving figure-ground relations between segregated units, as grouping into a shape by common lightness, consumed time, regardless of shape goodness. These results suggest that the time course of grouping varies as a function of the processes involved in it (e.g., segregation and shape formation) and the conditions prevailing for each process.     

Perceptual grouping and attention: Not all groupings are equal

We examined grouping under inattention using Driver, Davis, Russell, Turatto, & Freeman’s (2001) method. On each trial, two successive displays were briefly presented, each comprising a central target square surrounded by elements. The task was to judge whether the two targets were the same or different. The organization of the background elements stayed the same or changed, independently of the targets. In different conditions, background elements grouped into columns/rows by color similarity, a shape (a triangle/arrow, a square/cross, or a vertical/horizontal line) by color similarity, and a shape with no other elements in the background. We measured the influence of the background on the target same-different judgments. The results imply that background elements grouped into columns/rows by color similarity and into a shape when no segregation from other elements was involved and the shape was relatively “good.” In contrast, no background grouping was observed when resolving figure-ground relations for segregated units was required, as in grouping into a shape by color similarity. These results suggest that grouping is a multiplicity of processes that vary in their attentional demands. Regardless of attentional demands, the products of grouping are not available to awareness without attention.     

On the perception of shape from shading

The extraction of three-dimensional shape from shading is one of the most perceptually compelling, yet poorly understood, aspects of visual perception. In this paper, we report several new experiments on the manner in which the perception of shape from shading interacts with other visual processes such as perceptual grouping, preattentive search (“pop-out”), and motion perception. Our specific findings are as follows: (1) The extraction of shape from shading information incorporates at least two “assumptions” or constraints—first,that there is a single light source illuminating the whole scene, and second, that the light is shining from “above” in relation to retinal coordinates. (2) Tokens defined by shading can serve as a basis for perceptual grouping and segregation. (3) Reaction time for detecting a single convex shape does not increase with the number of items in the display. This “pop-out” effect must be based on shading rather than on differences in luminance polarity, since neither left-right differences nor step changes in luminance resulted in pop-out. (4) When the subjects were experienced, there were no search asymmetries for convex as opposed to concave tokens, but when the subjects were naive, cavities were much easier to detect than convex shapes. (5) The extraction of shape from shading can also provide an input to motion perception. And finally, (6) the assumption of “overhead illumination” that leads to perceptual grouping depends primarily on retinal rather than on “phenomenal” or gravitational coordinates. Taken collectively, these findings imply that the extraction of shape from shading is an “early” visual process that occurs prior to perceptual grouping, motion perception, and vestibular (as well as “cognitive”) correction for head tilt. Hence, there may be neural elements very early in visual processing that are specialized for the extraction of shape from shading.     

On the perception of shape from shading.

The extraction of three-dimensional shape from shading is one of the most perceptually compelling, yet poorly understood, aspects of visual perception. In this paper, we report several new experiments on the manner in which the perception of shape from shading interacts with other visual processes such as perceptual grouping, preattentive search ("pop-out"), and motion perception. Our specific findings are as follows: (1) The extraction of shape from shading information incorporates at least two "assumptions" or constraints--first, that there is a single light source illuminating the whole scene, and second, that the light is shining from "above" in relation to retinal coordinates. (2) Tokens defined by shading can serve as a basis for perceptual grouping and segregation. (3) Reaction time for detecting a single convex shape does not increase with the number of items in the display. This "pop-out" effect must be based on shading rather than on differences in luminance polarity, since neither left-right differences nor step changes in luminance resulted in pop-out. (4) When the subjects were experienced, there were no search asymmetries for convex as opposed to concave tokens, but when the subjects were naive, cavities were much easier to detect than convex shapes. (5) The extraction of shape from shading can also provide an input to motion perception. And finally, (6) the assumption of "overhead illumination" that leads to perceptual grouping depends primarily on retinal rather than on "phenomenal" or gravitational coordinates. Taken collectively, these findings imply that the extraction of shape from shading is an "early" visual process that occurs prior to perceptual grouping, motion perception, and vestibular (as well as "cognitive") correction for head tilt. Hence, there may be neural elements very early in visual processing that are specialized for the extraction of shape from shading.     

Grouping effects in flash-induced perceptual fading

We show that the flash-induced fading effect can be influenced by grouping based on colour and shape similarity. In flash-induced fading, peripherally presented elements perceptually disappear after a flash has been presented around or next to the element(s) (Kanai and Kamitani, 2003 Journal of Cognitive Neuroscience 15 664-672). In the present study, two elements (which could either be similar or different in both colour and shape) were presented at an eccentricity of 2.6 degrees from a fixation cross. After 2 s, a short flash (40 ms) was presented around one of the elements. The two elements remained on the screen for 2.5 s after the flash. We found a higher percentage of mutual perceptual disappearance when the two elements were similar in colour or in shape. These results converge with other findings showing that perceptual fading phenomena are influenced by effects of perceptual grouping. This indicates that, besides low-level adaptation, more global processes play a part in perceptual fading as well and should thus be taken into account in giving a full explanation of perceptual fading in general.     

Effects of similarity on apparent motion and perceptual grouping

Effects of similarity in colour, luminance, size, and shape on apparent motion and perceptual grouping were examined in part 1 in two parallel experiments on the same seven subjects. In both experiments, the effect of similarity was compared with that of proximity in competitive, bistable stimulus situations. A combination of a larger horizontal separation between the homogeneous stimulus elements and a smaller constant vertical separation between heterogeneous stimulus elements produced two kinds of apparent motion (or perceptual grouping) with equal probabilities. Such matched separations between homogeneous stimulus elements were obtained by the double staircase method in various stimulus conditions. In both experiments on apparent motion and perceptual grouping matched separation was found to increase as the difference between the heterogeneous stimulus elements increased. High correlations (0.71 to 0.94) of matched separations were found between apparent motion and perceptual grouping in four stimulus series: colour, luminance, size, and shape. Six of the seven subjects were also tested in part 2. Here, the effects of differences were found to work additively across different perceptual attributes in both phenomena, when multiple differences were combined in heterogeneous elements. The experimental results are discussed from the point of view that apparent motion is an example of perceptual constancy.     

Grouping puts figure-ground assignment in context by constraining propagation of edge assignment

Figure-ground organization involves the assignment of edges to a figural shape on one or the other side of each dividing edge. Established visual cues for edge assignment primarily concern relatively local rather than contextual factors. In the present article, we show that an assignment for a locally unbiased edge can be affected by an assignment of a remote contextual edge that has its own locally biased assignment. We find that such propagation of edge assignment from the biased remote context occurs only when the biased and unbiased edges are grouped. This new principle, whereby grouping constrains the propagation of figural edge assignment, emerges from both subjective reports and an objective short-term edge-matching task. It generalizes from moving displays involving grouping by common fate and collinearity, to static displays with grouping by similarity of edge-contrast polarity, or apparent occlusion. Our results identify a new contextual influence on edge assignment. They also identify a new mechanistic relation between grouping and figure-ground processes, whereby grouping between remote elements can constrain the propagation of edge assignment between those elements. Supplemental materials for this article may be downloaded from http://app.psychonomic-journals.org/content/supplemental.     

Sensitivity to attachment, alignment, and contrast polarity variation in local perceptual grouping

A number of leading theories (e.g., Grossberg & Mingolla, 1985; Kellman & Shipley, 1991; Rensink & Enns, 1995) commonly assume that perceptual grouping by contour alignment occurs preattentively across reversing contrast polarity elements. We examined this notion in seven visual search experiments. We found that only grouping by attachment supported preattentive visual search and that grouping by contour alignment required attention in order to operate. Both attachment grouping and grouping by contour alignment were sensitive to contrast reversals. Further results showed that contour alignment was a strong grouping cue only among elements with the same contrast sign but that it did not facilitate grouping across reversing contrast. These results suggest that grouping by contour alignment operates only on inputs of consistent contrast polarity.     

Associative grouping: Perceptual grouping of shapes by association

Perceptual grouping is usually defined by principles that associate distinct elements by virtue of image properties, such as proximity, similarity, and occurrence within common regions. What role does learning play in forming a perceptual group? This study provides evidence that learning of shape associations leads to perceptual grouping. Subjects were repeatedly exposed to pairs of unique shapes that co-occurred within a common region. The common region cue was later removed in displays composed of these shapes, and the subjects searched the display for two adjacent shapes of the same color. The subjects were faster at locating the color repetition when the adjacent shapes with the same color came from the same trained groups than when they were composed of two shapes from different trained groups. The effects were perceptual in nature: Learned pairings produced spatial distortions similar to those observed for groups defined by perceptual similarity. A residual grouping effect was observed even when the shapes in the trained group switched their relative positions but was eliminated when each shape was inverted. These results indicate that statistical co-occurrence with explicit grouping cues may form an important component of perceptual organization, determining perceived scene structure solely on the basis of past experience.     

Perceptual grouping in space and time: Evidence from the Ternus display

We report three experiments investigating the effect of perceptual grouping on the appearance of a bistable apparent-motion (Ternus) display. Subjects viewed a Ternus display embedded in an array of context elements that could potentially group with the Ternus elements. In contrast to several previous findings, we found that grouping influenced apparent motion perception. In Experiment 1, apparent motion perception was significantly affected via grouping by shape similarity, even when the visible persistence of the elements was controlled. In Experiment 2, elements perceived as moving without context were perceived as stationary when grouped with stationary context elements. In Experiment 3, elements perceived as stationary without context were perceived as moving when grouped with moving context elements. We argue that grouping in the spatial and temporal domains interact to yield perceptual experience of apparent-motion displays.     

Late influences on perceptual grouping: illusory figures

Two experiments demonstrate that grouping can be strongly influenced by the presence of figures defined by illusory contours. Rectangular arrays were constructed in which a central column of figures could group either with those on one side, on the basis of perception of figures defined by illusory contours, or with those on the other side, on the basis of physically present inducing elements. In all displays, subjects grouped according to the illusory figures significantly more often than for control displays that contained the same inducing elements, but rearranged so that illusory contours were degraded or eliminated. A second experiment showed that in objectively defined grouping tasks, subjects grouped faster by illusory figures than by inducing elements. These results indicate that grouping can occur after illusory contours have been perceived.     

Selective attention as tuning: the case of stroke weight

It has long been demonstrated that when grouping occurs, attention transfer between grouped elements is facilitated, as compared with attention transfer between elements-similarly distant-that are not grouped. This has been shown for grouping by connectedness, by orientation, and by color. The present article extends these findings to the case of similarity in coarseness. By using spatial cuing to elements drawn with different strokes, it is shown that the visual processing of elements that sharestroke heaviness with the cued element is more efficient than that of elements that do not. Three experiments, in which cue validity regarding the target's location and/or its stroke is manipulated, show that the facilitation has both an endogenous and an exogenous component. The findings are discussed in terms of visual tuning to the features of a stimulus, with tuning being the initial stage of visual processing required for identification and discrimination. It is proposed that grouping, rather than explaining the facilitation observed, can be explained by the notion of visual tuning to features. The findings also point to potential methodological pitfalls when different stroke weights are used, unintentionally, in visual displays.     

Grouping based on phenomenal similarity of achromatic color.

It is widely acknowledged that a precondition for the perception of the world of objects and events is an early process of organization, and it has generally been assumed that such organization is based on the Gestalt laws of grouping. However, the stage at which such grouping occurs, whether early or late, is an empirical question. It is demonstrated in two experiments that grouping by similarity of neutral color is based not on similarity of absolute luminance at the level of the proximal stimulus, but on phenomenal similarity of lightness resulting from the achievement of lightness constancy. An alternative explanation of such grouping based on the equivalence of luminance ratios between elements and background is ruled out by appropriate control conditions.     

Induced perceptual grouping

ABSTRACT— The term perceptual grouping is associated with classical principles such as similarity and proximity. This article reports induced perceptual grouping, a phenomenon that occurs when placement of a uniform set of items near a structured set induces grouping within the otherwise uniform set. For example, when items grouped pair-wise by similarity are placed near another set of unstructured items, an analogous pair-wise grouping links elements of the second set. Induced grouping affected reaction times in two different visual search tasks, with reaction times depending on whether the target properties were contained within a group or crossed group boundaries as defined by induced grouping due to similarity, proximity, or common fate. Induced grouping was reduced when grouping between the structured and unstructured sets was weakened by means of a common-region cue or decreased similarity. Induced grouping appears to reflect the computation of hierarchical structure in visual images.     

When does grouping happen?

Recent research on perceptual grouping is described with particular emphasis on identifying the level(s) at which grouping factors operate. Contrary to the classical view of grouping as an early, two-dimensional, image-based process, recent experimental results show that it is strongly influenced by phenomena related to perceptual constancy, such as binocular depth perception, lightness constancy, amodal completion, and illusory contours. These findings imply that at least some grouping processes operate at the level of phenomenal perception rather than at the level of the retinal image. Preliminary evidence is reported showing that grouping can affect perceptual constancy, suggesting that grouping processes must also operate at an early, preconstancy level. If so, grouping may be a ubiquitous, ongoing aspect of visual organization that occurs for each level of representation rather than as a single stage that can be definitively localized relative to other perceptual processes.     

Gestalt similarity principle,difference thresholds and pattern discriminability

Perceptual grouping within a visual array has been studied as a function of the difference limen between elements composing the array. Two experiments are reported in which the array elements differed in size or in shape. Although there is a relationship between perceptual emergence and amount of difference between elements of the display as expected, there was clear evidence that above threshold differences between elements could not be used to predict the emergence of a pattern hidden in the visual array.     

Perceptual grouping and distance estimates in typical and atypical development: comparing performance across perception, drawing and construction tasks

Perceptual grouping is a pre-attentive process which serves to group local elements into global wholes, based on shared properties. One effect of perceptual grouping is to distort the ability to estimate the distance between two elements. In this study, biases in distance estimates, caused by four types of perceptual grouping, were measured across three tasks, a perception, a drawing and a construction task in both typical development (TD: Experiment 1) and in individuals with Williams syndrome (WS: Experiment 2). In Experiment 1, perceptual grouping distorted distance estimates across all three tasks. Interestingly, the effect of grouping by luminance was in the opposite direction to the effects of the remaining grouping types. We relate this to differences in the ability to inhibit perceptual grouping effects on distance estimates. Additive distorting influences were also observed in the drawing and the construction task, which are explained in terms of the points of reference employed in each task. Experiment 2 demonstrated that the above distortion effects are also observed in WS. Given the known deficit in the ability to use perceptual grouping in WS, this suggests a dissociation between the pre-attentive influence of and the attentive deployment of perceptual grouping in WS. The typical distortion in relation to drawing and construction points towards the presence of some typical location coding strategies in WS. The performance of the WS group differed from the TD participants on two counts. First, the pattern of overall distance estimates (averaged across interior and exterior distances) across the four perceptual grouping types, differed between groups. Second, the distorting influence of perceptual grouping was strongest for grouping by shape similarity in WS, which contrasts to a strength in grouping by proximity observed in the TD participants.     

Perceptual grouping operates independently of attentional selection: Evidence from hemispatial neglect

To what extent can human observers process visual information that is not currently the focus of attention? We evaluated the extent to which unattended visual information (i.e., that which appears on the neglected side of space in individuals with hemispatial neglect) is perceptually organized and influences the perceptual processing of information on the attended side. To examine this, patients (and matched controls) judged whether successive, complex checkerboard stimuli (targets), presented entirely to their intact side of space, were the same or different. Concurrent with this demanding task, irrelevant distractor elements appeared on the unattended side and either changed or retained their perceptual grouping on successive displays, independently of changes in the ipsilesional task-relevant target. Changes in the grouping of the unattended task-irrelevant distractor elements produced congruency effects on the attended target-change judgment to the same extent in the neglect patients as in the control participants, and this was true even in those patients with severe attentional deficits. These results suggest that some perceptual processes, such as grouping, can operate in the absence of attention.