Interpolation processes in object perception: reply to Anderson (2007)

P. J. Kellman, P. Garrigan, & T. F. Shipley presented a theory of 3-D interpolation in object perception. Along with results from many researchers, this work supports an emerging picture of how the visual system connects separate visible fragments to form objects. In his commentary, B. L. Anderson challenges parts of that view, especially the idea of a common underlying interpolation component in modal and amodal completion (the identity hypothesis). Here the authors analyze Anderson's evidence and argue that he neither provides any reason to abandon the identity hypothesis nor offers a viable alternative theory. The authors offer demonstrations and analyses indicating that interpolated contours can appear modally despite absence of the luminance relations, occlusion geometry, and surface attachment that Anderson claims to be necessary. The authors elaborate crossing interpolations as key cases in which modal and amodal appearance must be consequences of interpolation. Finally, the authors dispute Anderson's assertion that vision researchers are misguided in using objective performance methods, and they argue that his challenges to relatability fail because contour and surface processes, as well as local and global influences, have been distinguished experimentally.     

Perception of partly occluded figures by baboons (Papio papio)

Comparative literature provides conflicting findings whether animals experience amodal completion. Five experiments were conducted to verify if baboons perceive partly occluded objects as complete. The first three experiments used a go/no-go procedure and a video monitor for stimulus presentation. These experiments failed to reveal amodal completion, suggesting that the stimuli were processed as 2-D images rather than 3-D objects. In contrast, completion was demonstrated in a fourth experiment with cardboard stimuli in a two-alternative forced-choice (2AFC) discrimination task presented in a Wisconsin General Test Apparatus. Although in experiment 5 the same 2AFC procedure was used as in experiment 4, completion was absent when the stimuli were shown with a computer graphic system. The results suggest that baboons share with humans the ability for amodal completion, but also underline some procedural factors that might affect the elicitation of this capacity.     

SPREADING OF VISUAL ATTENTION TO MODALLY VERSUS AMODALLY COMPLETED REGIONS

Abstract— Regions of objects that are partially obscured at the current retinal image are often perceptually filled in by the visual system (Kanizsa, 1979). In some cases (modal completion), this causes the filled-in region to appear tinged with the color and brightness of unobscured parts of that object, but m other cases (amodal completion), it does not (Michotte & Burke, 1951). It has recently been argued that modal and amodal completion both arise in preattentive vision, and may operate equivalently at that level (Davis & Driver, 1994, He & Nakayama, 1992, Shipley & Kellman, 1992). In this article, we show that they have different effects on attentive vision, with attention tending to spread to (and from) modally completed regions and their visible inducers, but not to (or from) comparable amodally completed regions and their inducers. This finding is consistent with visual attention operating on surfaces (e g, He & Nakayama, 1995) in a viewer-centered representation of the scene, after the operation of filling-in processes.     

Complete mergeability and amodal completion.

When image fragments are taken to correspond to the visible portions of a single occluded object, the object is said to 'amodally complete' behind the occluder. Kellman and Shipley (Kellman, P. J., & Shipley, T. F. (1991). A theory of visual interpolation in objective perception. Cognitive Psychology, 23, 144-221) argued that when the virtual contour extensions of such image fragments subtend an obtuse or right angle, the contours are 'relatable' and therefore complete. However, edge and surface relatability are neither necessary nor sufficient for completion to be perceived (Tse, P. U. (1999) Volume completion. Cognitive Psychology). Evidence is offered that completion is not driven directly by image cues such as contour relatability, but is driven, rather, by intermediate representations, such as volumes that are inferred from global image cue relationships. Evidence suggests that several factors, none of which is necessary for amodal completion to occur, contribute to the perceived strength of amodal completion, including similarity of pattern or substance, proximity, and good volume continuation or complete mergeability. Two partially occluded volumes are completely mergeable when they can be extended into occluded space along the trajectory defined by their visible surfaces such that they merge entirely with each other. Mergeability is not measurable in the image because it describes an inferred relationship among volumes that must themselves be inferred from the image.     

Amodal completion is modulated by lightness similarity

The strength of amodal completion is known to be modulated by contour relationships and global shape. Some researchers have shown that amodal completion also depends on surface similarity, but they have not distinguished the relative importance of similarity in surface representations either pre or post lightness constancy. In the experiments reported here, we aimed to determine whether amodal completion depends on processes that occur either before or after the establishment of lightness constancy. We used computer rendering techniques to vary the consistency of a cast shadow with a decrement in luminance on one side of a partially occluded surface. We found that perceived completion depended on the consistency of the decrement in surface luminance with the orientation of a cast shadow (Exp.?1). In Experiment ?2, we generated occluded surface fragments that could be either luminance-matched or lightness-matched to surface fragments on the opposite side of the occluder, and we found that the strength of amodal completion depended primarily on similarity in the perceived surface reflectance. In Experiments 3 and 4, we performed apparent-motion tasks to obtain converging evidence for grouping on the basis of perceived similarity in lightness. We found that matching surfaces in lightness significantly improved the apparent motion of surfaces behind an occluder, as compared with surfaces that were matched in contrast alone. These results suggest that amodal completion depends on the perceived similarity in lightness of partially occluded surface fragments.     

Visual cognition influences early vision: the role of visual short-term memory in amodal completion

A partly occluded visual object is perceptually filled in behind the occluding surface, a process known as amodal completion or visual interpolation. Previous research focused on the image-based properties that lead to amodal completion. In the present experiments, we examined the role of a higher-level visual process-visual short-term memory (VSTM)-in amodal completion. We measured the degree of amodal completion by asking participants to perform an object-based attention task on occluded objects while maintaining either zero or four items in visual working memory. When no items were stored in VSTM, participants completed the occluded objects; when four items were stored in VSTM, amodal completion was halted (Experiment 1). These results were not caused by the influence of VSTM on object-based attention per se (Experiment 2) or by the specific location of to-be-remembered items (Experiment 3). Items held in VSTM interfere with amodal completion, which suggests that amodal completion may not be an informationally encapsulated process, but rather can be affected by high-level visual processes.     

Conjunctive representations,the hippocampus,and contextual fear conditioning

The context in which events occur can be represented as both (1) a set of independent features, the feature representation view, and (2) a set of features bound into a unitary representation, the conjunction representation view. It is assumed that extrahippocampal (e.g., neocortical) areas provide a basis for feature representations, but the hippocampal formation makes an essential contribution to the automatic storage of conjunctive representations. We develop this dual-representation view and explore its implications for hippocampal contributions to contextual fear conditioning processes. To this end, we discuss how our framework can resolve some of the conflicts in the recent literature relating the hippocampus to contextual fear conditioning. We also present new data supporting the role of a key mechanism afforded by conjunctive representations—pattern completion (the ability of a subset of a memory pattern to activate the complete memory)—in contextual fear conditioning. As is implied by this mechanism, we report that fear can be conditioned to the memory representation of a context that is not actually present at the time of shock. Moreover, this result is predicted by our computational model of cortical and hippocampal function. We suggest that pattern completion demonstrated in animals and by our model provides a mechanistic bridge to human declarative memory.     

The effects of kinetic occlusion and categorization on amodal completion. A comment on Gerbino and Salmaso (1987)

The experimental study of Gerbino and Salmaso (1987) focused on the process of amodal completion, i.e. the process that leads to the impression of a total pattern in the absence of information about all parts of the pattern. One of the questions that they have attempted to answer with respect to this process is whether or not it depends on the categorization of the modal part (i.e. the part about which visual information is present) as a modification of a complete prototype. The major conclusion of their experiments is that amodal completion does not depend upon categorization. Furthermore, they interpret their results as indicating that neither the truncated form of the occluded part nor the unification/segregation of contour segments are phenomenally real. We designed some variants of the tunnel effect to demonstrate (1) that the explicit categorization of some parts of the input can have an influence on amodal completion and (2) that the truncated form of a pattern can be phenomenally real. Our results corroborate these hypotheses and can, therefore, interpreted as refuting two rather central claims of Gerbino and Salmaso (1987). The conclusion of our experiment must be that amodal completion is not as primary as they have supposed but can be influenced by kinetic occlusion and categorization stored in visual memory.     

Temporally unfolding neural representation of pictorial occlusion

The human visual system possesses a remarkable ability to reconstruct the shape of an object that is partly occluded by an interposed surface. Behavioral results suggest that, under some circumstances, this perceptual process (termed amodal completion) progresses from an initial representation of local image features to a completed representation of a shape that may include features that are not explicitly present in the retinal image. Recent functional magnetic resonance imaging (fMRI) studies have shown that the completed surface is represented in early visual cortical areas. We used fMRI adaptation, combined with brief, masked exposures, to track the amodal completion process as it unfolds in early visual cortical regions. We report evidence for an evolution of the neural representation from the image-based feature representation to the completed representation. Our method offers the possibility of measuring changes in cortical activity using fMRI over a time scale of a few hundred milliseconds.     

Amodal completion and relationalism

Amodal completion is usually characterized as the representation of those parts of the perceived object that we get no sensory stimulation from. In the case of the visual sense modality, for example, amodal completion is the representation of occluded parts of objects we see. I argue that relationalism about perception, the view that perceptual experience is constituted by the relation to the perceived object, cannot give a coherent account of amodal completion. The relationalist has two options: construe the perceptual relation as the relation to the entire perceived object or as the relation to the unoccluded parts of the perceived object. I argue that neither of these options are viable.

     

3-D vision and figure-ground separation by visual cortex

A neural network theory of three-dimensional (3-D) vision, called FACADE theory, is described. The theory proposes a solution of the classical figure-ground problem for biological vision. It does so by suggesting how boundary representations and surface representations are formed within a boundary contour system (BCS) and a feature contour system (FCS). The BCS and FCS interact reciprocally to form 3-D boundary and surface representations that are mutually consistent. Their interactions generate 3-D percepts wherein occluding and occluded object parts are separated, completed, and grouped. The theory clarifies how preattentive processes of 3-D perception and figure-ground separation interact reciprocally with attentive processes of spatial localization, object recognition, and visual search. A new theory of stereopsis is proposed that predicts how cells sensitive to multiple spatial frequencies, disparities, and orientations are combined by context-sensitive filtering, competition, and cooperation to form coherent BCS boundary segmentations. Several factors contribute to figure-ground pop-out, including: boundary contrast between spatially contiguous boundaries, whether due to scenic differences in luminance, color, spatial frequency, or disparity-partially ordered interactions from larger spatial scales and disparities to smaller scales and disparities; and surface filling-in restricted to regions surrounded by a connected boundary. Phenomena such as 3-D pop-out from a 2-D picture, Da Vinci stereopsis, 3-D neon color spreading, completion of partially occluded objects, and figure-ground reversals are analyzed. The BCS and FCS subsystems model aspects of how the two parvocellular cortical processing streams that join the lateral geniculate nucleus to prestriate cortical area V4 interact to generate a multiplexed representation of Form-And-Color-And-DEpth, orfacade, within area V4. Area V4 is suggested to support figure-ground separation and to interact with cortical mechanisms of spatial attention, attentive object learning, and visual search. Adaptive resonance theory (ART) mechanisms model aspects of how prestriate visual cortex interacts reciprocally with a visual object recognition system in inferotemporal (IT) cortex for purposes of attentive object learning and categorization. Object attention mechanisms of the What cortical processing stream through IT cortex are distinguished from spatial attention mechanisms of the Where cortical processing stream through parietal cortex. Parvocellular BCS and FCS signals interact with the model What stream. Parvocellular FCS and magnocellular motion BCS signals interact with the model Where stream. Reciprocal interactions between these visual, What, and Where mechanisms are used to discuss data about visual search and saccadic eye movements, including fast search of conjunctive targets, search of 3-D surfaces, selective search of like-colored targets, attentive tracking of multielement groupings, and recursive search of simultaneously presented targets.     

Amodal completion, depth stratification, and illusory figures: a test of Kanizsa's explanation.

Subjective contours have been explained by Kanizsa as being a consequence of amodal completion of incomplete figures. According to the theory of amodal completion, figural incompleteness triggers the emergence of an illusory object superimposed on the gaps in the inducers, which in turn hide parts of the pattern, thus suggesting that the plane of the illusory object must always be seen to be above the plane of the inducers. A figure was created in which subjective contours are seen despite the fact that the perceived depth relationships run counter to that required by the theory of amodal completion. In four experiments, this depth relationship is confirmed by using direct and indirect measures which assess both registered and apprehended depth. By emphasizing a logical inconsistency in the explanation based on amodal completion, the results show that amodal completion, at least in Kanizsa-like patterns, cannot be considered as a causal factor for subjective contour figures.     

Investigating global effects in visual occlusion: from a partly occluded square to the back of a tree-trunk.

'Classic' occlusion examples, such as a square partly occluded by a rectangle, have given rise to so-called local and global accounts of amodal completion. Without denying the influence of local configurations, I take the position that, in the long run, any theory of amodal completion should account for global properties. After a brief review of local and global accounts, two extensions of the stimulus domain are proposed to further illustrate the necessity of global accounts. The first is the domain of so-called fuzzy regularities, i.e., regularities which are not based on metrical identities. It is argued and demonstrated that observers are even susceptible to these fuzzy regularities and that they complete partly occluded shapes accordingly. The second extension is towards 3D object completion. Theories of object representation that describe intrinsic regularities of objects appear to be most suitable to predict relative preferences of alternative object completions. Consequently, fuzzy object completions, such as the completion of the back of a tree-trunk, can be explained better by global constraints.     

Perceptual causality in the amodal completion of kinetic structures

Perceptual causality as an amodal completion of kinetic structures was demonstrated. The experimental situation was as follows: a square appeared on the left, moved from left to right till it touched one side of a stationary rectangle (a perceptual screen), and disappeared; after a while, another square appeared at the other side of the rectangle and moved right. The first square moved faster than the second one. Sometimes, two moving objects were perceived, with an amodal launching effect: the first moving object seemed to collide with the second one behind the screen. When both the screen and the occluding interval were small, despite the discrepancy of velocities, a single moving object was perceived, but the object seemed to abruptly change its speed behind the screen. The perception of an amodal completion of movement raises some theoretical issues: the discrepancy between amodal completion of kinetic structures and that of static ones, the problem of perceptual identity, and figure-ground segregation in dynamic scene. The dilemma of the naive subject will also be discussed. Received: 11 June 1997 / Accepted: 14 October 1998     

Does the use of natural stimuli facilitate amodal completion in pigeons?

Three experiments were carried out to investigate whether amodal completion in pigeons can be facilitated by the use of colour photographs instead of highly artificial stimuli such as geometrical shapes. Ten pigeons were trained in a go/no-go procedure to discriminate between photographs of complete and of incomplete pigeon figures. In the subsequent test, the birds classified pictures of partly occluded pigeons as though they were complete (experiment 1). However, we found evidence that classification was based on spurious stimulus features that paralleled the intended class rule of figural completeness versus incompleteness. In particular, classification was shown to be guided by white background gaps that separated the parts of the fragmented pigeon figures (experiment 2), as well as by cues related to overall Gestalt (experiment 3). In summary, the present results indicate that the use of more natural stimuli such as photographs instead of geometrical shapes is insufficient for providing amodal completion in pigeons. It is suggested that a combination of various cues, including, eg, 3-D information and common motion in addition to surface and contour properties, may be required to induce a perceptual bias favouring visual completion of occluded portions.     

Object interpolation in three dimensions

Perception of objects in ordinary scenes requires interpolation processes connecting visible areas across spatial gaps. Most research has focused on 2-D displays, and models have been based on 2-D, orientation-sensitive units. The authors present a view of interpolation processes as intrinsically 3-D and producing representations of contours and surfaces spanning all 3 spatial dimensions. The authors propose a theory of 3-D relatability that indicates for a given edge which orientations and positions of other edges in 3 dimensions may be connected to it, and they summarize the empirical evidence for 3-D relatability. The theory unifies and illuminates a number of fundamental issues in object formation, including the identity hypothesis in visual completion, the relations of contour and surface processes, and the separation of local and global processing. The authors suggest that 3-D interpolation and 3-D relatability have major implications for computational and neural models of object perception.     

Tits use amodal completion in predator recognition: a field experiment

Amodal completion enables an animal to perceive partly concealed objects as an entirety, and to interact with them appropriately. Several studies, based upon either operant conditioning or filial imprinting techniques, have shown that various animals (both mammals and birds) can perform amodal completion. Before this study, the use of amodal completion by untrained animals in the recognition of objects had not been considered. Using two feeders, we observed in a field experiment the reaction of tits to the torso of a sparrowhawk (partly occluded or an ‘amputated’ dummy) in two different treatments (sparrowhawk torso vs. complete dummy pigeon; and torso vs. complete dummy sparrowhawk). It is clear that the birds considered the two torso variants as predators and kept away from both of them when the second feeder offered a ‘pigeon’ instead. On the other hand, when a ‘complete sparrowhawk’ was present on the second feeder, the number of visits to the occluded torso remained low; while the number of visits to the amputated one increased threefold. Birds risked perching near what was clearly an amputated torso; while the fear of a “hiding” (occluded) torso remained unchanged, when the second feeder did not provide a safe alternative. Such discrimination between torsos requires the ability for amodal completion. Our results demonstrate that in their recognition process, the birds not only use simple sign stimuli, but also complex cognitive functions.     

Using neural response properties to draw the distinction between modal and amodal representations

ABSTRACT

Barsalou has recently argued against the strategy of identifying amodal neural representations by using their cross-modal responses (i.e., their responses to stimuli from different modalities). I agree that there are indeed modal structures that satisfy this “cross-modal response” criterion (CM), such as distributed and conjunctive modal representations. However, I argue that we can distinguish between modal and amodal structures by looking into differences in their cross-modal responses. A component of a distributed cell assembly can be considered unimodal because its responses to stimuli from a given modality are stable, whereas its responses to stimuli from any other modality are not (i.e., these are lost within a short time, plausibly as a result of cell assembly dynamics). In turn, conjunctive modal representations, such as superior colliculus cells in charge of sensory integration, are multimodal because they have a stable response to stimuli from different modalities. Finally, some prefrontal cells constitute amodal representations because they exhibit what has been called ‘adaptive coding’. This implies that their responses to stimuli from any given modality can be lost when the context and task conditions are modified. We cannot assign them a modality because they have no stable relation with any input type.

Abbreviatons: CM: cross-modal response criterion; CCR: conjuntive cross-modal representations; fMRI: functional magnetic resonance imaging; MVPA: multivariate pattern analysis; pre-SMA: pre-supplementary motor area; PFC: prefrontal cortex; SC: superior colliculus; GWS: global workspace     

Mental representation in visual/haptic crossmodal memory: evidence from interference effects

Two experiments used visual-, verbal-, and haptic-interference tasks during encoding (Experiment 1) and retrieval (Experiment 2) to examine mental representation of familiar and unfamiliar objects in visual/haptic crossmodal memory. Three competing theories are discussed, which variously suggest that these representations are: (a) visual; (b) dual-code—visual for unfamiliar objects but visual and verbal for familiar objects; or (c) amodal. The results suggest that representations of unfamiliar objects are primarily visual but that crossmodal memory for familiar objects may rely on a network of different representations. The pattern of verbal-interference effects suggests that verbal strategies facilitate encoding of unfamiliar objects regardless of modality, but only haptic recognition regardless of familiarity. The results raise further research questions about all three theoretical approaches.     

Time course of amodal completion revealed by a shape discrimination task

We measured the extent of amodal completion as a function of stimulus duration over the range of 15–210 msec, for both moving and stationary stimuli. Completion was assessed using a performancebased measure: a shape discrimination task that is easy if the stimulus is amodally completed and difficult if it is not. Specifically, participants judged whether an upright rectangle was longer horizontally or vertically, when the rectangle was unoccluded, occluded at its corners by four negative-contrast squares, or occluded at its corners by four zero-contrast squares. In the zero-contrast condition, amodal completion did not occur because there were no occlusion cues; in the unoccluded condition, the entire figure was present. Thus, comparing performance in the negative-contrast condition to these two extremes provided a quantitative measure of amodal completion. This measure revealed a rapid but measurable time course for amodal completion. Moving and stationary stimuli took the same amount of time to be completed (≈ 75 msec), but moving stimuli had slightly stronger completion at long durations.