Recognizing rotated views of objects: Interpolation versus generalization by humans and pigeons

Pigeons and humans were trained to discriminate between pictures of three-dimensional objects that differed in global shape. Each pair of objects was shown at two orientations that differed by a depth rotation of 90° during training. Pictures of the objects at novel depth rotations were then tested for recognition. The novel test rotations were 30°, 45°, and 90° from the nearest trained orientation and were either interpolated between the trained orientations or extrapolated outside of the training range. For both pigeons and humans, recognition accuracy and/or speed decreased as a function of distance from the nearest trained orientation. However, humans, but not pigeons, were more accurate in recognizing novel interpolated views than novel extrapolated views. The results suggest that pigeons’ recognition was based on independent generalization from each training view, whereas humans showed view-combination processes that resulted in a benefit for novel views interpolated between the training views.     

Recognition by humans and pigeons of novel views of 3-D objects and their photographs

Humans and pigeons were trained to discriminate between 2 views of actual 3-D objects or their photographs. They were tested on novel views that were either within the closest rotational distance between the training views (interpolated) or outside of that range (extrapolated). When training views were 60 degrees apart, pigeons, but not humans, recognized novel views of actual objects better than their pictures. Further, both species recognized interpolated views of both stimulus types better than extrapolated views, but a single distinctive geon enhanced recognition of novel views only for humans. When training views were 90 degrees apart, pigeons recognized interpolated views better than extrapolated views with actual objects but not with photographs. Thus, pigeons may represent actual objects differently than their pictures.     

Pigeons see correspondence between objects and their pictures

The extent to which nonhumans recognize the correspondence between static pictures and the objects they represent remains an interesting and controversial issue. Pictures displayed on computers are used extensively for research on behavioral and neural mechanisms of cognition in birds, yet attempts to show that birds recognize the objects seen in pictures have produced mixed and inconclusive results. We trained pigeons to discriminate between two identically colored but differently shaped three-dimensional objects seen directly or as pictures, and we found clear bidirectional transfer of the learned object discrimination. Transfer from objects to pictures occurred even when pigeons were trained with 12 views and only novel views of the objects were presented in transfer. This study provides the strongest evidence yet that pigeons can recognize the correspondence between objects and pictures.     

VIEWPOINT DEPENDENCE IN SCENE RECOGNITION

Abstract— Two experiments investigated the viewpoint dependence of spatial memories. In Experiment 1, participants learned the locations of objects on a desktop from a single perspective and then took part in a recognition test, test scenes included familiar and novel views of the layout. Recognition latency was a linear function of the angular distance between a test view and the study view. In Experiment 2, participants studied a layout from a single view and then learned to recognize the layout from three additional training views. A final recognition test showed that the study view and the training views were represented in memory, and that latency was a linear function of the angular distance to the nearest study or training view. These results indicate that interobject spatial relations are encoded in a viewpoint-dependent manner, and that recognition of novel views requires normalization to the most similar representation in memory. These findings parallel recent results in visual object recognition     

The role of edges in object recognition by pigeons

In three experiments, we explored how pigeons use edges, corresponding to orientation and depth discontinuities, in visual recognition tasks. In experiment 1, we compared the pigeon's ability to recognize line drawings of four different geons when trained with shaded images. The birds were trained with either a single view or five different views of each object. Because the five training views had markedly different appearances and locations of shaded surfaces, reflectance edges, etc, the pigeons might have been expected to rely more on the orientation and depth discontinuities that were preserved over rotation and in the line drawings. In neither condition, however, was there any transfer from the rendered images to the outline drawings. In experiment 2, some pigeons were trained with line drawings and shaded images of the same objects associated with the same response (consistent condition), whereas other pigeons were trained with a line drawing and a shaded image of two different objects associated with the same response (inconsistent condition). If the pigeons perceived any correspondence between the stimulus types, then birds in the consistent condition should have learned the discrimination more quickly than birds in the inconsistent condition. But, there was no difference in performance between birds in the consistent and inconsistent conditions. In experiment 3, we explored pigeons' processing of edges by comparing their discrimination of shaded images or line drawings of four objects. Once trained, the pigeons were tested with planar rotations of those objects. The pigeons exhibited different patterns of generalization depending on whether they were trained with line drawings or shaded images. The results of these three experiments suggest that pigeons may place greater importance on surface features indicating materials, such as food or water. Such substances do not have definite boundaries cued by edges which are thought to be central to human recognition.     

Perceiving Real-World Viewpoint Changes

Retinal images vary as observers move through the environment, but observers seem to have little difficulty recognizing objects and scenes across changes in view. Although real-world view changes can be produced both by object rotations (orientation changes) and by observer movements (viewpoint changes), research on recognition across views has relied exclusively on display rotations. However, research on spatial reasoning suggests a possible dissociation between orientation and viewpoint. Here we demonstrate that scene recognition in the real world depends on more than the retinal projection of the visible array; viewpoint changes have little effect on detection of layout changes, but equivalent orientation changes disrupt performance significantly. Findings from our three experiments suggest that scene recognition across view changes relies on a mechanism that updates a viewer-centered representation during observer movements, a mechanism not available for orientation changes. These results link findings from spatial tasks to work on object and scene recognition and highlight the importance of considering the mechanisms underlying recognition in real environments.     

The visual representation of three-dimensional, rotating objects.

Depth rotations can reveal new object parts and result in poor recognition of "static" objects (Biederman & Gerhardstein, 1993). Recent studies have suggested that multiple object views can be associated through temporal contiguity and similarity (Edelman & Weinshall, 1991; Lawson, Humphreys & Watson, 1994; Wallis, 1996). Motion may also play an important role in object recognition since observers recognize novel views of objects rotating in the picture plane more readily than novel views of statically re-oriented objects (Kourtzi & Shiffrar, 1997). The series of experiments presented here investigated how different views of a depth-rotated object might be linked together even when these views do not share the same parts. The results suggest that depth rotated object views can be linked more readily with motion than with temporal sequence alone to yield priming of novel views of 3D objects that fall in between "known" views. Motion can also enhance path specific view linkage when visible object parts differ across views. Such results suggest that object representations depend on motion processes.     

Viewpoint-dependent recognition of familiar faces

The question whether object representations in the human brain are object-centered or viewer-centered has motivated a variety of experiments with divergent results. A key issue concerns the visual recognition of objects seen from novel views. If recognition performance depends on whether a particular view has been seen before, it can be interpreted as evidence for a viewer-centered representation. Earlier experiments used unfamiliar objects to provide the experimenter with complete control over the observer's previous experience with the object. In this study, we tested whether human recognition shows viewpoint dependence for the highly familiar faces of well-known colleagues and for the observer's own face. We found that observers are poorer at recognizing their own profile, whereas there is no difference in response time between frontal and profile views of other faces. This result shows that extensive experience and familiarity with one's own face is not sufficient to produce viewpoint invariance. Our result provides strong evidence for viewer-centered representations in human visual recognition even for highly familiar objects.     

Three-month-old infants' object recognition across changes in viewpoint using an operant learning procedure

Object knowledge refers to the understanding that all objects share certain properties. Various components of object knowledge (e.g., object occlusion, object causality) have been examined in human infants to determine its developmental origins. Viewpoint invariance--the understanding that an object viewed from different viewpoints is still the same object--is one area of object knowledge, however, that has received less attention. To this end, infants' capacity for viewpoint-invariant perception of multi-part objects was investigated. Three-month-old infants were tested for generalization to an object displayed on a mobile that differed only in orientation (i.e., viewpoint) from a training object. Infants were given experience with a wide range of object views (Experiment 1) or a more restricted range during training (Experiment 2). The results showed that infants generalized between a horizontal and vertical viewpoint (Experiment 1) that they could clearly discriminate between in other contexts (i.e., with restricted view experience, Experiment 2). Overall, the outcome shows that training experience with multiple viewpoints plays an important role in infants' ability to develop a general percept of an object's 3D structure and promotes viewpoint-invariant perception of multi-part objects; in contrast, restricting training experience impedes viewpoint-invariant recognition of multi-part objects.     

Seeing things from a different angle: the pigeon's recognition of single geons rotated in depth

In 2 experiments involving computer-rendered versions of single shapes or "geons," the extent to which depth rotation affects the visual discrimination performance of pigeons in both go/no-go and forced-choice tasks was documented. The pigeons were able to recognize geons at most rotations in depth; however, the pigeons' recognition performance was better at the training view than at most other views. Aspects of these results are both consistent with and problematic for object-centered and viewer-centered theories of object recognition.     

Stereo disparity facilitates view generalization during shape recognition for solid multipart objects

Current theories of object recognition in human vision make different predictions about whether the recognition of complex, multipart objects should be influenced by shape information about surface depth orientation and curvature derived from stereo disparity. We examined this issue in five experiments using a recognition memory paradigm in which observers (N = 134) memorized and then discriminated sets of 3D novel objects at trained and untrained viewpoints under either mono or stereo viewing conditions. In order to explore the conditions under which stereo-defined shape information contributes to object recognition we systematically varied the difficulty of view generalization by increasing the angular disparity between trained and untrained views. In one series of experiments, objects were presented from either previously trained views or untrained views rotated (15°, 30°, or 60°) along the same plane. In separate experiments we examined whether view generalization effects interacted with the vertical or horizontal plane of object rotation across 40° viewpoint changes. The results showed robust viewpoint-dependent performance costs: Observers were more efficient in recognizing learned objects from trained than from untrained views, and recognition was worse for extrapolated than for interpolated untrained views. We also found that performance was enhanced by stereo viewing but only at larger angular disparities between trained and untrained views. These findings show that object recognition is not based solely on 2D image information but that it can be facilitated by shape information derived from stereo disparity.     

Varieties of object constancy

Three experiments are described in which two pictures of isolated man-made objects were presented in succession. The subjects' task was to decide, as rapidly as possible, whether the two pictured objects had the same name. With a stimulus-onset asynchrony (SOA) of above 200 msec two types of facilitation were observed: (1) the response latency was reduced if the pictures showed the same object, even though seen from different viewpoints (object benefit); (2) decision time was reduced further if the pictures showed the same object from the same angle of view (viewpoint benefit). These facilitation effects were not affected by projecting the pictures to different retinal locations. Significant benefits of both types were also obtained when the projected images differed in size. However, in these circumstances there was a small but significant performance decrement in matching two similar views of a single object, but not if the views were different. Conversely, the object benefit, but not the viewpoint benefit, was reduced when the SOA was only 100 msec. The data suggest the existence of (at least) two different visual codes, one non-retinotopic but viewer-centred, the other object-centred.     

PIGEONS ARE SENSITIVE TO THE SPATIAL ORGANIZATION OF COMPLEX VISUAL STIMULI

Abstract— Two experiments investigated the role of spatial organisation in the discrimination and generalization of complex visual stimuli by pigeons. In Experiment 1, after pigeons had been trained to discriminate line drawings of four objects, they were tested with novel pictures in which the same component parts of the objects were spatially rearranged. The spatially scrambled pictures led to a dramatic drop in recognition accuracy, hut responding remained above chance. In Experiment 2, pigeons reached a high level of discriminative performance when required to choose among four different spatial arrangements of the same object parts. These results confirm Cerella's (1980) conclusion that pigeons discriminate the component parts of complex visual stimuli, but. unless it is assumed that the scrambling deleted or created emergent features, the results disconfirm his conclusion that spatial organization plays no role in pigeons' picture perception.     

Spatial updating during locomotion does not eliminate viewpoint-dependent visual object processing

Two experiments were conducted to investigate whether locomotion to a novel test view would eliminate viewpoint costs in visual object processing. Participants performed a sequential matching task for object identity or object handedness, using novel 3-D objects displayed in a head-mounted display. To change the test view of the object, the orientation of the object in 3-D space and the test position of the observer were manipulated independently. Participants were more accurate when the test view was the same as the learned view than when the views were different no matter whether the view change of the object was 50° or 90°. With 50° rotations, participants were more accurate at novel test views caused by participants’ locomotion (object stationary) than caused by object rotation (observer stationary) but this difference disappeared when the view change was 90°. These results indicate that facilitation of spatial updating during locomotion occurs within a limited range of viewpoints, but that such facilitation does not eliminate viewpoint costs in visual object processing.     

Viewpoint dependence and object discriminability

In an attempt to reconcile results of previous studies, several theorists have suggested that object recognition performance should range from viewpoint invariant to highly viewpoint dependent depending on how easy it is to differentiate the objects in a given recognition situation. The present study assessed recognition across depth rotations of a single general class of novel objects in three contexts that varied in difficulty. In an initial experiment, recognition in the context involving the most discriminable object differences was viewpoint invariant, but recognition in the least discriminable context and recognition in the intermediate context were equally viewpoint dependent. In a second experiment, utilizing gray-scale versions of the same stimuli, almost identical viewpoint-cost functions were obtained in all three contexts. These results suggest that differences in the geometry of stimulus objects, rather than task difficulty, lie at the heart of previously discrepant findings.     

Pigeons' recognition of partially occluded objects depends on specific training experience

DiPietro et al (2002 Perception 31 1299-1312) reported a dramatic improvement in pigeons' recognition of partially occluded objects after the birds had been trained to recognize objects that were placed on top of another surface. Here, we investigated whether training with partially erased stimuli or with notched stimuli that had a thin gap between the object and another surface would similarly enhance pigeons' recognition of partially occluded objects. We found that erased training had no effect on the birds' recognition of partially occluded objects. Training pigeons to recognize notched objects improved their performance with the same objects when they were partially occluded; but this improvement did not transfer to novel objects, a result that DiPietro et al reported after on-top training. Together, the present results and those of DiPietro et al implicate prior experience as a key factor in pigeons' recognition of partially occluded objects. Training experiences which improve recognition of partially occluded objects may do so because they improve decomposition of complex two-dimensional scenes by pigeons into separate entities.     

Effects of occlusion on pigeons' visual object recognition

Casual observation suggests that pigeons and other animals can recognize occluded objects; yet laboratory research has thus far failed to show that pigeons can do so. In a series of experiments, we investigated pigeons' ability to 'name' shaded, textured stimuli by associating each with a different response. After first learning to recognize four unoccluded objects, pigeons had to recognize the objects when they were partially occluded by another surface or when they were placed on top of another surface; in each case, recognition was weak. Following training with the unoccluded stimuli and with the stimuli placed on top of the occluder, pigeons' recognition of occluded objects dramatically improved. Pigeons' improved recognition of occluded objects was not limited to the trained objects but transferred to novel objects as well. Evidently, the recognition of occluded objects requires pigeons to learn to discriminate the object from the occluder; once this discrimination is mastered, occluded objects can be better recognized.     

Viewpoint and orientation influence picture recognition in the blind

In the first three experiments, subjects felt solid geometrical forms and matched raised-line pictures to the objects. Performance was best in experiment 1 for top views, with shorter response latencies than for side views, front views, or 3-D views with foreshortening. In a second experiment with blind participants, matching accuracy was not significantly affected by prior visual experience, but speed advantages were found for top views, with 3-D views also yielding better matching accuracy than side views. There were no performance advantages for pictures of objects with a constant cross section in the vertical axis. The early-blind participants had lower performance for side and frontal views. The objects were rotated to oblique orientations in experiment 3. Early-blind subjects performed worse than the other subjects given object rotation. Visual experience with pictures of objects at many angles could facilitate identification at oblique orientations. In experiment 5 with blindfolded sighted subjects, tangible pictures were used as targets and as choices. The results yielded superior overall performance for 3-D views (mean, M = 74% correct) and much lower matching accuracy for top views as targets (M = 58% correct). Performance was highest when the target and matching viewpoint were identical, but 3-D views (M = 96% correct) were still far better than top views. The accuracy advantage of the top views also disappeared when more complex objects were tested in experiment 6. Alternative theoretical implications of the results are discussed.     

Viewpoint dependence in visual and haptic object recognition

On the whole, people recognize objects best when they see the objects from a familiar view and worse when they see the objects from views that were previously occluded from sight. Unexpectedly, we found haptic object recognition to be viewpoint-specific as well, even though hand movements were unrestricted. This viewpoint dependence was due to the hands preferring the back "view" of the objects. Furthermore, when the sensory modalities (visual vs. haptic) differed between learning an object and recognizing it, recognition performance was best when the objects were rotated back-to-front between learning and recognition. Our data indicate that the visual system recognizes the front view of objects best, whereas the hand recognizes objects best from the back.