A stereo advantage in generalizing over changes in viewpoint on object recognition tasks

In four experiments, we examined whether generalization to unfamiliar views was better under stereo viewing or under nonstereo viewing across different tasks and stimuli. In the first three experiments, we used a sequential matching task in which observers matched the identities of shaded tube-like objects. Across Experiments 1-3, we manipulated the presentation method of the nonstereo stimuli (having observers wear an eye patch vs. showing observers the same screen image) and the magnitude of the viewpoint change (30 degrees vs. 38 degrees). In Experiment 4, observers identified "easy" and "hard" rotating wire-frame objects at the individual level under stereo and nonstereo viewing conditions. We found a stereo advantage for generalizing to unfamiliar views in all the experiments. However, in these experiments, performance remained view dependent even under stereo viewing. These results strongly argue against strictly 2-D image-based models of object recognition, at least for the stimuli and recognition tasks used, and suggest that observers used representations that contained view-specific local depth information.     

Recognizing rotated views of objects: Interpolation versus generalization by humans and pigeons

Pigeons and humans were trained to discriminate between pictures of three-dimensional objects that differed in global shape. Each pair of objects was shown at two orientations that differed by a depth rotation of 90° during training. Pictures of the objects at novel depth rotations were then tested for recognition. The novel test rotations were 30°, 45°, and 90° from the nearest trained orientation and were either interpolated between the trained orientations or extrapolated outside of the training range. For both pigeons and humans, recognition accuracy and/or speed decreased as a function of distance from the nearest trained orientation. However, humans, but not pigeons, were more accurate in recognizing novel interpolated views than novel extrapolated views. The results suggest that pigeons’ recognition was based on independent generalization from each training view, whereas humans showed view-combination processes that resulted in a benefit for novel views interpolated between the training views.     

A stereo disadvantage for recognizing rotated familiar objects

We tested recognition of familiar objects in two different conditions: mono, where stimuli were displayed as flat, 2-D images, and stereo, where objects were displayed with stereoscopic depth information. In three experiments, participants performed a sequential matching task, where an object was rotated by up to 180° between presentations. When the 180° rotation resulted in large changes in depth for object components, recognition performance in the mono condition showed better performance at 180° rotations than at smaller rotations, but stereo presentations showed a monotonic increase in response time with rotation. However, 180° rotations that did not result in much depth variation showed similar patterns of results for mono and stereo conditions. These results suggest that in some circumstances, the lack of explicit 3-D information in 2-D images may influence the recognition of familiar objects when they are depicted on flat computer monitors.     

Dissociating the effects of angular disparity and image similarity in mental rotation and object recognition

Performance is often impaired linearly with increasing angular disparity between two objects in tasks that measure mental rotation or object recognition. But increased angular disparity is often accompanied by changes in the similarity between views of an object, confounding the impact of the two factors in these tasks. We examined separately the effects of angular disparity and image similarity on handedness (to test mental rotation) and identity (to test object recognition) judgments with 3-D novel objects. When similarity was approximately equated, an effect of angular disparity was only found for handedness but not identity judgments. With a fixed angular disparity, performance was better for similar than dissimilar image pairs in both tasks, with a larger effect for identity than handedness judgments. Our results suggest that mental rotation involves mental transformation procedures that depend on angular disparity, but that object recognition is predominately dependent on the similarity of image features.     

Recognizing familiar objects by hand and foot: Haptic shape perception generalizes to inputs from unusual locations and untrained body parts

The limits of generalization of our 3-D shape recognition system to identifying objects by touch was investigated by testing exploration at unusual locations and using untrained effectors. In Experiments 1 and 2, people found identification by hand of real objects, plastic 3-D models of objects, and raised line drawings placed in front of themselves no easier than when exploration was behind their back. Experiment 3 compared one-handed, two-handed, one-footed, and two-footed haptic object recognition of familiar objects. Recognition by foot was slower (7 vs. 13 s) and much less accurate (9 % vs. 47 % errors) than recognition by either one or both hands. Nevertheless, item difficulty was similar across hand and foot exploration, and there was a strong correlation between an individual’s hand and foot performance. Furthermore, foot recognition was better with the largest 20 of the 80 items (32 % errors), suggesting that physical limitations hampered exploration by foot. Thus, object recognition by hand generalized efficiently across the spatial location of stimuli, while object recognition by foot seemed surprisingly good given that no prior training was provided. Active touch (haptics) thus efficiently extracts 3-D shape information and accesses stored representations of familiar objects from novel modes of input.     

VIEWPOINT DEPENDENCE IN SCENE RECOGNITION

Abstract— Two experiments investigated the viewpoint dependence of spatial memories. In Experiment 1, participants learned the locations of objects on a desktop from a single perspective and then took part in a recognition test, test scenes included familiar and novel views of the layout. Recognition latency was a linear function of the angular distance between a test view and the study view. In Experiment 2, participants studied a layout from a single view and then learned to recognize the layout from three additional training views. A final recognition test showed that the study view and the training views were represented in memory, and that latency was a linear function of the angular distance to the nearest study or training view. These results indicate that interobject spatial relations are encoded in a viewpoint-dependent manner, and that recognition of novel views requires normalization to the most similar representation in memory. These findings parallel recent results in visual object recognition     

Generalization to novel views from view combination

Four experiments examined whether or not exposure to two views (A and B) of a novel object improves generalization to a third view (C) through view combination on tasks that required symmetry or recognition memory decisions. The results of Experiment 1 indicated that exposure to either View A or View B alone produced little or no generalization to View C on either task. The results of Experiment 2 indicated that exposure to both View A and View B did improve generalization to View C, but only for symmetrical objects. Experiment 3 replicated this generalization advantage for symmetrical but not asymmetrical objects, when objects were well learned at study. The results of Experiment 4 showed that Views A and B did not have to be presented consecutively to facilitate responses to View C. Together, the pattern of results suggests that generalization to novel views does occur through view combination of temporally separated views, but it is more likely to be observed with symmetrical objects.     

When are viewpoint costs greater for silhouettes than for shaded images?

Previous studies of object recognition have shown efficient recognition of silhouettes, suggesting that much of the information used to recognize objects resides in the outline. These studies, however, have used objects that contain many components, which provide redundant information. In this study, we examined recognition of silhouettes of less-complex objects, so that redundant information was reduced. We found that viewpoint generalization costs (the decrement of performance when recognizing nonstudied views) were greater for silhouettes than for shaded images, even when the same qualitative components were visible in the outline of both studied and nonstudied views. Thus, silhouettes do not always allow for view generalization as efficiently as do shaded images.     

The visual representation of three-dimensional, rotating objects.

Depth rotations can reveal new object parts and result in poor recognition of "static" objects (Biederman & Gerhardstein, 1993). Recent studies have suggested that multiple object views can be associated through temporal contiguity and similarity (Edelman & Weinshall, 1991; Lawson, Humphreys & Watson, 1994; Wallis, 1996). Motion may also play an important role in object recognition since observers recognize novel views of objects rotating in the picture plane more readily than novel views of statically re-oriented objects (Kourtzi & Shiffrar, 1997). The series of experiments presented here investigated how different views of a depth-rotated object might be linked together even when these views do not share the same parts. The results suggest that depth rotated object views can be linked more readily with motion than with temporal sequence alone to yield priming of novel views of 3D objects that fall in between "known" views. Motion can also enhance path specific view linkage when visible object parts differ across views. Such results suggest that object representations depend on motion processes.     

The effect of distinctive parts on recognition of depth-rotated objects by pigeons (Columba livia) and humans.

To explore whether effects observed in human object recognition represent fundamental properties of visual perception that are general across species, the authors trained pigeons (Columba livia) and humans to discriminate between pictures of 3-dimensional objects that differed in shape. Novel pictures of the depth-rotated objects were then tested for recognition. Across conditions, the object pairs contained either 0, 1, 3, or 5 distinctive parts. Pigeons showed viewpoint dependence in all object-part conditions, and their performance declined systematically with degree of rotation from the nearest training view. Humans showed viewpoint invariance for novel rotations between the training views but viewpoint dependence for novel rotations outside the training views. For humans, but not pigeons, viewpoint dependence was weakest in the 1-part condition. The authors discuss the results in terms of structural and multiple-view models of object recognition.     

The importance of being upright: Use of environmental and viewer-centered reference frames in shape discriminations of novel three-dimensional objects

We investigated the frame of reference that people use to make shape discriminations when their heads are either upright or tilted. Observers madesame-different judgments of pairs of novel threedimensional objects that were aligned along their length within the frontal-parallel plane and rotated in depth around an axis parallel to their own axes of elongation. The aligned objects were displayed vertically, tilted 45°, or horizontally with respect to the environmental upright, but the distance of each pair from the upright was irrelevant to resolving the angular disparity between the stimuli for thesame-different judgment. Nevertheless, when observers’ heads were upright, the time to encode the stimuli was a linear function of the distance of the stimuli from the environmental upright, whereas when observers’ heads were tilted 45°, encoding times for tilted and vertical stimuli did not differ and were faster than the times to encode horizontal stimuli. We interpreted these data to mean that observers either rotate or reference the top of an object to the environmental upright, and they can use either a gravitational or retinal reference frame to do so when either they or the objects are not upright.     

Spatial updating during locomotion does not eliminate viewpoint-dependent visual object processing

Two experiments were conducted to investigate whether locomotion to a novel test view would eliminate viewpoint costs in visual object processing. Participants performed a sequential matching task for object identity or object handedness, using novel 3-D objects displayed in a head-mounted display. To change the test view of the object, the orientation of the object in 3-D space and the test position of the observer were manipulated independently. Participants were more accurate when the test view was the same as the learned view than when the views were different no matter whether the view change of the object was 50° or 90°. With 50° rotations, participants were more accurate at novel test views caused by participants’ locomotion (object stationary) than caused by object rotation (observer stationary) but this difference disappeared when the view change was 90°. These results indicate that facilitation of spatial updating during locomotion occurs within a limited range of viewpoints, but that such facilitation does not eliminate viewpoint costs in visual object processing.     

The role of edges in object recognition by pigeons

In three experiments, we explored how pigeons use edges, corresponding to orientation and depth discontinuities, in visual recognition tasks. In experiment 1, we compared the pigeon's ability to recognize line drawings of four different geons when trained with shaded images. The birds were trained with either a single view or five different views of each object. Because the five training views had markedly different appearances and locations of shaded surfaces, reflectance edges, etc, the pigeons might have been expected to rely more on the orientation and depth discontinuities that were preserved over rotation and in the line drawings. In neither condition, however, was there any transfer from the rendered images to the outline drawings. In experiment 2, some pigeons were trained with line drawings and shaded images of the same objects associated with the same response (consistent condition), whereas other pigeons were trained with a line drawing and a shaded image of two different objects associated with the same response (inconsistent condition). If the pigeons perceived any correspondence between the stimulus types, then birds in the consistent condition should have learned the discrimination more quickly than birds in the inconsistent condition. But, there was no difference in performance between birds in the consistent and inconsistent conditions. In experiment 3, we explored pigeons' processing of edges by comparing their discrimination of shaded images or line drawings of four objects. Once trained, the pigeons were tested with planar rotations of those objects. The pigeons exhibited different patterns of generalization depending on whether they were trained with line drawings or shaded images. The results of these three experiments suggest that pigeons may place greater importance on surface features indicating materials, such as food or water. Such substances do not have definite boundaries cued by edges which are thought to be central to human recognition.     

Orientation-specific possibility priming for novel three-dimensional objects

Priming effects on the object possibility task, in which participants decide whether line drawings could or could not be possible three-dimensional objects, may be supported by the same processes and representations used in recognizing and identifying objects. Three experiments manipulating objects’ picture-plane orientation provided limited support for this hypothesis. Like old/new recognition performance, possibility priming declined as study-test orientation differences increased from 0° to 60°. However, while significant possibility priming was not observed for larger orientation differences, recognition performance continued to decline following 60°–180° orientation shifts. These results suggest that possibility priming and old/new recognition may rely on common viewpoint-specific representations but that access to these representations in the possibility test occurs only when study and test views are sufficiently similar (i.e., rotated less than 60°).     

Constraints on view combination: effects of self-occlusion and differences among familiar and novel views

The use of multiple familiar views of objects to facilitate recognition of novel views has been addressed in a number of behavioral studies, but the results have not been conclusive. The present study was a comprehensive examination of view combination for different types of novel views (internal or external to the studied views) and different objects (amoeboid objects and objects composed of geons; objects with and without self-occlusion across rotation). The authors found that the advantage gained from the study of 2 views was more than the generalization from each of the studied views presented alone. This facilitation occurred only for internal views but not external views. In addition, the benefits from the study of 2 views diminished when (a) the studied views did not share the same visible features and when (b) the studied views were separated by a small angular difference.     

The ability to recognize objects from bottlenose dolphin (Tursiops truncatus) echoes generalizes across multiple orientations in humans and neural networks

Object constancy, the ability to recognize objects despite changes in orientation, has not been well studied in the auditory modality. Dolphins use echolocation for object recognition, and objects ensonified by dolphins produce echoes that can vary significantly as a function of orientation. In this experiment, human listeners had to classify echoes from objects varying in material, shape, and size that were ensonified with dolphin signals. Participants were trained to discriminate among the objects using an 18-echo stimulus from a 10° range of aspect angles, then tested with novel aspect angles across a 60° range. Participants were typically successful recognizing the objects at all angles (M = 78 %). Artificial neural networks were trained and tested with the same stimuli with the purpose of identifying acoustic cues that enable object recognition. A multilayer perceptron performed similarly to the humans and revealed that recognition was enabled by both the amplitude and frequency of echoes, as well as the temporal dynamics of these features over the course of echo trains. These results provide insight into representational processes underlying echoic recognition in dolphins and suggest that object constancy perceived through the auditory modality is likely to parallel what has been found in the visual domain in studies with both humans and animals.     

Simplicity and generalization: Short-cutting abstraction in children's object categorizations

Development in any domain is often characterized by increasingly abstract representations. Recent evidence in the domain of shape recognition provides one example; between 18 and 24 months children appear to build increasingly abstract representations of object shape [Smith, L. B. (2003). Learning to recognize objects. Psychological Science, 14, 244-250]. Abstraction is in part simplification because it requires the removal of irrelevant information. At the same time, part of generalization is ignoring irrelevant differences. The resulting prediction is this: simplification may enable generalization. Four experiments asked whether simple training instances could shortcut the process of abstraction and directly promote appropriate generalization. Toddlers were taught novel object categories with either simple or complex training exemplars. We found that children who learned with simple objects were able to generalize according to shape similarity, typically relevant for early object categories, better than those who learned with complex objects. Abstraction is the product of learning; using simplified - already abstracted instances - can short-cut that learning, leading to robust generalization.     

Flipping and spinning: Spatial transformation procedures in the identification of rotated natural objects

The proposal that identification of inverted objects is accomplished by either a relatively slow rotation in the picture plane or a faster rotation in the depth plane about the horizontal axis was tested. In Experiment 1, subjects decided whether objects at 0° or 180° corresponded to previously learned normal views of the upright objects, or were mirror images. Instructions to mentally flip an inverted object in the depth plane to the upright produced faster decision times than did instructions to mentally spin the object in the picture plane. In Experiment 2, the effects of orientation were compared across an object-naming task and a normal-mirror task for six orientations from 0° to 300°. In the normal-mirror task, objects at 180° were cued for rotation in the picture plane or in the depth plane in equal numbers. The naming function for one group of subjects did not differ from the normalmirror function where inverted objects had been mentally rotated to the upright. For both functions, response time (RT) increased linearly from 0° to 180° and the slopes did not differ. The naming function for a second group of subjects did not differ from the normal-mirror function where inverted objects had been mentally flipped to the upright. For both functions, RT increased linearly at a similar rate from 0° to 120°, but decreased from 120° to 180°. The results are discussed in terms of theories of orientation-specific identification.     

The role of sequence order in determining view canonicality for novel wire-frame objects

Objects are best recognized from so-called “canonical” views. The characteristics of canonical views of arbitrary objects have been qualitatively described using a variety of different criteria, but little is known regarding how these views might be acquired during object learning. We address this issue, in part, by examining the role of object motion in the selection of preferred views of novel objects. Specifically, we adopt a modeling approach to investigate whether or not the sequence of views seen during initial exposure to an object contributes to observers’ preferences for particular images in the sequence. In two experiments, we exposed observers to short sequences depicting rigidly rotating novel objects and subsequently collected subjective ratings of view canonicality (Experiment 1) and recall rates for individual views (Experiment 2). Given these two operational definitions of view canonicality, we attempted to fit both sets of behavioral data with a computational model incorporating 3-D shape information (object foreshortening), as well as information relevant to the temporal order of views presented during training (the rate of change for object foreshortening). Both sets of ratings were reasonably well predicted using only 3-D shape; the inclusion of terms that capture sequence order improved model performance significantly.     

Effects of distance between objects and distance from the vertical axis on shape identity judgments

In three experiments, we independently manipulated the angular disparity between objects to be compared and the angular distance between the central axis of the objects and the vertical axis in a mental rotation paradigm. There was a linear increase in reaction times that was attributable to both factors. This result held whether the objects were rotated (with respect to each other and to the upright) within the frontal-parallel plane (Experiment 1) or in depth (Experiment 2), although the effects of both factors were greater for objects rotated in depth than for objects rotated within the frontal-parallel plane (Experiment 3). In addition, the factors interacted when the subjects had to search for matching ends of the figures (Experiments 1 and 2), but they were additive when the ends that matched were evident (Experiment 3). These data may be interpreted to mean that subjects normalize or reference an object with respect to the vertical upright as well as compute the rotational transformations used to determine shape identity.