3-D shape perception

In this paper, we analyze and test three theories of 3-D shape perception: (1) Helmholtzian theory, which assumes that perception of the shape of an object involves reconstructing Euclidean structure of the object (up to size scaling) from the object’s retinal image after taking into account the object’s orientation relative to the observer, (2) Gibsonian theory, which assumes that shape perception involves invariants (projective or affine) computed directly from the object’s retinal image, and (3) perspective invariants theory, which assumes that shape perception involves a new kind of invariants of perspective transformation. Predictions of these three theories were tested in four experiments. In the first experiment, we showed that reliable discrimination between a perspective and nonperspective image of a random polygon is possible even when information only about the contour of the image is present. In the second experiment, we showed that discrimination performance did not benefit from the presence of a textured surface, providing information about the 3-D orientation of the polygon, and that the subjects could not reliably discriminate between the 3-D orientation of the textured surface and that of a shape. In the third experiment, we compared discrimination for solid shapes that either had flat contours (cuboids) or did not have visible flat contours (cylinders). The discrimination was very reliable in the case of cuboids but not in the case of cylinders. In the fourth experiment, we tested the effectiveness of planar motion in perception of distances and showed that the discrimination threshold was large and similar to thresholds when other cues to 3-D orientation were used. All these results support perspective invariants as a model of 3-D shape perception.     

Fechner, information, and shape perception

How do retinal images lead to perceived environmental objects? Vision involves a series of spatial and material transformations—from environmental objects to retinal images, to neurophysiological patterns, and finally to perceptual experience and action. A rationale for understanding functional relations among these physically different systems occurred to Gustav Fechner: Differences in sensation correspond to differences in physical stimulation. The concept of information is similar: Relationships in one system may correspond to, and thus represent, those in another. Criteria for identifying and evaluating information include (a)?resolution, or the precision of correspondence; (b)?uncertainty about which input (output) produced a given output (input); and (c)?invariance, or the preservation of correspondence under transformations of input and output. We apply this framework to psychophysical evidence to identify visual information for perceiving surfaces. The elementary spatial structure shared by objects and images is the second-order differential structure of local surface shape. Experiments have shown that human vision is directly sensitive to this higher-order spatial information from interimage disparities (stereopsis and motion parallax), boundary contours, texture, shading, and combined variables. Psychophysical evidence contradicts other common ideas about retinal information for spatial vision and object perception.     

The perception of surface curvature from optical motion

Observers viewed the optical flow field of a rotating quadric surface patch and were required to match its perceived structure by adjusting the shape of a stereoscopically presented surface. In Experiment 1, the flow fields included rigid object rotations and constant flow fields with patterns of image acceleration that had no possible rigid interpretation. In performing their matches, observers had independent control of two parameters that determined the surface shape. One of these, called the shape characteristic, is defined as the ratio of the two principle curvatures and is independent of object size. The other, called curvedness, is defined as the sum of the squared principle curvatures and depends on the size of the object. Adjustments of shape characteristic were almost perfectly accurate for both motion conditions. Adjustments of curvedness, on the other hand, were systematically overestimated and were not highly correlated with the simulated curvedness of the depicted surface patch. In Experiment 2, the same flow fields were masked with a global pattern of curl, divergence, or shear, which disrupted the first-order spatial derivatives of the image velocity field, while leaving the second-order spatial derivatives invariant. The addition of these masks had only negligible effects on observers’ performance. These findings suggest that observers’ judgments of three-dimensional surface shape from motion are primarily determined by the second-order spatial derivatives of the instantaneous field of image displacements.     

The influence of illumination direction on the pictorial reliefs of Lambertian surfaces

In order to assess the influence of illumination direction on shape constancy, we studied the pictorial relief of computer images of globular 3-D objects. We used two globally convex objects, one with a furrow and one with a dimple. Observers adjusted local surface attitude probes at 200-250 different locations in the image such that they seemed to lie on the pictorial surface. We manipulated the viewing direction and the illumination direction in a 2 x 2 orthogonal design. Viewing directions were chosen such that the image contained only a few, or no, contour singularities. Changes in the illumination direction were found to induce systematic changes in the settings for both viewing directions. Effects were especially pronounced for images that had no contour singularities. The results showed that a change in the illumination direction can change the local shape of the pictorial relief in addition to the bas-relief ambiguities of scaling and shearing in depth. We found that concavities in the pictorial relief are associated with the darker areas in the image. The deviation from shape constancy cannot be explained by bas-relief ambiguity since the required transformation between the shapes is nonlinear.     

Temporally unfolding neural representation of pictorial occlusion

The human visual system possesses a remarkable ability to reconstruct the shape of an object that is partly occluded by an interposed surface. Behavioral results suggest that, under some circumstances, this perceptual process (termed amodal completion) progresses from an initial representation of local image features to a completed representation of a shape that may include features that are not explicitly present in the retinal image. Recent functional magnetic resonance imaging (fMRI) studies have shown that the completed surface is represented in early visual cortical areas. We used fMRI adaptation, combined with brief, masked exposures, to track the amodal completion process as it unfolds in early visual cortical regions. We report evidence for an evolution of the neural representation from the image-based feature representation to the completed representation. Our method offers the possibility of measuring changes in cortical activity using fMRI over a time scale of a few hundred milliseconds.     

记忆过程中海马CA1区神经元的集群放电特征

观察空间工作记忆过程中海马CA1区神经元群的放电特征。应用多通道神经元集群放电记录技术, 同步观察和记录清醒大鼠在执行延迟选择任务时的行为轨迹以及海马CA1区神经元的放电活动。发现：海马CA1区位置细胞的位置野是在学习过程中逐渐形成并可消退; 部分位置细胞的放电对未来目标定向性行为具有预测作用; 在空间工作记忆过程中, 神经元放电之间的相关性加强, 神经元之间以及神经元与局部场电位之间存在相位编码方式。结果提示海马CA1区神经元参与对空间信息的初级编码和加工, 并为未来行为决策提供有效信息, 而且海马对信息的加工是通过局部神经网络进行, 时间编码可能是海马信息加工的重要方式之一。     

Does the brain's ventral visual pathway compute object shape?

A rich behavioral literature has shown that human object recognition is supported by a representation of shape that is tolerant to variations in an object's appearance. Such 'global' shape representations are achieved by describing objects via the spatial arrangement of their local features, or structure, rather than by the appearance of the features themselves. However, accumulating evidence suggests that the ventral visual pathway – the primary substrate underlying object recognition – may not represent global shape. Instead, ventral representations may be better described as a basis set of local image features. We suggest that this evidence forces a reevaluation of the role of the ventral pathway in object perception and posits a broader network for shape perception that encompasses contributions from the dorsal pathway.     

The role of spatial frequency in the processing of hierarchically organized stimuli

Can spatial frequency differences between local and global forms account for differences in the way different levels of structure are analyzed? We examined this question by having subjects identify local or global forms of hierarchical stimuli that had beencontrast balanced. Contrast balancing eliminates low spatial frequencies, so that both local and global forms must be identified on the basis of high spatial frequency information. Response times (RTs) to global (but not local) forms were slowed for contrast-balanced stimuli, suggesting that low spatial frequencies mediate the global RT advantage typically found. In contrast, interference between local and global forms was little affected by contrast balancing or by shifts of attention between local and global forms, suggesting that it does not result from inhibitory interactions between spatial frequency channels or from temporal precedence of low versus high spatial frequency information. Finally, shifts of attention between local and global forms were also little affected by contrast balancing, suggesting that they were not based on spatial frequency.     

The perception of lightness in 3-D curved objects

Lightness constancy in complex scenes requires that the visual system take account of information concerning variations of illumination falling on visible surfaces. Three experiments on the perception of lightness for three-dimensional (3-D) curved objects show that human observers are better able to perform this accounting for certain scenes than for others. The experiments investigate the effect of object curvature, illumination direction, and object shape on lightness perception. Lightness constancy was quite good when a rich local gray-level context was provided. Deviations occurred when both illumination and reflectance changed along the surface of the objects. Does the perception of a 3-D surface and illuminant layout help calibrate lightness judgments? Our results showed a small but consistent improvement between lightness matches on ellipsoid shapes, relative to flat rectangle shapes, under illumination conditions that produce similar image gradients. Illumination change over 3-D forms is therefore taken into account in lightness perception.     

Getting the big picture: Development of spatial scaling abilities

Spatial scaling is an integral aspect of many spatial tasks that involve symbol-to-referent correspondences (e.g., map reading, drawing). In this study, we asked 3–6-year-olds and adults to locate objects in a two-dimensional spatial layout using information from a second spatial representation (map). We examined how scaling factor and reference features, such as the shape of the layout or the presence of landmarks, affect performance. Results showed that spatial scaling on this simple task undergoes considerable development, especially between 3 and 5 years of age. Furthermore, the youngest children showed large individual variability and profited from landmark information. Accuracy differed between scaled and un-scaled items, but not between items using different scaling factors (1:2 vs. 1:4), suggesting that participants encoded relative rather than absolute distances.     

Three-dimensional Müller-Lyer illusion

Three-dimensional (3-D) variants of the Müller-Lyer pattern were created to address the question of where along the path of information flow in the visual system the illusion might occur. These variants, which yielded a robust illusion, included dihedral angles in place of the arrowheads of the classical pattern. The enormous difference in the shape of the resulting retinal image, compared with that of the classical pattern, makes it difficult to explain the present illusion by resorting to image-processing theories such as selective filtering (Ginsburg, 1984, 1986) or depth processing (Gregory, 1963, 1966, 1968). It was also shown that this 3-D illusion is homologous with the classical illusion, and that the two may thus share a common causal mechanism. A new type of 3-D figure, which yielded the same retinal image as did the classical pattern, was then employed. However, since the figure was 3-D, its shape in spatial coordinates was very different compared to that of the classical pattern. The magnitude of the illusion obtained with this figure was half that of the classical pattern. This finding suggests that the illusion might be caused by processes that occur after the computation of depth. All three experiments indicated that the illusion may be produced later in the processing stream than has previously been suggested.     

Perception of local orientation from shaded images

The perception of local orientation from shaded images was examined. In Experiment 1, subjects viewed a boundaryless Gaussian hill and judged local orientation using both a gauge figure and a pointing method. One subject reported an internally consistent surface which was incompatible with the judged light-source direction and model used to generate the image. The remaining subjects reported a surface similar to the generating one, and analysis of their results indicated a contour of zero difference between response and generating slants. This contour of zero slant difference was explored in three subsequent experiments using the pointing technique. These experiments investigated possible influences of luminance artifact (Experiment 2), perception of global orientation (Experiment 3), and self-occluding contours (Experiment 4). All three of these experiments yielded results similar to those of Experiment 1, with distinct contours of zero slant difference. This contour was explored for relationships with the simulated slant of the generating surface and the differential structure of image intensity. This analysis indicated that the contour of zero slant difference was approximately a line of constant slant which shared large regions of adjacency to the zero crossings of the second directional derivative of image intensity.     

Dominance of global visual properties at birth

Six experiments are reported that were aimed at demonstrating the presence in newborns of a perceptual dominance of global over local visual information in hierarchical patterns, similar to that observed in adults (D. Navon, 1977, 1981). The first four experiments showed that, even though both levels of visual information were detectable by the newborn (Experiments 1A and 1B), global cues enjoyed some advantage over local cues (Experiments 2 and 3). Experiments 4A and 4B demonstrated that the global bias was strictly dependent on the low spatial frequency content of the stimuli and vanished after selective removal of low spatial frequencies. The results are interpreted as suggesting parallels between newborns' visual processing and processing later in development.     

The importance of information localization in scene gist recognition

People can recognize the meaning or gist of a scene from a single glance, and a few recent studies have begun to examine the sorts of information that contribute to scene gist recognition. The authors of the present study used visual masking coupled with image manipulations (randomizing phase while maintaining the Fourier amplitude spectrum; random image structure evolution [RISE]; J. Sadr & P. Sinha, 2004) to explore whether and when unlocalized Fourier amplitude information contributes to gist perception. In 4 experiments, the authors found that differences between scene categories in the Fourier amplitude spectrum are insufficient for gist recognition or gist masking. Whereas the global 1/f spatial frequency amplitude spectra of scenes plays a role in gist masking, local phase information is necessary for gist recognition and for the strongest gist masking. Moreover, the ability to recognize the gist of a target image was influenced by mask recognizability, suggesting that conceptual masking occurs even at the earliest stages of scene processing.     

The role of spatial and surface cues in the age-processing of unfamiliar faces

Two experiments investigated the importance of spatial and surface cues in the age-processing of unfamiliar faces aged between one and 80 years. Three manipulations known to affect face recognition were used, individually and in various combinations: inversion, negation, and blurring. Faces were presented either in whole or in part. Age-estimation performance was largely unaffected by most of these manipulations; age-processing appears to be a highly robust process, due to the numerous cues available. Experiment 1 showed that, in contrast to face recognition, age-perception appears to be substantially unimpaired by inversion or negation. Experiment 2 suggests that age-estimates can be made on the basis of either surface information (the 2D disposition of the internal facial features, together with texture information) or shape information (head-shape plus feature configuration, as long as shape-from-shading information is present).     

Haptic form perception: relative salience of local and global features.

When we examine objects haptically, do we weight their local and global features as we do visually, or do we place relatively greater emphasis on local shape? In Experiment 1, subjects made either haptic or visual comparisons of pairs of geometric objects (from a set of 16) differing in local and global shape. Relative to other objects, those with comparable global shape but different local features were judged less similar by touch than by vision. Separate groups of subjects explored the same objects while wearing either thick gloves (to discourage contour-following) or splinted gloves (to prevent enclosure). Ratings of similarity were comparable in these two conditions, suggesting that neither exploratory procedure was necessary, by itself, for the extraction of either local or global shape. In Experiment 2, haptic exploration time was restricted to 1, 4, 8, or 16 sec. Limiting exploration time affected relative similarity in objects differing in their local but not their global shape. Together, the findings indicate that the hepatic system initially weights local features more heavily than global ones, that this differential weighting decreases over time, and that neither contour-following nor enclosure is exclusively associated with the differential emphasis on local versus global features.     

Visual search at different spatial scales

Treisman and Gelade's (1980) feature-integration model claims that the search for separate ("primitive") stimulus features is parallel, but that the conjunctions of those features require serial scan. Recently, evidence has accumulated that parallel processing is not limited to these "primitive" stimulus features, but that combinations of features can also produce parallel search. In the experiments reported here, the processing of feature conjunctions was studied when the stimulus features of a combination were at different spatial scales. The patterns in the search array were composed of three cross-shaped or T-shaped (local) elements, which formed an oblique bar (the global pattern) 45 deg or 135 deg in orientation. When the target and distractors differed from each other at one spatial scale only (either in the bar orientation or in the shape of the local elements), target detection was independent of the number of distractors, i.e., the search was parallel. In the conjunction task, in which the target and distractors were defined as the combinations of the bar orientation and the element shape, i.e., both spatial scales were relevant, the detection of the target required slow serial scrutiny of the search array. It is possible that the conjunction search could not be performed in parallel because switches between the two scales (or spatial frequency channels) are linked to attention and the task required the use of both scales in order to find the target.     

Vertical disparity affects shape and size judgments across surfaces separated in depth

Vertical binocular disparity provides a useful source of information allowing three-dimensional (3-D) shape to be recovered from horizontal binocular disparity. In order to influence metric shape judgments, a large field of view is required, suggesting that vertical disparity may play a limited role in the perception of objects projecting small retinal images. This limitation could be overcome if vertical disparity information could be pooled over wide areas of 3-D space. This was investigated by assessing the effect of vertical disparity scaling of a large surround surface on the perceived size and 3-D shape of a small, central object. Observers adjusted the size and shape of a virtual, binocularly defined ellipsoid to match those of a real, hand-held tennis ball. The virtual ball was presented at three distances (200, 325, and 450 mm). Vertical disparities in a large surround surface were manipulated to be consistent with a distance of 160 mm or infinity. Both shape and size settings were influenced by this manipulation. This effect did not depend on presenting the surround and target objects at the same distance. These results suggest that the influence of vertical disparity on the perceived distance to a surface also affects the estimated distance of other visible surfaces. Vertical disparities are therefore important in the perception of metric depth, even for objects that in themselves subtend only small retinal images.     

The spatial mismatch effect is based on global configuration and not on perceptual records within the visual cache

If configurations of objects are presented in a S1-S2 matching task for the identity of objects a spatial mismatch effect occurs. Changing the (irrelevant) spatial layout lengthens response times. We investigated what causes this effect. We observed a reliable mismatch effect that was not influenced by a secondary task during maintenance. Neither articulatory suppression (Experiment 1), nor unattended (Experiments 2 and 6) or attended visual material (Experiment 3) reduced the effect, and this was independent of the length of the retention interval (Experiment 6). The effect was also rather independent of the visual appearance of the local elements. It was of similar size with color patches (Experiment 4) and with completely different surface information when testing was cross modal (Experiment 5), and the name-ability of the global configuration was not relevant (Experiments 6 and 7). In contrast, the figurative similarity of the configurations of S1 and S2 systematically influenced the size of the spatial mismatch effect (Experiment 7). We conclude that the spatial mismatch effect is caused by a mismatch of the global shape of the configuration stored together with the objects of S1 and not by a mismatch of templates of perceptual records maintained in a visual cache.