Perception of three-dimensional form from patterns of optical texture

The research described in the present article was designed to investigate how patterns of optical texture provide information about the three-dimensional structure of objects in space. Four experiments were performed in which observers were asked to judge the perceived depth of simulated ellipsoid surfaces under a variety of experimental conditions. The results revealed that judged depth increases linearly with simulated depth although the slope of this relation varies significantly among different types of texture patterns. Random variations in the sizes and shapes of individual surface elements have no detectable effect on observers' judgments. The perception of three-dimensional form is quite strong for surfaces displayed under parallel projection, but the amount of apparent depth is slightly less than for identical surfaces displayed under polar projection. Finally, the perceived depth of a surface is eliminated if the optical elements in a display are not sufficiently elongated or if they are not approximately aligned with one another. A theoretical explanation of these findings is proposed based on the neural network analysis of Grossberg and Mingolla.     

The absence of depth constancy in contour stereograms

Stereoscopic surfaces constructed from Kanizsa-type illusory contours or explicit luminance contours were tested for three-dimensional (3-D) shape constancy. The curvature of the contours and the apparent viewing distance between the surface and the observer were manipulated. Observers judged which of two surfaces appeared more curved. Experiment 1 allowed eye movements and revealed a bias in 3-D shape judgment with changes in apparent viewing distance, such that surfaces presented far from the observer appeared less curved than surfaces presented close to the observer. The lack of depth constancy was approximately the same for illusory-contour surfaces and for explicit-contour surfaces. Experiment 2 showed that depth constancy for explicit-contour surfaces improved slightly when fixation was required and eye movements were restricted. These experiments suggest that curvature in depth is misperceived, and that illusory-contour surfaces are particularly sensitive to this distortion.     

Visual discrimination of local surface depth and orientation

Many theoretical analyses of 3-dimensional form perception assume that visible surfaces in the environment are perceptually represented in terms of local mappings of metric depth and/or orientation. Although this approach is often taken for granted in the study of human vision, there have been relatively few attempts to demonstrate its psychological validity empirically. In an effort to shed new light on this issue, our research has been designed to investigate the accuracy with which observers can discriminate metric depth and orientation intervals on smoothly curved surfaces. Observers were presented with visual images of surfaces defined by shading and/or texture, on which two pairs of points were designated with small dots. In Experiment 1, their task was to identify which pair of points had a greater difference in depth; in Experiment 2 they were required to judge which pair had a greater difference in orientation. The Weber fractions obtained for these tasks were 10 to 100 times greater than those that have been reported for other types of sensory discrimination, indicating that the perception of metric structure from these displays is surprisingly coarse grained.     

Conjoint measurement of gloss and surface texture

The image of a material's surface varies not only with viewing and illumination conditions, but also with the material's surface properties, including its 3-D texture and specularity. Previous studies on the visual perception of surface material have typically focused on single material properties, ignoring possible interactions. In this study, we used a conjoint-measurement design to determine how observers represent perceived 3-D texture ("bumpiness") and specularity ("glossiness") and modeled how each of these two surface-material properties affects perception of the other. Observers made judgments of bumpiness and glossiness of surfaces that varied in both surface texture and specularity. We quantified how changes in each surface-material property affected judgments of the other and found that a simple additive model captured visual perception of texture and specularity and their interaction. Conjoint measurement is potentially a powerful tool for analyzing perception of surface material in realistic environments.     

Stereoscopic surface interpolation supports lightness constancy

The human visual system has a remarkable ability to construct surface representations from sparse stereoscopic, as well as texture and motion, information. In impoverished displays where few points are used to define regions in depth, the brain often interpolates depth estimates across intervening blank regions to create a compelling sense of a solid surface. The set of experiments described here examined stereoscopic interpolation using a novel technique based on lightness constancy. The effectiveness of this method is notable because it stands as the only technique to date that unequivocally examines the perception of interpolated surfaces, and not surfaces inferred subjectively from depth information in the stimulus. Further, these data support the growing evidence that a primary function of the stereoscopic system is to define three-dimensional surface structure.     

Static depth cues do affect the perceived direction of motion

Models of motion perception usually assume that the visual system references spatial displacements to retinal coordinates, and not to three-dimensional coordinates recovered by a parallel process. The present studies investigated whether moving elements viewed in the context of a static random-dot stereogram could lead to the appearance of motion in depth. Observers judged the velocity of a monocular element translating horizontally in the stereo context as 'same as' or 'different to' that of a standard. Based on velocity constancy, if there was apparent motion in depth, the relative velocity judgments would yield a predictable pattern of errors. The first experiment compared two stereo contexts: a sloped surface versus a fronto-parallel plane at zero disparity. The results indicated an overall increase in the perceived velocity of the element moving in the sloped surface context. A similar pattern of results was found when surfaces differing in incline were compared. Experiment 2 explored the case of fronto-parallel planes at crossed and uncrossed disparities. Here depth differences did not systematically affect observers' judgments. It was concluded that in some cases motion analysis can be affected by three-dimensional disparity information and not by angular displacement alone.     

Vertical disparity affects shape and size judgments across surfaces separated in depth

Vertical binocular disparity provides a useful source of information allowing three-dimensional (3-D) shape to be recovered from horizontal binocular disparity. In order to influence metric shape judgments, a large field of view is required, suggesting that vertical disparity may play a limited role in the perception of objects projecting small retinal images. This limitation could be overcome if vertical disparity information could be pooled over wide areas of 3-D space. This was investigated by assessing the effect of vertical disparity scaling of a large surround surface on the perceived size and 3-D shape of a small, central object. Observers adjusted the size and shape of a virtual, binocularly defined ellipsoid to match those of a real, hand-held tennis ball. The virtual ball was presented at three distances (200, 325, and 450 mm). Vertical disparities in a large surround surface were manipulated to be consistent with a distance of 160 mm or infinity. Both shape and size settings were influenced by this manipulation. This effect did not depend on presenting the surround and target objects at the same distance. These results suggest that the influence of vertical disparity on the perceived distance to a surface also affects the estimated distance of other visible surfaces. Vertical disparities are therefore important in the perception of metric depth, even for objects that in themselves subtend only small retinal images.     

Is the anisotropy of perceived 3-D shape invariant across scale?

A number of studies have resulted in the finding of a 3-D perceptual anisotropy, whereby spatial intervals oriented in depth are perceived to be smaller than physically equal intervals in the frontoparallel plane. In this experiment, we examined whether this anisotropy is scale invariant. The stimuli were L shapes created by two rods placed flat on a level grassy field, with one rod defining a frontoparallel interval, and the other, a depth interval. Observers monocularly and binocularly viewed L shapes at two scales such that they were projectively equivalent under monocular viewing. Observers judged the aspect ratio (depth/width) of each shape. Judged aspect ratio indicated a perceptual anisotropy that was invariant with scale for monocular viewing, but not for binocular viewing. When perspective is kept constant, monocular viewing results in perceptual anisotropy that is invariant across these two scales and presumably across still larger scales. This scale invariance indicates that the perception of shape under these conditions is determined independently of the perception of size.     

The linear perspective information in ground surface representation and distance judgment

Most ground surfaces contain various types of texture gradient information that serve as depth cues for space perception. We investigated how linear perspective, a type of texture gradient information on the ground, affects judged absolute distance and eye level. Phosphorescent elements were used to display linear perspective information on the floor in an otherwise dark room. We found that observers were remarkably receptive to such information. Changing the configuration of the linear perspective information from parallel to converging resulted in relatively larger judged distances and lower judged eye levels. These findings support the proposals that (1) the visual system has a bias for representing an image of converging lines as one of parallel lines on a downward-slanting surface and (2) the convergence point of a converging-lines image represents the eye level. Finally, we found that the visual system may be less sensitive to the manipulation of compression gradient information than of linear perspective information.     

Kinetic occlusion: further studies of the boundary-flow cue

Previous work has demonstrated that human beings employ a processing assumption, the boundary-flow constraint, in perceiving the order of depth at an edge. Subjects perceive depth order of surfaces on the basis of the relative motions of an image boundary and a projected surface texture on either side of the boundary. In the present study, adult subjects viewed computer-generated kinematograms in which boundary-flow information provided the only cue for depth order. The results of Experiments 1 and 2 indicate that common motion between boundary and texture and differential motion between boundary and texture can independently generate the perception of ordered depths of surfaces. In Experiment 3, we examined the interaction of two processes involved in the extraction of depth order from boundary-flow displays: (1) the propagation of foreground and background surfaces from texture to boundary; and (2) the computation of depth order of surfaces on either side of the boundary. The results indicate that while the mechanism that computes depth from boundary-flow information functions reliably when the mean distance between texture and boundary is 8.1(0), surface propagation may be disrupted for distances of this magnitude.     

An investigation of surface reconstruction from binocular disparity based on standard regularization theory: comparison between "membrane" and "thin-plate" potential energy models

Visible surfaces of three-dimensional objects are reconstructed from two-dimensional retinal images in the early stages of human visual processing. In the computational model of surface reconstruction based on the standard regularization theory, an energy function is minimized. Two types of model have been proposed, called "membrane" and "thin-plate" after their function formulas, in which the first or the second derivative of depth information is used. In this study, the threshold of surface reconstruction from binocular disparity was investigated using a sparse random dot stereogram, and the predictive accuracy of these models was evaluated. It was found that the thin-plate model reconstructed surfaces more accurately than the membrane model and showed good agreement with experimental results. The likelihood that these models imitate human processing of visual information is discussed in terms of the size of receptive fields in the visual pathways of the human cortex.     

The perception of surface orientation from multiple sources of optical information

An orientation matching task was used to evaluate observers’ sensitivity to local surface orientation at designated probe points on randomly shaped 3-D objects that were optically defined by texture, lambertian shading, or specular highlights. These surfaces could be stationary or in motion, and they could be viewed either monocularly or stereoscopically, in all possible combinations. It was found that the deformations of shading and/or highlights (either over time or between the two eyes’ views) produced levels of performance similar to those obtained for the optical deformations of textured surfaces. These findings suggest that the human visual system utilizes a much richer array of optical information to support its perception of shape than is typically appreciated.     

Perception of three-dimensional shape from specular highlights, deformations of shading, and other types of visual information

There have been numerous computational models developed in an effort to explain how the human visual system analyzes three-dimensional (3D) surface shape from patterns of image shading, but they all share some important limitations. Models that are applicable to individual static images cannot correctly interpret regions that contain specular highlights, and those that are applicable to moving images have difficulties when a surface moves relative to its sources of illumination. Here we describe a psychophysical experiment that measured the sensitivity of human observers to small differences of 3D shape over a wide variety of conditions. The results provide clear evidence that the presence of specular highlights or the motions of a surface relative to its light source do not pose an impediment to perception, but rather, provide powerful sources of information for the perceptual analysis of 3D shape.     

Effects of surface texture and grip force on the discrimination of hand-held loads

In this paper, we report the results from two experiments in which subjects were required to discriminate horizontal load forces applied to a manipulandum held with a precision grip. The roughness (and hence friction) of the grip surfaces and required grip force were manipulated. In the first experiment, subjects were instructed to judge the load while maintaining hand position and not letting the manipulandum slip. It was found that performance was influenced by surface texture; a given load was judged to be greater when the surface texture was smooth than when it was rough. This result is consistent with a previous study based on lifting objects and indicates that the effect of surface texture applies to loads in general and not just to gravitational loads (i.e., weight). To test whether the load acting on a smooth object is judged to be greater because the grip force required to prevent it from slipping is larger, a second experiment was carried out. Subjects used a visual feedback display to maintain the same grip force for smooth and rough manipulandum surfaces. In this case, there was no effect of surface texture on load perception. These results provide evidence that perceived load depends on the grip force used to resist the load. The implications of these results in terms of central and peripheral factors underlying load discrimination are considered.     

Shape from shading from images rendered with various surface types and light fields

Shape constancy is referred to as the tendency for the perceived shape of an object to remain unchanged even under changed viewing and illumination conditions. We investigated, in two experiments, whether shape constancy would hold for images of 3-D solid objects defined by shading only, whose renderings differed in terms of surface material type (bi-directional reflectance distribution functions), light field, light direction, shape, and specularity. Observers were presented with the image of a sphere or an ellipsoid and required to set perceived orientation and cross-section profile on designated points of the image. Results showed that shape judgments varied with all the aforementioned variables except specularity. Shape estimates were more precise with specular than asperity scattering surfaces, collimated than hemispherical diffuse lighting conditions, lower than higher elevations, spherical than ellipsoidal shapes, but not different between surfaces having differing specularity. These results suggest that shape judgments are made largely on the basis of the overall intensity distribution of shading, and that the portions of intensity distribution that are due to nonstructural variables such as surface material type or light field are not excluded in the process of shape estimation, as if being due to structural components. It is concluded that little constancy is expected in the perception of shape from shading.     

Projective invariance and picture perception

Four experiments test the assumption that, in the visual perception of pictures, observers have reliable and direct access to the equivalence of shapes in projective geometry. The assumption is that perception of projective equivalence is the basis of shape constancy ('the projective thesis'). Observers matched or reproduced abstract planar shapes under conditions of rotation in the picture plane, and pictured rotation in depth. Departure from projective equivalence was assessed in each study by measuring the planar analogue of cross ratio. Projective equivalence was not found to be perceived uniformly where Euclidean equivalence was not judged uniformly, either in recognition tasks or in production tasks. When the projective thesis is put to a suitably general test, confidence in the thesis is undermined.     

Integration of stereo and texture cues in the formation of discontinuities during three-dimensional surface interpolation

A series of stereograms are presented which demonstrate that texture boundaries can strongly influence the perception of discontinuities between neighbouring three-dimensional (3-D) surfaces portrayed by means of stereo cues. In these demonstration figures, no stereo information is available in the immediate vicinity of the boundary between the two 3-D stereo surfaces because all texture in that region is removed in one eye's view. On the other hand, various forms of texture boundary information are provided in the resulting monocular region. This stimulus paradigm is used to explore the question: what influence does texture boundary information have on the nature of the perceived 3-D surface that is interpolated between two stimulus regions which carry stereo cues? It is shown that if a clear-cut texture boundary is present in the monocular region then this is used by the human visual system to fix the perceived location of 3-D crease and step surface discontinuities between the stereo regions. Collett (1985) explored this issue with a similar methodology and reported weak and unreliable assistance from monocular texture boundaries in helping shape 3-D stereo surface discontinuities. The strong and robust phenomena demonstrated here seem to rely on two main differences between the present stimuli and those of Collett. In the present stimuli, figurally continuous textures containing strong texture boundaries are used, together with a technique for minimising the complications, including binocular rivalry, that arise from the borders of the stimulus regions present in only one half of each stereogram.     

The perception of depth and slant from texture in three-dimensional scenes.

The perception of depth and slant in three-dimensional scenes specified by texture was investigated in five experiments. Subjects were presented with computer-generated scenes of a ground and ceiling plane receding in depth. Compression, convergence, and grid textures were examined. The effect of the presence or absence of a gap in the center of the display was also assessed. Under some conditions perceived slant and depth from compression were greater than those found with convergence. The relative effectiveness of compression in specifying surface slant was greater for surfaces closer to ground planes (80 degrees slant) than for surfaces closer to frontal parallel planes (40 degrees slant). The usefulness of compression was also observed with single-plane displays and with displays with surfaces oriented to reduce information regarding the horizon.     

Projective invariance and the kinetic depth effect.

Seven experiments test the assumption that, in the kinetic depth effect, observers have reliable and direct access to the equivalence of shapes in projective geometry. The assumption is implicit in 'inverse optics' approaches to visual form perception. Observers adjusted a comparison shape to match a standard shape; both standard and comparison were portrayed as in continuous rotation in space, using a graphics computer. The shapes were either plane quadrilaterals or solid prisms. The angular difference of the planes of the shapes was varied, as was the dot density of a texture in those planes. Departure from projective equivalence was measured in six studies by measuring the planar analogue of cross ratio, and in a seventh by measuring the cross ratio for points in space. Projective equivalence was not found to be perceived uniformly, except in one experiment that did not involve rotation in depth. Otherwise changes in orientation of up to 180 degrees about a single coordinate axis had no significant effect on matches in shape, while changes in orientation about more than one coordinate axis produced significant effects. The addition of texture and a change in rotation speed did not correct departures from projective equivalence.