An investigation of surface reconstruction from binocular disparity based on standard regularization theory: comparison between "membrane" and "thin-plate" potential energy models

Visible surfaces of three-dimensional objects are reconstructed from two-dimensional retinal images in the early stages of human visual processing. In the computational model of surface reconstruction based on the standard regularization theory, an energy function is minimized. Two types of model have been proposed, called "membrane" and "thin-plate" after their function formulas, in which the first or the second derivative of depth information is used. In this study, the threshold of surface reconstruction from binocular disparity was investigated using a sparse random dot stereogram, and the predictive accuracy of these models was evaluated. It was found that the thin-plate model reconstructed surfaces more accurately than the membrane model and showed good agreement with experimental results. The likelihood that these models imitate human processing of visual information is discussed in terms of the size of receptive fields in the visual pathways of the human cortex.     

Shape from shading in pigeons

Light is the origin of vision. The pattern of shading reflected from object surfaces is one of several optical features that provide fundamental information about shape and surface orientation. To understand how surface and object shading is processed by birds, six pigeons were tested with differentially illuminated convex and concave curved surfaces in five experiments using a go/no-go procedure. We found that pigeons rapidly learned this type of visual discrimination independent of lighting direction, surface coloration and camera perspective. Subsequent experiments varying the pattern of the lighting on these surfaces through changes in camera perspective, surface height, contrast, material specularity, surface shape, light motion, and perspective movement were consistent with the hypothesis that the pigeons were perceiving these illuminated surfaces as three-dimensional surfaces containing curved shapes. The results suggest that the use of relative shading for objects in a visual scene creates highly salient features for shape processing in birds.     

The visual perception of length along intrinsically curved surfaces

The ability of observers to perceive three-dimensional (3-D) distances or lengths along intrinsically curved surfaces was investigated in three experiments. Three physically curved surfaces were used: convex and/or concave hemispheres (Experiments 1 and 3) and a hyperbolic paraboloid (Experiment 2). The first two experiments employed a visual length-matching task, but in the final experiment the observers estimated the surface lengths motorically by varying the separation between their two index fingers. In general, the observers' judgments of surface length in both tasks (perceptual vs. motoric matching) were very precise but were not necessarily accurate. Large individual differences (overestimation, underestimation, etc.) in the perception of length occurred. There were also significant effects of viewing distance, type of surface, and orientation of the spatial intervals on the observers' judgments of surface length. The individual differences and failures of perceptual constancy that were obtained indicate that there is no single relationship between physical and perceived distances on 3-D surfaces that is consistent across observers.     

The visual perception of smoothly curved surfaces from minimal apparent motion sequences.

A series of four experiments was designed to investigate the minimal amounts of information required to perceive the structure of a smoothly curved surface from its pattern of projected motion. In Experiments 1 and 2, observers estimated the amplitudes of sinusoidally corrugated surfaces relative to their periods. Observers' judgments varied linearly with the depicted surface amplitudes, but the amount of perceived relative depth was systematically overestimated by approximately 30%. The observers' amplitude judgments were also influenced to a lesser extent by the amount of rotary displacement of a surface at each frame transition, and by increasing the length of the apparent motion sequences from two to eight frames. The latter effect of sequence length was quite small, however, accounting for less than 3% of the variance in the observers' judgments. Experiments 3 and 4 examined observers' discrimination thresholds for sinusoidally corrugated surfaces of variable amplitude and for ellipsoid surfaces of variable eccentricity. The results revealed that observers could reliably detect differences of surface structure as small as 5%. The length of the apparent motion sequences had no detectable effect on these tasks, although there were significant effects of angular displacement and surface orientation. These results are considered with respect to the analysis of affine structure from motion proposed by Todd and Bressan (1990).     

Disparity and shading cues cooperate for surface interpolation

In two experiments, we tested whether disparity and shading cues cooperated for surface interpolation. Observers adjusted a probe dot to/lie on a surface specified either by a sparse disparity field, a continuous stereo shading or monocular shading gradient, or both cues. Observers' adjustments were very consistent with disparity information but their adjustments were much more variable with shading information. However, observers significantly improved their precision when both cues were present, relative to when only disparity information was present. These results cannot be explained by assuming that separate modules analyze disparity and shading information, even if observers optimally combined these cues. Rather, we attribute this improvement to a process through which the shading gradient constrains the disparity field in regions where disparities cannot be directly measured. This cooperative process may be based on the natural covariation existing between these cues produced by the retinal projection of smooth surfaces.     

Integration of motion parallax with binocular disparity specifying different surface shapes

We investigated the interaction between motion parallax and binocular disparity cues in the perception of surface shape and depth magnitude by the use of the random dot stimuli in which these cues specified sinusoidal depth surfaces undulating with different spatial frequencies. When ambiguous motion parallax is inconsistent with unambiguous disparity cue, the reasonable solution for the visual system is to convert the motion signal to the flow on the surface specified by disparity. Two experiments, however, found that the visual system did not always use this reasonable solution; observers often perceived the surface specified by a composite of the two cues, or the surface specified by parallax alone. In the perception of this composite of the two cues, the apparent depth magnitude increased with the increase of the depth magnitude specified by both cues. This indicates that the visual system can combine the depth magnitude information from parallax and disparity in an additive fashion. The interference with parallax by disparity implies that the parallax processing is not independent of the disparity processing.     

Shape constancy from novel views

Prior experiments on shape constancy from novel views are inconclusive: Some show that shapes of objects can be recognized reliably from novel views, whereas others show just the opposite. Our analysis of prior results suggests that shape constancy from novel views is reliable when the object has properties that constrain its shape: The object has volumetric primitives, it has surfaces, it is symmetrical, it is composed of geons, its contours are planar, and its images provide useful topological information about its three-dimensional structure. To test the role of some of these constraints, we performed a set of experiments. Solid shapes (polyhedra) were shown on a computer monitor by means of kinetic depth effect. Experiment 1 showed that shape constancy can be reliably achieved when a polyhedron is represented by its contours (most of the constraints are present), but not when it is represented by vertices or by a polygonal line connecting the vertices in a random order (all the constraints are absent). Experiments 2 and 3 tested the role of individual constraints. Results of these experiments show that shape constancy from novel views is reliable when the object has planar contours and when the shapes of the contours together with topological information about the relations among the contours constrain the possible interpretations of the shape. Symmetry of the object and the topological stability of its image also contribute to shape constancy.     

Perceptual latencies to discriminate surface orientation in stereopsis

The difference in sensitivity to stereoscopic surfaces oriented horizontally or vertically (the stereoscopic orientation anisotropy) can be redescribed as a difference in sensitivity to shear or compression transformations that relate the binocular images. The present experiment was designed to test this by dissociating the image transformation from the orientation of the surface. Surfaces were presented in isolation or in the presence of a surrounding frame that formed step and gradient discontinuities in the disparity field. Without discontinuities, observers required considerably more time to discriminate between surfaces differing in compression than between those differing in shear, irrespective of surface orientation. Disparity discontinuities facilitated the perception of the disparity gradients; minimum stimulus durations were reduced by over an order of magnitude when the reference frame was present. These results support the hypothesis that the disparity field is decomposed into different primitives during the recovery of depth and surface structure.     

The structure of three-dimensional object representations in human vision: evidence from whole-part matching

This article examines how the human visual system represents the shapes of 3-dimensional (3D) objects. One long-standing hypothesis is that object shapes are represented in terms of volumetric component parts and their spatial configuration. This hypothesis is examined in 3 experiments using a whole-part matching paradigm in which participants match object parts to whole novel 3D object shapes. Experiments 1 and 2, consistent with volumetric image segmentation, show that whole-part matching is faster for volumetric component parts than for either open or closed nonvolumetric regions of edge contour. However, the results of Experiment 3 show that an equivalent advantage is found for bounded regions of edge contour that correspond to object surfaces. The results are interpreted in terms of a surface-based model of 3D shape representation, which proposes edge-bounded 2-dimensional polygons as basic primitives of surface shape.     

Static depth cues do affect the perceived direction of motion

Models of motion perception usually assume that the visual system references spatial displacements to retinal coordinates, and not to three-dimensional coordinates recovered by a parallel process. The present studies investigated whether moving elements viewed in the context of a static random-dot stereogram could lead to the appearance of motion in depth. Observers judged the velocity of a monocular element translating horizontally in the stereo context as 'same as' or 'different to' that of a standard. Based on velocity constancy, if there was apparent motion in depth, the relative velocity judgments would yield a predictable pattern of errors. The first experiment compared two stereo contexts: a sloped surface versus a fronto-parallel plane at zero disparity. The results indicated an overall increase in the perceived velocity of the element moving in the sloped surface context. A similar pattern of results was found when surfaces differing in incline were compared. Experiment 2 explored the case of fronto-parallel planes at crossed and uncrossed disparities. Here depth differences did not systematically affect observers' judgments. It was concluded that in some cases motion analysis can be affected by three-dimensional disparity information and not by angular displacement alone.     

How is motion disparity integrated with binocular disparity in depth perception?

Two experiments presented motion disparity conflicting with binocular disparity to examine how these cues determined apparent depth order (convex, concave) and depth magnitude. In each experiment, 8 subjects estimated the depth order and depth magnitude. The first experiment showed the following. (1) The visual system used one of these cues exclusively in selecting a depth order for each display. (2) The visual system integrated the depth magnitude information from these cues by a weighted additive fashion if it selected the binocular disparity in depth order perception and if the depth magnitude specified by motion disparity was small relative to that specified by binocular disparity. (3) The visual system ignored the depth magnitude information of binocular disparity if it selected the motion disparity in depth order perception. The second experiment showed that these three points were consistent whether the subject’s head movement or object movement generated motion disparity.     

Perception of three-dimensional form from patterns of optical texture

The research described in the present article was designed to investigate how patterns of optical texture provide information about the three-dimensional structure of objects in space. Four experiments were performed in which observers were asked to judge the perceived depth of simulated ellipsoid surfaces under a variety of experimental conditions. The results revealed that judged depth increases linearly with simulated depth although the slope of this relation varies significantly among different types of texture patterns. Random variations in the sizes and shapes of individual surface elements have no detectable effect on observers' judgments. The perception of three-dimensional form is quite strong for surfaces displayed under parallel projection, but the amount of apparent depth is slightly less than for identical surfaces displayed under polar projection. Finally, the perceived depth of a surface is eliminated if the optical elements in a display are not sufficiently elongated or if they are not approximately aligned with one another. A theoretical explanation of these findings is proposed based on the neural network analysis of Grossberg and Mingolla.     

Three gradients and the perception of flat and curved surfaces

Researchers of visual perception have long been interested in the perceived slant of a surface and in the gradients that purportedly specify it. Slant is the angle between the line of sight and the tangent to the planar surface at any point, also called the surface normal. Gradients are the sources of information that grade, or change, with visual angle as one looks from one's feet upward to the horizon. The present article explores three gradients--perspective, compression, and density--and the phenomenal impression of flat and curved surfaces. The perspective gradient is measured at right angles to the axis of tilt at any point in the optic array; that is, when looking down a hallway at the tiles of a floor receding in the distance, perspective is measured by the x-axis width of each tile projected on the image plane orthogonal to the line of sight. The compression gradient is the ratio of y/x axis measures on the projected plane. The density gradient is measured by the number of tiles per unit solid visual angle. For flat surfaces and many others, perspective and compression gradients decrease with distance, and the density gradient increases. We discuss the manner in which these gradients change for various types of surfaces. Each gradient is founded on a different assumption about textures on the surfaces around us. In Experiment 1, viewers assessed the three-dimensional character of projections of flat and curved surfaces receding in the distance. They made pairwise judgments of preference and of dissimilarity among eight stimuli in each of four sets. The presence of each gradient was manipulated orthogonally such that each stimulus had zero, one, two, or three gradients appropriate for either a flat surface or a curved surface. Judgments were made were made for surfaces with both regularly shaped and irregularly shaped textures scattered on them. All viewer assessment were then scaled in one dimension. Multiple correlation and regression on the scale values revealed that greater than 98% of the variance in scale values was accounted for by the gradients. For the flat surfaces a mean of 65% of the variance was accounted for by the perspective gradient, 28% by the density gradient, and 6% by the compression gradient. For curved surfaces, on the other hand, a mean of 96% of the variance was accounted for by the compression gradient, and less than 2% by either the perspective gradient or the density gradient.(ABSTRACT TRUNCATED AT 400 WORDS)     

How owls structure visual information

Recent studies on perceptual organization in humans claim that the ability to represent a visual scene as a set of coherent surfaces is of central importance for visual cognition. We examined whether this surface representation hypothesis generalizes to a non-mammalian species, the barn owl (Tyto alba). Discrimination transfer combined with random-dot stimuli provided the appropriate means for a series of two behavioural experiments with the specific aims of (1) obtaining psychophysical measurements of figure–ground segmentation in the owl, and (2) determining the nature of the information involved. In experiment 1, two owls were trained to indicate the presence or absence of a central planar surface (figure) among a larger region of random dots (ground) based on differences in texture. Without additional training, the owls could make the same discrimination when figure and ground had reversed luminance, or were camouflaged by the use of uniformly textured random-dot stereograms. In the latter case, the figure stands out in depth from the ground when positional differences of the figure in two retinal images are combined (binocular disparity). In experiment 2, two new owls were trained to distinguish three-dimensional objects from holes using random-dot kinematograms. These birds could make the same discrimination when information on surface segmentation was unexpectedly switched from relative motion to half-occlusion. In the latter case, stereograms were used that provide the impression of stratified surfaces to humans by giving unpairable image features to the eyes. The ability to use image features such as texture, binocular disparity, relative motion, and half-occlusion interchangeably to determine figure–ground relationships suggests that in owls, as in humans, the structuring of the visual scene critically depends on how indirect image information (depth order, occlusion contours) is allocated between different surfaces. Electronic Publication     

Orientation Invariance and Geometric Primitives in Shape Recognition

Although it is generally assumed that vision is orientation invariant, that is, that shapes can be recognized regardless of viewing angle, there is little evidence that speaks directly to this issue, and what evidence there is fails to support orientation invariance. We propose an explanation for the previous results in terms of the kinds of shape primitives used by the visual system in achieving orientation invariance: Whereas contours are used at stages of vision that ore not orientation invariant, surfaces and/or volumes are used at stages of vision that are orientation invariant. The stimuli in previously reported studies were wire forms, which can represented only in terms of contour. In four experiments, testing both short-term and long-term memory for shape, we replicated the previous failures of orientation invariance using wire forms, but found relatively good or perfect orientation invariance with equivalently shaped surfaces.     

Planar motion permits perception of metric structure in stereopsis

A fundamental problem in the study of spatial perception concerns whether and how vision might acquire information about the metric structure of surfaces in three-dimensional space from motion and from stereopsis. Theoretical analyses have indicated that stereoscopic perceptions of metric relations in depth require additional information about egocentric viewing distance; and recent experiments by James Todd and his colleagues have indicated that vision acquires only affine but not metric structure from motion—that is, spatial relations ambiguous with regard to scale in depth. The purpose of the present study was to determine whether the metric shape of planar stereoscopic forms might be perceived from congruence under planar rotation. In Experiment 1, observers discriminated between similar planar shapes (ellipses) rotating in a plane with varying slant from the frontal-parallel plane. Experimental conditions varied the presence versus absence of binocular disparities, magnification of the disparity scale, and moving versus stationary patterns. Shape discriminations were accurate in all conditions with moving patterns and were near chance in conditions with stationary patterns; neither the presence nor the magnification of binocular disparities had any reliable effect. In Experiment 2, accuracy decreased as the range of rotation decreased from 80° to 10°. In Experiment 3, small deviations from planarity of the motion produced large decrements in accuracy. In contrast with the critical role of motion in shape discrimination, motion hindered discriminations of the binocular disparity scale in Experiment 4. In general, planar motion provides an intrinsic metric scale that is independent of slant in depth and of the scale of binocular disparities. Vision is sensitive to this intrinsic optical metric.     

Planar motion permits perception of metric structure in stereopsis.

A fundamental problem in the study of spatial perception concerns whether and how vision might acquire information about the metric structure of surfaces in three-dimensional space from motion and from stereopsis. Theoretical analyses have indicated that stereoscopic perceptions of metric relations in depth require additional information about egocentric viewing distance; and recent experiments by James Todd and his colleagues have indicated that vision acquires only affine but not metric structure from motion--that is, spatial relations ambiguous with regard to scale in depth. The purpose of the present study was to determine whether the metric shape of planar stereoscopic forms might be perceived from congruence under planar rotation. In Experiment 1, observers discriminated between similar planar shapes (ellipses) rotating in a plane with varying slant from the frontal-parallel plane. Experimental conditions varied the presence versus absence of binocular disparities, magnification of the disparity scale, and moving versus stationary patterns. Shape discriminations were accurate in all conditions with moving patterns and were near chance in conditions with stationary patterns; neither the presence nor the magnification of binocular disparities had any reliable effect. In Experiment 2, accuracy decreased as the range of rotation decreased from 80 degrees to 10 degrees. In Experiment 3, small deviations from planarity of the motion produced large decrements in accuracy. In contrast with the critical role of motion in shape discrimination, motion hindered discriminations of the binocular disparity scale in Experiment 4. In general, planar motion provides an intrinsic metric scale that is independent of slant in depth and of the scale of binocular disparities. Vision is sensitive to this intrinsic optical metric.     

The perception of length on curved and flat surfaces

In three experiments, observers judged the apparent extents of spatial intervals along the surface of a curved cylinder or a flat plane that was binocularly viewed in a natural, indoor environment. The observers' judgments of surface lengths were precise and reliable but were also inaccurate and subject to relatively large constant errors. These distortions differed among the observers, but they tended to perceive lengths oriented along the curved dimension of the cylinder as being longer than physically equivalent lengths in the noncurved dimension. This phenomenon did not occur when the observers judged curved and noncurved paths on the flat surface. In addition, some observers' judgments of length were affected by changes in the distance to the cylinder, whereas others were affected by the cylinder's orientation in space. These results demonstrate that the perception of length on surfaces is highly dependent on the particular context in which the length occurs.     

Postfusional latency in stereoscopic slant perception and the primitives of stereopsis

Random dot stereograms of slanted surfaces were constructed, each representing one or two slanted surfaces in different relative arrangements and with different axes. Latency to fusion and from fusion to stereoscopic resolution was measured for each stimulus. It was found that latency to fusion was always very brief but that latency to stereoscopic resolution varied markedly, depending upon the orientation and arrangement of the stereoscopic surfaces. A gradient of discontinuities at a surface boundary produced an instant slant response for that surface, whereas a gradient of absolute disparities across the surface did not, except under conditions where vertical declination (a form of orientation disparity) was present. We conclude that stereopsis is not based on the primitives used in matching the images for fusion and that it is, at least initially, a response to disparity discontinuities which play no role in the fusion process. We also conclude that vertical declination is responded to globally as a slant around a horizontal axis but that other forms of orientation disparity are ineffective. The evidence from our experiments does not support the existence of a stereoscopic ability to respond globally to differences in magnification (or spatial frequency). It is suggested that stereoscopic perception of slant around a vertical axis is slow because it results from the integration of local processes.