Interpolation in structure from motion

We investigated surface interpolation in displays of structure from motion (SFM). To do so, we introduced a new method for measuring surface perception in dynamic displays—theSFM probe. An SFM probe is a dot that moves rigidly with the dots on a simulated surface, and whose distance from that surface can be adjusted with a joystick or similar control. The displays we studied were random-dot cylinders containing a vertical strip devoid of feature points (thegap). Subjects adjusted an SFM probe, presented in the gap, until the probe dot appeared to be on the surface. Variability in probe-dot placement decreased with increasing texture density on the cylinder and increased with increasing gap width. Subjects showed a consistent bias to place the probe dot outside the cylinder. This bias increased with increasing texture density for the SFM displays. (The opposite bias was found in a static two-dimensional interpolation task with an arc whose curvature matched that of the cylinder: Subjects placed the probe dot inside the arc.) This outside bias is inconsistent with several theoretical approaches to surface interpolation.     

Interpolation across surface discontinuities in structure from motion Asad Satdpour

Interpolation across orientation discontinuities in simulated three-dimensional (3-D). surfaces was studied in three experiments with the use of structure-from-motion (SFM). displays. The displays depicted dots on two slanted planes with a region devoid of dots (a gap). between them. If extended through the gap at constant slope, the planes would meet at a dihedral edge. Subjects were required to place an SFM probe dot, located within the gap, on the perceived surface. Probe dot placements indicated that subjects perceived a smooth surface connecting the planes rather than a surface with a discontinuity. Probe dot placements varied with slope of the planes, density of the dots, and gap size, but not with orientation (horizontal or vertical). of the dihedral edge or of the axis of rotation. Smoothing was consistent with models of 2-D interpolation proposed by Ullman (1976). and Kellman and Shipley (1991). and with a model of 3-D interpolation proposed by Grimson (1981).     

Temporal integration in structure from motion

A temporal integration model is proposed that predicts the results reported in 4 psychophysical experiments. The main findings were (a) the initial part of a structure-from-motion (SFM) sequence influences the orientation evoked by the final part of that sequence (an effect lasting for more than 1 s), and (b) for oscillating SFM sequences, perceived slant is affected by the oscillation frequency and by the sign of the final gradient. For contracting optic flows (i.e., rotations away from the image plane), the sequence with the lowest oscillation frequency appeared more slanted; for expanding optic flows (i.e., rotations toward the image plane), the sequence with the highest oscillation frequency appeared more slanted.     

The influence of structure from motion on motion correspondence

The visual system has a remarkable ability to reconstruct 3-D structure from moving 2-D features. The processing of structure from motion is generally thought to consist of two stages. First, the direction and speed of features is measured (2-D velocity measurement) and, second, 3-D structure is reconstructed from the measured 2-D velocities (3-D structure recovery). Most models have assumed that these stages occur in a bottom-up fashion. Here, however, we present evidence that the 3-D structure-recovery stage influences the 2-D velocity-measurement stage. We developed a stimulus in which two perceptual modes of motion correspondence (one-way translation versus oscillation), and two perceptual modes of 3-D surface structure (flat surface versus cylinder) could be achieved. We found that the likelihood of perceiving both one-way motion and cylindrical structure increased in similar ways with increasing frame duration. In subsequent experiments we found, first, that a higher likelihood of perceiving one-way motion did not affect the likelihood of perceiving cylindrical structure; and, second, that a higher likelihood of perceiving cylindrical structure increased the likelihood of perceiving one-way motion. These results suggest that the higher, 3-D structure-recovery stage may influence the lower, 2-D motion-correspondence stage. This result is not in accordance with most computational models that assume that there is only one-way, feedforward information processing from the 2-D velocity (energy)-measurement stage to the 3-D structure-recovery stage. Perhaps, one of the roles of feedback processing is to seek consensus of the information processed in different stages.     

Three-dimensional structure from long-range apparent motion

Experiments are reported which show that three-dimensional structure can be perceived from two-dimensional image motions carried by objects defined solely by the differences in binocular and/or temporal correlation (ie disparity or motion discontinuities). This demonstrates that the kinetic depth effect is independent of motion detection in the luminance domain and that its relevant input comes from detectors based on some form of identity preservation of objects or features over time, ie the long-range processes of apparent motion.     

Perception of three-dimensional structure from motion

The ability to perceive the 3-D shape of objects solely from motion cues is referred to as structure-from-motion perception. Recent experiments indicate how this remarkable perceptual attribute is computed by the brains of primates. This computation proceeds in at least two stages, one in which motion measurements are made and another in which moving surfaces are reconstructed. The middle temporal area (MT) in the macaque monkey appears to play a pivotal role in the latter step and suggests a previously unappreciated function for this well-known cortical region, which had previously been thought to play a more rudimentary role in simply signaling the direction of motion of images.     

Discriminating constant from variable angular velocities in structure from motion

We investigated accuracy in discriminating between constant and variable angular velocities for orthographic projections of three-dimensional rotating objects. The reported judgments of “constant” or “variable” angular velocity were only slightly influenced by the projected angular velocities, but they were greatly affected by the variations of the deformation, a first-order component of the optic flow. When viewing either a rotating ellipsoidal volume or a planar surface that accelerated and decelerated over the course of rotation, observers’ tendencies to report a variable angular velocity were increased when the temporal phase of the acceleration pattern increased the range of variation of the median deformation; the tendencies were decreased when the same acceleration pattern was used to decrease the range of variation of the median deformation. These results provide evidence contrary to the hypothesis that the visual system performs a mathematically correct analysis of the optic flow.     

Perceived motion in structure from motion: Pointing responses to the axis of rotation

We investigated the ability to match finger orientation to the direction of the axis of rotation in structure-from-motion displays. Preliminary experiments verified that subjects could accurately use the index finger to report direction. The remainder of the experiments studied the perception of the axis of rotation from full rotations of a group of discrete points, the profiles of a rotating ellipsoid, and two views of a group of discrete points. Subjects’ responses were analyzed by decomposing the pointing responses into their slant and tilt components. Overall, the results indicated that subjects were sensitive to both slant and tilt. However, when the axis of rotation was near the viewing direction, subjects had difficulty reporting tilt with profiles and two views and showed a large bias in their slant judgments with two views and full rotations. These results are not entirely consistent with theoretical predictions. The results, particularly for two views, suggest that additional constraints are used by humans in the recovery of structure from motion.     

The perceptual buildup of three-dimensional structure from motion

We present a set of psychophysical experiments that measure the accuracy of perceived three-dimensional (3-D) structure derived from relative motion in the changing two-dimensional image. The experiments are motivated in part by a computational model proposed by Ullman (1984), called the incremental rigidity scheme, in which an accurate 3-D structure is built up incrementally, by considering images of moving objects over an extended time period. Our main conclusions are: First, the human visual system can derive an accurate model of the relative depths of moving points, even in the presence of noise in their image positions; second, the accuracy of the 3-D model improves with time, eventually reaching a plateau; and third, the 3-D structure currently perceived appears to depend on previous 3-D models. Through computer simulations, we relate the results of our psychophysical experiments with the predictions of Ullman's model.     

The interplay between stereopsis and structure from motion

In a series of psychophysical experiments, an adaptation paradigm was employed to study the influence of stereopsis on perception of rotation in an ambiguous kinetic depth (KD) display. Without prior adaptation or stereopsis, a rotating globe undergoes spontaneous reversals in perceived direction of rotation, with successive durations of perceived rotation being random variables. Following 90 sec of viewing a stereoscopic globe undergoing unambiguous rotation, the KD globe appeared to rotate in a direction opposite that experienced during the stereoscopic adaptation period. This adaptation aftereffect was short-lived, and it occurred only when the adaptation and test figures stimulated the same retinal areas, and only when the adaptation and test figures rotated about the same axis. The aftereffect was just as strong when the test and adaptation figures had different shapes, as long as the adaptation figure contained multiple directions of motion imaged at different retinal disparities. Nonstereoscopic adaptation figures had no effect on the perceived direction of rotation of the ambiguous KD figure. These results imply that stereopsis and motion strongly interact in the specification of structure from motion, a result that complements earlier work on this problem.     

Distortions of depth-order relations and parallelism in structure from motion

Four experiments related human perception of depth-order relations in structure-from-motion dis-plays to current Euclidean and affine theories of depth recovery from motion. Discrimination between parallel and nonparallel lines and relative-depth judgments was observed for orthographic projections of rigidly oscillating random-dot surfaces. We found that (1) depth-order relations were perceived veridically for surfaces with the same slant magnitudes, but were systematically biased for surfaces with different slant magnitudes. (2) Parallel (virtual) lines defined by probe dots on surfaces with different slant magnitudes were judged to be nonparallel. (3) Relative-depth judgments were internally inconsistent for probe dots on surfaces with different slant magnitudes. It is argued that both veridical performance and systematic misperceptions may be accounted for by a heuristic analysis of the first-order optic flow.     

The spatial and temporal characteristics of perceiving 3-D structure from motion.

In four experiments, a scalar judgment of perceived depth was used to examine the spatial and temporal characteristics of the perceptual buildup of three-dimensional (3-D) structure from optical motion as a function of the depth in the simulated object, the speed of motion, the number of elements defining the object, the smoothness of the optic flow field, and the type of motion. In most of the experiments, the objects were polar projections of simulated half-ellipsoids undergoing a curvilinear translation about the screen center. It was found that the buildup of 3-D structure was: (1) jointly dependent on the speed at which an object moved and on the range through which the object moved; (2) more rapid for deep simulated objects than for shallow objects; (3) unaffected by the number of points defining the object, including the maximum apparent depth within each simulated object-depth condition; (4) not disrupted by nonsmooth optic flow fields; and (5) more rapid for rotating objects than for curvilinearly translating objects.     

Planar motion permits perception of metric structure in stereopsis.

A fundamental problem in the study of spatial perception concerns whether and how vision might acquire information about the metric structure of surfaces in three-dimensional space from motion and from stereopsis. Theoretical analyses have indicated that stereoscopic perceptions of metric relations in depth require additional information about egocentric viewing distance; and recent experiments by James Todd and his colleagues have indicated that vision acquires only affine but not metric structure from motion--that is, spatial relations ambiguous with regard to scale in depth. The purpose of the present study was to determine whether the metric shape of planar stereoscopic forms might be perceived from congruence under planar rotation. In Experiment 1, observers discriminated between similar planar shapes (ellipses) rotating in a plane with varying slant from the frontal-parallel plane. Experimental conditions varied the presence versus absence of binocular disparities, magnification of the disparity scale, and moving versus stationary patterns. Shape discriminations were accurate in all conditions with moving patterns and were near chance in conditions with stationary patterns; neither the presence nor the magnification of binocular disparities had any reliable effect. In Experiment 2, accuracy decreased as the range of rotation decreased from 80 degrees to 10 degrees. In Experiment 3, small deviations from planarity of the motion produced large decrements in accuracy. In contrast with the critical role of motion in shape discrimination, motion hindered discriminations of the binocular disparity scale in Experiment 4. In general, planar motion provides an intrinsic metric scale that is independent of slant in depth and of the scale of binocular disparities. Vision is sensitive to this intrinsic optical metric.     

Recovery of 3-D shape from binocular disparity and structure from motion

Four experiments were conducted to examine the integration of depth information from binocular stereopsis and structure from motion (SFM), using stereograms simulating transparent cylindrical objects. We found that the judged depth increased when either rotational or translational motion was added to a display, but the increase was greater for rotating (SFM) displays. Judged depth decreased as texture element density increased for static and translating stereo displays, but it stayed relatively constant for rotating displays. This result indicates that SFM may facilitate stereo processing by helping to resolve the stereo correspondence problem. Overall, the results from these experiments provide evidence for a cooperative relationship between. SFM and binocular disparity in the recovery of 3-D relationships from 2-D images. These findings indicate that the processing of depth information from SFM and binocular disparity is not strictly modular, and thus theories of combining visual information that assume strong modularity-or-independence cannot accurately characterize all instances of depth perception from multiple sources.     

The perception of shape and curvedness from binocular stereopsis and structure from motion

The integration of binocular stereopsis and kinetic depth was measured for two distinct aspects of 3-Dstructure: (1)shape index, which is a measure of scale-independent structure, and (2)curvedness, which is a measure of scale-dependent structure. We found that motion contributes significantly more to judged shape index than it does to judged curvedness, and stereo contributes significantly more to judged curvedness than it does to judged shape index. This suggests that the differences in the relative contribution of motion and stereo reported here occurred because these two sources do not equally specify the scale-dependent and scale-independent aspects of surface structure. Furthermore, these results seem to be inconsistent with integration models in which the different visual cues all initially contribute to the same single representation of 3-Dstructure.     

The spatial and temporal characteristics of perceiving 3-D structure from motion

In four experiments, a scalar judgment of perceived depth was used to examine the spatial and temporal characteristics of the perceptual buildup of three-dimensional (3-D) structure from optical motion as a function of the depth in the simulated object, the speed of motion, the number of elements defining the object, the smoothness of the optic flow field, and the type of motion. In most of the experiments, the objects were polar projections of simulated half-ellipsoids under-going a curvilinear translation about the screen center. It was found that the buildup of 3-D structure was: (1) jointly dependent on the speed at which an object moved and on the range through which the object moved; (2) more rapid for deep simulated objects than for shallow objects; (3) unaffected by the number of points defining the object, including the maximum apparent depth within each simulated object-depth condition; (4) not disrupted by nonsmooth optic flow fields; and (5) more rapid for rotating objects than for curvilinearly translating objects.     

A counterexample to the rigidity assumption in the visual perception of structure from motion

It has been proposed that the human visual system prefers perceptions of objects that are rigid or undergo minimum form change. A counterexample is presented in which a rigid two-dimensional figure rotating in the frontal plane is perceived as a distorting three-dimensional shape. It is argued that this perception results from the stimulation of automatic processes for perceiving size change, and that these processes are not subject to a general rigidity assumption.     

The perception of 3-dimensional affine structure from minimal apparent motion sequences

The research described in the present article was designed to identify the minimal conditions for the visual perception of 3-dimensional structure from motion by comparing the theoretical limitations of ideal observers with the perceptual performance of actual human subjects on a variety of psychophysical tasks. The research began with a mathematical analysis, which showed that 2-frame apparent motion sequences are theoretically sufficient to distinguish between rigid and nonrigid motion and to identify structural properties of an object that remain invariant under affine transformations, but that 3 or more distinct frames are theoretically necessary to adequately specify properties of euclidean structure such as the relative 3-dimensional lengths or angles between nonparallel line segments. A series of four experiments was then performed to verify the psychological validity of this analysis. The results demonstrated that the determination of structure from motion in actual human observers may be restricted to the use of first order temporal relations, which are available within 2-frame apparent motion sequences. That is to say, the accuracy of observers' judgments did not improve in any of these experiments as the number of distinct frames in an apparent motion sequence was increased from 2 to 8, and performance on tasks involving affine structure was of an order of magnitude greater than performance on similar tasks involving euclidean structure.