The perceptual buildup of three-dimensional structure from motion

We present a set of psychophysical experiments that measure the accuracy of perceived three-dimensional (3-D) structure derived from relative motion in the changing two-dimensional image. The experiments are motivated in part by a computational model proposed by Ullman (1984), called the incremental rigidity scheme, in which an accurate 3-D structure is built up incrementally, by considering images of moving objects over an extended time period. Our main conclusions are: First, the human visual system can derive an accurate model of the relative depths of moving points, even in the presence of noise in their image positions; second, the accuracy of the 3-D model improves with time, eventually reaching a plateau; and third, the 3-D structure currently perceived appears to depend on previous 3-D models. Through computer simulations, we relate the results of our psychophysical experiments with the predictions of Ullman's model.     

Robust recovery of human motion from video using Kalman filters and virtual humans

In sport science, as in clinical gait analysis, optoelectronic motion capture systems based on passive markers are widely used to recover human movement. By processing the corresponding image points, as recorded by multiple cameras, the human kinematics is resolved through multistage processing involving spatial reconstruction, trajectory tracking, joint angle determination, and derivative computation. Key problems with this approach are that marker data can be indistinct, occluded or missing from certain cameras, that phantom markers may be present, and that both 3D reconstruction and tracking may fail. In this paper, we present a novel technique, based on state space filters, that directly estimates the kinematical variables of a virtual mannequin (biomechanical model) from 2D measurements, that is, without requiring 3D reconstruction and tracking. Using Kalman filters, the configuration of the model in terms of joint angles, first and second order derivatives is automatically updated in order to minimize the distances, as measured on TV-cameras, between the 2D measured markers placed on the subject and the corresponding back-projected virtual markers located on the model. The Jacobian and Hessian matrices of the nonlinear observation function are computed through a multidimensional extension of Stirling's interpolation formula. Extensive experiments on simulated and real data confirmed the reliability of the developed system that is robust against false matching and severe marker occlusions. In addition, we show how the proposed technique can be extended to account for skin artifacts and model inaccuracy.     

PERCEIVED CONTINUITY OF OCCLUDED VISUAL OBJECTS

Abstract— The human visual system does not rigidly preserve the properties of the retinal image as neural signals are transmitted to higher areas of the brain Instead, it generates a representation that captures stable surface properties despite a retinal image that is often fragmented in space and time because of occlusion caused by object and observer motion The recovery of this coherent representation depends at least in part on input from an abstract representation of three-dimensional (3-D) surface layout In the two experiments reported, a stereoscopic apparent motion display was used to investigate the perceived continuity of a briefly interrupted visual object When a surface appeared in front of the object's location during the interruption, the object was more likely to be perceived as persisting through the interruption (behind an occluder) than when the surface appeared behind the object's location under otherwise identical stimulus conditions The results reveal the influence of 3-D surface-based representations even in very simple visual tasks.     

Visual motion and attentional capture

Previous work has shown that abrupt visual onsets capture attention. This occurs even with stimuli that are equiluminant with the background, which suggests that the appearance of a new perceptual object, not merely a change in luminance, captures attention. Three experiments are reported in which this work was extended by investigating the possible role of visual motion in attentional capture. Experiment 1 revealed that motion can efficiently guide attention when it is perfectly informative about the location of a visual search target, but that it does not draw attention when it does not predict the target’s position. This result was obtained with several forms of motion, including oscillation, looming, and nearby moving contours. To account for these and other results, we tested anew-object account of attentional capture in Experiment 2 by using a global/local paradigm. When motion segregated a local letter from its perceptual group, the local letter captured attention as indexed by an effect on latency of response to the task-relevant global configuration. Experiment 3 ruled out the possibility that the motion in Experiment 2 captured attention merely by increasing the salience of the moving object. We argue instead that when motion segregates a perceptual element from a perceptual group, a new perceptual object is created, and this event captures attention. Together, the results suggest that motion as such does not capture attention but that the appearance of a new perceptual object does.     

Pattern orientation, not motion, determines the two-dimensional tilt illusion

In the usual tilt illusion (TI) configuration, an inducing stimulus which has a single orientation is used psychophysically to explore orientation analysis in the human visual system. Recently, this approach has been extended to the use of inducing stimuli which have two orientations. Such a two-dimensional (2-D) stimulus permits investigation of the low-level analysis of visual patterns. Prior experimentation has left it unclear whether it is the spatial or the motion properties of a moving crossed-grating plaid which determine two-dimensional tilt illusions (2-D TIs) because these two parameters previously were perfectly correlated. In the present experiments pattern orientation and motion were decoupled. It is shown that 2-D TIs are determined by the spatial properties of an inducing annulus and not by its motion properties. The results also support the existence of a mechanism which extracts axes of symmetry, and which is difficult to account for in terms of local cross-orientation domain inhibition.     

Validation of a full body 3-D dynamic linked segment model

In studies related to human movement, linked segment models (LSM's) are often used to quantify forces and torques, generated in body joints. Some LSM's represent only a few body segments. Others, for instance used in studies on the control of whole body movements, include all body segments. As a consequence of the complexity of 3-dimensional (3-D) analyses, most LSM's are restricted to one plane of motion. However, in asymmetric movements this may result in a loss of relevant information. The aim of the current study was to develop and validate a 3-D LSM including all body segments. Braces with markers, attached to all body segments, were used to record the body movements. The validation of the model was accomplished by comparing the measured with the estimated ground reaction force and by comparing the torques at the lumbo-sacral joint that resulted from a bottom-up and a top-down mechanical analysis. For both comparisons, reasonable to good agreement was found. Sources of error that could not be analysed this way, were subjected to an additional sensitivity analysis. It was concluded that the internal validity of the current model is quite satisfactory.     

Discrimination of 3-D shape and 3-D curvature from motion in active vision

We examined the ability of human observers to discriminate between different 3-D quadratic surfaces defined by motion, and with head position fed back to the stimulus to provide an up-to-date dynamical perspective view. We tested whether 3-D shape or 3-D curvature would affect discrimination performance. It appeared that discrimination of 3-D quadratic shape clearly depended on shape but not on the amount of curvature. Even when the amount of curvature was randomized, subjects’ performance was not altered. On the other hand, the discrimination of 3-D curvature clearly depended linearly on curvature with Weber fractions of 20% on the average and, to a small degree, on 3-D shape. The experiment shows that observers can easily separate 3-D shape and 3-D curvature, and that Koenderink’s shape index and curvedness provide a convenient way to specify shape. These results warn us against using just any arbitrary 3-D shape in 3-D shape perception tasks and indicate, for example, that emphasizing 3-D shape in computer displays by exaggerating curvature does not have any effect.     

The influence of structure from motion on motion correspondence

The visual system has a remarkable ability to reconstruct 3-D structure from moving 2-D features. The processing of structure from motion is generally thought to consist of two stages. First, the direction and speed of features is measured (2-D velocity measurement) and, second, 3-D structure is reconstructed from the measured 2-D velocities (3-D structure recovery). Most models have assumed that these stages occur in a bottom-up fashion. Here, however, we present evidence that the 3-D structure-recovery stage influences the 2-D velocity-measurement stage. We developed a stimulus in which two perceptual modes of motion correspondence (one-way translation versus oscillation), and two perceptual modes of 3-D surface structure (flat surface versus cylinder) could be achieved. We found that the likelihood of perceiving both one-way motion and cylindrical structure increased in similar ways with increasing frame duration. In subsequent experiments we found, first, that a higher likelihood of perceiving one-way motion did not affect the likelihood of perceiving cylindrical structure; and, second, that a higher likelihood of perceiving cylindrical structure increased the likelihood of perceiving one-way motion. These results suggest that the higher, 3-D structure-recovery stage may influence the lower, 2-D motion-correspondence stage. This result is not in accordance with most computational models that assume that there is only one-way, feedforward information processing from the 2-D velocity (energy)-measurement stage to the 3-D structure-recovery stage. Perhaps, one of the roles of feedback processing is to seek consensus of the information processed in different stages.     

Accuracy and precision of binocular 3-D motion perception

In principle, information for 3-D motion perception is provided by the differences in position and motion between left- and right-eye images of the world. It is known that observers can precisely judge between different 3-D motion trajectories, but the accuracy of binocular 3-D motion perception has not been studied. The authors measured the accuracy of 3-D motion perception. In 4 different tasks, observers were inaccurate, overestimating trajectory angle, despite consistently choosing similar angles (high precision). Errors did not vary consistently with target distance, as would be expected had inaccuracy been due to misestimates of viewing distance. Observers appeared to rely strongly on the lateral position of the target, almost to the exclusion of the use of depth information. For the present tasks, these data suggest that neither an accurate estimate of 3-D motion direction nor one of passing distance can be obtained using only binocular cues to motion in depth. ((c) 2003 APA, all rights reserved)     

Learning to see stereokinetic effects

The Saturn illusion is a stereokinetic effect that occurs when a flat pattern composed of a full ellipse with two symmetrical semirings is rotated slowly in the frontoparallel plane. Subjects report seeing an egg-shaped object inserted into a circular ring, and the two objects move solidly into 3-D space as a single rigid body. Inexperienced observers show a conspicuous delay before reaching this percept. Two experiments are reported in which it is shown that this incubation time progressively decreases with repeated exposures to the stimulus pattern. A certain amount of time (14 s on average) is, however, required to obtain the effect, even after six successive exposures. It is argued that this time, which is independent of the speed of rotation and is not further reducible, is a fixed entity and is needed to compute the most rigid 3-D solution from deformations in the 2-D image. The results are discussed in relation to current theories of perception of structure from motion.     

Contextual influences of dimension, speed, and direction of motion on subjective time perception

Research has indicated that the direction of motion and the speed of motion can influence the subjective estimates of temporal duration of two-dimensional (2-D) stimuli expanding and contracting within the picture plane. In this study, we investigated whether the contextual cues of stimulus/movement-plane dimensionality (2-D stimuli with implied movement in the picture plane or depth-rendered “3-D” stimuli with implied movement in the depth plane) influence and interact with speed and implied movement direction during interval estimation. Participants viewed a series of standard stimulus durations followed by a test stimulus duration and determined whether the test and standard durations differed. The results indicated that moving stimuli were overestimated relative to stationary stimuli, regardless of the direction of motion or dimensionality. Also, faster-moving stimuli were overestimated relative to slower-moving stimuli. Importantly, an interaction between movement direction and dimensional cues indicated that the loom/recede distinction occurs for 2-D but not for 3-D stimuli. It is possible that the loom/recede distinction for the 2-D condition may be an artifact arising from reduced or from a lack of perceived motion in 2-D “recede” conditions, rather than a specific overestimation for looming stimuli.     

An effective attentional set for a specific colour does not prevent capture by infrequently presented motion distractors

An organism's survival depends on the ability to rapidly orient attention to unanticipated events in the world. Yet, the conditions needed to elicit such involuntary capture remain in doubt. Especially puzzling are spatial cueing experiments, which have consistently shown that involuntary shifts of attention to highly salient distractors are not determined by stimulus properties, but instead are contingent on attentional control settings induced by task demands. Do we always need to be set for an event to be captured by it, or is there a class of events that draw attention involuntarily even when unconnected to task goals? Recent results suggest that a task-irrelevant event will capture attention on first presentation, suggesting that salient stimuli that violate contextual expectations might automatically capture attention. Here, we investigated the role of contextual expectation by examining whether an irrelevant motion cue that was presented only rarely (～3–6% of trials) would capture attention when observers had an active set for a specific target colour. The motion cue had no effect when presented frequently, but when rare produced a pattern of interference consistent with attentional capture. The critical dependence on the frequency with which the irrelevant motion singleton was presented is consistent with early theories of involuntary orienting to novel stimuli. We suggest that attention will be captured by salient stimuli that violate expectations, whereas top-down goals appear to modulate capture by stimuli that broadly conform to contextual expectations.     

Mom’s shadow: structure-from-motion in newly hatched chicks as revealed by an imprinting procedure

The ability to recognize three-dimensional objects from two-dimensional (2-D) displays was investigated in domestic chicks, focusing on the role of the object’s motion. In Experiment 1 newly hatched chicks, imprinted on a three-dimensional (3-D) object, were allowed to choose between the shadows of the familiar object and of an object never seen before. In Experiments 2 and 3 random-dot displays were used to produce the perception of a solid shape only when set in motion. Overall, the results showed that domestic chicks were able to recognize familiar shapes from 2-D motion stimuli. It is likely that similar general mechanisms underlying the perception of structure-from-motion and the extraction of 3-D information are shared by humans and animals. The present data shows that they occur similarly in birds as known for mammals, two separate vertebrate classes; this possibly indicates a common phylogenetic origin of these processes.     

Using a Kinect sensor to acquire biological motion: Toolbox and evaluation

Biological motion (BM) is the movement of animate entities, which conveys rich social information. To obtain pure BM, researchers nowadays predominantly use point-light displays (PLDs), which depict BM through a set of light points (e.g., 12 points) placed at distinct joints of a moving human body. Most prevalent BM stimuli are created by state-of-the-art motion capture systems. Although these stimuli are highly precise, the motion capture system is expensive and bulky, and its process of constructing a PLD-based BM is time-consuming and complex. These factors impede the investigation of BM mechanisms. In this study, we propose a free Kinect-based biological motion capture (KBC) toolbox based on the Kinect Sensor 2.0 in C++. The KBC toolbox aims to help researchers acquire PLD-based BM in an easy, low-cost, and user-friendly way. We conducted three experiments to examine whether KBC-generated BM can genuinely reflect the processing characteristics of BM: (1)?Is BM from this source processed globally in vision? (2)?Does its BM (e.g., from the feet) retain detailed local information? and (3)?Does the BM convey emotional information? We obtained positive results in response to all three questions. Therefore, we think that the KBC toolbox can be useful in generating BM for future research.     

3-D structure perceived from dynamic information: a new theory

Image movement provides one of the most potent two-dimensional cues for depth. From motion cues alone, the brain is capable of deriving a three-dimensional representation of distant objects. For many decades, theoretical and empirical investigations into this ability have interpreted these percepts as faithful copies of the projected 3-D structures. Here we review empirical findings showing that perceived 3-D shape from motion is not veridical and cannot be accounted for by the current models. We present a probabilistic model based on a local analysis of optic flow. Although such a model does not guarantee a correct reconstruction of 3-D shape, it is shown to be consistent with human performance.     

Similarity structure in visual speech perception and optical phonetic signals

A complete understanding of visual phonetic perception (lipreading) requires linking perceptual effects to physical stimulus properties. However, the talking face is a highly complex stimulus, affording innumerable possible physical measurements. In the search for isomorphism between stimulus properties and phoneticeffects, second-order isomorphism was examined between theperceptual similarities of video-recorded perceptually identified speech syllables and the physical similarities among the stimuli. Four talkers produced the stimulus syllables comprising 23 initial consonants followed by one of three vowels. Six normal-hearing participants identified the syllables in a visual-only condition. Perceptual stimulus dissimilarity was quantified using the Euclidean distances between stimuli in perceptual spaces obtained via multidimensional scaling. Physical stimulus dissimilarity was quantified using face points recorded in three dimensions by an optical motion capture system. The variance accounted for in the relationship between the perceptual and the physical dissimilarities was evaluated using both the raw dissimilarities and the weighted dissimilarities. With weighting and the full set of 3-D optical data, the variance accounted for ranged between 46% and 66% across talkers and between 49% and 64% across vowels. The robust second-order relationship between the sparse 3-D point representation of visible speech and the perceptual effects suggests that the 3-D point representation is a viable basis for controlled studies of first-order relationships between visual phonetic perception and physical stimulus attributes.     

An investigation of the line motion effect in a callosotomy patient

When a line is flashed instantaneously between two markers it can appear to propagate from one marker to the other. This illusion is known as the line motion effect. We investigated this effect in the two hemispheres of a callosotomy ("split-brain") patient. We found that both hemispheres perceived the line motion effect, and that flashing one of the markers biased the direction of motion away from that marker regardless of which hemisphere received the stimulus. In contrast, matching the width of the line to the width of one of the markers biased the direction of motion away from the marker only when it appeared in the left visual hemifield. This suggests that multiple mechanisms can contribute to the line motion effect, and that some of these mechanisms rely on different neural structures.     

Do new objects capture attention?

The visual system relies on several heuristics to direct attention to important locations and objects. One of these mechanisms directs attention to sudden changes in the environment. Although a substantial body of research suggests that this capture of attention occurs only for the abrupt appearance of a new perceptual object, more recent evidence shows that some luminance-based transients (e.g., motion and looming) and some types of brightness change also capture attention. These findings show that new objects are not necessary for attention capture. The present study tested whether they are even sufficient. That is, does a new object attract attention because the visual system is sensitive to new objects or because it is sensitive to the transients that new objects create? In two experiments using a visual search task, new objects did not capture attention unless they created a strong local luminance transient.     

Age-related reduction in 3-D visual motion priming

In 3 experiments, younger and older adults judged the perceived motion of three-dimensional (3-D) figures that rotated in depth either unambiguously or ambiguously. Both groups were found to be equivalent in judging the direction of single rotations of the simulated 3-D objects (Experiment 1). In Experiments 2 and 3, a single unambiguous rotation (prime) was followed 0-3200 ms later by an ambiguous rotation (target). Motion priming was indicated by the disambiguation of the second rotation by the first rotation. 3-D motion priming was initially found to be similar in young and old, but it rapidly reduced in the older participants compared to the younger ones. Using a nonluminance depth cue--occlusion--to induce 3-D motion, diminished contrast sensitivity in the elderly was ruled out as a cause of the reduced priming. The results show that 3-D motion priming exhibits robust age-related decline. An age-related decrease in temporal persistence may account for the reduction in 3-D motion priming in older adults.