Heading backward: Perceived direction of movement in contracting and expanding optical flow fields

Two goals were pursued in an investigation of possible visual sources for directionality judgments of ego-motion. First, J. J. Gibson’s (1950) global radial outflow hypothesis was contrasted with a simple extrapolation strategy. Second, backing-direction judgments capitalizing on the informational equivalence of global radial outflow created during forward ego-motion and global radial inflow during backward ego-motion were explored. In comparing the accuracy of heading and backing judgments, new insights about global flow and extrapolation strategies were found. Consistent with the hypothesis of an extrapolation strategy, Experiment 1 demonstrated that backing judgments were more accurate than heading judgments when linear observer motion was simulated by means of a point-light flow field. In this case, accuracy was higher with two-point-light displays (extrapolation) than with more complex displays (global flow). Experiment 2 showed that in cases where extrapolation was not possible, such as circular motion, no advantage of backing judgments could be found and judgments were generally less accurate. We conclude that, whenever possible, observers use extrapolation to determine their heading/backing. Only when global flow is the only good source of information do they rely on it.     

Time-to-passage judgments on circular trajectories are based on relative optical acceleration

Current theories of arrival time have difficulty explaining performance in the common but neglected case of nonlinear approach. Global tau, a variable supposed to guide time-to-passage (TTP) judgments of objects approaching on linear trajectories, does not apply to circular movement. However, TTP judgments are surprisingly accurate in such cases. We simulated movement through a three-dimensional cloud of point-lights on various circular trajectories. Arrival-time judgments were found to be above chance when observers had to determine which of two expansionless targets would pass them first. Similar to the inside bias observed in heading studies on circular trajectories, observers showed a strong bias to select the target on the inside of their own curved motion path as passing by first. Analysis of the projected target motion revealed that targets on the inside had lower optical velocities and relatively high optical acceleration rates. Empirical TTP judgments agreed best with a strategy based on relative optical velocity changes.     

Time-to-passage judgments on circular trajectories are based on relative optical acceleration.

Current theories of arrival time have difficulty explaining performance in the common but neglected case of nonlinear approach. Global tau, a variable supposed to guide time-to-passage (TTP) judgments of objects approaching on linear trajectories, does not apply to circular movement. However, TTP judgments are surprisingly accurate in such cases. We simulated movement through a three-dimensional cloud of point-lights on various circular trajectories. Arrival-time judgments were found to be above chance when observers had to determine which of two expansionless targets would pass them first. Similar to the inside bias observed in heading studies on circular trajectories, observers showed a strong bias to select the target on the inside of their own curved motion path as passing by first. Analysis of the projected target motion revealed that targets on the inside had lower optical velocities and relatively high optical acceleration rates. Empirical TTP judgments agreed best with a strategy based on relative optical velocity changes.     

Perceiving curvilinear heading in the presence of moving objects

Four experiments were directed at understanding the influence of multiple moving objects on curvilinear (i.e., circular and elliptical) heading perception. Displays simulated observer movement over a ground plane in the presence of moving objects depicted as transparent, opaque, or black cubes. Objects either moved parallel to or intersected the observer's path and either retreated from or approached the moving observer. Heading judgments were accurate and consistent across all conditions. The significance of these results for computational models of heading perception and for information in the global optic flow field about observer and object motion is discussed.     

The influence of object-image velocity change on perceived heading in minimal environments

When human observers move forward and rotate their eyes, a complex pattern of light flows across the retina. This pattern is referred to as retinal flow. A model has been proposed to explain how humans perceive their direction of self-movement (or heading) from (1) static depth, (2) direction of image motion, and (3) whether image velocity undergoes acceleration or deceleration (Wang & Cutting, 1999). However, findings from past research in which sparse or minimalist stimuli were used have suggested that not all of the information to which participants are sensitive is captured within the scope of this model. In particular it has been suggested that the magnitude or size of image velocity change may be of significance beyond simply whether image velocity could be categorized as speeding up (i.e., accelerating) or slowing down (i.e., decelerating). In two experiments, the influence of this factor on heading judgments under minimal conditions was investigated. Evidence was found in support of the idea that the rate of image velocity change can influence judgments of the direction of self-movement in minimalist conditions.     

Differential effects of shared attention on perception of heading and 3-D object motion

When a person moves in a straight line through a stationary environment, the images of object surfaces move in a radial pattern away from a single point. This point, known as thefocus of expansion (FOE), corresponds to the person’s direction of motion. People judge their heading from image motion quite well in this situation. They perform most accurately when they can see the region around the FOE, which contains the most useful information for this task. Furthermore, a large moving object in the scene has no effect on observer heading judgments unless it obscures the FOE. Therefore, observers may obtain the most accurate heading judgments by focusing their attention on the region around the FOE. However, in many situations (e.g., driving), the observer must pay attention to other moving objects in the scene (e.g., cars and pedestrians) to avoid collisions. These objects may be located far from the FOE in the visual field. We tested whether people can accurately judge their heading and the three-dimensional (3-D) motion of objects while paying attention to one or the other task. The results show that differential allocation of attention affects people’s ability to judge 3-D object motion much more than it affects their ability to judge heading. This suggests that heading judgments are computed globally, whereas judgments about object motion may require more focused attention.     

Tracking without perceiving: a dissociation between eye movements and motion perception

Can people react to objects in their visual field that they do not consciously perceive? We investigated how visual perception and motor action respond to moving objects whose visibility is reduced, and we found a dissociation between motion processing for perception and for action. We compared motion perception and eye movements evoked by two orthogonally drifting gratings, each presented separately to a different eye. The strength of each monocular grating was manipulated by inducing adaptation to one grating prior to the presentation of both gratings. Reflexive eye movements tracked the vector average of both gratings (pattern motion) even though perceptual responses followed one motion direction exclusively (component motion). Observers almost never perceived pattern motion. This dissociation implies the existence of visual-motion signals that guide eye movements in the absence of a corresponding conscious percept.     

Differential effects of shared attention on perception of heading and 3-D object motion

When a person moves in a straight line through a stationary environment, the images of object surfaces move in a radial pattern away from a single point. This point, known as the focus of expansion (FOE), corresponds to the person's direction of motion. People judge their heading from image motion quite well in this situation. They perform most accurately when they can see the region around the FOE, which contains the most useful information for this task. Furthermore, a large moving object in the scene has no effect on observer heading judgments unless it obscures the FOE. Therefore, observers may obtain the most accurate heading judgments by focusing their attention on the region around the FOE. However, in many situations (e.g., driving), the observer must pay attention to other moving objects in the scene (e.g., cars and pedestrians) to avoid collisions. These objects may be located far from the FOE in the visual field. We tested whether people can accurately judge their heading and the three-dimensional (3-D) motion of objects while paying attention to one or the other task. The results show that differential allocation of attention affects people's ability to judge 3-D object motion much more than it affects their ability to judge heading. This suggests that heading judgments are computed globally, whereas judgments about object motion may require more focused attention.     

Perception of self-motion from visual flow

Accurate and efficient control of self-motion is an important requirement for our daily behavior. Visual feedback about self-motion is provided by optic flow. Optic flow can be used to estimate the direction of self-motion (‘heading’) rapidly and efficiently. Analysis of oculomotor behavior reveals that eye movements usually accompany self-motion. Such eye movements introduce additional retinal image motion so that the flow pattern on the retina usually consists of a combination of self-movement and eye movement components. The question of whether this ‘retinal flow’ alone allows the brain to estimate heading, or whether an additional ‘extraretinal’ eye movement signal is needed, has been controversial. This article reviews recent studies that suggest that heading can be estimated visually but extraretinal signals are used to disambiguate problematic situations. The dorsal stream of primate cortex contains motion processing areas that are selective for optic flow and self-motion. Models that link the properties of neurons in these areas to the properties of heading perception suggest possible underlying mechanisms of the visual perception of self-motion.     

Perception of translational heading from optical flow

Radial patterns of optical flow produced by observer translation could be used to perceive the direction of self-movement during locomotion, and a number of formal analyses of such patterns have recently appeared. However, there is comparatively little empirical research on the perception of heading from optical flow, and what data there are indicate surprisingly poor performance, with heading errors on the order of 5 degrees-10 degrees. We examined heading judgments during translation parallel, perpendicular, and at oblique angles to a random-dot plane, varying observer speed and dot density. Using a discrimination task, we found that heading accuracy improved by an order of magnitude, with 75%-correct thresholds of 0.66 degrees in the highest speed and density condition and 1.2 degrees generally. Performance remained high with displays of 63-10 dots, but it dropped significantly with only 2 dots; there was no consistent speed effect and no effect of angle of approach to the surface. The results are inconsistent with theories based on the local focus of outflow, local motion parallax, multiple fixations, differential motion parallax, and the local maximum of divergence. But they are consistent with Gibson's (1950) original global radial outflow hypothesis for perception of heading during translation.     

Perception of heading without retinal optic flow

How do we determine where we are heading during visually controlled locomotion? Psychophysical research has shown that humans are quite good at judging their travel direction, or heading, from retinal optic flow. Here we show that retinal optic flow is sufficient, but not necessary, for determining heading. By using a purely cyclopean stimulus (random dot cinematogram), we demonstrate heading perception without retinal optic flow. We also show that heading judgments are equally accurate for the cyclopean stimulus and a conventional optic flow stimulus, when the two are matched for motion visibility. The human visual system thus demonstrates flexible, robust use of available visual cues for perceiving heading direction.     

Why eye movements and perceptual factors have to be controlled in studies on "representational momentum"

In order to study memory of the final position of a smoothly moving target, Hubbard (e.g., Hubbard and Bharucha, 1988) presented smooth stimulus motion and used motor responses. In contrast, Freyd (e.g., Freyd and Finke, 1984) presented implied stimulus motion and used the method of constant stimuli. The same forward error was observed in both paradigms. However, the processes underlying the error may be very different. When smooth stimulus motion is followed by smooth pursuit eye movements, the forward error is associated with asynchronous processing of retinal and extraretinal information. In the absence of eye movements, no forward displacement is observed with smooth motion. In contrast, implied motion produces a forward error even without eye movements, suggesting that observers extrapolate the next target step when successive target presentations are far apart. Finally, motor responses produce errors that are not observed with perceptual judgments, indicating that the motor system may compensate for neuronal latencies.     

Heading judgments in minimal environments: the value of a heuristic when invariants are rare

Observers made systematic heading judgments in two experiments simulating their translation through an environment with only two trees. When those trees converged or decelerated apart, observers tended to follow the invariant information and make heading judgments outside the near member of the pair. When those trees accelerated apart, however, observers tended to follow the heuristic information and make judgments outside the far member, although this result was tempered by the angular separation between the trees and their relative acceleration. The simultaneous existence and use of invariants and heuristics are discussed in terms of different metatheoretical approaches to perception.     

Comment and Reply Why eye movements and perceptual factors have to be controlled in studies on “representational momentum”

In order to study memory of the final position of a smoothly moving target, Hubbard (e.g., Hubbard & Bharucha, 1988) presented smooth stimulus motion and used motor responses. In contrast, Freyd (e.g., Freyd & Finke, 1984) presented implied stimulus motion and used the method of constant stimuli. The same forward error was observed in both paradigms. However, the processes underlying the error may be very different. When smooth stimulus motion is followed by smooth pursuit eye movements, the forward error is associated with asynchronous processing of retinal and extraretinal information. In the absence of eye movements, no forward displacement is observed with smooth motion. In contrast, implied motion produces a forward error even without eye movements, suggesting that observers extrapolate the next target step when successive target presentations are far apart. Finally, motor responses produce errors that are not observed with perceptual judgments, indicating that the motor system may compensate for neuronal latencies.     

The perception of surface curvature from optical motion

Observers viewed the optical flow field of a rotating quadric surface patch and were required to match its perceived structure by adjusting the shape of a stereoscopically presented surface. In Experiment 1, the flow fields included rigid object rotations and constant flow fields with patterns of image acceleration that had no possible rigid interpretation. In performing their matches, observers had independent control of two parameters that determined the surface shape. One of these, called the shape characteristic, is defined as the ratio of the two principle curvatures and is independent of object size. The other, called curvedness, is defined as the sum of the squared principle curvatures and depends on the size of the object. Adjustments of shape characteristic were almost perfectly accurate for both motion conditions. Adjustments of curvedness, on the other hand, were systematically overestimated and were not highly correlated with the simulated curvedness of the depicted surface patch. In Experiment 2, the same flow fields were masked with a global pattern of curl, divergence, or shear, which disrupted the first-order spatial derivatives of the image velocity field, while leaving the second-order spatial derivatives invariant. The addition of these masks had only negligible effects on observers’ performance. These findings suggest that observers’ judgments of three-dimensional surface shape from motion are primarily determined by the second-order spatial derivatives of the instantaneous field of image displacements.     

Perceiving heading with different retinal regions and types of optic flow

We examined the ability to use optic flow to judge heading when different parts of the retina are stimulated and when the specified heading is in different directions relative to the display. To do so, we manipulated retinal eccentricity (the angle between the fovea and the center of the stimulus) and heading eccentricity (the angle between the specified heading and the center of the stimulus) independently. Observers viewed two sequences of moving dots that simulated translation through a random cloud of dots. They reported whether the direction of translation—the heading—in the second sequence was to the left or right of the direction in the first sequence. The results revealed a large and consistent effect of heading eccentricity: Judgments were much more accurate with radial flow fields (small heading eccentricities) than with lamellar fields (large heading eccentricities), regardless of the part of the retina being stimulated. The results also revealeda smaller and less consistent effect of retinal eccentricity: With radial flow (small heading eccentricities), judgments were more accurate when the stimulus was presented near the fovea. The variation of heading thresholds from radial to lamellar flow fields is predicted by a simple model of two-dimensional motion discrimination. The fact that the predictions are accurate implies that the human visual system is equally efficient at processing radial and lamellar flow fields. In addition, efficiency is reasonably constant no matter what part of the retina is being stimulated.     

Shedding some light on catching in the dark: perceptual mechanisms for catching fly balls

To catch a lofted ball, a catcher must pick up information that guides locomotion to where the ball will land. The acceleration of tangent of the elevation angle of the ball (AT) has received empirical support as a possible source of this information. Little, however, has been said about how the information is detected. Do catchers fixate on a stationary point, or do they track the ball with their gaze? Experiment 1 revealed that catchers use eye and head movements to track the ball. This means that if AT is picked up retinally, it must be done by means of background motion. Alternatively, AT could be picked up by extraretinal mechanisms, such as the vestibular and proprioceptive systems. In Experiment 2, catchers reliably ran to intercept luminous fly balls in the dark, that is, in absence of a visual background, under both binocular and monocular viewing conditions. This indicates that the optical information is not detected by a retinal mechanism alone.     

Retinal flow is sufficient for steering during observer rotation

How do people control locomotion while their eyes are simultaneously rotating? A previous study found that during simulated rotation, they can perceive a straight path of self-motion from the retinal flow pattern, despite conflicting extraretinal information, on the basis of dense motion parallax and reference objects. Here we report that the same information is sufficient for active control ofjoystick steering. Participants steered toward a target in displays that simulated a pursuit eye movement. Steering was highly inaccurate with a textured ground plane (motion parallax alone), but quite accurate when an array of posts was added (motion parallax plus reference objects). This result is consistent with the theory that instantaneous heading is determined from motion parallax, and the path of self-motion is determined by updating heading relative to environmental objects. Retinal flow is thus sufficient for both perceiving self-motion and controlling self-motion with a joystick; extraretinal and positional information can also contribute, but are not necessary.     

Extraretinal and retinal amplitude and phase errors during Filehne illusion and path perception

Pursuit eye movements give rise to retinal motion. To judge stimulus motion relative to the head, the visual system must correct for the eye movement by using an extraretinal, eye-velocity signal. Such correction is important in a variety of motion estimation tasks including judgments of object motion relative to the head and judgments of self-motion direction from optic flow. The Filehne illusion (where a stationary object appears to move opposite to the pursuit) results from a mismatch between retinal and extraretinal speed estimates. A mismatch in timing could also exist. Speed and timing errors were investigated using sinusoidal pursuit eye movements. We describe a new illusion--the slalom illusion--in which the perceived direction of self-motion oscillates left and right when the eyes move sinusoidally. A linear model is presented that determines the gain ratio and phase difference of extraretinal and retinal signals accompanying the Filehne and slalom illusions. The speed mismatch and timing differences were measured in the Filehne and self-motion situations using a motion-nulling procedure. Timing errors were very small for the Filehne and slalom illusions. However, the ratios of extraretinal to retinal gain were consistently less than 1, so both illusions are the consequence of a mismatch between estimates of retinal and extraretinal speed. The relevance of the results for recovering the direction of self-motion during pursuit eye movements is discussed.     

Steering, optic flow, and the respective importance of depth and retinal motion distribution

Movement through an environment produces an optical spatiotemporal pattern, known as a flow field. When visually guiding movement using a flow field, do humans make use of information about the distance of constituent elements? Employing a novel active steering task, we examined the use of depth (height-in-scene and disparity) and the role of the retinal motion distribution in the perceptual control of heading from flow. We found that retinal motion distribution, rather than depth order, has the primary role in determining the accuracy of steering.