Optic flow helps humans learn to navigate through synthetic environments

Self-movement through an environment generates optic flow, a potential source of heading information. But it is not certain that optic flow is sufficient to support navigation, particularly navigation along complex, multi-legged paths. To address this question, we studied human participants who navigated synthetic environments with and without salient optic flow. Participants used a keyboard to control realistic simulation of self-movement through computer-rendered, synthetic environments. Because these environments comprised series of identically textured virtual corridors and intersections, participants had to build up some mental representation of the environment in order to perform. The impact of optic flow on learning was examined in two experiments. In experiment 1, participants learned to navigate multiple T-junction mazes with and without accompanying optic flow. Optic flow promoted faster learning, mainly by preventing disorientation and backtracking in the maze. In experiment 2, participants found their way around a virtual city-block environment, experiencing two different kinds of optic flow as they went. By varying the rate at which the display was updated, we created optic flow that was either fluid or choppy. Here, fluid optic flow (as compared with choppy optic flow) enabled participants to locate a remembered target position more accurately. When other cues are unavailable, optic flow can be a significant aid in wayfinding. Among other things, optic flow can facilitate path integration, which involves updating a mental representation of place by combining the trajectories of previously travelled paths [corrected].     

Perception of heading without retinal optic flow

How do we determine where we are heading during visually controlled locomotion? Psychophysical research has shown that humans are quite good at judging their travel direction, or heading, from retinal optic flow. Here we show that retinal optic flow is sufficient, but not necessary, for determining heading. By using a purely cyclopean stimulus (random dot cinematogram), we demonstrate heading perception without retinal optic flow. We also show that heading judgments are equally accurate for the cyclopean stimulus and a conventional optic flow stimulus, when the two are matched for motion visibility. The human visual system thus demonstrates flexible, robust use of available visual cues for perceiving heading direction.     

Stability in young infants' discrimination of optic flow

Although considerable progress has been made in understanding how adults perceive their direction of self-motion, or heading, from optic flow, little is known about how these perceptual processes develop in infants. In 3 experiments, the authors explored how well 3- to 6-month-old infants could discriminate between optic flow patterns that simulated changes in heading direction. The results suggest that (a) prior to the onset of locomotion, the majority of infants discriminate between optic flow displays that simulate only large (> 22 deg.) changes in heading, (b) there is minimal development in sensitivity between 3 and 6 months, and (c) optic flow alone is sufficient for infants to discriminate heading. These data suggest that spatial abilities associated with the dorsal visual stream undergo prolonged postnatal development and may depend on locomotor experience.     

Perception of self-motion from visual flow

Accurate and efficient control of self-motion is an important requirement for our daily behavior. Visual feedback about self-motion is provided by optic flow. Optic flow can be used to estimate the direction of self-motion (‘heading’) rapidly and efficiently. Analysis of oculomotor behavior reveals that eye movements usually accompany self-motion. Such eye movements introduce additional retinal image motion so that the flow pattern on the retina usually consists of a combination of self-movement and eye movement components. The question of whether this ‘retinal flow’ alone allows the brain to estimate heading, or whether an additional ‘extraretinal’ eye movement signal is needed, has been controversial. This article reviews recent studies that suggest that heading can be estimated visually but extraretinal signals are used to disambiguate problematic situations. The dorsal stream of primate cortex contains motion processing areas that are selective for optic flow and self-motion. Models that link the properties of neurons in these areas to the properties of heading perception suggest possible underlying mechanisms of the visual perception of self-motion.     

The discrimination of heading from optic flow is not retinally invariant.

Three experiments were conducted to determine whether the discrimination of heading from optic flow is retinally invariant and to determine the importance of acuity in accounting for heading eccentricity effects. In the first experiment, observers were presented with radial flow fields simulating forward translation through a three-dimensional volume of dots. The flow fields subtended 10 degrees of visual angle and were presented at 0 degree, 10 degrees, 20 degrees, and 40 degrees of retinal eccentricity. The observers were asked to indicate whether the simulated movement was to the right or the left of a target that appeared at the end of the display sequence. Eye movements were monitored with an electrooculogram apparatus. In a second experiment, static acuity thresholds were derived for each of the observers at the same retinal eccentricities. There was a significant increase in heading detection thresholds with retinal eccentricity (from 0.92 degree at 0 degree retinal eccentricity to 3.47 degrees at 40 degrees). An analysis of covariance indicated that the variation in sensitivity to radial flow, as a function of retinal eccentricity, is independent of acuity. Similar results were obtained when the Vernier acuity of observers was measured. These results suggest that the discrimination of heading from radial flow is not retinally invariant.     

Perceiving self-motion in depth: The role of stereoscopic motion and changing-size cues

During self-motions, different patterns of optic flow are presented to the left and right eyes. Previous research has, however, focused mainly on the self-motion information contained in a single pattern of optic flow. The present experiments investigated the role that binocular disparity plays in the visual perception of self-motion, showing that the addition of stereoscopic cues to optic flow significantly improves forward linear vection in central vision. Improvements were also achieved by adding changingsize cues to sparse (but not dense) flow patterns. These findings showed that assumptions in the heading literature that stereoscopic cues facilitate self-motion only when the optic flow has ambiguous depth ordering do not apply to vection. Rather, it was concluded that both stereoscopic and changingsize cues provide additional motion-in-depth information that is used in perceiving self-motion.     

Perceiving heading with different retinal regions and types of optic flow

We examined the ability to use optic flow to judge heading when different parts of the retina are stimulated and when the specified heading is in different directions relative to the display. To do so, we manipulated retinal eccentricity (the angle between the fovea and the center of the stimulus) and heading eccentricity (the angle between the specified heading and the center of the stimulus) independently. Observers viewed two sequences of moving dots that simulated translation through a random cloud of dots. They reported whether the direction of translation—the heading—in the second sequence was to the left or right of the direction in the first sequence. The results revealed a large and consistent effect of heading eccentricity: Judgments were much more accurate with radial flow fields (small heading eccentricities) than with lamellar fields (large heading eccentricities), regardless of the part of the retina being stimulated. The results also revealeda smaller and less consistent effect of retinal eccentricity: With radial flow (small heading eccentricities), judgments were more accurate when the stimulus was presented near the fovea. The variation of heading thresholds from radial to lamellar flow fields is predicted by a simple model of two-dimensional motion discrimination. The fact that the predictions are accurate implies that the human visual system is equally efficient at processing radial and lamellar flow fields. In addition, efficiency is reasonably constant no matter what part of the retina is being stimulated.     

The discrimination of heading from optic flow is not retinally invariant

Three experiments were conducted to determine whether the discrimination of heading from optic flow is retinally invariant and to determine the importance of acuity in accounting for heading eccentricity effects. In the first experiment, observers were presented with radial flow fields simulating forward translation through a three-dimensional volume of dots. The flow fields subtended 10° of visual angle and were presented at 0°, 10°, 20°, and 40° of retinal eccentricity. The observers were asked to indicate whether the simulated movement was to the right or the left of a target that appeared at the end of the display sequence. Eye movements were monitored with an electrooculogram apparatus. In a second experiment, static acuity thresholds were derived for each of the observers at the same retinal eccentricities. There was a significant increase in heading detection thresholds with retinal eccentricity (from 0.92° at 0° retinal eccentricity to 3.47° at 40°). An analysis of covariance indicated that the variation in sensitivity to radial flow, as a function of retinal eccentricity, is independent of acuity. Similar results were obtained when the Vernier acuity of observers was measured. These results suggest that the discrimination of heading from radial flow is not retinally invariant.     

Judgments of path, not heading, guide locomotion

To steer a course through the world, people are almost entirely dependent on visual information, of which a key component is optic flow. In many models of locomotion, heading is described as the fundamental control variable; however, it has also been shown that fixating points along or near one's future path could be the basis of an efficient control solution. Here, the authors aim to establish how well observers can pinpoint instantaneous heading and path, by measuring their accuracy when looking at these features while traveling along straight and curved paths. The results showed that observers could identify both heading and path accurately (approximately 3 degrees ) when traveling along straight paths, but on curved paths they were more accurate at identifying a point on their future path (approximately 5 degrees ) than indicating their instantaneous heading (approximately 13 degrees ). Furthermore, whereas participants could track changes in the tightness of their path, they were unable to accurately track the rate of change of heading. In light of these results, the authors suggest it is unlikely that heading is primarily used by the visual system to support active steering.     

Perceiving curvilinear heading in the presence of moving objects

Four experiments were directed at understanding the influence of multiple moving objects on curvilinear (i.e., circular and elliptical) heading perception. Displays simulated observer movement over a ground plane in the presence of moving objects depicted as transparent, opaque, or black cubes. Objects either moved parallel to or intersected the observer's path and either retreated from or approached the moving observer. Heading judgments were accurate and consistent across all conditions. The significance of these results for computational models of heading perception and for information in the global optic flow field about observer and object motion is discussed.     

A neural model of how the brain computes heading from optic flow in realistic scenes

Visually-based navigation is a key competence during spatial cognition. Animals avoid obstacles and approach goals in novel cluttered environments using optic flow to compute heading with respect to the environment. Most navigation models try either explain data, or to demonstrate navigational competence in real-world environments without regard to behavioral and neural substrates. The current article develops a model that does both. The ViSTARS neural model describes interactions among neurons in the primate magnocellular pathway, including V1, MT⁺, and MSTd. Model outputs are quantitatively similar to human heading data in response to complex natural scenes. The model estimates heading to within 1.5° in random dot or photo-realistically rendered scenes, and within 3° in video streams from driving in real-world environments. Simulated rotations of less than 1°/s do not affect heading estimates, but faster simulated rotation rates do, as in humans. The model is part of a larger navigational system that identifies and tracks objects while navigating in cluttered environments.     

Reference frames in spatial updating when body-based cues are absent

The current study investigated the reference frame used in spatial updating when idiothetic cues to self-motion were minimized (desktop virtual reality). In Experiment 1, participants learned a layout of eight objects from a single perspective (learning heading) in a virtual environment. After learning, they were placed in the same virtual environment and used a keyboard to navigate to two of the learned objects (visible) before pointing to a third object (invisible). We manipulated participants’ starting orientation (initial heading) and final orientation (final heading) before pointing, to examine the reference frame used in this task. We found that participants used the initial heading and the learning heading to establish reference directions. In Experiment 2, the procedure was almost the same as in Experiment 1 except that participants pointed to objects relative to an imagined heading that differed from their final heading in the virtual environment. In this case, pointing performance was only affected by alignment with the learning heading. We concluded that the initial heading played an important role in spatial updating without idiothetic cues, but the representation established at this heading was transient and affected by the interruption of spatial updating; the learning heading, on the other hand, corresponded to an enduring representation which was used consistently.     

Cooperativity, priming, and 3-D surface detection from optic flow

Three experiments were conducted to determine whether the mechanisms responsible for the detection of three-dimensional (3-D) surfaces from optic flow operate in a cooperative manner. The first experiment was conducted to determine whether a hysteresis effect occurs for 3-D surface detection from optic flow. The results of the first experiment demonstrated a hysteresis effect with lower thresholds occurring for decreasing texture density than for increasing texture density. The second experiment used a priming methodology to determine whether this form of cooperativity was based on preactivation of shear detectors or preactivation of 2-D motion detectors. The results suggest that only shear detectors were primed. The third experiment utilized a similar methodology to determine whether a surface representation would produce a priming effect. We found no evidence that the priming effect found in the second experiment was the result of preactivation of a generic representation of the test stimuli. The results of the experiments, considered together, suggest priming of the mechanisms responsible for recovering shear.     

Use of cognitive versus perceptual heading during imagined locomotion depends on the response mode

Three experiments investigated whether the systematic errors previously observed in a triangle-completion task were caused by failures to form and update a cognitive heading or by use of perceived heading (even though an updated cognitive heading was available) during the response. These errors were replicated when participants indicated the origin of triangular paths they had imagined walking by turning their bodies toward the origin, but not when they responded verbally. The results indicate that participants are capable of updating their cognitive heading using imagined movements and suggest that the systematic errors previously observed were a result of the strong attachment of responses such as turns to a perceptual representation of the physical body.     

Frames of reference in mobile augmented reality displays

In 3 experiments, the authors investigated spatial updating in augmented reality environments. Participants learned locations of virtual objects on the physical floor. They were turned to appropriate facing directions while blindfolded before making pointing judgments (e.g., "Imagine you are facing X. Point to Y"). Experiments manipulated the angular difference between the learning heading and the imagined heading and between the actual heading and the imagined heading. The effect of actual-imagined on pointing latency was observed for naive users but not for users with brief training or instructions concerning the fact that objects can move with body movements. The results indicated that naive users used an environment-stabilized reference frame to access information arrays, but with experience and instruction the nature of the representation changed from an environment stabilized to a body stabilized reference frame.     

Responsiveness to terrestrial optic flow in infancy: does locomotor experience play a role?

Human infants show a peak in postural compensation to optic flow at approximately nine months of age. The current experiment tested whether the magnitude of visual-postural coupling in 9-month-olds increases when terrestrial optic flow is added to a moving room. A secondary objective was to explore whether locomotor experience plays any role in enhancing responsiveness to the additional terrestrial information. Ninety-one infants (experienced creepers, nascent creepers, and prelocomotors) were exposed to two conditions of optic flow: global optic flow (G) and global optic flow minus terrestrial optic flow (G-T). The additional terrestrial optic flow led to significantly higher visual-postural coupling. Consistent with previous findings, locomotor experience had no effect on responsiveness to the G-T condition, though there was weak evidence that the nascent creepers were more strongly influenced by the difference between flow conditions than the other infants. Unexpectedly, the prelocomotor females showed significantly lower visual-postural coupling than the prelocomotor males. These findings support the notion that the ground provides an important source of information for the control of posture and locomotion. The findings also suggest that locomotor experience most likely helps to functionalize smaller (partial), rather than larger (global), optic flow fields for postural control.     

Static scene analysis for the perception of heading

In the current study, we explored observers' use of two distinct analyses for determining their direction of motion, or heading: a scene-based analysis and a motion-based analysis. In two experiments, subjects viewed sequentially presented, paired digitized images of real-world scenes and judged the direction of heading; the pairs were presented with various interstimulus intervals (ISIs). In Experiment 1, subjects could determine heading when the two frames were separated with a 1,000-ms ISI, long enough to eliminate apparent motion. In Experiment 2, subjects performed two tasks, a path-of-motion task and a memory-load task, under three different ISIs, 50 ms, 500 ms, and 1,000 ms. Heading accuracy decreased with an increase in ISI. Increasing memory load influenced heading judgments only for the longer ISI when motion-based information was not available. These results are consistent with the hypothesis that the scene-based analysis has a coarse spatial representation, is a sustained temporal process, and is capacity limited, whereas the motion-based analysis has a fine spatial resolution, is a transient temporal process, and is capacity unlimited.     

Role of optical flow field asymmetry in the perception of heading during linear motion

During linear translation through a stationary environment, the pattern of optical flow generated on each retina is symmetrical when the head is aligned with the heading, but during lateral gaze the optical flow is asymmetric. We assessed whether human subjects could use the magnitude of this asymmetry to determine the direction of heading during passive translation through a 3-D environment. When allowed to move their heads in order to look in the direction of self-motion, subjects indicated their heading precisely and accurately. When the head was locked in alignment with the misaligned body, and gaze adjustments were not allowed, responses were quite precise, but showed a large bias which increased with increasing heading angle.     

Effects of optic flow on the kinematics of human gait: a comparison of young and older adults

This experiment studied the effect of imposed optic flow on human locomotion. Six young and 6 older adults were exposed to various patterns of optic flow while walking in a moving hallway. Results showed few cases of impaired postural control (staggers, parachute reactions). No falls were recorded. Kinematic patterns of gait were altered when vision was absent or inconsistent optic flow was presented: Ninety two percent of the subjects' mean step velocity differed from their step velocities under normal vision. Compared with imposed central flow, peripheral optic flow was not dominant in inducing kinematic changes. Characteristic gait profiles were obtained, depending on flow direction. Global backward flow tended to slow down step velocity, whereas subjects' step velocity increased during conditions of forward flow. The results suggest that subjects attempted to match their own walking speed to the velocity of the moving visual scenes. It is concluded that in an uncluttered environment, imposed optic flow has a modulating rather than a destabilizing effect on human locomotion.