Spatio-temporal priority revisited: the role of feature identity and similarity for object correspondence in apparent motion

We live in a dynamic environment in which objects change location over time. To maintain stable object representations the visual system must determine how newly sampled information relates to existing object representations, the correspondence problem. Spatiotemporal information is clearly an important factor that the visual system takes into account when solving the correspondence problem, but is feature information irrelevant as some theories suggest? The Ternus display provides a context in which to investigate solutions to the correspondence problem. Two sets of three horizontally aligned disks, shifted by one position, were presented in alternation. Depending on how correspondence is resolved, these displays are perceived either as one disk "jumping" from one end of the row to the other (element motion) or as a set of three disks shifting back and forth together (group motion). We manipulated a feature (e.g., color) of the disks such that, if features were taken into account by the correspondence process, it would bias the resolution of correspondence toward one version or the other. Features determined correspondence, whether they were luminance-defined or not. Moreover, correspondence could be established on the basis of similarity, when features were not identical between alternations. Finally, the stronger the feature information supported a certain correspondence solution the more it dominated spatiotemporal information.     

Differential activation solution to the motion correspondence problem

The correspondence problem arises in motion perception when more than one motion path is possible for discontinuously presented visual elements. Ullman's (1979) "minimal mapping" solution to the correspondence problem, for which costs are assigned to competing motion paths on the basis of element affinities (e.g., greater affinity for elements that are closer together), is distinguished from a solution based on the differential activation of directionally selective motion detectors. The differential activation account was supported by evidence that path length affects detector activation in a paradignm for which motion correspondence is not a factor. Effects on detector activation in this paradigm also were the basis for the successful prediction of path luminance effects on solutions to the motion correspondence problem. Finally, the differential activation account was distinguished from minimal mapping theory by an experiment showing that the perception of an element moving simultaneously in two directions does not depend on whether the two motions are matched in path-length determined affinity; it is sufficient that the activation of detectors responding to each of the two motion directions is above the threshold level required for the motions to be perceived. Implications of the differential activation solution are discussed for the stability of perceived motions once they are established, and the adaptation of perceived and unperceived motions.     

Perceiving motion and rigid structure from optic flow: A combined weak-perspective and polar-perspective approach

Structure-from-motion algorithms based on weak-perspective projection have many interesting properties and could serve as a basis for a model of human perception of motion and structure from motion (M&SFM). There is some psychophysical evidence, however, that points to discrepancies between what can be accomplished with these algorithms and the performance of human subjects in certain M&SFM tasks. In light of this evidence, this paper presents a mechanism that both takes advantage of all the possibilities offered by a weak-perspective approach and behaves in a manner that is in close correspondence with human performance in M&SFM tasks. It consists of a novel weak-perspective—based method operating at small visual angles and a complementary, perspective-projection—based method operating at larger visual angles.     

Correspondence matching in long-range apparent motion precedes featural analysis

It has been suggested that correspondence matching in long-range motion is mediated by a perceptually high-level, 'intelligent' system. This suggestion is based on findings that long-range motion can be perceived between stimuli that could not be detected by lower-level motion mechanisms acting on Fourier motion energy, and that correspondence matching is affected by featural similarities between motion tokens that would be invisible to low-level (Fourier) motion detectors. Here, the effects of spatial-frequency content, color, and binocular disparity on correspondence matching are investigated. It is shown that the effects of featural matches between motion tokens develop only over time and lag behind the effects of the relative proximity between motion tokens in the retinal projection. This suggests that correspondence matching in long-range apparent motion is mediated by a mechanism which acts initially on the retinal coordinates of the motion tokens only, but may be biased to favor matching tokens that are featurally similar through a slower top-down influence by higher-level processes.     

Visible persistence and form correspondence in Ternus apparent motion.

Visual stimuli remain visible for some time after their physical offset (visible persistence). Visible persistence has been hypothesized to play an important role in determining the pattern of correspondence matching in the Ternus apparent-motion display. In this display, one or more elements reappears in overlapping locations at different times, whereas another element appears alternately to the right or the left of these elements. Usually either the elements are perceived to move coherently as a group (group motion), or one element may be perceived to hop over one or more other elements (element motion). According to the visible-persistence account of the perceptual organization of the Ternus display, element motion is seen when the temporal gap between elements in overlapping locations is small enough to be bridged by visible persistence; if it is not, group motion is seen. We conducted four experiments to test this visible-persistence account. In Experiments 1 and 2, a form correspondence cue (line length) was introduced to bias the visual system toward the element-motion interpretation, while visible persistence was either reduced or eliminated. The element-motion percept dominated despite the elimination of visible persistence. In Experiments 3 and 4, we found that Ternus elements presented without interruption, and thus presumably persisting over time, can be perceived in group motion. Together, the results indicate that visible persistence is neither necessary nor sufficient to account for the pattern of correspondence matches in the Ternus display.     

Object-based apparent motion

The interpretation of a dynamic visual scene requires integrating information within frames (grouping and completion) and across frames (correspondence matching). Fragmentary views of objects were used in five experiments. These views could not be matched with each other by any similarity transformation on the basis of their explicit visual features, but their completed versions were related by a rotational transformation. When the fragmentary images were successively presented to observers, it was found that they produced apparent motion in the picture plane and in depth. Thus, apparent motion is capable of establishing correspondence at the level of perceptually recovered objects in three-dimensional space.     

Some additional predictions and further tests of the Marr-Ullman model of motion perception

The Marr-Ullman model for motion detection in the human visual system functions by means of the dual input of polarity-specific edge detectors and luminance change detectors. Moulden and Begg (1986) found a polarity-specific motion aftereffect which they claimed provided support for this dual input model. The logic of their experiment is examined, and it is shown that several additional predictions arise from the Marr-Ullman model, which were not supported by Moulden and Begg's study. A more powerful experiment was carried out and these additional predictions were disconfirmed, although the polarity-specific effect did emerge. A consideration of alternative explanations of this effect led to a second experiment in which an attempt was made to discover the actual determinants of the effect. This revealed that polarity-specific units are unlikely to play any part in the phenomenon. It was concluded, in the light of this and other evidence, that one of a class of alternative models is more likely to be the actual mechanism for motion perception. However, careful consideration of the Marr-Ullman model indicated that it may be untestable in principle if various differentially weighted levels of neural integration are envisaged.     

Visual apparent motion can be modulated by task-irrelevant lexical information

Previous studies have repeatedly demonstrated the impact of Gestalt structural grouping principles upon the parsing of motion correspondence in ambiguous apparent motion. Here, by embedding Chinese characters in a visual Ternus display that comprised two stimulus frames, we showed that the perception of visual apparent motion can be modulated by activation of task-irrelevant lexical representations. Each frame had two disks, with the second disk of the first frame and the first disk of the second frame being presented at the same location. Observers could perceive either "element motion," in which the endmost disk is seen as moving back and forth while the middle disk at the central position remains stationary, or "group motion," in which both disks appear to move laterally as a whole. More reports of group motion, as opposed to element motion, were obtained when the embedded characters formed two-character compound words than when they formed nonwords, although this lexicality effect appeared to be attenuated by the use of the same characters at the overlapping position across the two frames. Thus, grouping of visual elements in a changing world can be guided by both structural principles and prior world knowledge, including lexical information.     

Perception of elliptic biological motion

We tested the ability of the mature visual system for discrimination between types of elliptic biological motion on the basis of event kinematics. Healthy adult volunteers were presented with point-light displays depicting elliptic motion when only a single dot, a moving point-light arm, or a whole point-light human figure was visible. The displays were created in accordance with the two-thirds power kinematic law (natural motion), whereas the control displays violated this principle (unnatural motion). On each trial, participants judged whether the display represented natural or unnatural motion. The findings indicate that adults are highly sensitive to violation of the two-thirds power kinematic law. Notably, participants can easily discriminate between natural and unnatural motions without recognising the stimuli, which suggests that people implicitly use kinematic information. Most intriguing, event recognition seems to diminish the capacity to judge whether event kinematics is unnatural. We discuss possible ways for a cross-talk between perception and production of biological movement, and the brain mechanisms involved in biological motion processing.     

The dynamical foundations of motion pattern formation: stability,selective adaptation,and perceptual continuity

A dynamical model is used to show that global motion pattern formation for several different apparent motion stimuli can be embodied in the stable distribution of activation over a population of concurrently activated, directionally selective motion detectors. The model, which is based on motion detectors being interactive, noisy, and self-stabilizing, accounts for such phenomena as bistability, spontaneous switching, hysteresis, and selective adaptation. Simulations show that dynamical solutions to the motion correspondence problem for a bistable stimulus (two qualitatively different patterns are formed) apply as well to the solution for a monostable stimulus (only one pattern is formed) and highlight the role of interactions among sequentially stimulated detectors in establishing the state dependence and, thereby, the temporal persistence of percepts.     

Visible persistence and form correspondence in Ternus apparent motion

Visual stimuli remain visible for some time after their physical offset (visible persistence) Visible persistence has been hypothesized to play an important role in determining the pattern of correspondence matching in the Ternus apparent-motion display. In this display, one or more elements reappears in overlapping locations at different times, whereas another element appears alternately to the right or the left of these elements. Usually either the elements are perceived to move coherently as a group (group motion), or one element may be perceived to hop over one or more other elements (element motion). According to the visible-persistence account of the perceptual organization of the Ternus display, element motion is seen when the temporal gap between elements in overlapping locations is small enough to be bridged by visible persistence; if it is not, group motion is seen. We conducted four experiments to test this visible-persistence account. In Experiments 1 and 2, a form correspondence cue (line length) was introduced to bias the visual system toward the element-motion interpretation, while visible persistence was either reduced or eliminated. The element-motion percept dominated despite the elimination of visible persistence. In Experiments 3 and 4, we found that Ternus elements presented without interruption, and thus presumably persisting over time, can be perceived in group motion. Together, the results indicate that visible persistence is neither necessary nor sufficient to account for the pattern of correspondence matches in the Ternus display.     

Color correspondence in apparent motion

To maintain figural identity during motion perception, the visual system must match images over space and time. Correct matching requires a metric for identifying "corresponding" images, those representing the same physical object. To test whether matching is based on achromatic (black/white) polarity and chromatic (red/green) color, observers viewed an ambiguous motion display and judged the path of apparent motion. Matching preserved black/white identity regardless of whether frames were viewed binocularly or dichoptically. Red/green identity was also preserved, but coherence of motion depended in part on the number of frames in the motion sequence and on the background luminance. These results suggest that correspondence is computed by a weighted metric containing terms for image features coded early in visual processing.     

The perception of biological motion across apertures

To understand the visual analysis of biological motion, subjects viewed dynamic, stick figure renditions of a walker, car, or scissors through apertures. As a result of the aperture problem, the motion of each visible edge was ambiguous. Subjects readily identified the human figure but were unable to identify the car or scissors through invisible apertures. Recognition was orientation specific and robust across a range of stimulus durations, and it benefited from limb orientation cues. The results support the theory that the visual system performs spatially global analyses to interpret biological motion displays.     

Apparent motion of stimuli presented stroboscopically during pursuit movement of the eye

This paper reports the first phase of a research program on visual perception of motion patterns characteristic of living organisms in locomotion. Such motion patterns in animals and men are termed here as biological motion. They are characterized by a far higher degree of complexity than the patterns of simple mechanical motions usually studied in our laboratories. In everyday perceptions, the visual information from biological motion and from the corresponding figurative contour patterns (the shape of the body) are intermingled. A method for studying information from the motion pattern per se without interference with the form aspect was devised. In short, the motion of the living body was represented by a few bright spots describing the motions of the main joints. It is found that 10–12 such elements in adequate motion combinations in proximal stimulus evoke a compelling impression of human walking, running, dancing, etc. The kinetic-geometric model for visual vector analysis originally developed in the study of perception of motion combinations of the mechanical type was applied to these biological motion patterns. The validity of this model in the present context was experimentally tested and the results turned out to be highly positive.     

Biological motion cues trigger reflexive attentional orienting

The human visual system is extremely sensitive to biological signals around us. In the current study, we demonstrate that biological motion walking direction can induce robust reflexive attentional orienting. Following a brief presentation of a central point-light walker walking towards either the left or right direction, observers' performance was significantly better on a target in the walking direction compared with that in the opposite direction even when participants were explicitly told that walking direction was not predictive of target location. Interestingly, the effect disappeared when the walker was shown upside-down. Moreover, the reflexive attentional orienting could be extended to motions of other biological entities but not inanimate objects, and was not due to the viewpoint effect of the point-light figure. Our findings provide strong evidence that biological motion cues can trigger reflexive attentional orienting, and highlight the intrinsic sensitivity of the human visual attention system to biological signals.     

Visual motion disambiguation by a subliminal sound

There is growing interest in the effect of sound on visual motion perception. One model involves the illusion created when two identical objects moving towards each other on a two-dimensional visual display can be seen to either bounce off or stream through each other. Previous studies show that the large bias normally seen toward the streaming percept can be modulated by the presentation of an auditory event at the moment of coincidence. However, no reports to date provide sufficient evidence to indicate whether the sound bounce-inducing effect is due to a perceptual binding process or merely to an explicit inference resulting from the transient auditory stimulus resembling a physical collision of two objects. In the present study, we used a novel experimental design in which a subliminal sound was presented either 150 ms before, at, or 150 ms after the moment of coincidence of two disks moving towards each other. The results showed that there was an increased perception of bouncing (rather than streaming) when the subliminal sound was presented at or 150 ms after the moment of coincidence compared to when no sound was presented. These findings provide the first empirical demonstration that activation of the human auditory system without reaching consciousness affects the perception of an ambiguous visual motion display.     

The integration of auditory and visual motion signals at threshold

To interpret our environment, we integrate information from all our senses. For moving objects, auditory and visual motion signals are correlated and provide information about the speed and the direction of the moving object. We investigated at what level the auditory and the visual modalities interact and whether the human brain integrates only motion signals that are ecologically valid. We found that the sensitivity for identifying motion was improved when motion signals were provided in both modalities. This improvement in sensitivity can be explained by probability summation. That is, auditory and visual stimuli are combined at a decision level, after the stimuli have been processed independently in the auditory and the visual pathways. Furthermore, this integration is direction blind and is not restricted to ecologically valid motion signals.     

Neural dynamics of motion perception: Direction fields,apertures, and resonant grouping

A neural network model of global motion segmentation by visual cortex is described. Called the motion boundary contour system (BCS), the model clarifies how ambiguous local movements on a complex moving shape are actively reorganized into a coherent global motion signal, Unlike many previous researchers, we analyze how a coherent motion signal is imparted to all regions of a moving figure, not only to regions at which unambiguous motion signals exist. The model hereby suggests a solution to the global aperture problem. The motion BCS describes how preprocessing of motion signals by a motion oriented contrast (MOC) filter is joined to long-range cooperative grouping mechanisms in a motion cooperative-competitive (MOCC) loop to control phenomena such as motion capture. The motion BCS is computed in parallel with the static BCS of Grossberg and Mingolla (1985a, 1985b, 1987). Homologous properties of the motion BCS and the static BCS, specialized to process motion directions and static orientations, respectively, support a unified explanation of many data about static form perception and motion form perception that have heretofore been unexplained or treated separately. Predictions about microscopic computational differences of the parallel cortical streams V1 → MT and V1 → V2 → MT are made— notably, the magnocellular thick stripe and parvocellular interstripe streams. It is shown how the motion BCS can compute motion directions that may be synthesized from multiple orientations with opposite directions of contrast. Interactions of model simple cells, complex cells, hyper-complex cells, and bipole cells are described, with special emphasis given to new functional roles in direction disambiguation for endstopping at multiple processing stages and to the dynamic interplay of spatially short-range and long-range interactions.     

A depictive neural model for the representation of motion verbs

In this paper, we present a depictive neural model for the representation of motion verb semantics in neural models of visual awareness. The problem of modelling motion verb representation is shown to be one of function application, mapping a set of given input variables defining the moving object and the path of motion to a defined output outcome in the motion recognition context. The particular function-applicative implementation and consequent recognition model design presented are seen as arising from a noun-adjective recognition model enabling the recognition of colour adjectives as applied to a set of shapes representing objects to be recognised. The presence of such a function application scheme and a separately implemented position identification and path labelling scheme are accordingly shown to be the primitives required to enable the design and construction of a composite depictive motion verb recognition scheme. Extensions to the presented design to enable the representation of transitive verbs are also discussed.     

Surface discontinuity is critical in a moving observer's perception of objects' depth order and relative motion from retinal image motion.

The visual system perceptually decomposes retinal image motion into three basic components that are ecologically significant for the human observer: object depth, object motion, and self motion. Using this conceptual framework, we explored the relationship between them by examining perception of objects' depth order and relative motion during self motion. We found that the visual system obeyed what we call the parallax-sign constraint, but in different ways depending on whether the retinal image motion contained velocity discontinuity or not. When velocity discontinuity existed (e.g. in dynamic occlusion, transparent motion), the subject perceptually interpreted image motion as relative motion between surfaces with stable depth order. When velocity discontinuity did not exist, he/she perceived depth-order reversal but no relative motion. The results suggest that the existence of surface discontinuity or of multiple surfaces indexed by velocity discontinuity inhibits the reversal of global depth order.