Are objects required for object-files? Roles of segmentation and spatiotemporal continuity in computing object persistence

Two central tasks of visual processing are (1) to segment undifferentiated retinal images into discrete objects, and (2) to represent those objects as the same persisting individuals over time and motion. Here we explore the interaction of these two types of processing in the context of object files—mid-level visual representations that “stick” to moving objects on the basis of spatiotemporal properties. Object files can be revealed by object-specific preview benefits (OSPBs), wherein a “preview” of information on a moving object speeds the recognition of that information at a later point when it appears again on the same object (compared to when it reappears on a different moving object), beyond display-wide priming. Here we explore the degree of segmentation required to establish object files in the first place. Surprisingly, we find that no explicit segmentation is required until after the previews disappear, when using purely motion-defined objects (consisting of random elements on a random background). Moreover, OSPBs are observed in such displays even after moderate (but not long) delays between the offset of the preview information and the onset of the motion. These effects indicate that object files can be established without initial static segmentation cues, so long as there is spatiotemporal continuity between the previews and the eventual appearance of the objects. We also find that top-down strategies can sometimes mimic OSPBs, but that these strategies can be eliminated by novel manipulations. We discuss how these results alter our understanding of the nature of object files, and also why researchers must take care to distinguish “true OSPBs” from “illusory OSPBs”.     

The relationship between object files and conscious perception

Object files (OFs) are hypothesized mid-level representations which mediate our conscious perception of persisting objects-e.g. telling us 'which went where'. Despite the appeal of the OF framework, not previous research has directly explored whether OFs do indeed correspond to conscious percepts. Here we present at least one case wherein conscious percepts of 'which went where' in dynamic ambiguous displays diverge from the analogous correspondence computed by the OF system. Observers viewed a 'bouncing/streaming' display in which two identical objects moved such that they could have either bounced off or streamed past each other. We measured two dependent variables: (1) an explicit report of perceived bouncing or streaming; and (2) an implicit 'object-specific preview benefit' (OSPB), wherein a 'preview' of information on a specific object speeds the recognition of that information at a later point when it appears again on the same object (compared to when it reappears on a different object), beyond display-wide priming. When the displays were manipulated such that observers had a strong bias to perceive streaming (on over 95% of the trials), there was nevertheless a strong OSPB in the opposite direction-such that the object files appeared to have 'bounced' even though the percept 'streamed'. Given that OSPBs have been taken as a hallmark of the operation of object files, the five experiments reported here suggest that in at least some specialized (and perhaps ecologically invalid) cases, conscious percepts of 'which went where' in dynamic ambiguous displays can diverge from the mapping computed by the object-file system.     

Visual memory for feature bindings: The disruptive effect of responding to new perceptual input

In the current study, we examined how short-term memory for location–identity feature bindings is influenced by subsequent cognitive and perceptual processing demands. Previous work has shown that memory performance for feature bindings can be disrupted by the presentation of subsequent visual information, particularly when this information is similar to that held in memory. The present study demonstrates that memory performance for feature bindings can be profoundly disrupted by also requiring a response to visual information presented subsequent to the visual memory array. Across five experiments, memory for a location–identity binding was substantially impaired following a localization response to a following item that matched the location but mismatched the identity of the memory target. The results point to an important role for action in the episodic integration processes that control short-term visual memory performance.     

Temporal structure coding with and without awareness

In order to interpret a constantly changing environment, visual events far apart in space and time must be integrated into a unified percept. While spatial properties of invisible signals are known to be encoded without awareness, the fate of temporal properties remains largely unknown. Here, we probed temporal integration for two distinct motion stimuli that were either visible or rendered invisible using continuous flash suppression. We found that when invisible, both the direction of apparent motion and the gender of point-light walkers were processed only when defined across short time periods (i.e., respectively 100 ms and 1000 ms). This limitation was not observed under full visibility. These similar findings at two different hierarchical levels of processing suggest that temporal integration windows shrink in the absence of perceptual awareness. We discuss this phenomenon as a key prediction of the global neuronal workspace and the information integration theories of consciousness.     

Perceptual and decisional contributions to audiovisual interactions in the perception of apparent motion: a signal detection study

Motion information available to different sensory modalities can interact at both perceptual and post-perceptual (i.e., decisional) stages of processing. However, to date, researchers have only been able to demonstrate the influence of one of these components at any given time, hence the relationship between them remains uncertain. We addressed the interplay between the perceptual and post-perceptual components of information processing by assessing their influence on performance within the same experimental paradigm. We used signal detection theory to discriminate changes in perceptual sensitivity (d') from shifts in response criterion (c) in performance on a detection (Experiment 1) and a classification (Experiment 2) task regarding the direction of auditory apparent motion streams presented in noise. In the critical conditions, a visual motion distractor moving either leftward or rightward was presented together with the auditory motion. The results demonstrated a significant decrease in sensitivity to the direction of the auditory targets in the crossmodal conditions as compared to the unimodal baseline conditions that was independent of the relative direction of the visual distractor. In addition, we also observed significant shifts in response criterion, which were dependent on the relative direction of the distractor apparent motion. These results support the view that the perceptual and decisional components involved in audiovisual interactions in motion processing can coexist but are largely independent of one another.     

Further evidence that object-based correspondence effects are primarily modulated by object location not by grasping affordance

Tipper, Paul and Hayes found object-based correspondence effects for door-handle stimuli for shape judgments but not colour. They reasoned that a grasping affordance is activated when judging dimensions related to a grasping action (shape), but not for other dimensions (colour). Cho and Proctor, however, found the effect with respect to handle position when the bases of the door handles were centred (so handles were positioned left or right; the base-centred condition) but not when the handles were centred (the object-centred condition), suggesting that the effect is driven by object location, not grasping affordance. We conducted an independent replication of Cho and Proctor's design, but with behavioural and event-related potential measures. Participants made shape judgments in Experiment 1 and colour judgments in Experiment 2 on the same door-handle objects. Correspondence effects on response time and errors were obtained in both experiments for the base-centred condition but not the object-centred condition. Effects were absent in the P1 and N1 data, which are consistent with the hypothesis of little binding between visual processing of grasping component and action. These findings question the grasping-affordance view but support a spatial-coding view, suggesting that correspondence effects are modulated primarily by object location.     

Is motion extrapolation employed in multiple object tracking? Tracking as a low-level, non-predictive function

In a series of five experiments, we investigated whether visual tracking mechanisms utilize prediction when recovering multiple reappearing objects. When all objects abruptly disappeared and reappeared mid-trajectory, it was found that (a) subjects tracked better when objects reappeared at their loci of disappearance than when they reappeared in their extrapolated trajectories, (b) disappearance episodes ranging from 150 to 900 ms had virtually no differential effect on performance, (c) tracking deteriorated monotonically as a function of displacement magnitude during disappearance, and (d) tracking did not depend on whether objects moved in predictable paths. Even objects that reappeared backward in their trajectories were tracked dramatically better than objects that reappeared in their extrapolated trajectories. When all objects disappeared and reappeared in ways that implicated the presence of an occluder (i.e., with occlusion and disocclusion cues along fixed contours), tracking again was not predictive, and performance deteriorated with increased displacement. When objects reappeared predictably in 75% of trials, they were still tracked better when they reappeared at their points of disappearance. Theoretical implications of a non-predictive multiple object tracking mechanism are discussed.     

Common mechanisms in apparent motion perception and visual pattern matching

There are close functional similarities between apparent motion perception and visual pattern matching. In particular, striking functional similarities have been demonstrated between perception of rigid objects in apparent motion and purely mental transformations of visual size and orientation used in comparisons of objects with respect to shape but regardless of size and orientation. In both cases, psychophysical data suggest that differences in visual size are resolved as differences in depth, such that transformation of size is done by translation in depth. Furthermore, the process of perceived or imagined translation appears to be stepwise additive such that a translation over a long distance consists of a sequence of smaller translations, the durations of these steps being additive. Both perceived and imagined rotation also appear to be stepwise additive, and combined transformations of size and orientation appear to be done by alternation of small steps of pure translation and small steps of pure rotation. The functional similarities suggest that common mechanisms underlie perception of apparent motion and purely mental transformations. In line with this suggestion, functional brain imaging has isolated neural structures in motion area MT used in mental transformation of size.     

Object correspondence across brief occlusion is established on the basis of both spatiotemporal and surface feature cues

The correspondence problem is a classic issue in vision and cognition. Frequent perceptual disruptions, such as saccades and brief occlusion, create gaps in perceptual input. How does the visual system establish correspondence between objects visible before and after the disruption? Current theories hold that object correspondence is established solely on the basis of an object’s spatiotemporal properties and that an object’s surface feature properties (such as color or shape) are not consulted in correspondence operations. In five experiments, we tested the relative contributions of spatiotemporal and surface feature properties to establishing object correspondence across brief occlusion. Correspondence operations were strongly influenced both by the consistency of an object’s spatiotemporal properties across occlusion and by the consistency of an object’s surface feature properties across occlusion. These data argue against the claim that spatiotemporal cues dominate the computation of object correspondence. Instead, the visual system consults multiple sources of relevant information to establish continuity across perceptual disruption.     

Object manipulation facilitates kind-based object individuation of shape-similar objects

Five experiments investigated the importance of shape and object manipulation when 12-month-olds were given the task of individuating objects representing exemplars of kinds in an event-mapping design. In Experiments 1 and 2, results of the study from Xu, Carey, and Quint (2004, Experiment 4) were partially replicated, showing that infants were able to individuate two natural-looking exemplars from different categories, but not two exemplars from the same category. In Experiment 3, infants failed to individuate two shape-similar exemplars (from Pauen, 2002a) from different categories. However, Experiment 4 revealed that allowing infants to manipulate objects shortly before the individuation task enabled them to individuate shape-similar objects from different categories. In Experiment 5, allowing object manipulation did not induce infants to individuate natural-looking objects from the same category. These findings suggest that object manipulation facilitates kind-based individuation of shape-similar objects by 12-month-olds.     

Learning to like it: Aesthetic perception of bodies,movements and choreographic structure

Appreciating human movement can be a powerful aesthetic experience. We have used apparent biological motion to investigate the aesthetic effects of three levels of movement representation: body postures, movement transitions and choreographic structure. Symmetrical (ABCDCBA) and asymmetrical (ABCDBCA) sequences of apparent movement were created from static postures, and were presented in an artificial grammar learning paradigm. Additionally, “good” continuation of apparent movements was manipulated by changing the number of movement path reversals within a sequence. In an initial exposure phase, one group of participants saw only symmetrical sequences, while another group saw only asymmetrical sequences. In a subsequent test phase, both groups rated all sequences on an aesthetic evaluation scale. We found that posture, movement, and choreographic structure all influenced aesthetic ratings. Separate ratings for the static body postures presented individually showed that both groups preferred a posture that maximized spatial symmetry. Ratings for the experimental sequences showed that both groups gave higher ratings to symmetrical sequences with “good” continuation and lower ratings to sequences with many path reversals. Further, participants who had been initially familiarized with asymmetrical sequences showed increased liking for asymmetrical sequences, suggesting a structural mere exposure effect. Aesthetic preferences thus depend on body postures, apparent movement continuation and choreographic structure. We propose a hierarchical model of aesthetic perception of human movement with distinct processing levels for body postures, movements and choreographic structure.     

Priming infants to attend to color and pattern information in an individuation task

Wilcox (Cognition 72 (1999) 125) reported that infants are more sensitive to form than surface features when individuating objects in occlusion events: it is not until 7.5 months that infants spontaneously use pattern information, and 11.5 months that they spontaneously use color information, as the basis for object individuation. The present research assessed the extent to which infants' sensitivity to surface features could be increased under more supportive conditions. More specifically, we examined whether younger infants could be primed to draw on color and pattern features in an individuation task if they were first shown the functional value of attending to color and pattern information (i.e. the color or the pattern of an object predicted the function it would engage in). Five experiments were conducted with infants 4.5 to 9.5 months of age. The main findings were that 9.5- and 7.5-month-olds could be primed to use color information, and 5.5- and 4.5-month-olds could be primed to attend to pattern information, after viewing the function events. The results are discussed in terms of the kinds of experiences that can lead to increased sensitivity to surface features and the mechanisms that support feature priming in young infants.     

Conceptual knowledge attenuates viewpoint dependency in visual object recognition

Learning verbal semantic knowledge for objects has been shown to attenuate recognition costs incurred by changes in view from a learned viewpoint. Such findings were attributed to the semantic or meaningful nature of the learned verbal associations. However, recent findings demonstrate surprising benefits to visual perception after learning even noninformative verbal labels for stimuli. Here we test whether learning verbal information for novel objects, independent of its semantic nature, can facilitate a reduction in viewpoint-dependent recognition. To dissociate more general effects of verbal associations from those stemming from the semantic nature of the associations, participants learned to associate semantically meaningful (adjectives) or nonmeaningful (number codes) verbal information with novel objects. Consistent with a role of semantic representations in attenuating the viewpoint-dependent nature of object recognition, the costs incurred by a change in viewpoint were attenuated for stimuli with learned semantic associations relative to those associated with nonmeaningful verbal information. This finding is discussed in terms of its implications for understanding basic mechanisms of object perception as well as the classic viewpoint-dependent nature of object recognition.