Using movement and intentions to understand human activity

During perception, people segment continuous activity into discrete events. They do so in part by monitoring changes in features of an ongoing activity. Characterizing these features is important for theories of event perception and may be helpful for designing information systems. The three experiments reported here asked whether the body movements of an actor predict when viewers will perceive event boundaries. Body movements were recorded using a magnetic motion tracking system and compared with viewers’ segmentation of his activity into events. Changes in movement features were strongly associated with segmentation. This was more true for fine-grained than for coarse-grained boundaries, and was strengthened when the stimulus displays were reduced from live-action movies to simplified animations. These results suggest that movement variables play an important role in the process of segmenting activity into meaningful events, and that the influence of movement on segmentation depends on the availability of other information sources.     

Event perception: a mind-brain perspective

People perceive and conceive of activity in terms of discrete events. Here the authors propose a theory according to which the perception of boundaries between events arises from ongoing perceptual processing and regulates attention and memory. Perceptual systems continuously make predictions about what will happen next. When transient errors in predictions arise, an event boundary is perceived. According to the theory, the perception of events depends on both sensory cues and knowledge structures that represent previously learned information about event parts and inferences about actors' goals and plans. Neurological and neurophysiological data suggest that representations of events may be implemented by structures in the lateral prefrontal cortex and that perceptual prediction error is calculated and evaluated by a processing pathway, including the anterior cingulate cortex and subcortical neuromodulatory systems.     

Activation of human motion processing areas during event perception

Observers are able to segment continuous everyday activity into meaningful parts. This ability may be related to processing low-level visual cues, such as changes in motion. To address this issue, the present study combined measurement of evoked responses to event boundaries with functional identification of the extrastriate motion complex (MT+) and the frontal eye field (FEF), two regions related to motion perception and eye movements. The results provided strong evidence that MT+ is activated by event boundaries: Individuals' MT+ regions showed strong responses to event boundaries, and MT+ was collocated with a lateral posterior region that responded at event boundaries. The evidence regarding the FEF was less conclusive: The FEF showed reliable but relatively reduced responses to event boundaries, but the FEF was medial and superior to a frontal area that responded at event boundaries. These results suggest that motion cues, and possibly eye movements, may play key roles in event structure perception.     

Visual representation of malleable and rigid objects that deform as they rotate

Most studies and theories of object recognition have addressed the perception of rigid objects. Yet, physical objects may also move in a nonrigid manner. A series of priming studies examined the conditions under which observers can recognize novel views of objects moving nonrigidly. Observers were primed with 2 views of a rotating object that were linked by apparent motion or presented statically. The apparent malleability of the rotating prime object varied such that the object appeared to be either malleable or rigid. Novel deformed views of malleable objects were primed when falling within the object's motion path. Priming patterns were significantly more restricted for deformed views of rigid objects. These results suggest that moving malleable objects may be represented as continuous events, whereas rigid objects may not. That is, object representations may be "dynamically remapped" during the analysis of the object's motion.     

Discrete events as units of perceived time

In visual images, we perceive both space (as a continuous visual medium) and objects (that inhabit space). Similarly, in dynamic visual experience, we perceive both continuous time and discrete events. What is the relationship between these units of experience? The most intuitive answer may be similar to the spatial case: time is perceived as an underlying medium, which is later segmented into discrete event representations. Here we explore the opposite possibility--that our subjective experience of time itself can be influenced by how durations are temporally segmented, beyond more general effects of change and complexity. We show that the way in which a continuous dynamic display is segmented into discrete units (via a path shuffling manipulation) greatly influences duration judgments, independent of psychophysical factors previously implicated in time perception, such as overall stimulus energy, attention and predictability. It seems that we may use the passage of discrete events--and the boundaries between them--in our subjective experience as part of the raw material for inferring the strength of the underlying "current" of time.     

Perceiving Stability in a Changing World: Combining Shots and Intergrating Views in Motion Pictures and the Real World

One of the central debates in the study of media concerns the relationship between our perception of real-world events and mediated events such as motion pictures. A number of authors have argued convincingly that both rely on similar natural perceptual processes. Here we extend this argument by reviewing research on scene perception showing that the similarity between film and natural perception includes a common tendency to perceive events as continuous in the face of large view-to-view inconsistencies. This research shows that observers are frequently unable to detect large unexpected changes that occur between views, even in objects within their focus of attention. This is true both in mediated stimuli such as motion pictures and during staged real-world interactions. Thus, perception of continuity may be an inference that proceeds in spite of impossible between-view changes, both in motion pictures and real-world scenes.     

Perception of motion affects language processing

Recently developed accounts of language comprehension propose that sentences are understood by constructing a perceptual simulation of the events being described. These simulations involve the re-activation of patterns of brain activation that were formed during the comprehender's interaction with the world. In two experiments we explored the specificity of the processing mechanisms required to construct simulations during language comprehension. Participants listened to (and made judgments on) sentences that described motion in a particular direction (e.g. "The car approached you"). They simultaneously viewed dynamic black-and-white stimuli that produced the perception of movement in the same direction as the action specified in the sentence (i.e. towards you) or in the opposite direction as the action specified in the sentence (i.e. away from you). Responses were faster to sentences presented concurrently with a visual stimulus depicting motion in the opposite direction as the action described in the sentence. This suggests that the processing mechanisms recruited to construct simulations during language comprehension are also used during visual perception, and that these mechanisms can be quite specific.     

Segmentation in the perception and memory of events

People make sense of continuous streams of observed behavior in part by segmenting them into events. Event segmentation seems to be an ongoing component of everyday perception. Events are segmented simultaneously at multiple timescales, and are grouped hierarchically. Activity in brain regions including the posterior temporal and parietal cortex and lateral frontal cortex increases transiently at event boundaries. The parsing of ongoing activity into events is related to the updating of working memory, to the contents of long-term memory, and to the learning of new procedures. Event segmentation might arise as a side effect of an adaptive mechanism that integrates information over the recent past to improve predictions about the near future.     

Aging and the segmentation of narrative film

The perception of event structure in continuous activity is important for everyday comprehension. Although the segmentation of experience into events is a normal concomitant of perceptual processing, previous research has shown age differences in the ability to perceive structure in naturalistic activity, such as a movie of someone washing a car. However, past research has also shown that older adults have a preserved ability to comprehend events in narrative text, which suggests that narrative may improve the event processing of older adults. This study tested whether there are age differences in event segmentation at the intersection of continuous activity and narrative: narrative film. Younger and older adults watched and segmented a narrative film, The Red Balloon, into coarse and fine events. Changes in situational features, such as changes in characters, goals, and objects predicted segmentation. Analyses revealed little age-difference in segmentation behavior. This suggests the possibility that narrative structure supports event understanding for older adults.     

Categories and boundaries in speech and music*

Perceptual categories and boundaries arise when Ss respond to continuous variation on a physical dimension in a discontinuous fashion. It is more difficult to discriminate between members of the same category than to discriminate between members of different categories, even though the amount of physical difference between both pairs is the same. Speech stimuli have been the sole class of auditory signals to yield such perception; for example, each different consonant phoneme serves as a category label. Experiment I demonstrates that categories and boundaries occur for both speechand nonspeech stimuli differing in rise time. Experiment II shows that rise time cues categorical differences in both complex and simple nonspeech waveforms. Taken together, these results suggest that certain aspects of speech perception are intimately related to processes and mechanisms exploited in other domains. The many categories in speech may be based on categories that occur elsewhere in auditory perception.     

EVENT SEGMENTATION

ABSTRACT— One way to understand something is to break it up into parts. New research indicates that segmenting ongoing activity into meaningful events is a core component of perception and that this has consequences for memory and learning. Behavioral and neuroimaging data suggest that event segmentation is automatic and that people spontaneously segment activity into hierarchically organized parts and subparts. This segmentation depends on the bottom-up processing of sensory features such as movement and on the top-down processing of conceptual features such as actors' goals. How people segment activity affects what they remember later; as a result, those who identify appropriate event boundaries during perception tend to remember more and to learn more proficiently.     

Individuation of Actions from Continuous Motion

Three studies explored how infants parse a stream of motion into distinct actions. Results show that infants (a) can perceptually discriminate different actions performed by a puppet and (b) can individuate and enumerate heterogeneous sequences of such actions (e.g., jump-fall-jump) when the actions are separated by brief motionless pauses, but (c) are not able to individuate such actions when embedded within a continuous stream of motion. Combined with previous research showing that infants can individuate homogeneous actions from an ongoing stream of motion, these findings suggest that infants can use repeating patterns of motion in the perceptual input to define action boundaries. Results have implications as well for infants' conceptual structure for actions.     

Perception of whether an object can be carried through an aperture depends on anticipated speed

We investigated whether anticipated speed of locomotion through an aperture influences perception of whether an object can be carried through that aperture. Participants reported whether they would be able to carry objects through an aperture (a) if they were to attempt to walk through the aperture and (b) if they were to attempt to run through the aperture. Furthermore, they did so when the object was held but not seen and when the object was seen but not held. In general, perception was influenced by object width and by anticipated speed but not by perceptual modality. Perceptual boundaries occurred at smaller object widths when participants anticipated running through the aperture than when they anticipated walking through the aperture. The results build on work showing that perception of affordances is influenced by kinetic potential as well as geometric properties and that perception may be supported by the detection of modality-neutral stimulation patterns.     

The impact of continuity editing in narrative film on event segmentation

Filmmakers use continuity editing to engender a sense of situational continuity or discontinuity at editing boundaries. The goal of this study was to assess the impact of continuity editing on how people perceive the structure of events in a narrative film and to identify brain networks that are associated with the processing of different types of continuity editing boundaries. Participants viewed a commercially produced film and segmented it into meaningful events, while brain activity was recorded with functional magnetic resonance imaging (MRI). We identified three degrees of continuity that can occur at editing locations: edits that are continuous in space, time, and action; edits that are discontinuous in space or time but continuous in action; and edits that are discontinuous in action as well as space or time. Discontinuities in action had the biggest impact on behavioral event segmentation, and discontinuities in space and time had minor effects. Edits were associated with large transient increases in early visual areas. Spatial-temporal changes and action changes produced strikingly different patterns of transient change, and they provided evidence that specialized mechanisms in higher order perceptual processing regions are engaged to maintain continuity of action in the face of spatiotemporal discontinuities. These results suggest that commercial film editing is shaped to support the comprehension of meaningful events that bridge breaks in low-level visual continuity, and even breaks in continuity of spatial and temporal location.     

Action Interrupted: Movement and Breakpoints in the Processing of Motion Violations in Toddlers and Adults

From early in development, segmenting events unfolding in the world in meaningful ways renders input more manageable and facilitates interpretation and prediction. Yet, little is known about how children process action structure in events composed of multiple coarse-grained actions. More importantly, little is known about the time course of action processing in young children or about the specific features that recruit attention. This is particularly true when we consider action that pauses unexpectedly—as actions sometimes do—thereby violating the expectation of a continuous unfolding of motion. We assessed visual preference to intact and disrupted actions embedded within a multiaction event in toddlers and adults. In 1 condition, pauses were inserted at intact action boundaries, whereas in the other condition, they disrupted action. Attention in both groups was recruited to the disrupted relative to intact events. Time-course analyses, however, revealed developmental differences in sensitivity to the movement features (e.g., motion, pauses, and transitions) of disrupted events.     

The formation of rhythmic categories and metric priming

Two experiments on categorical rhythm perception are reported, the object of which was to investigate how listeners perceive discrete rhythmic categories while listening to rhythms performed on a continuous time scale. This is studied by considering the space of all temporal patterns (all possible rhythms made up of three intervals) and how they, in perception, are partitioned into categories, ie where the boundaries of these categories are located. This process of categorisation is formalised as the mapping from the continuous space of a series of time intervals to a discrete, symbolic domain of integer-ratio sequences. The methodological framework uses concepts from mathematics and psychology (eg convexity and entropy) that allow precise characterisations of the empirical results. In the first experiment, twenty-nine participants performed an identification task with 66 rhythmic stimuli (a systematic sampling of the performance space). The results show that listeners do not just perceive the time intervals between onsets of sounds as placed in a homogeneous continuum. Instead, they can reliably identify rhythmic categories, as a chronotopic time clumping map reveals. In a second experiment, the effect of metric priming was studied by presenting the same stimuli but preceded with a duple or triple metre subdivision. It is shown that presenting patterns in the context of a metre has a large effect on rhythmic categorisation: the presence of a specific musical metre primes the perception of specific rhythmic patterns.     

Perceiving causality after the fact: postdiction in the temporal dynamics of causal perception

In simple dynamic events we can easily perceive not only motion, but also higher-level properties such as causality, as when we see one object collide with another. Several researchers have suggested that such causal perception is an automatic and stimulus-driven process, sensitive only to particular sorts of visual information, and a major research project has been to uncover the nature of these visual cues. Here, rather than investigating what information affects causal perception, we instead explore the temporal dynamics of when certain types of information are used. Surprisingly, we find that certain visual events can determine whether we perceive a collision in an ambiguous situation even when those events occur after the moment of potential 'impact' in the putative collision has already passed. This illustrates a type of postdictive perception: our conscious perception of the world is not an instantaneous moment-by-moment construction, but rather is formed by integrating information presented within short temporal windows, so that new information which is obtained can influence the immediate past in our conscious awareness. Such effects have been previously demonstrated for low-level motion phenomena, but the present results demonstrate that postdictive processes can influence higher-level event perception. These findings help to characterize not only the 'rules' of causal perception, but also the temporal dynamics of how and when those rules operate.     

Patterns of neural response to emotive stimuli distinguish the different symptom dimensions of obsessive-compulsive disorder

Despite its heterogeneous symptomatology, obsessive-compulsive disorder (OCD) is currently conceptualized as a unitary diagnostic entity. Recent factor-analytic studies have identified several OCD symptom dimensions that are associated with different demographic variables, comorbidity, patterns of genetic transmission, and treatment response. Functional abnormalities in neural systems important for emotion perception, including the orbitofrontal cortex, lateral prefrontal cortex, anterior cingulate gyrus, and limbic regions, have been reported in OCD. In this review, we discuss the extent to which neurobiological markers may distinguish these different symptom dimensions and whether specific symptom dimensions, such as contamination/washing, are associated with abnormalities in emotion and, in particular, disgust, perception in OCD. Also discussed are findings that indicate that anxiety can be induced in healthy volunteers in response to OCD symptom-related material, and that associated increases in activity within neural systems important for emotion perception occur to washing- and hoarding-related material in particular in these subjects. Further examination of neural responses during provocation of different symptom dimensions in OCD patients will help determine the extent to which specific abnormalities in neural systems underlying emotion perception are associated with different symptom dimensions and predict treatment response in OCD.     

Visual target detection is impaired at event boundaries

Boundaries between meaningful events are key moments in comprehending human action. At these points, viewers may focus on the event's contents at the expense of other information. We tested whether visual detection was impaired at those moments perceivers judged to be boundaries between events. Short animated football clips were used as stimulus material, and event boundaries were imposed by having the ball change possession. In a first experiment, we found that possession changes were perceived to be event boundaries. In a second experiment, participants were asked to keep track of 4 of 10 players and to watch for 120 ms probes appearing either at an event boundary or a nonboundary. Probe detection was less accurate at event boundaries. This result suggests that the segmentation of ongoing activity into events corresponds with the regulation of attention over time.