Online prediction of others' actions: the contribution of the target object, action context and movement kinematics

Previous research investigated the contributions of target objects, situational context and movement kinematics to action prediction separately. The current study addresses how these three factors combine in the prediction of observed actions. Participants observed an actor whose movements were constrained by the situational context or not, and object-directed or not. After several steps, participants had to indicate how the action would continue. Experiment 1 shows that predictions were most accurate when the action was constrained and object-directed. Experiments 2A and 2B investigated whether these predictions relied more on the presence of a target object or cues in the actor's movement kinematics. The target object was artificially moved to another location or occluded. Results suggest a crucial role for kinematics. In sum, observers predict actions based on target objects and situational constraints, and they exploit subtle movement cues of the observed actor rather than the direct visual information about target objects and context.     

But I thought it was Mickey Mouse: the effects of new postevent information on 18-month-olds' memory

Two experiments examined the effects of postevent information on 18-month-olds' event memory. Experiment 1 (N=60) explored whether children's memory was reinstated when action information was eliminated from the reinstatement and only object information was introduced. Experiment 2 (N=48) examined children's recall when either (a). information about the objects' target actions was replaced with new action information or (b). the original training objects were replaced with new objects. In an elicited-imitation paradigm, children were trained to perform six target actions, watched a video reinstatement 10 weeks later, and were tested for recall 24 h after reinstatement. Two results were found. First, a video reminder eliminating action information reinstated children's memory as effectively as a video containing object and action information. Second, children were reminded of their past training when during reinstatement action information was preserved and new objects were presented but were not reminded when object information was preserved and new actions were presented.     

Motion tracking of parents’ infant‐ versus adult‐directed actions reveals general and action‐specific modulations

Parents tend to modulate their movements when demonstrating actions to their infants. Thus far, these modulations have primarily been quantified by human raters and for entire interactions, thereby possibly overlooking the intricacy of such demonstrations. Using optical motion tracking, the precise modulations of parents’ infant‐directed actions were quantified and compared to adult‐directed actions and between action types. Parents demonstrated four novel objects to their 14‐month‐old infants and adult confederates. Each object required a specific action to produce a unique effect (e.g. rattling). Parents were asked to demonstrate an object at least once before passing it to their demonstration partner, and they were subsequently free to exchange the object as often as desired. Infants’ success at producing the objects’ action‐effects was coded during the demonstration session and their memory of the action‐effects was tested after a several‐minute delay. Indicating general modulations across actions, parents repeated demonstrations more often, performed the actions in closer proximity and demonstrated action‐effects for longer when interacting with their infant compared to the adults. Meanwhile, modulations of movement size and velocity were specific to certain action‐effect pairs. Furthermore, a ‘just right’ modulation of proximity was detected, since infants’ learning, memory, and parents’ prior evaluations of their infants’ motor abilities, were related to demonstrations that were performed neither too far from nor too close to the infants. Together, these findings indicate that infant‐directed action modulations are not solely overall exaggerations but are dependent upon the characteristics of the to‐be learned actions, their effects, and the infant learners.     

Hitting ability and perception of object’s size: evidence for a negative relation

We examined the relation between motor performance and perception of object’s size in near space. The general task was to repeatedly hit a target by means of pointing movements and to estimate target’s size. In contrast to the results of previous studies, Experiment 1 and Experiment 2 revealed a negative relation between action ability and perceived target size: Participants who hit the target relatively often and whose motor variability was relatively low judged targets to be smaller than did participants whose motor performance was relatively poor. In Experiment 3, the size judgments were made in the presence of the target before, as well as after, pointing movements. The target was judged as smaller when it was easy, rather than difficult, to hit before as well as after the movement. Altogether, these results indicate that under certain conditions, an increased action ability reduces the apparent size of the actions’ target objects.     

Infant perception of causal motion produced by humans and inanimate objects

Both the movements of people and inanimate objects are intimately bound up with physical causality. Furthermore, in contrast to object movements, causal relationships between limb movements controlled by humans and their body displacements uniquely reflect agency and goal-directed actions in support of social causality. To investigate the development of sensitivity to causal movements, we examined the looking behavior of infants between 9 and 18 months of age when viewing movements of humans and objects. We also investigated whether individual differences in gender and gross motor functions may impact the development of the visual preferences for causal movements. In Experiment 1, infants were presented with walking stimuli showing either normal body translation or a “moonwalk” that reversed the horizontal motion of body translations. In Experiment 2, infants were presented with unperformable actions beyond infants’ gross motor functions (i.e., long jump) either with or without ecologically valid body displacement. In Experiment 3, infants were presented with rolling movements of inanimate objects that either complied with or violated physical causality. We found that female infants showed longer looking times to normal walking stimuli than to moonwalk stimuli, but did not differ in their looking time to movements of inanimate objects and unperformable actions. In contrast, male infants did not show sensitivity to causal movement for either category. Additionally, female infants looked longer at social stimuli of human actions than male infants. Under the tested circumstances, our findings indicate that female infants have developed a sensitivity to causal consistency between limb movements and body translations of biological motion, only for actions with previous visual and motor exposures, and demonstrate a preference toward social information.     

Direct and indirect effects of action on object classification

We report three experiments in which name verification responses to either objects (Experiments 1 and 2) or hand movements (Experiment 3) were compared with action decisions, where participants verified whether an object is typically used in the way described by a verbal label. In Experiments 1 and 2, we report that action decisions show more consistent and larger effects of the congruency of either a handgrip or a type of movement than do name verification responses, although there was some effect of the congruency of the handgrip on name verification. In Experiment 3, we demonstrate that the congruency of the object being moved affects both action and name verification responses to hand movements. We discuss the]data relative to accounts of how actions and names are accessed by visually presented objects and in relation to work on the information called upon in classification tasks.     

Movements or targets: What makes an action in action–effect learning?

According to ideomotor theory, actions become linked to the sensory feedback they contingently produce, so that anticipating the feedback automatically evokes the action it typically results from. Numerous recent studies have provided evidence in favour of such action–effect learning but left an important issue unresolved. It remains unspecified to what extent action–effect learning is based on associating effect-representations to representations of the performed movements or to representations of the targets at which the behaviour aimed at. Two experiments were designed to clarify this issue. In an acquisition phase, participants learned the contingency between key presses and effect tones. In a following test phase, key–effect and movement–effect relations were orthogonally assessed by changing the hand–key mapping for one half of the participants. Experiment 1 showed precedence for target–effect over movement–effect learning in a forced-choice RT task. In Experiment 2, target–effect learning was also shown to influence the outcome of response selection in a free-choice task. Altogether, the data indicate that both movement–effect and target–effect associations contribute to the formation of action–effect linkages—provided that movements and targets are likewise contingently related to the effects.     

Beyond the information given: Infants’ transfer of actions learned through imitation

Five experiments were conducted to investigate infants’ ability to transfer actions learned via imitation to new objects and to examine what components of the original context are critical to such transfer. Infants of 15 months observed an experimenter perform an action with one or two toys and then were offered a novel toy that was not demonstrated for them. In all experiments, infants performed target actions with the novel toy more frequently than infants who were offered the same toy but had seen no prior demonstrations. Infants exhibited transfer even when the specific part to be manipulated looked different across the toys, even when they had not performed the actions with the demonstration toys themselves, even when the actions produced no effects on the demonstrations, and even when the actions were demonstrated with only a single exemplar toy. Transfer was especially robust when infants not only observed but also practiced the target actions on the demonstration trials. Learning action affordances (“means”) seems to be a central aspect of human imitation, and the propensity to apply these learned action affordances in new object contexts may be an important basis for technological innovation and invention.     

Goal congruency in bimanual object manipulation

In 3 experiments, the authors investigated the impact of action goals on the production of discrete bimanual responses. Similar to a bartender putting 2 glasses simultaneously on a shelf, participants placed 2 objects into either parallel or opposite orientations by carrying out either mirror-symmetrical or mirror-asymmetrical movements. In Experiment 1, performance was strongly affected by the congruency of the intended object orientations but was essentially unaffected by movement symmetry. Experiment 2 replicated this instrumental goal-congruency effect (and the absence of motor-symmetry effects) when actions were cued in advance. Experiment 3 revealed substantial motor-symmetry effects, provided the movements themselves became the action goal. The authors concluded that performance in bimanual choice reaction tasks is constrained by the creation and maintenance of goal codes rather than by properties inherent in the neuromuscular system that carries out these responses. These goals can relate to either body-intrinsic states or to body-extrinsic states according to the actor's current intentions.     

How infant-directed actions enhance infants’ attention,learning, and exploration: Evidence from EEG and computational modeling

When teaching infants new actions, parents tend to modify their movements. Infants prefer these infant-directed actions (IDAs) over adult-directed actions and learn well from them. Yet, it remains unclear how parents’ action modulations capture infants’ attention. Typically, making movements larger than usual is thought to draw attention. Recent findings, however, suggest that parents might exploit movement variability to highlight actions. We hypothesized that variability in movement amplitude rather than higher amplitude is capturing infants’ attention during IDAs. Using EEG, we measured 15-month-olds’ brain activity while they were observing action demonstrations with normal, high, or variable amplitude movements. Infants’ theta power (4–5 Hz) in fronto-central channels was compared between conditions. Frontal theta was significantly higher, indicating stronger attentional engagement, in the variable compared to the other conditions. Computational modelling showed that infants’ frontal theta power was predicted best by how surprising each movement was. Thus, surprise induced by variability in movements rather than large movements alone engages infants’ attention during IDAs. Infants with higher theta power for variable movements were more likely to perform actions successfully and to explore objects novel in the context of the given goal. This highlights the brain mechanisms by which IDAs enhance infants’ attention, learning, and exploration.     

Performance of unimanual and bimanual multiphased prehensile movements

By manipulating task action demands in 2 experiments, the author investigated whether the context-dependent effects seen in unimanual multiphase movements are also present in bimanual movements. Participants (N = 14) in Experiment 1 either placed or tossed objects into targets. The results indicated that the intention to perform a subsequent action with an object could influence the performance of an earlier movement in a sequence in both unimanual and bimanual tasks. Furthermore, assimilation effects were found when the subsequent tasks being performed by the 2 hands were incongruent. In Experiment 2, the author investigated in 12 participants whether planning in a multiphase movement includes some representation of the accuracy demands of the subsequent task. The accuracy demands of a subsequent task did not appear to influence initial movement planning. Instead, the present results support the notion that it is the action requirements of the subsequent movement that lead to context-dependent effects.     

Learning From Gesture and Action: An Investigation of Memory for Where Objects Went and How They Got There

Speakers often use gesture to demonstrate how to perform actions—for example, they might show how to open the top of a jar by making a twisting motion above the jar. Yet it is unclear whether listeners learn as much from seeing such gestures as they learn from seeing actions that physically change the position of objects (i.e., actually opening the jar). Here, we examined participants' implicit and explicit understanding about a series of movements that demonstrated how to move a set of objects. The movements were either shown with actions that physically relocated each object or with gestures that represented the relocation without touching the objects. Further, the end location that was indicated for each object covaried with whether the object was grasped with one or two hands. We found that memory for the end location of each object was better after seeing the physical relocation of the objects, that is, after seeing action, than after seeing gesture, regardless of whether speech was absent (Experiment 1) or present (Experiment 2). However, gesture and action built similar implicit understanding of how a particular handgrasp corresponded with a particular end location. Although gestures miss the benefit of showing the end state of objects that have been acted upon, the data show that gestures are as good as action in building knowledge of how to perform an action.     

Visual categorization of social interactions

Prominent theories of action recognition suggest that during the recognition of actions the physical patterns of the action is associated with only one action interpretation (e.g., a person waving his arm is recognized as waving). In contrast to this view, studies examining the visual categorization of objects show that objects are recognized in multiple ways (e.g., a VW Beetle can be recognized as a car or a beetle) and that categorization performance is based on the visual and motor movement similarity between objects. Here, we studied whether we find evidence for multiple levels of categorization for social interactions (physical interactions with another person, e.g., handshakes). To do so, we compared visual categorization of objects and social interactions (Experiments 1 and 2) in a grouping task and assessed the usefulness of motor and visual cues (Experiments 3, 4, and 5) for object and social interaction categorization. Additionally, we measured recognition performance associated with recognizing objects and social interactions at different categorization levels (Experiment 6). We found that basic level object categories were associated with a clear recognition advantage compared to subordinate recognition but basic level social interaction categories provided only a little recognition advantage. Moreover, basic level object categories were more strongly associated with similar visual and motor cues than basic level social interaction categories. The results suggest that cognitive categories underlying the recognition of objects and social interactions are associated with different performances. These results are in line with the idea that the same action can be associated with several action interpretations (e.g., a person waving his arm can be recognized as waving or greeting).     

The influence of environmental context in interpersonal observation–execution

Cyclical upper-limb movements involuntarily deviate from a primary movement direction when the actor concurrently observes incongruent biological motion. We examined whether environmental context influences such motor interference during interpersonal observation–execution. Participants executed continuous horizontal arm movements while observing congruent horizontal or incongruent curvilinear biological movements with or without the presence of an object positioned as an obstacle or distractor. When participants were observing a curvilinear movement, an object located within the movement space became an obstacle, and, thus, the curvilinear trajectory was essential to reach into horizontal space. When acting as a distractor, or with no object, the curvilinear trajectory was no longer essential. For observing horizontal movements, objects were located at the same relative locations as in the curvilinear movement condition. We found greater involuntary movement deviation when observing curvilinear than horizontal movements. Also, there was an influence of context only when observing horizontal movements, with greater deviation exhibited in the presence of a large obstacle. These findings suggest that the influence of environmental context is underpinned by the (mis-)matching of observed and executed actions as incongruent biological motion is primarily coded via bottom-up sensorimotor processes, whilst the congruent condition incorporates surrounding environmental features to modulate the bottom-up sensorimotor processes.     

Did you see that? Dissociating advanced visual information and ball flight constrains perception and action processes during one-handed catching

The integration of separate, yet complimentary, cortical pathways appears to play a role in visual perception and action when intercepting objects. The ventral system is responsible for object recognition and identification, while the dorsal system facilitates continuous regulation of action. This dual-system model implies that empirically manipulating different visual information sources during performance of an interceptive action might lead to the emergence of distinct gaze and movement pattern profiles. To test this idea, we recorded hand kinematics and eye movements of participants as they attempted to catch balls projected from a novel apparatus that synchronised or de-synchronised accompanying video images of a throwing action and ball trajectory. Results revealed that ball catching performance was less successful when patterns of hand movements and gaze behaviours were constrained by the absence of advanced perceptual information from the thrower's actions. Under these task constraints, participants began tracking the ball later, followed less of its trajectory, and adapted their actions by initiating movements later and moving the hand faster. There were no performance differences when the throwing action image and ball speed were synchronised or de-synchronised since hand movements were closely linked to information from ball trajectory. Results are interpreted relative to the two-visual system hypothesis, demonstrating that accurate interception requires integration of advanced visual information from kinematics of the throwing action and from ball flight trajectory.     

Parallels Between Action‐Object Mapping and Word‐Object Mapping in Young Children

Across a series of four experiments with 3‐ to 4‐year‐olds we demonstrate how cognitive mechanisms supporting noun learning extend to the mapping of actions to objects. In Experiment 1 (n = 61) the demonstration of a novel action led children to select a novel, rather than a familiar object. In Experiment 2 (n = 78) children exhibited long‐term retention of novel action‐object mappings and extended these actions to other category members. In Experiment 3 (n = 60) we showed that children formed an accurate sensorimotor record of the novel action. In Experiment 4 (n = 54) we demonstrate limits on the types of actions mapped to novel objects. Overall these data suggest that certain aspects of noun mapping share common processing with action mapping and support a domain‐general account of word learning.     

Target- and effect-directed actions towards temporal goals: Similar mechanisms?

The goal of an action can consist of generating a change in the environment (to produce an effect) or changing one's own situation in the environment (to move to a physical target). To investigate whether the mechanisms of effect-directed and target-directed action control are similar, participants performed continuous reversal movements. They either synchronized movement reversals with regularly presented tones (temporal targets) or produced tones at reversals isochronously (temporal effects). In both goal conditions an irrelevant goal characteristic was integrated into the goal representation (loudness, Experiment 1). When targets and effects were presented within the same reversal movement, similarities were enhanced (Experiment 2). When the task posed spatial demands in addition to temporal demands, target- and effect-directed movement kinematics changed equally with tempo (Experiment 3). Correlations between target-directed and effect-directed movements in temporal variability indicated similar timing mechanisms (Experiments 1 and 2). Only gradual differences between target- and effect-directed movements were observed. We conclude that the same mechanisms of action control, including the anticipation of upcoming events, underlie effect-directed and target-directed movements. Ideomotor theories of action control should incorporate action targets as goals similar to action effects. (PsycINFO Database Record (c) 2012 APA, all rights reserved).     

Recognition of dance-like actions: Memory for static posture or dynamic movement?

Dance-like actions are complex visual stimuli involving multiple changes in body posture across time and space. Visual perception research has demonstrated a difference between the processing of dynamic body movement and the processing of static body posture. Yet, it is unclear whether this processing dissociation continues during the retention of body movement and body form in visual working memory (VWM). When observing a dance-like action, it is likely that static snapshot images of body posture will be retained alongside dynamic images of the complete motion. Therefore, we hypothesized that, as in perception, posture and movement would differ in VWM. Additionally, if body posture and body movement are separable in VWM, as form- and motion-based items, respectively, then differential interference from intervening form and motion tasks should occur during recognition. In two experiments, we examined these hypotheses. In Experiment 1, the recognition of postures and movements was tested in conditions in which the formats of the study and test stimuli matched (movement–study to movement–test, posture–study to posture–test) or mismatched (movement–study to posture–test, posture–study to movement–test). In Experiment 2, the recognition of postures and movements was compared after intervening form and motion tasks. These results indicated that (1) the recognition of body movement based only on posture is possible, but it is significantly poorer than recognition based on the entire movement stimulus, and (2) form-based interference does not impair memory for movements, although motion-based interference does. We concluded that, whereas static posture information is encoded during the observation of dance-like actions, body movement and body posture differ in VWM.     

Incongruent imagery interferes with action initiation

It has been suggested that representing an action through observation and imagery share neural processes with action execution. In support of this view, motor-priming research has shown that observing an action can influence action initiation. However, there is little motor-priming research showing that imagining an action can modulate action initiation. The current study examined whether action imagery could prime subsequent execution of a reach and grasp action. Across two motion analysis tracking experiments, 40 participants grasped an object following congruent or incongruent action imagery. In Experiment 1, movement initiation was faster following congruent compared to incongruent imagery, demonstrating that imagery can prime the initiation of grasping. In Experiment 2, incongruent imagery resulted in slower movement initiation compared to a no-imagery control. These data show that imagining a different action to that which is performed can interfere with action production. We propose that the most likely neural correlates of this interference effect are brain regions that code imagined and executed actions. Further, we outline a plausible mechanistic account of how priming in these brain regions through imagery could play a role in action cognition.     

Cognitive representations of functional interactions with objects

Two studies addressed people’s knowledge about the movements underlying functional interactions with objects, when the interactions were described by simple verbal labels expressing environmental goals. In Experiment 1, subjects rated each action with respect to six dimensions: which portion of the limb moved, distance moved, forcefulness, effectors involved, size of the contact surface, and resemblance to grasp. Ratings were systematic and fell on two distinct underlying factors related to limb movement and effector (usually the hand) configuration. In Experiment 2, subjects sorted a subset of the actions by similarity of movement. Clustering and multidimensional scaling solutions indicated that the six initial dimensions contributed to similarity judgments, along with additional parameters. The results support the existence of cognitively accessible, but still relatively specific, representations of functional actions, with potential implications for motor and memory performance.