The mirror reflects both ways: Action influences perception of others

Substantial evidence links perception of others’ bodies and mental representation of the observer’s own body; however, the overwhelming majority of this evidence is unidirectional, showing influence from perception to action. It has been proposed that the influence also runs from action to perception, but to date the evidence is scant. Here we report that ordinary motor actions performed by the subject affect concurrent psychophysical judgments of human-body stimuli. Subjects remained unaware of the connection between the action and the main task. The results show that perception can change as a result of the observer’s ongoing actions.     

Shifts in kinesthesis through time and after active and passive movement.

The position sense of a stationary arm was investigated subsequent to an horizontally adductive movement with axis the shoulder joint. The right arm was the treated arm: it reached a test position actively, using minimal voluntary effort, or passively from each of 10 starting positons. The blind-folded S localized the index finger of the treated arm by attempting to touch it with the index finger of his left hand. The results indicate that subsequent to active movement the final position of a limb is more accurately known than a position resulting from passive movement. A second finding is that concomitant with both forms of limb placement there is a unidirectional drift of perceived limb position over trials.     

Differential reliance of chimpanzees and humans on automatic and deliberate control of motor actions

Humans are often unaware of how they control their limb motor movements. People pay attention to their own motor movements only when their usual motor routines encounter errors. Yet little is known about the extent to which voluntary actions rely on automatic control and when automatic control shifts to deliberate control in nonhuman primates. In this study, we demonstrate that chimpanzees and humans showed similar limb motor adjustment in response to feedback error during reaching actions, whereas attentional allocation inferred from gaze behavior differed. We found that humans shifted attention to their own motor kinematics as errors were induced in motor trajectory feedback regardless of whether the errors actually disrupted their reaching their action goals. In contrast, chimpanzees shifted attention to motor execution only when errors actually interfered with their achieving a planned action goal. These results indicate that the species differed in their criteria for shifting from automatic to deliberate control of motor actions. It is widely accepted that sophisticated motor repertoires have evolved in humans. Our results suggest that the deliberate monitoring of one’s own motor kinematics may have evolved in the human lineage.     

Effects of interlimb practice on coding and learning of movement sequences

An interlimb practice paradigm was designed to determine the role that visual–spatial (Cartesian) and motor (joint angles, activation patterns) coordinates play in the coding and learning of complex movement sequences. Participants practised a 16-element movement sequence by moving a lever to sequentially presented targets with one limb on Day 1 and the contralateral limb on Day 2. Practice involved the same sequence with either the same visual–spatial or motor coordinates on the two days. A unilateral practice condition (control) was also tested where both coordinate systems were changed but the same limb was used. Retention tests were conducted on Day 3. Regardless of the order in which the limbs were used during practice, results indicated that keeping the visual–spatial coordinates the same during acquisition resulted in superior retention. This provides strong evidence that the visual–spatial code plays a dominant role in complex movement sequences, and this code is represented in an effector-independent manner.     

Accurate reaching after active but not passive movements of the hand: evidence for forward modeling

Converging behavioral findings support recent models of motor control suggesting that estimates of the future positions of a limb as well as the expected sensory consequences of a planned movement may be derived, in part, from efference copies of motor commands. These estimates are referred to as forward models. However, relatively little behavioral evidence has been obtained for proposed forward models that provide on-line estimates of current position. We report data from a patient (JD) who reached accurately to visualized targets with and without vision of her hand despite substantial proprioceptive loss. Additionally, we administered a double-start reaching test to examine the possibility that efference copy information could be used to estimate current limb position. JD reached accurately, without vision, to a final target after actively reaching to a landmark, but exhibited severely impaired reaching after passive movements to the landmark. This finding suggests that forward modeling of efference copy signals may provide relatively accurate estimates of current limb position for the purpose of motor planning. The possibility that such estimates may also contribute to the awareness of body position and to self-recognition is discussed.     

Perception of the end position of a limb loaded with a weight

We examined the effect of loading a weight on the perception of the end position of the limb. The participants haptically perceived where the end of their limb was located while they swung the limb or statically held it. The results showed that the loading of a weight influenced participants' perception only when they swung the limb; when the weight was attached to a participant's hand, the position was perceived to be farther from the body than when the weight was attached to the forearm or no weight was attached. We also found that the end position was generally underestimated when the participants swung their limb. On the other hand, when the participants statically held their limb, the perceived position was not affected by the loading of the weight and was rather precise. These results suggest that the perceptual system for limb end position is sensitive to changes in the surrounding haptic information caused by moving the limb.     

Time delays prior to movement alter the drawing kinematics of elderly adults

Position sense has been found to decay as a function of the time delay the limb remains in a static position prior to movement onset. Position sense has also been found to deteriorate as a function of aging, with increased reliance on vision by the elderly. This study investigated whether the pointing kinematics of elderly adults were differentially affected by delay compared to young adults, and whether visual information could compensate for the effects of delay. Young and elderly adults kept the limb in a static position for 1, 6, or 10 s prior to movement onset, both with and without vision of the limb, initial position, and the movement trajectory. Across groups, delay resulted in increased overall movement duration, decreased peak velocity including a shorter relative time to peak velocity, with decreased distance and duration of the primary submovement. Delay and lack of vision differentially decreased distance of the primary submovement for elderly adults. Vision was able to compensate to some degree for the effects of delay across age groups. The findings provide evidence that decays in position sense as a function of time create difficulties in incorporating the initial limb position in motor planning process in elderly adults.     

Representation and learning in motor action – Bridges between experimental research and cognitive robotics

To gain a better understanding of the functionality of representation and categorization in action and interaction, it is fundamental that researchers understand how movements are represented in long-term memory. It is our position that human motor control requires that our actions be planned and represented in terms of intended perceptual effects and future task demands, and that the individual has a well-structured mental representation of the task so that the movement can be carried out successfully. Basic Action Concepts (BACs) are identified as major building blocks of cognitive representation in long-term memory, which are cognitive tools used to master the functional demands of movement tasks. In this paper, we consider relevant issues in research methodology and present an experimental method that can be used to assess action-relevant representational structures. This method permits us to observe the strong relationship between cognitive representation and performance in manual action. For example, the specific differences in the mental representations of participants are strongly related to skill level, as well as biomechanical and task constraints. We then discuss results from our learning experiments, where we have examined the development and changes in cognitive representation over time. From these experiments we have found that cognitive reference structures include task-specific spatial information, which provides the basis for action control in skilled voluntary movement. We have implemented these results on various robotic platforms. We argue that the insights gained from various experimental approaches in the field of cognitive psychology and motor control enable researchers to explore the possibilities and limitations of artificial control architectures in robot systems. Finally, we argue that this is not a unidirectional process. Researchers from the field of cognitive psychology and motor control can profit from the advances in technological systems, which enhance the understanding of human motor control in skilled voluntary action.     

Response preparation and stimulus-response congruence to fractionating reaction time of upper and lower limbs

We measured the fractionating reaction time for upper and lower limbs of 18 young men, using an electromyogram to clarify the interaction of response preparation and stimulus-response congruence on premotor and motor time. The reaction time was divided into two periods with respect to the onset of the electromyogram, premotor and motor components. Manipulating the probability of the Go versus NoGo signals varied three different amounts of response preparation. Prior to the imperative stimulus, precue information about the position of a stimulus was presented. In the stimulus-response congruent task, subjects were required to release a key on the same side of the body using either an upper or lower limb in response to an imperative Go stimulus. In the stimulus-response incongruent task, subjects were required to act in a reverse manner. Subjects were not supposed to respond to the NoGo stimulus. The premotor time of the upper and lower limb responses was affected by the respective effects of response preparation and stimulus-response congruence. The motor time of the upper and lower limb responses increased as the probability of the Go signal decreased but not in relation to stimulus-response congruence. These results suggest that response preparation affects not only information processing time but also peripheral motor activity due to motor programming, and that the stimulus-response congruence effect on motor time disappears when information about the position of the imperative stimulus is precued.     

Active and passive scene recognition across views.

Recent evidence suggests that scene recognition across views is impaired when an array of objects rotates relative to a stationary observer, but not when the observer moves relative to a stationary display [Simons, D.J., Wang, R.F., 1998. Perceiving real-world viewpoint changes. Psychological Science 9, 315-320]. The experiments in this report examine whether the relatively poorer performance by stationary observers across view changes results from a lack of perceptual information for the rotation or from the lack of active control of the perspective change, both of which are present for viewpoint changes. Three experiments compared performance when observers passively experienced the view change and when they actively caused the change. Even with visual information and active control over the display rotation, change detection performance was still worse for orientation changes than for viewpoint changes. These findings suggest that observers can update a viewer-centered representation of a scene when they move to a different viewing position, but such updating does not occur during display rotations even with visual and motor information for the magnitude of the change. This experimental approach, using arrays of real objects rather than computer displays of isolated individual objects, can shed light on mechanisms that allow accurate recognition despite changes in the observer's position and orientation.     

Self produced and observed actions influence emotion: the roles of action fluency and eye gaze

Affective responses to objects can be influenced by cognitive processes such as perceptual fluency. Here we investigated whether the quality of motor interaction with an object influences affective response to the object. Participants grasped and moved objects using either a fluent action or a non-fluent action (avoiding an obstacle). Liking ratings were higher for objects in the fluent condition. Two further studies investigated whether the fluency of another person’s actions influences affective response. Observers watched movie clips of the motor actions described above, in conditions where the observed actor could be seen to be looking towards the grasped object, or where the actor’s head and gaze were not visible. Two results were observed: First, when the actor’s gaze cannot be seen, liking ratings of the objects are reduced. Second, action fluency of observed actions does influence liking ratings, but only when the actor’s gaze towards the object is visible. These findings provide supporting evidence for the important role of observed eye gaze in action simulation, and demonstrate that non-emotive actions can evoke empathic states in observers. This research was supported by Economic and Social Research Council grant RES-000-23-0429 awarded to S. P. Tipper and A. E. Hayes.     

Throwing in the dark: improved prediction of action outcomes following motor training without vision of the action

Traditional models of action understanding emphasise the idea that long-term exposure to a wide array of visual patterns of particular actions allows for effective action anticipation or prediction. More recently, a greater emphasis has been placed on the motor system’s role in the perceptual understanding and prediction of action outcomes. There have been attempts to isolate the contributions of visual and motor experience in action prediction, but to date, these studies have relied on comparisons of motor-visual experience to visual-only (observational) experience. We conducted a learning study, where visual experience was directly manipulated during practice. Novice participants practised throwing darts to 3 specific areas of a dartboard. A group trained without vision of their action, only feedback about the final landing position, significantly improved in their ability to predict the landing position of a thrown dart, from temporally occluded video clips. The performance of this ‘no-vision’ group did not differ from a Full-vision group and was significantly more accurate than an observation-only and a no-practice control group (with the latter two groups not improving pre- to post-practice). These results suggest that motor experience specifically modulates the perceptual prediction of action outcomes. This is thought to occur through simulative mechanisms, whereby observed actions are mapped onto the observer’s own motor representations.     

Interference effects demonstrate distinct roles for visual and motor imagery during the mental representation of human action

Four experiments were completed to characterize the utilization of visual imagery and motor imagery during the mental representation of human action. In Experiment 1, movement time functions for a motor imagery human locomotion task conformed to a speed-accuracy trade-off similar to Fitts' Law, whereas those for a visual imagery object motion task did not. However, modality-specific interference effects in Experiment 2 demonstrate visual and motor imagery as cooperative processes when the action represented is tied to visual coordinates in space. Biomechanic-specific motor interference effects found in Experiment 3 suggest one basis for separation of processing channels within motor imagery. Finally, in Experiment 4 representations of motor actions were found to be generated using only visual imagery under certain circumstances: namely, when the imaginer represented the motor action of another individual while placed at an opposing viewpoint. These results suggest that the modality of representation recruited to generate images of human action is dependent on the dynamic relationship between the individual, movement, and environment.     

Control of Equilibrium Position and Stiffness Through Postural Modules

If muscles are viewed as spring-like torque generators, then the integral of torque with respect to joint angle is the potential energy of that muscle. An energy function for the musculoskeletal system can be defined by summing the energy contribution of each muscle and the potential energy stored in the limb. Any local minimum in this energy landscape is a possible equilibrium position for the limb. The gradient of this function with respect to joint angles is a torque field, and the task of postural control is to find a set of muscle activations to produce a desired field. We consider one technique by which this approximation may be achieved: A postural module is defined as a synergy of muscles that produce a class of torque functions that converge at a constant equilibrium position, but whose stiffness at this position varies as a function of activation of the postural module. For a single-joint system, we show that through control of two such modules it is possible to produce any stiffness at any desired equilibrium position. To extend this scheme to a multijoint system, we initially derive the mechanical constraints on the shape of the restoring force field when a multijoint limb is displaced from equilibrium. Next, we consider voluntary control of the force field when the human arm is displaced from equilibrium: Mussa-Ivaldi, Hogan, and Bizzi (1985) have suggested that subjects are unable to voluntarily change the shape and orientation of the field, although they can readily scale it. This suggests existence of a limitation on the independent recruitment of arm muscles. We show, through simulation, that the inability to voluntarily control the shape and orientation of the restoring force field can be attributed to an organization of postural modules that act as local stiffness controllers. We propose that through coactivation, postural modules coarsely encode the work space and serve as an intermediate control system in the motor control hierarchy.     

A Computational Model of the Simplest Motor Program

A computational procedure (program) is defined to generate control signals for the motoneuron pools of agonist and antagonist muscles that will move a limb segment from one stationary position to another. The program accounts for the ability to move different distances with different inertial loads and for the influence of instructions concerning movement speed and accuracy. These motor commands allow the program to produce EMG patterns as well as force and kinematic trajectories that are consistent with much of the data found in the literature of these movements. The program is premised on the notion that kinematically defined tasks are accomplished by programming commands to the motoneuron pools, based on only a few cognitively recognized kinematic and dynamic features of the task. Most of the features found in EMG and kinematic patterns can be considered consequences of the program's algorithmic procedures rather than specifically planned features of those movements.     

Cognitive representations of functional interactions with objects

Two studies addressed people’s knowledge about the movements underlying functional interactions with objects, when the interactions were described by simple verbal labels expressing environmental goals. In Experiment 1, subjects rated each action with respect to six dimensions: which portion of the limb moved, distance moved, forcefulness, effectors involved, size of the contact surface, and resemblance to grasp. Ratings were systematic and fell on two distinct underlying factors related to limb movement and effector (usually the hand) configuration. In Experiment 2, subjects sorted a subset of the actions by similarity of movement. Clustering and multidimensional scaling solutions indicated that the six initial dimensions contributed to similarity judgments, along with additional parameters. The results support the existence of cognitively accessible, but still relatively specific, representations of functional actions, with potential implications for motor and memory performance.     

Influence of Perspective of Action Observation Training on Residual Limb Control in Naïve Prosthesis Usage

Prior work in amputees and partial limb immobilization have shown improved neural and behavioral outcomes in using their residual limb with prosthesis when undergoing observation-based training with a prosthesis-using actor compared to an intact limb. It was posited that these improvements are due to an alignment of user with the actor. It may be affected by visual angles that allow emphasis of critical joint actions which may promote behavioral changes. The purpose of this study was to examine how viewing perspective of observation-based training effects prosthesis adaptation in naïve device users. Twenty nonamputated prosthesis users learned how to use an upper extremity prosthetic device while viewing a training video from either a sagittal or coronal perspective. These views were chosen as they place visual emphasis on different aspects of task performance to the device. The authors found that perspective of actions has a significant role in adaptation of the residual limb while using upper limb prostheses. Perspectives that demonstrate elbow adaptations to prosthesis usage may enhance the functional motor outcomes of action observation therapy. This work has potential implications on how prosthetic device operation is conveyed to persons adapting to prostheses through action observation based therapy.     

Tie my hands, tie my eyes

Previous studies have demonstrated that motor abilities allow us not only to execute our own actions and to predict their consequences, but also to predict others' actions and their consequences. But just how deeply are motor abilities implicated in action observation? If an observer is prevented from acting while witnessing others' actions, will this impact on their making sense of others' behavior? We recorded proactive eye movements while participants observed an actor grasping objects. The participants' hands were either freely resting on the table or tied behind their back. Proactivity of gaze behavior was dramatically impaired when participants observed others' actions with their hands tied. Since we don't literally perceive actions with our hands, the effect may be explained by the hypothesis that effective observation of action depends not only on motor abilities but on being in a position to exercise them. This suggests, for the first time, that actions are observed best when we are actually in the position to perform them.