A technique for continuous measurement of body movement from video

The movements that we make with our body vary continuously along multiple dimensions. However, many of the tools and techniques presently used for coding and analyzing hand gestures and other body movements yield categorical outcome variables. Focusing on categorical variables as the primary quantitative outcomes may mislead researchers or distort conclusions. Moreover, categorical systems may fail to capture the richness present in movement. Variations in body movement may be informative in multiple dimensions. For example, a single hand gesture has a unique size, height of production, trajectory, speed, and handshape. Slight variations in any of these features may alter how both the speaker and the listener are affected by gesture. In this paper, we describe a new method for measuring and visualizing the physical trajectory of movement using video. This method is generally accessible, requiring only video data and freely available computer software. This method allows researchers to examine features of hand gestures, body movement, and other motion, including size, height, curvature, and speed. We offer a detailed account of how to implement this approach, and we also offer some guidelines for situations where this approach may be fruitful in revealing how the body expresses information. Finally, we provide data from a small study on how speakers alter their hand gestures in response to different characteristics of a stimulus to demonstrate the utility of analyzing continuous dimensions of motion. By creating shared methods, we hope to facilitate communication between researchers from varying methodological traditions.     

A program for parsing mouse movements into component submovements

Most current movement control theories include the idea that movement toward a target can be broken into several submovements. The complexity of analyzing a movement into its constituent submovement structure and the additional complexity imposed by the problem of noise in the data and hand tremor seem to be daunting to researchers. This paper discusses a program that can ameliorate both of these problems and parse movements into their constituent submovements. It also contains a graphing feature that is useful as a visual tool for analyzing submovement structure. The programs are easily modifiable, so that researchers can specify their own parsing rules on the basis of different assumptions about movement control and use the parser for data from different experimental tasks.     

Social attitudes modulate automatic imitation

In naturalistic interpersonal settings, mimicry or “automatic imitation” generates liking, affiliation, cooperation and other positive social attitudes. The purpose of this study was to find out whether the relationship between social attitudes and mimicry is bidirectional: Do social attitudes have a direct and specific effect on mimicry? Participants were primed with pro-social, neutral or anti-social words in a scrambled sentence task. They were then tested for mimicry using a stimulus-response compatibility procedure. In this procedure, participants were required to perform a pre-specified movement (e.g. opening their hand) on presentation of a compatible (open) or incompatible (close) hand movement. Reaction time data were collected using electromyography (EMG) and the magnitude of the mimicry/automatic imitation effect was calculated by subtracting reaction times on compatible trials from those on incompatible trials. Pro-social priming produced a larger automatic imitation effect than anti-social priming, indicating that the relationship between mimicry and social attitudes is bidirectional, and that social attitudes have a direct and specific effect on the tendency to imitate behavior without intention or conscious awareness.     

言语与手部运动关系的研究回顾

言语与手部运动之间存在复杂的联系。该文总结了两类手部运动（伴随言语发生的手势运动和抓握运动）与言语之间关系的行为和脑科学研究成果。发现：（1）伴随言语产生的意义手势可促进言语加工,特别是词汇的提取过程;（2）观察手的抓握运动影响言语产生时唇的运动和声音成分;（3）对词语的知觉影响抓握运动的早期计划阶段;（4）言语产生可增加手运动皮层的兴奋性。作者由此认为,言语加工与手势间的联系不仅表现为神经通路的重叠和相互激活,而且可能在外显行为上也相互影响     

Goal-Directed Aiming: Correcting a Force-Specification Error With the Right and Left Hands

In 2 experiments, the authors examined manual aiming asymmetries as well as the ability of participants to adjust their aiming trajectories following an unexpected change to the inertial resistance to movement. In Experiment 1, participants (N = 11) were able to rapidly adjust their movement trajectories to conform to the new movement requirements. They were faster and more consistent when aiming with their right hand than with their left hand, regardless of whether or not the movement was perturbed. In Experiment 2, participants' (N = 11) vision of the hand was manipulated so that the role of visual feedback in the corrective process could be examined. Vision had an impact not only on performance but also on the characteristics of the movement trajectories. Manual asymmetries in aiming were associated with a right hand superiority during the final corrective stages of the movement.     

Fake hand in movement: Visual motion cues from the rubber hand are processed for kinesthesia

The feeling that a fake (e.g. rubber) hand belongs to a person's own body can be elicited by synchronously stroking the fake hand and the real hand, with the latter hidden from view. Here, we sought to determine whether visual motion signals from that incorporated rubber hand would provide relevant cues for sensing movement (i.e. kinesthesia). After 180 s of visuo-tactile synchronous or asynchronous stroking, the fake hand was moved along the lateral or the sagittal axis. After synchronous stroking, movement of the rubber hand induced illusory movement of the static (real) hand in the same direction; the illusion was slightly more frequent and more intense when the fake hand was moved along the sagittal axis. We therefore conclude that visual signals of motion originating from the rubber hand are integrated for kinesthesia by the central nervous system just as visual signals from the real hand are.     

Cognitive processing and motor execution in the lexical decision task: a developmental study

We investigated lexical decision making in children and adults by analyzing spatiotemporal characteristics of responses involving a hand movement. Children’s and adults’ movement trajectories were assessed in three tasks: a lexical decision task (LDT), a pointing task that involved minimal cognitive processing, and a symbol task requiring a simple binary decision. Cognitive interference on motor performance was quantified by analyzing movement characteristics in the LDT and symbol task relative to the pointing task. Across age groups, movements in the LDT were less smooth, slower, and more strongly curved to the opposite response option, and these interference effects decreased steadily with age. Older children showed stronger interference effects than did adults, even though their reaction times were similar to adults’ performance. No comparable effects were found in the symbol task, indicating that task characteristics such as response mapping and decision selection alone are not able to explain the developmental differences observed in the LDT. Our results indicate substantial overlap between cognitive processing and motor execution in the LDT in children that is not captured by computational models of visual word recognition and cognitive development.     

Period Duration of Physical and Im aginary Movement Sequences Affects Contralateral Amplitude Modulation

The hypothesis was tested that the strength of cross-manual effects of voluntary amplitude modulations in bimanual tasks increases when less preparation time is available during the execution of a sequence of movements. By means of the continuation procedure, various period durations (600, 800, 1200, and 1800 msec) of movement sequences were imposed. Subjects performed bimanual periodic arm movements on two digitizers: constant-amplitude movements (short or long) with the one hand, and movements of constant short or constant long amplitudes and movements of alternating short and long amplitudes with the other hand. Period-duration-dependent modulations of the amplitude were observed in movements of instructed constant amplitudes when the other hand performed alternating-amplitude movements, but not when the other hand performed constant-amplitude movements. Not only physical performance, but also the imaginary production of alternating-amplitude movements resulted in period-duration-dependent cross-manual effects, though reduced in size. The pattern of results is in agreement with a recently proposed two-level model of cross-manual effects according to which cross-talk can occur at the programming level as well as at the execution level.     

Do children find movements which cross the body midline difficult?

Young children asked to make hand movements which cross the body are likely to respond with a movement confined to the same body side as the hand used. This crosslateral inhibition effect (CIE) features in Kephart's (1971) perceptualmotor theory; there is an extensive, earlier, literature on CIE in adults following cerebral insult; since the twenties, attempts have been made to relate developmental, clinical and neuropathological findings. In the present study 120 children aged from 3 to 8 years faced and copied hand movements of a model in a procedure following Head's (1920) Hand, Eye and Ear Test. CIE was confirmed, but differences between groups did not convincingly relate to age. CIE was greatest not at 3, 4 or 5 but at 6 years; further, 4-year-olds subjects made more preferred hand crosslateral responses than any other group. Analysis of imitation performance showed that older subjects applied a “proximity” strategy to both body part touched and hand use, whereas younger subjects disregarded the latter. It is argued that developmental CIE stems from procedural constraints and an agerelated choice of response strategy, and not learning related to a lateral bifurcation of perceptual-motor organization.     

Achieving Coordination in Prehension: Joint Freezing and Postural Contributions

The focus of the present study was on the intersegmental relationships that emerge when both task and oganismic constraints are imposed upon the coordination system. Seven right-handed subjects were required to reach and grasp a cup (hand transport phase) and place it on a designated target (cup transport phase), using either their preferred or nonpreferred hand. The kinematics of the movement were examined as a function of task (grasping a full cup versus grasping an empty one) and organismic (preferred or nonpreferred hand) constraints. During the hand transport phase, a task constraint effect was revealed through an increase in the low-velocity phase for the full cup condition. This constraint coexisted with a decrease in angular motion of the shoulder and elbow joints, indicating subjects reduced the number of variables to be independently controlled in the final homing-in stage of the movement. Accompanying this decrease in angular change was an increase in the displacement of the trunk. During the cup transport phase, the trunk was shown to contribute significantly more to the movement in the full cup condition and for the left hand movements, thereby increasing the stability of the movement system. These findings are in agreement with Bernstein's (1967) notion of fixating parts of the body as an initial solution to a movement problem, and they lend support to the concept of a proximodistal organization of coordination.     

The role of peripheral and central visual information for the directional control of manual aiming movements.

Seeing one's hand in visual periphery has been shown to optimize the directional accuracy of a sweeping hand movement, which is consistent with Paillard's (1980; Paillard & Amblard, 1985) two-channels model of visual information processing. However, contrary to this model, seeing one's hand in central vision, even for a brief period of time, also resulted in optimal directional accuracy. One goal of the present study was to test two opposing hypotheses proposed to explain the latter finding. As a second goal, we wanted to determine whether additional support could be found for the existence of a visual kinetic channel. The results indicated that seeing one's hand in central vision, even for a very short delay, resulted in the same accuracy as being permitted to see one's hand for the duration of the whole movement. This suggests that seeing one's hand around the target might enable one to code its location and that of the target within a single frame of reference and, thus, facilitate movement planning. In addition, the results of the present study indicated that seeing one's hand in motion while in visual periphery permitted a better directional accuracy than when this information was not available. This suggests that the movement vector, which is planned prior to movement initiation, can be quickly updated following movement initiation.     

Eye-hand coupling during closed-loop drawing: evidence of shared motor planning?

Previous paradigms have used reaching movements to study coupling of eye-hand kinematics. In the present study, we investigated eye-hand kinematics as curved trajectories were drawn at normal speeds. Eye and hand movements were tracked as a monkey traced ellipses and circles with the hand in free space while viewing the hand's position on a computer monitor. The results demonstrate that the movement of the hand was smooth and obeyed the 2/3 power law. Eye position, however, was restricted to 2-3 clusters along the hand's trajectory and fixed approximately 80% of the time in one of these clusters. The eye remained stationary as the hand moved away from the fixation for up to 200 ms and saccaded ahead of the hand position to the next fixation along the trajectory. The movement from one fixation cluster to another consistently occurred just after the tangential hand velocity had reached a local minimum, but before the next segment of the hand's trajectory began. The next fixation point was close to an area of high curvature along the hand's trajectory even though the hand had not reached that point along the path. A visuo-motor illusion of hand movement demonstrated that the eye movement was influenced by hand movement and not simply by visual input. During the task, neural activity of pre-motor cortex (area F4) was recorded using extracellular electrodes and used to construct a population vector of the hand's trajectory. The results suggest that the saccade onset is correlated in time with maximum curvature in the population vector trajectory for the hand movement. We hypothesize that eye and arm movements may have common, or shared, information in forming their motor plans.     

Tracking hand movements captures the response dynamics of the evaluative priming effect

We tested the response dynamics of the evaluative priming effect (i.e. facilitation of target responses following evaluatively congruent compared with evaluatively incongruent primes) using a mouse tracking procedure that records hand movements during the execution of categorisation tasks. In Experiment 1, when participants performed the evaluative categorisation task but not the non-evaluative semantic categorisation task, their mouse trajectories for evaluatively incongruent trials curved more toward the opposite response than those for evaluatively congruent trials, indicating the emergence of evaluative priming effects based on response competition. In Experiment 2, implementing a task-switching procedure in which evaluative and non-evaluative categorisation tasks were intermixed, we obtained reliable evaluative priming effects in the non-evaluative semantic categorisation task as well as in the evaluative categorisation task when participants assigned attention to the evaluative stimulus dimension. Analyses of hand movements revealed that the evaluative priming effects in the evaluative categorisation task were reflected in the mouse trajectories, while evaluative priming effects in the non-evaluative categorisation tasks were reflected in initiation times (i.e. the time elapsed between target onset and first mouse movement). Based on these findings, we discuss the methodological benefits of the mouse tracking procedure and the underlying processes of evaluative priming effects.     

Newborns' preference for goal-directed actions

The central role of sensory-motor representations in cognitive functions is almost universally accepted. However, determining the link between motor execution and its sensory counterpart and when, during ontogenesis, this link originates are still under investigation. The aim of the present study is to investigate whether at birth this link is already present and 2-day-old newborns are able to discriminate between visual cues indicating goal-directed or non-goal-directed actions. Here, with a preferential looking technique, a hand grasping a ball was the observed movement and we orthogonally manipulated the three factors necessary to successfully reach the goal: (a) presence of the ball, (b) direction of the arm movement, and (c) hand shaping. Results indicated that newborns orient more frequently and look longer at a hand shape for whole hand prehension but only when the movement is directed away from the body and toward the external world. In addition, newborns prefer the away from the body movement only when the object is present. We argue that newborns prefer a movement directed toward the external world only when it may develop into a purposeful movement because of the presence of the to-be-grasped object. Overall, our results support the existence of primitive sensory-motor associations since the first days after birth.     

The coupling of head,reach and grasp movement in nine months old infant prehension

In 9-month-old-infants adjustments in the reaching pattern to sudden changes in object location were examined. An attractive ball was presented to the infants at their midline and on some trials (perturbation trials) the ball suddenly changed position 15 cm to the right or left during the reach. For the perturbed trials the movement times approximately doubled compared to the control trials and significantly fewer balls were grasped. The results indicate that infants need to finish the first movement before being able to redirect the reach to a new destination. The correlation between the latency of the head and hand adjustment to the perturbation were 0.85 and 0.78 for movements to the left and to the right, respectively, indicating a tight coupling. The time between the start of the perturbation and peak velocity (TPPV) was significantly shorter for the head movement than for the hand movement, indicating that the head is leading the hand.     

利手与内隐左右空间情感效价关系的眼动研究

为探明手动作流畅性和情感材料呈现空间在不同利手者左右空间情感偏好中的关系,本研究将情绪Stroop范式和眼动测量相结合,通过反应速度和眼动数据将动作流畅性和空间情感注意偏向相分离,并考察其交互作用。结果发现右利手个体的反应速度存在优势手效应,不同利手者在使用左手时表现出对优势手同侧空间的内隐情感偏好,表明右利手个体的反应速度存在优势手流畅性的主导作用,手动作流畅性和内隐空间情感偏好的作用可以分离。     

Effects of motor intention on the perception of somatosensory events: A behavioural and functional magnetic resonance imaging study

The intention to execute a movement can modulate our perception of sensory events, and this modulation is observed ahead of both ocular and upper limb movements. However, theoretical accounts of these effects, and also the empirical data, are often contradictory. Accounts of “active touch”, and the premotor theory of attention, have emphasized how movement intention leads to enhanced perceptual processing at the target of a movement, or on the to-be-moved effector. By contrast, recent theories of motor control emphasize how internal “forward” model (FM) estimates may be used to cancel or attenuate sensory signals that arise as a result of self-generated movements. We used behavioural and functional brain imaging (functional magnetic resonance imaging, fMRI) to investigate how perception of a somatosensory stimulus differed according to whether it was delivered to a hand that was about to execute a reaching movement or the alternative, nonmoving, hand. The results of our study demonstrate that a somatosensory stimulus delivered to a hand that is being prepared for movement is perceived to have occurred later than when that same stimulus is delivered to a nonmoving hand. This result indicates that it takes longer for a tactile stimulus to be detected when it is delivered to a moving limb and may correspond to a change in perceptual threshold. Our behavioural results are paralleled by the results of our fMRI study that demonstrated that there were significantly reduced blood-oxygen-level-dependent (BOLD) responses within the parietal operculum and insula following somatosensory stimulation of the hand being prepared for movement, compared to when an identical stimulus was delivered to a nonmoving hand. These findings are consistent with the prediction of FM accounts of motor control that postulate that central sensory suppression of somatosensation accompanies self-generated limb movements, and with previous reports indicating that effects of sensory suppression are observed in higher order somatosensory regions.     

Effects of motor intention on the perception of somatosensory events: a behavioural and functional magnetic resonance imaging study

The intention to execute a movement can modulate our perception of sensory events, and this modulation is observed ahead of both ocular and upper limb movements. However, theoretical accounts of these effects, and also the empirical data, are often contradictory. Accounts of "active touch", and the premotor theory of attention, have emphasized how movement intention leads to enhanced perceptual processing at the target of a movement, or on the to-be-moved effector. By contrast, recent theories of motor control emphasize how internal "forward" model (FM) estimates may be used to cancel or attenuate sensory signals that arise as a result of self-generated movements. We used behavioural and functional brain imaging (functional magnetic resonance imaging, fMRI) to investigate how perception of a somatosensory stimulus differed according to whether it was delivered to a hand that was about to execute a reaching movement or the alternative, nonmoving, hand. The results of our study demonstrate that a somatosensory stimulus delivered to a hand that is being prepared for movement is perceived to have occurred later than when that same stimulus is delivered to a nonmoving hand. This result indicates that it takes longer for a tactile stimulus to be detected when it is delivered to a moving limb and may correspond to a change in perceptual threshold. Our behavioural results are paralleled by the results of our fMRI study that demonstrated that there were significantly reduced blood-oxygen-level-dependent (BOLD) responses within the parietal operculum and insula following somatosensory stimulation of the hand being prepared for movement, compared to when an identical stimulus was delivered to a nonmoving hand. These findings are consistent with the prediction of FM accounts of motor control that postulate that central sensory suppression of somatosensation accompanies self-generated limb movements, and with previous reports indicating that effects of sensory suppression are observed in higher order somatosensory regions.     

On the hand transport component of prehensile movements

Jeannerod (1981) proposed that prehensile movements involve two independent visuomotor channels that are responsible for hand transport and hand aperture. In many studies, the movement of a marker placed on the wrist has been used as an index of hand transport because wrist movement is unaffected by the movements of the digits responsible for hand aperture. In the present study, the spatial paths of the wrist, index finger, and thumb of 5 adults, each performing 50 reaching movements, were measured with a WATSMART movement tracking system, and their variability was analyzed. The measures of movement variability suggest that the motor system is more concerned with thumb position than with wrist position during hand transport. Although the wrist is a technically convenient index of hand transport, the thumb may be a more appropriate index from the point of view of motor control     

On the Hand Transport Component of Prehensile Movements

Jeannerod (1981) proposed that prehensile movements involve two independent visuomotor channels that are responsible for hand transport and hand aperture. In many studies, the movement of a marker placed on the wrist has been used as an index of hand transport because wrist movement is unaffected by the movements of the digits responsible for hand aperture. In the present study, the spatial paths of the wrist, index finger, and thumb of 5 adults, each performing 50 reaching movements, were measured with a WATSMART movement tracking system, and their variability was analyzed. The measures of movement variability suggest that the motor system is more concerned with thumb position than with wrist position during hand transport. Although the wrist is a technically convenient index of hand transport, the thumb may be a more appropriate index from the point of view of motor control.