Emotion perception from dynamic and static body expressions in point-light and full-light displays

Research on emotion recognition has been dominated by studies of photographs of facial expressions. A full understanding of emotion perception and its neural substrate will require investigations that employ dynamic displays and means of expression other than the face. Our aims were: (i) to develop a set of dynamic and static whole-body expressions of basic emotions for systematic investigations of clinical populations, and for use in functional-imaging studies; (ii) to assess forced-choice emotion-classification performance with these stimuli relative to the results of previous studies; and (iii) to test the hypotheses that more exaggerated whole-body movements would produce (a) more accurate emotion classification and (b) higher ratings of emotional intensity. Ten actors portrayed 5 emotions (anger, disgust, fear, happiness, and sadness) at 3 levels of exaggeration, with their faces covered. Two identical sets of 150 emotion portrayals (full-light and point-light) were created from the same digital footage, along with corresponding static images of the 'peak' of each emotion portrayal. Recognition tasks confirmed previous findings that basic emotions are readily identifiable from body movements, even when static form information is minimised by use of point-light displays, and that full-light and even point-light displays can convey identifiable emotions, though rather less efficiently than dynamic displays. Recognition success differed for individual emotions, corroborating earlier results about the importance of distinguishing differences in movement characteristics for different emotional expressions. The patterns of misclassifications were in keeping with earlier findings on emotional clustering. Exaggeration of body movement (a) enhanced recognition accuracy, especially for the dynamic point-light displays, but notably not for sadness, and (b) produced higher emotional-intensity ratings, regardless of lighting condition, for movies but to a lesser extent for stills, indicating that intensity judgments of body gestures rely more on movement (or form-from-movement) than static form information.     

Pauses and syntax in American sign language

Research on spoken languages has shown that the durations of silent pauses in a sentence are strongly related to the syntactic structure of the sentence. A similar analysis of the pauses (holds) in a passage in American Sign Language reveals that sequences of signs are also interspersed with holds of different lengths: long holds appear to indicate the ends of sentences; shorter holds, the break between two conjoined sentences; and the shortest holds, breaks between internal constituents. Thus, pausal analysis is a guide to parsing sentences in ASL.     

Infants perceive human point-light displays as solid forms

While five-month-old infants show orientation-specific sensitivity to changes in the motion and occlusion patterns of human point-light displays, it is not known whether infants are capable of binding a human representation to these displays. Furthermore, it has been suggested that infants do not encode the same physical properties for humans and material objects. To explore these issues we tested whether infants would selectively apply the principle of solidity to upright human displays. In the first experiment infants aged six and nine months were repeatedly shown a human point-light display walking across a computer screen up to 10 times or until habituated. Next, they were repeatedly shown the walking display passing behind an in-depth representation of a table, and finally they were shown the human display appearing to pass through the table top in violation of the solidity of the hidden human form. Both six- and nine-month-old infants showed significantly greater recovery of attention to this final phase. This suggests that infants are able to bind a solid vertical form to human motion. In two further control experiments we presented displays that contained similar patterns of motion but were not perceived by adults as human. Six- and nine-month-old infants did not show recovery of attention when a scrambled display or an inverted human display passed through the table. Thus, the binding of a solid human form to a display in only seems to occur for upright human motion. The paper considers the implications of these findings in relation to theories of infants' developing conceptions of objects, humans and animals.     

Recognition of point-light biological motion displays by young children

We tested the ability of children 3-5 years of age to recognise biological motion displays. Children and adults were presented with moving point-light configurations depicting a walking person, four-legged animals (dogs), and a bird. Participants were able to reliably recognise displays with biological motion, but failed in the identification of a static (four consecutive frames taken from each sequence) version. The results indicate that, irrespective of the highly reduced and unusual structural information available in point-light displays, biological motion is sufficient for reliable recognition of human and non-human forms at an age as early as 3 years. Moreover, 5-year-olds exhibit the ceiling level of recognition. The findings are discussed in the context of the neuropsychological and brain mechanisms involved in biological motion perception.     

Perception of a throwing action from point-light demonstrations

80 adults (40 men and 40 women) aged 18 to 25 yr. and 80 boys aged 14 to 15 yr. observed a throwing action presented in point-light form at a frontoparallel viewpoint either as a video film or computer simulation of the actor. The display consisted of the relative motion of the wrist, elbow, and shoulder joints of the throwing arm. Subjects were asked to report what they observed. The action was recognised immediately by 68% of all subjects, and this value increased to 90% with one more viewing of the demonstration. Response was virtually the same for both types of display, and there were no differences between adults and children. The results were similar to those from other studies from which researchers have concluded that kinematic information alone is sufficient to permit perception of fundamental patterns of movement.     

Cerebral asymmetry in the perception of American sign language.

American Sign Language (ASL) offers a valuable opportunity for the study of cerebral asymmetries, since it incorporates both language structure and complex spatial relations: processing the former has generally been considered a left-hemisphere function, the latter, a right-hemisphere one. To study such asymmetries, congenitally deaf, native ASL users and normally-hearing English speakers unfamiliar with ASL were asked to identify four kinds of stimuli: signs from ASL, handshapes never used in ASL, Arabic digits, and random geometric forms. Stimuli were presented tachistoscopically to a visual hemifield and subjects manually responded as rapidly as possible to specified targets. Both deaf and hearing subjects showed left-visual-field (hence, presumably right-hemisphere) advantages to the signs and to the non-ASL hands. The hearing subjects, further, showed a left-hemisphere advantage to the Arabic numbers, while the deaf subjects showed no reliable visual-field differences to this material. We infer that the spatial processing required of the signs predominated over their language processing in determining the cerebral asymmetry of the deaf for these stimuli.     

Young children can extend motion verbs to point-light displays

In the first study using point-light displays (lights corresponding to the joints of the human body) to examine children's understanding of verbs, 3-year-olds were tested to see if they could perceive familiar actions that corresponded to motion verbs (e.g., walking). Experiment 1 showed that children could extend familiar motion verbs (e.g., walking and dancing) to videotaped point-light actions shown in the intermodal preferential looking paradigm. Children watched the action that matched the requested verb significantly more than they watched the action that did not match the verb. In Experiment 2, the findings of Experiment 1 were validated by having children spontaneously produce verbs for these actions. The use of point-light displays may illuminate the factors that contribute to verb learning.     

Perceiving performer identity and intended expression intensity in point-light displays of dance

This study investigated agent and expression intensity recognition in point-light displays depicting dancing performances. In a first session, participants danced with two different expression intensities to music, solo or in dyads. In a subsequent session, they watched point-light displays of 1–5-s duration, depicting their own, their partner’s or another participant’s recorded actions, and were asked to identify the agent (self vs. partner vs. stranger) and the intended expression intensity (expressive vs. inexpressive) of the performer. The results indicate that performer identity and expression intensity could be discerned reliably from displays as short as 1 s. The accuracy in judgment increased with exposure duration and, for performer identification, with higher expression intensity. Judgment accuracy in agent and expression intensity recognition tasks correlated with self-report empathy indices. Accuracy correlated also with confidence in judgment, but only in the intensity recognition task. The results are discussed in relation to perceptual and neural mechanisms underlying action and intention recognition.     

Cues for self-recognition in point-light displays of actions performed in synchrony with music

Self–other discrimination was investigated with point-light displays in which actions were presented with or without additional auditory information. Participants first executed different actions (dancing, walking and clapping) in time with music. In two subsequent experiments, they watched point-light displays of their own or another participant’s recorded actions, and were asked to identify the agent (self vs. other). Manipulations were applied to the visual information (actions differing in complexity, and degradation from 15 to 2 point-lights within the same clapping action) and to the auditory information (self-generated vs. externally-generated vs. none). Results indicate that self-recognition was better than chance in all conditions and was highest when observing relatively unconstrained patterns of movement. Auditory information did not increase accuracy even with the most ambiguous visual displays, suggesting that judgments of agent identity depend much more on motor cues than on auditory (action-generated) or audiovisual (synchronization) information.     

The perception of emotion from body movement in point-light displays of interpersonal dialogue

We examined whether it is possible to identify the emotional content of behaviour from point-light displays where pairs of actors are engaged in interpersonal communication. These actors displayed a series of emotions, which included sadness, anger, joy, disgust, fear, and romantic love. In experiment 1, subjects viewed brief clips of these point-light displays presented the right way up and upside down. In experiment 2, the importance of the interaction between the two figures in the recognition of emotion was examined. Subjects were shown upright versions of (i) the original pairs (dyads), (ii) a single actor (monad), and (iii) a dyad comprising a single actor and his/her mirror image (reflected dyad). In each experiment, the subjects rated the emotional content of the displays by moving a slider along a horizontal scale. All of the emotions received a rating for every clip. In experiment 1, when the displays were upright, the correct emotions were identified in each case except disgust; but, when the displays were inverted, performance was significantly diminished for some emotions. In experiment 2, the recognition of love and joy was impaired by the absence of the acting partner, and the recognition of sadness, joy, and fear was impaired in the non-veridical (mirror image) displays. These findings both support and extend previous research by showing that biological motion is sufficient for the perception of emotion, although inversion affects performance. Moreover, emotion perception from biological motion can be affected by the veridical or non-veridical social context within the displays.     

Processing of biological motion point-light displays by baboons (Papio papio)

Humans apply complex conceptual judgments to point-light displays (PLDs) representing biological motion (BM), but how animals process this kind of display remains uncertain. Four baboons (Papio papio) were trained to discriminate BM from nonbiological motion PLDs using an operant computerized test system. Transfer tests were given after training with novel BM stimuli representing humans or baboons (Experiment 1), with inverted PLDs (Experiment 2), and with BM stimuli in which body parts had been spatially disorganized (Experiment 3). Very limited transfer was obtained with the novel and inverted displays in Experiments 1 and 2, but transfer was much higher after spatial disorganization in Experiment 3. It is suggested that the baboons did not retrieve and interpret the articulated shape of the human or monkey body from the BM PLD stimuli, but rather focused their attention on the configural properties of subparts of the stimuli. Limits in perceptual grouping and restricted abilities in picture-object equivalence might explain why the baboons did not map BM PLD displays onto what they represent.     

Counting in sign language

Do the visuomanual modality and the structure of the sequence of numbers in sign language have an impact on the development of counting and its use by deaf children? The sequence of number signs in Belgian French Sign Language follows a base-5 rule while the number sequence in oral French follows a base-10 rule. The accuracy and use of sequence number string were investigated in hearing children varying in age from 3 years 4 months to 5 years 8 months and in deaf children varying in age from 4 years to 6 years 2 months. Three tasks were used: abstract counting, object counting, and creation of sets of a given cardinality. Deaf children exhibited age-related lags in their knowledge of the number sequence; they made different errors from those of hearing children, reflecting the rule-bound nature of sign language. Remarkably, their performance in object counting and creating sets of given cardinality was similar to that of hearing children who had a longer sequence number string, indicating a better use of counting than predicted by their knowledge of the linguistic sequence of numbers.     

A study of timing in a manual and a spoken language: American sign language and English

A comparative analysis of the time variables in the production of speech and sign reveals that signers modify their global physical rate primarily by altering the time they spend articulating, whereas speakers do so by changing the time they spend pausing. When signers increase or decrease their pause time, however little they do so, they alter the number and the length of the pauses equally, whereas speakers primarily alter the number of pauses and leave their pause durations relatively constant. An analysis of the durations of signs reveals that signs are longer at the end of sentences than within sentences and that the first occurrence of a sign is longer than the second when syntactic location is controlled (both these findings have already been reported for spoken language). The inherent duration of a sign can be accounted for almost totally by the movement characteristic; handshape, location, and number of hands in a sign are of little importance. Finally, signers retain their regular quiet-breathing respiratory pattern across signing rates and inhale at locations independent of syntactic importance. In this they are quite unlike speakers, who breathe at syntactic breaks.This research was supported in part by Grant R03 MH 28133, Department of Health, Education, and Welfare.     

Processing of formational, semantic, and iconic information in American sign language

Three experiments examined short-term encoding processes of deaf signers for different aspects of signs from American Sign Language. Experiment 1 compared short-term memory for lists of formationally similar signs with memory for matched lists of random signs. Just as acoustic similarity of words interferes with short-term memory or word sequences, formational similarity of signs had a marked debilitating effect on the ordered recall of sequences of signs. Experiment 2 evaluated the effects of the semantic similarity of the signs on short-term memory: Semantic similarity had no significant effect on short-term ordered recall of sequences of signs. Experiment 3 studied the role that the iconic (representational) value of signs played in short-term memory. Iconicity also had no reliable effect on short-term recall. These results provide support for the position that deaf signers code signs from American Sign Language at one level in terms of linguistically significant formational parameters. The semantic and iconic information of signs, however, seems to have little effect on short-term memory.     

Perception of biological motion: A stimulus set of human point-light actions

We present a set of stimuli representing human actions under point-light conditions, as seen from different viewpoints. The set contains 22 fairly short, well-delineated, and visually “loopable” actions. For each action, we provide movie files from five different viewpoints as well as a text file with the three spatial coordinates of the point lights, allowing researchers to construct customized versions. The full set of stimuli may be downloaded fromwww.psychonomic.org/archive/.     

Techniques for the production of point-light and fully illuminated video displays from identical recordings

Illumination of only a few key points on a moving human body or face is enough to convey a compelling perception of human motion. A full understanding of the perception ofbiological motion from point-light displays requires accurate comparison with the perception of motion in normal, fully illuminated versions of the same images. Traditionally, these two types of stimuli (point-light and fully illuminated) have been filmed separately, allowing the introduction of uncontrolled variation across recordings. This is undesirable for accurate comparison of perceptual performance across the two types of display. This article describes simple techniques, using proprietary software, that allow production of point-light and fully illuminated video displays from identical recordings. These techniques are potentially useful for many studies of motion perception, by permitting precise comparison of perceptual performances across point-light displays and their fully illuminated counterparts with accuracy and comparative ease.