A Systematic Investigation of Gesture Kinematics in Evolving Manual Languages in the Lab

Silent gestures consist of complex multi-articulatory movements but are now primarily studied through categorical coding of the referential gesture content. The relation of categorical linguistic content with continuous kinematics is therefore poorly understood. Here, we reanalyzed the video data from a gestural evolution experiment (Motamedi, Schouwstra, Smith, Culbertson, & Kirby, 2019), which showed increases in the systematicity of gesture content over time. We applied computer vision techniques to quantify the kinematics of the original data. Our kinematic analyses demonstrated that gestures become more efficient and less complex in their kinematics over generations of learners. We further detect the systematicity of gesture form on the level of thegesture kinematic interrelations, which directly scales with the systematicity obtained on semantic coding of the gestures. Thus, from continuous kinematics alone, we can tap into linguistic aspects that were previously only approachable through categorical coding of meaning. Finally, going beyond issues of systematicity, we show how unique gesture kinematic dialects emerged over generations as isolated chains of participants gradually diverged over iterations from other chains. We, thereby, conclude that gestures can come to embody the linguistic system at the level of interrelationships between communicative tokens, which should calibrate our theories about form and linguistic content.     

Communicative gestures of the hand and arm when 4-year-old sons and their mothers interact

This study investigated a wide range of communicative hand/arm gestures of 4-year-old males when interacting with their mothers. The types of gesture categories observed were in keeping with the predicted encoding ability of this aged child. Pantomimic and deictic gestures were observed in significantly greater numbers than semantic modifying and relational gestures. Although it was found that the mothers' gestural usage reflected the type of gesture categories seen in the children's group, no correlation was found between gesture usage of individual mother-child pairs.     

Get by With a Little Help From a Word: Multimodal Input Facilitates 26-Month-Olds' Ability to Map and Generalize Arbitrary Gestural Labels

In the early stages of word learning, children demonstrate considerable flexibility in the type of symbols they will accept as object labels. However, around the 2nd year, as children continue to gain language experience, they become focused on more conventional symbols (e.g., words) as opposed to less conventional symbols (e.g., gestures). During this period of symbolic narrowing, the degree to which children are able to learn other types of labels, such as arbitrary gestures, remains a topic of debate. Thus, the purpose of the current set of experiments was to determine whether a multimodal label (word + gesture) could facilitate 26-month-olds' ability to learn an arbitrary gestural label. We hypothesized that the multimodal label would exploit children's focus on words thereby increasing their willingness to interpret the gestural label. To test this hypothesis, we conducted two experiments. In Experiment 1, 26-month-olds were trained with a multimodal label (word + gesture) and tested on their ability to map and generalize both the arbitrary gesture and the multimodal label to familiar and novel objects. In Experiment 2, 26-month-olds were trained and tested with only the gestural label. The findings revealed that 26-month-olds are able to map and generalize an arbitrary gesture when it is presented multimodally with a word, but not when it is presented in isolation. Furthermore, children's ability to learn the gestural labels was positively related to their reported productive vocabulary, providing additional evidence that children's focus on words actually helped, not hindered, their gesture learning.     

Hand gestures and perceived influence in small group interaction

A laboratory study was carried out to establish the relative importance of verbal and gestural behavior, as well as their interaction, for perceived social influence in more or less competitive small groups. Forty women (psychology students) participated in leaderless small group discussions of different sizes (four-member and eight-member): at the end, each member rated the perceived influence in decision-making of every other member. Verbal dominance coding is based on traditional quantitative conversational dominance (number of talk turns). Gestural coding (conversational, ideational, object-adaptor, self-adaptor gestures) is based on classical gesture classifications. Beside a substantial effect of verbal dominance, the main result is that frequency of object-adaptors and conversational (only in large groups) and ideational (in both small and large groups) gestures increases perceived influence scores particularly when the verbal dominance of the speaker is low.     

Gestural development and its relation to a child's early vocabulary

Gesture and language are tightly connected during the development of a child's communication skills. Gestures mostly precede and define the way of language development; even opposite direction has been found. Few recent studies have focused on the relationship between specific gestures and specific word categories, emphasising that the onset of one gesture type predicts the onset of certain word categories or of the earliest word combinations.The aim of this study was to analyse predicative roles of different gesture types on the onset of first word categories in a child's early expressive vocabulary. Our data show that different types of gestures predict different types of word production. Object gestures predict open-class words from the age of 13 months, and gestural routines predict closed-class words and social terms from 8 months. Receptive vocabulary has a strong mediating role for all linguistically defined categories (open- and closed-class words) but not for social terms, which are the largest word category in a child's early expressive vocabulary. Accordingly, main contribution of this study is to define the impact of different gesture types on early expressive vocabulary and to determine the role of receptive vocabulary in gesture-expressive vocabulary relation in the Croatian language.     

Enhancement of naming in nonfluent aphasia through gesture

In a number of studies that have examined the gestural disturbance in aphasia and the utility of gestural interventions in aphasia therapy, a variable degree of facilitation of verbalization during gestural activity has been reported. The present study examined the effect of different unilateral gestural movements on simultaneous oral-verbal expression, specifically naming to confrontation. It was hypothesized that activation of the phylogenetically older proximal motor system of the hemiplegic right arm in the execution of a communicative but nonrepresentational pointing gesture would have a facilitatory effect on naming ability. Twenty-four aphasic patients, representing five aphasic subtypes, including Broca's, Transcortical Motor, Anomic, Global, and Wernicke's aphasics were assessed under three gesture/naming conditions. The findings indicated that gestures produced through activation of the proximal (shoulder) musculature of the right paralytic limb differentially facilitated naming performance in the nonfluent subgroup, but not in the Wernicke's aphasics. These findings may be explained on the view that functional activation of the archaic proximal motor system of the hemiplegic limb, in the execution of a communicative gesture, permits access to preliminary stages in the formative process of the anterior action microgeny, which ultimately emerges in vocal articulation.     

Gesture and metaphor comprehension: Electrophysiological evidence of cross-modal coordination by audiovisual stimulation

In recent years, studies have suggested that gestures influence comprehension of linguistic expressions, for example, eliciting an N400 component in response to a speech/gesture mismatch. In this paper, we investigate the role of gestural information in the understanding of metaphors. Event related potentials (ERPs) were recorded while participants viewed video clips of an actor uttering metaphorical expressions and producing bodily gestures that were congruent or incongruent with the metaphorical meaning of such expressions. This modality of stimuli presentation allows a more ecological approach to meaning integration. When ERPs were calculated using gesture stroke as time-lock event, gesture incongruity with metaphorical expression modulated the amplitude of the N400 and of the late positive complex (LPC). This suggests that gestural and speech information are combined online to make sense of the interlocutor’s linguistic production in an early stage of metaphor comprehension. Our data favor the idea that meaning construction is globally integrative and highly context-sensitive.     

Communicative intentions in wild chimpanzees: persistence and elaboration in gestural signalling

We examine evidence for communicative intent during conspecific interactions in wild chimpanzees (Budongo Forest, Uganda), focusing on persistence in gestural communication. Previous research indicates that great apes have large gestural repertoires and produce gestural communication in a flexible and intentional manner, including the production of gesture sequences. Although there is a lack of consensus on the form and function of sequences, there is some evidence that sequences are produced when signallers fail to receive any response from a recipient. Here, we provide first systematic evidence for communicative persistence in wild chimpanzees. Rather than examining only the presence or absence of a response, we used the most commonly observed response to assign meanings to gestures and examined sequence production in relation to response congruency. Chimpanzees ceased communication if successful, but persevered when unsuccessful. Chimpanzees repeated gestures when a response partially matched their goal but substituted the original gesture when a response was incongruent. Persistence was also mediated by recipient intent to respond, with more sequences produced within competitive than affiliative contexts. Gestures within sequences were homogenous in semantic meaning and signallers continued until the response matched the assigned meaning of the initial gesture. Gestural sequence production was not primarily affective; gesture intensity (in terms of modality) did not increase within sequences. Chimpanzee gestural sequences emerged to achieve specific outcomes; given variability in recipient behaviour following initial gestures, signallers were flexible in their persistence towards these goals.     

Evocation of functional and volumetric gestural knowledge by objects and words

We distinguish between grasping gestures associated with using an object for its intended purpose (functional) and those used to pick up an object (volumetric) and we develop a novel experimental framework to show that both kinds of knowledge are automatically evoked by objects and by words denoting those objects. Cued gestures were carried out in the context of depicted objects or visual words. On incongruent trials, the cued gesture was not compatible with gestures typically associated with the contextual item. On congruent trials, the gesture was compatible with the item's functional or volumetric gesture. For both gesture types, response latency was longer for incongruent trials indicating that objects and words elicited both functional and volumetric manipulation knowledge. Additional evidence, however, clearly supports a distinction between these two kinds of gestural knowledge. Under certain task conditions, functional gestures can be evoked without the associated activation of volumetric gestures. We discuss the implication of these results for theories of action evoked by objects and words, and for interpretation of functional imaging results.     

The You-I event: on the genesis of self-awareness

I present empirical evidence suggesting that an infant first becomes aware of herself as the focal center of a caregiver's attending. Yet that does not account for her awareness of herself as agent. To address this question, I bring in research on neonatal imitation, as well as studies demonstrating the existence of a neural system in which parts of the same brain areas are activated when observing another's action and when executing a similar one. Applying these findings, I consider gestural exchanges between infant and caregiver, such as reciprocal smiles and imitative vocalizations. Lacking self-awareness at first, the infant is unaware of her own agency. By returning her unwitting gesture, the caregiver singles out for her—thanks to neural matching—the gesture's kinesthesis. Moreover, the caregiver's smile, imitative vocalization, or other gesture is the form that focusing takes. The kinesthesis of the infant's gesture, in being singled out, is experienced by the infant as what the caregiver is focusing on. It is experienced as being within the focal center. In this way, the infant becomes aware of herself as a bodily entity acting toward the caregiver. Exchanges that involve matching are at first essential, I argue, in making the infant present to herself in action. Matching will cease to be necessary, but self-awareness continues to depend fundamentally on others until the acquisition of language, when the child becomes capable of talking to herself as if she were the caregiver.     

Great ape gestures: intentional communication with a rich set of innate signals

Great apes give gestures deliberately and voluntarily, in order to influence particular target audiences, whose direction of attention they take into account when choosing which type of gesture to use. These facts make the study of ape gesture directly relevant to understanding the evolutionary precursors of human language; here we present an assessment of ape gesture from that perspective, focusing on the work of the “St Andrews Group” of researchers. Intended meanings of ape gestures are relatively few and simple. As with human words, ape gestures often have several distinct meanings, which are effectively disambiguated by behavioural context. Compared to the signalling of most other animals, great ape gestural repertoires are large. Because of this, and the relatively small number of intended meanings they achieve, ape gestures are redundant, with extensive overlaps in meaning. The great majority of gestures are innate, in the sense that the species’ biological inheritance includes the potential to develop each gestural form and use it for a specific range of purposes. Moreover, the phylogenetic origin of many gestures is relatively old, since gestures are extensively shared between different genera in the great ape family. Acquisition of an adult repertoire is a process of first exploring the innate species potential for many gestures and then gradual restriction to a final (active) repertoire that is much smaller. No evidence of syntactic structure has yet been detected.     

Form and function in early communication: language and pointing gestures

Pointing gestures of verbally advanced 2-year-olds were contrasted with those of less advanced peers, in order to examine the relationships of gesture to language during the acquisition of each. Hypotheses regarding the replacement of gestural functions by speech as verbal skills improve, regarding developmental correspondences between the two communicative domains, and regarding the independence of language acquisition from nonverbal developments were drawn from evolutionary, structuralist, and nativist viewpoints, respectively. Both formal and functional aspects of each communicative skill were measured, and were shown to be largely unrelated, particularly in the gestural domain. No evidence that language replaced gesture for communication in ontogeny was obtained. Correspondences between gesture and language occurred only between functional aspects of each, and the independence of developing language from gestural advances was suggested by the findings.     

The gestural repertoire of the wild bonobo (<Emphasis Type="Italic">Pan paniscus</Emphasis>): a mutually understood communication system

In animal communication, signallers and recipients are typically different: each signal is given by one subset of individuals (members of the same age, sex, or social rank) and directed towards another. However, there is scope for signaller–recipient interchangeability in systems where most signals are potentially relevant to all age–sex groups, such as great ape gestural communication. In this study of wild bonobos (Pan paniscus), we aimed to discover whether their gestural communication is indeed a mutually understood communicative repertoire, in which all individuals can act as both signallers and recipients. While past studies have only examined the expressed repertoire, the set of gesture types that a signaller deploys, we also examined the understood repertoire, the set of gestures to which a recipient reacts in a way that satisfies the signaller. We found that most of the gestural repertoire was both expressed and understood by all age and sex groups, with few exceptions, suggesting that during their lifetimes all individuals may use and understand all gesture types. Indeed, as the number of overall gesture instances increased, so did the proportion of individuals estimated to both express and understand a gesture type. We compared the community repertoire of bonobos to that of chimpanzees, finding an 88 % overlap. Observed differences are consistent with sampling effects generated by the species’ different social systems, and it is thus possible that the repertoire of gesture types available to Pan is determined biologically.     

Language-based access to gestural components of conceptual knowledge

We report two experiments in which production of articulated hand gestures was used to reveal the nature of gestural knowledge evoked by sentences referring to manipulable objects. Two gesture types were examined: functional gestures (executed when using an object for its intended purpose) and volumetric gestures (used when picking up an object simply to move it). Participants read aloud a sentence that referred to an object but did not mention any form of manual interaction (e.g., Jane forgot the calculator) and were cued after a delay of 300 or 750 ms to produce the functional or volumetric gesture associated with the object, or a gesture that was unrelated to the object. At both cue delays, functional gestures were primed relative to unrelated gestures, but no significant priming was found for volumetric gestures. Our findings elucidate the types of motor representations that are directly linked to the meaning of words referring to manipulable objects in sentences.     

Evaluation and integration of speech and pointing gestures during referential understanding

Two experiments investigated the relative influence of speech and pointing gesture information in the interpretation of referential acts. Children averaging 3 and 5 years of age and adults viewed a videotape containing the independent manipulation of speech and gestural forms of reference. A man instructed the subjects to choose a ball or a doll by vocally labeling the referent and/or pointing to it. A synthetic speech continuum between two alternatives was crossed with the pointing gesture in a factorial design. Based on research in other domains, it was predicted that all age groups would utilize gestural information, although both speech and gestures were predicted to influence children less than adults. The main effects and interactions of speech and gesture in combination with quantitative models of performance showed the following similarities in information processing between preschoolers and adults: (1) referential evaluation of gestures occurs independently of the evaluation of linguistic reference; (2) speech and gesture are continuous, rather than discrete, sources of information; (3) 5-year-olds and adults combine the two types of information in such a way that the least ambiguous source has the most impact on the judgment. Greater discriminability of both speech and gesture information for adults compared to preschoolers indicated small quantitative progressions with development in the ability to extract and utilize referential signals.     

Gestural communication of the gorilla (Gorilla gorilla): repertoire, intentionality and possible origins

Social groups of gorillas were observed in three captive facilities and one African field site. Cases of potential gesture use, totalling 9,540, were filtered by strict criteria for intentionality, giving a corpus of 5,250 instances of intentional gesture use. This indicated a repertoire of 102 gesture types. Most repertoire differences between individuals and sites were explicable as a consequence of environmental affordances and sampling effects: overall gesture frequency was a good predictor of universality of occurrence. Only one gesture was idiosyncratic to a single individual, and was given only to humans. Indications of cultural learning were few, though not absent. Six gestures appeared to be traditions within single social groups, but overall concordance in repertoires was almost as high between as within social groups. No support was found for the ontogenetic ritualization hypothesis as the chief means of acquisition of gestures. Many gestures whose form ruled out such an origin, i.e. gestures derived from species-typical displays, were used as intentionally and almost as flexibly as gestures whose form was consistent with learning by ritualization. When using both classes of gesture, gorillas paid specific attention to the attentional state of their audience. Thus, it would be unwarranted to divide ape gestural repertoires into ‘innate, species-typical, inflexible reactions’ and ‘individually learned, intentional, flexible communication’. We conclude that gorilla gestural communication is based on a species-typical repertoire, like those of most other mammalian species but very much larger. Gorilla gestures are not, however, inflexible signals but are employed for intentional communication to specific individuals.     

Object-related knowledge and the production of gestures with imagined objects by preschool children

Previous research suggests that the gestural representation of preschool age children has a symbolic quality in the absence of real objects. It has shown a developmental progression from use of concrete body parts to more abstract imaginary-object gestural representations during the preschool years. The present study examined whether object-related knowledge is involved in the production of imaginary-object gestures. 35 children (12 3-, 11 4-, and 12 5-yr.-olds) performed gestural tasks in which they were asked to pretend to use common objects, e.g., "pretend to brush your teeth with a toothbrush." In addition, they were asked to describe the relationship between the performed gesture and object when they were pretending. Analysis indicated that, when an imaginary-object gesture was performed, it was through an evoked prior knowledge of the function and context of the use of objects. This may suggest that object-related knowledge is a key factor involved in developmental changes from body part to imaginary-object gestures and that it is involved in freeing preschool children from object substitution, which is prescribed by perceptual support.     

The origin of language and relative roles of voice and gesture in early communication development

Both vocalization and gesture are universal modes of communication and fundamental features of language development. The gestural origins theory proposes that language evolved out of early gestural use. However, evidence reported here suggests vocalization is much more prominent in early human communication than gesture is. To our knowledge no prior research has investigated the rates of emergence of both gesture and vocalization across the first year in human infants. We evaluated the rates of gestures and speech-like vocalizations (protophones) in 10 infants at 4, 7, and 11 months of age using parent-infant laboratory recordings. We found that infant protophones outnumbered gestures substantially at all three ages, ranging from >35 times more protophones than gestures at 3 months, to >2.5 times more protophones than gestures at 11 months. The results suggest vocalization, not gesture, is the predominant mode of communication in human infants in the first year.     

Gesture and early bilingual development

The relationship between speech and gestural proficiency was investigated longitudinally (from 2 years to 3 years 6 months, at 6-month intervals) in 5 French-English bilingual boys with varying proficiency in their 2 languages. Because of their different levels of proficiency in the 2 languages at the same age, these children's data were used to examine the relative contribution of language and cognitive development to gestural development. In terms of rate of gesture production, rate of gesture production with speech, and meaning of gesture and speech, the children used gestures much like adults from 2 years on. In contrast, the use of iconic and beat gestures showed differential development in the children's 2 languages as a function of mean length of utterance. These data suggest that the development of these kinds of gestures may be more closely linked to language development than other kinds (such as points). Reasons why this might be so are discussed.     

Do captive mandrills invent new gestures?

Studies of intraspecific behavioral variability have documented cases where behaviors are present in some populations or groups but are absent in others. In some cases these differences cannot be explained by recourse to environmental or genetic variation, and may instead represent “traditions”. Despite many examples of animal traditions in acoustic communication, relatively few examples exist of gestural traditions. Here I report on a study of communication across eight captive groups of mandrills (Mandrillus sphinx) in which a prominent gesture (Hand extension) was unique to two groups. Habitat variability, genetic differences, and sampling bias were not sufficient to account for the gesture’s limited distribution across the study groups. Within the two groups where the gesture did occur only the juveniles in the group performed it, consistently directing it toward adults. Quantitative analysis of the contexts and responses associated with the gesture suggested that juveniles utilized it to provoke adults. Moreover, the gesture appeared to minimize the risk juveniles incurred while inciting adults, suggesting that repeated social interactions shaped the gesture’s form. Interestingly, both the groups where the gesture emerged contained few juveniles. With limited play partners, these juveniles may have resorted to harassing adults as an alternative social play outlet. The creation of this novel gesture may thus be due to the combined influence of a shortage of play partners and of the increased free time for playful social exploration afforded by captivity. Although juveniles frequently “eavesdropped” on dyadic interactions involving the gesture and would subsequently initiate an interaction with the recipient of the gesture, there was no definitive evidence for social transmission; the gesture could instead have been independently invented by each juvenile. Electronic supplementary material The online version of this article (doi:) contains supplementary material, which is available to authorized users.