Communicative intentions in wild chimpanzees: persistence and elaboration in gestural signalling

We examine evidence for communicative intent during conspecific interactions in wild chimpanzees (Budongo Forest, Uganda), focusing on persistence in gestural communication. Previous research indicates that great apes have large gestural repertoires and produce gestural communication in a flexible and intentional manner, including the production of gesture sequences. Although there is a lack of consensus on the form and function of sequences, there is some evidence that sequences are produced when signallers fail to receive any response from a recipient. Here, we provide first systematic evidence for communicative persistence in wild chimpanzees. Rather than examining only the presence or absence of a response, we used the most commonly observed response to assign meanings to gestures and examined sequence production in relation to response congruency. Chimpanzees ceased communication if successful, but persevered when unsuccessful. Chimpanzees repeated gestures when a response partially matched their goal but substituted the original gesture when a response was incongruent. Persistence was also mediated by recipient intent to respond, with more sequences produced within competitive than affiliative contexts. Gestures within sequences were homogenous in semantic meaning and signallers continued until the response matched the assigned meaning of the initial gesture. Gestural sequence production was not primarily affective; gesture intensity (in terms of modality) did not increase within sequences. Chimpanzee gestural sequences emerged to achieve specific outcomes; given variability in recipient behaviour following initial gestures, signallers were flexible in their persistence towards these goals.     

Enhancement of naming in nonfluent aphasia through gesture

In a number of studies that have examined the gestural disturbance in aphasia and the utility of gestural interventions in aphasia therapy, a variable degree of facilitation of verbalization during gestural activity has been reported. The present study examined the effect of different unilateral gestural movements on simultaneous oral-verbal expression, specifically naming to confrontation. It was hypothesized that activation of the phylogenetically older proximal motor system of the hemiplegic right arm in the execution of a communicative but nonrepresentational pointing gesture would have a facilitatory effect on naming ability. Twenty-four aphasic patients, representing five aphasic subtypes, including Broca's, Transcortical Motor, Anomic, Global, and Wernicke's aphasics were assessed under three gesture/naming conditions. The findings indicated that gestures produced through activation of the proximal (shoulder) musculature of the right paralytic limb differentially facilitated naming performance in the nonfluent subgroup, but not in the Wernicke's aphasics. These findings may be explained on the view that functional activation of the archaic proximal motor system of the hemiplegic limb, in the execution of a communicative gesture, permits access to preliminary stages in the formative process of the anterior action microgeny, which ultimately emerges in vocal articulation.     

The relation between gesture and language in aphasic communication

Effective communication in aphasia depends not only on use of preserved linguistic capacities but also (and perhaps primarily) on the capacity to exploit alternative modalities of communication, such as gesture. To ascertain the capacity of aphasic patients to use gesture in their spontaneous communication, informally structured interviews were conducted with two Wernicke's aphasics and two Broca's aphasics, as well as with four normal controls. The performances of the patient groups were compared on the physical parameters of gesture, the points in the communication where gestures occurred, and several facets of the semantics and pragmatics of gesture. Generally speaking, the gestures of the aphasics closely paralleled their speech output: on most indices, the performance of the Wernicke's aphasics more closely resembled that of the normal controls. Wernicke's aphasics differed from normals in the clarity of their language and gestures: While individual linguistic units were often clear, the relation among units was not. In contrast, the Broca's aphasics equaled or surpassed the normal controls in the clarity of their communications. The results offer little support for the view that aphasic patients spontaneously enhance their communicative efficacy through the use of gesture; these findings can, however, be interpreted as evidence in favor of a “central organizer” which controls critical features of communication, irrespective of the modality of expression.     

Communicative gestures of the hand and arm when 4-year-old sons and their mothers interact

This study investigated a wide range of communicative hand/arm gestures of 4-year-old males when interacting with their mothers. The types of gesture categories observed were in keeping with the predicted encoding ability of this aged child. Pantomimic and deictic gestures were observed in significantly greater numbers than semantic modifying and relational gestures. Although it was found that the mothers' gestural usage reflected the type of gesture categories seen in the children's group, no correlation was found between gesture usage of individual mother-child pairs.     

Differential use of vocal and gestural communication by chimpanzees (Pan troglodytes) in response to the attentional status of a human (Homo sapiens)

This study examined the communicative behavior of 49 captive chimpanzees (Pan troglodytes), particularly their use of vocalizations, manual gestures, and other auditory- or tactile-based behaviors as a means of gaining an inattentive audience's attention. A human (Homo sapiens) experimenter held a banana while oriented either toward or away from the chimpanzee. The chimpanzees' behavior was recorded for 60 s. Chimpanzees emitted vocalizations faster and were more likely to produce vocalizations as their 1st communicative behavior when a human was oriented away from them. Chimpanzees used manual gestures more frequently and faster when the human was oriented toward them. These results replicate the findings of earlier studies on chimpanzee gestural communication and provide new information about the intentional and functional use of their vocalizations.     

Nonverbal communication as a compensative strategy for severely nonfluent aphasics? A quantitative approach

Seven severely nonfluent aphasics and one relative or close friend were investigated in an interview situation. The conversation was videotaped and analyzed for the use of nonverbal communication between aphasic and partner. The results indicate that the group of aphasics used significantly more frequently and for a significantly longer period of time nonverbal channels of communication than their healthy partners. The aphasic patients also used significantly fewer speech-focused movements and significantly more codified gestures. Nonverbal elements were more frequently used as speech substitutes by the group of aphasics.     

Form and function in early communication: language and pointing gestures

Pointing gestures of verbally advanced 2-year-olds were contrasted with those of less advanced peers, in order to examine the relationships of gesture to language during the acquisition of each. Hypotheses regarding the replacement of gestural functions by speech as verbal skills improve, regarding developmental correspondences between the two communicative domains, and regarding the independence of language acquisition from nonverbal developments were drawn from evolutionary, structuralist, and nativist viewpoints, respectively. Both formal and functional aspects of each communicative skill were measured, and were shown to be largely unrelated, particularly in the gestural domain. No evidence that language replaced gesture for communication in ontogeny was obtained. Correspondences between gesture and language occurred only between functional aspects of each, and the independence of developing language from gestural advances was suggested by the findings.     

Why the spontaneous images created by the hands during talk can help make TV advertisements more effective

The design of effective communications depends upon an adequate model of the communication process. The traditional model is that speech conveys semantic information and bodily movement conveys information about emotion and interpersonal attitudes. But McNeill (2000) argues that this model is fundamentally wrong and that some bodily movements, namely spontaneous hand movements generated during talk (iconic gestures), are integral to semantic communication. But can we increase the effectiveness of communication using this new theory? Focusing on advertising we found that advertisements in which the message was split between speech and iconic gesture (possible on TV) were significantly more effective than advertisements in which meaning resided purely in speech or language (radio/ newspaper). We also found that the significant differences in communicative effectiveness were maintained across five consecutive trials. We compared the communicative power of professionally made TV advertisements in which a spoken message was accompanied either by iconic gestures or by pictorial images, and found the iconic gestures to be more effective. We hypothesized that iconic gestures are so effective because they illustrate and isolate just the core semantic properties of a product. This research suggests that TV advertisements can be made more effective by incorporating iconic gestures with exactly the right temporal and semantic properties.     

A longitudinal examination of the transition to symbolic communication in the second year of life

Between 10 and 24 months of age, children progress from communicating through conventional signals to communicating through symbols in a variety of situations. The present study investigates this transition analysing mother–child communication frames and the child's communicative acts, and tracing the developmental changes in both frames and communicative acts. Four children (2 girls, 2 boys) and their mothers were observed longitudinally and extensively, from 10 to 24 months of age, using a multiple case‐study method. Through the detailed investigation of these single cases, clear developmental rajectories were found, showing that conventional frames and representational gestures ‘bridge’ the transition from attentional to symbolic communication. There were significant differences between dyads and children in the developmental trajectories for several frames and communicative acts. The results not only confirm previous findings on the development of verbal and gestural communication, but provide new insights into the transition to symbolic communication in the second year of life, first because of the use of frame analysis and second because of a focus on individual trajectories in development. Copyright © 2003 John Wiley & Sons, Ltd.     

The repertoire and intentionality of gestural communication in wild chimpanzees

A growing body of evidence suggests that human language may have emerged primarily in the gestural rather than vocal domain, and that studying gestural communication in great apes is crucial to understanding language evolution. Although manual and bodily gestures are considered distinct at a neural level, there has been very limited consideration of potential differences at a behavioural level. In this study, we conducted naturalistic observations of adult wild East African chimpanzees (Pan troglodytes schweinfurthii) in order to establish a repertoire of gestures, and examine intentionality of gesture production, use and comprehension, comparing across manual and bodily gestures. At the population level, 120 distinct gesture types were identified, consisting of 65 manual gestures and 55 bodily gestures. Both bodily and manual gestures were used intentionally and effectively to attain specific goals, by signallers who were sensitive to recipient attention. However, manual gestures differed from bodily gestures in terms of communicative persistence, indicating a qualitatively different form of behavioural flexibility in achieving goals. Both repertoire size and frequency of manual gesturing were more affiliative than bodily gestures, while bodily gestures were more antagonistic. These results indicate that manual gestures may have played a significant role in the emergence of increased flexibility in great ape communication and social bonding.     

The relationship between limb apraxia and the spontaneous use of communicative gesture in aphasia

This study investigated the relationship between limb apraxia, as assessed by a formal clinical test, and the production of spontaneous communicative gesture, as measured by a newly designed rating scale--the Nonvocal Communication Scale (NCS). Subjects were aphasic adult males with cerebrovascular lesions of the left hemisphere. The performance of aphasic patients on the praxis test and the NCS was independent of demographic, neuroanatomic, linguistic, or cognitive variables, except for global aphasics who were low-scoring across the board. There was a significant positive correlation, however, between praxis ability and spontaneous gestural communication. Clinical implications of these findings are discussed.     

Degree of Language Experience Modulates Visual Attention to Visible Speech and Iconic Gestures During Clear and Degraded Speech Comprehension

Visual information conveyed by iconic hand gestures and visible speech can enhance speech comprehension under adverse listening conditions for both native and non‐native listeners. However, how a listener allocates visual attention to these articulators during speech comprehension is unknown. We used eye‐tracking to investigate whether and how native and highly proficient non‐native listeners of Dutch allocated overt eye gaze to visible speech and gestures during clear and degraded speech comprehension. Participants watched video clips of an actress uttering a clear or degraded (6‐band noise‐vocoded) action verb while performing a gesture or not, and were asked to indicate the word they heard in a cued‐recall task. Gestural enhancement was the largest (i.e., a relative reduction in reaction time cost) when speech was degraded for all listeners, but it was stronger for native listeners. Both native and non‐native listeners mostly gazed at the face during comprehension, but non‐native listeners gazed more often at gestures than native listeners. However, only native but not non‐native listeners' gaze allocation to gestures predicted gestural benefit during degraded speech comprehension. We conclude that non‐native listeners might gaze at gesture more as it might be more challenging for non‐native listeners to resolve the degraded auditory cues and couple those cues to phonological information that is conveyed by visible speech. This diminished phonological knowledge might hinder the use of semantic information that is conveyed by gestures for non‐native compared to native listeners. Our results demonstrate that the degree of language experience impacts overt visual attention to visual articulators, resulting in different visual benefits for native versus non‐native listeners.     

Hand gestures and perceived influence in small group interaction

A laboratory study was carried out to establish the relative importance of verbal and gestural behavior, as well as their interaction, for perceived social influence in more or less competitive small groups. Forty women (psychology students) participated in leaderless small group discussions of different sizes (four-member and eight-member): at the end, each member rated the perceived influence in decision-making of every other member. Verbal dominance coding is based on traditional quantitative conversational dominance (number of talk turns). Gestural coding (conversational, ideational, object-adaptor, self-adaptor gestures) is based on classical gesture classifications. Beside a substantial effect of verbal dominance, the main result is that frequency of object-adaptors and conversational (only in large groups) and ideational (in both small and large groups) gestures increases perceived influence scores particularly when the verbal dominance of the speaker is low.     

Cross-species referential signalling events in domestic dogs (<Emphasis Type="Italic">Canis familiaris</Emphasis>)

Referential gestures are used by a signaller to draw a recipient’s attention to a specific object, individual or event in the environment. These gestures have received much research attention in relation to human and non-human primates with great apes being shown to possess impressive gestural repertoires. Domestic dogs (Canis familiaris) meanwhile provide an ideal non-primate candidate for investigating referential signalling due to their unique relationship with humans that centres on non-verbal communication with frequent interaction. Here we observed 37 pet dogs in their own homes. Owners recorded 242 videos containing 47 potential referential gesture events. We analysed those recordings to reveal evidence of 19 referential gestures performed by domestic dogs during everyday communicative bouts with humans, showing that the gestures conform to the five features of referential signalling. Our study exposes impressive gesturing abilities in a non-primate mammal; especially when viewed in the context of the cross-species rather than intraspecific communication.     

用户运动信息反馈形式对体感操作用户体验的影响——一项基于Kinect的可用性研究

体感交互是新兴的人机交互方式;当前,相关的人因学研究相对缺乏。本研究采用典型任务结合主观评价法,首次探索了用户运动信息反馈形式对体感交互用户体验的影响。笔者从易学性、控制性、情感与心理负荷四个方面,比较了有、无人形反馈形式对游戏体感菜单操作可用性的影响。结果发现,二者在心理负荷上无差异,无人形反馈在易学性上优于有人形,但有人形反馈在控制感上优于无人形,且更受用户喜爱。基于该结果,笔者建议在体感操作界面上使用有人形反馈。     

体感交互人因学研究回顾与展望

人-机体感交互是个体使用肢体动作直接与机器进行的交互,是新近发展起来的自然交互方式。人因学研究是体感交互研究中的核心问题之一。笔者从体感交互的发展历史、已有体感交互方式、基于体感设备Kinect的人因学评估三个方面系统总结了体感交互的已有人因学研究,并对体感交互中存在的人因学问题与潜在研究方向做了论述。     

The gestural repertoire of the wild bonobo (<Emphasis Type="Italic">Pan paniscus</Emphasis>): a mutually understood communication system

In animal communication, signallers and recipients are typically different: each signal is given by one subset of individuals (members of the same age, sex, or social rank) and directed towards another. However, there is scope for signaller–recipient interchangeability in systems where most signals are potentially relevant to all age–sex groups, such as great ape gestural communication. In this study of wild bonobos (Pan paniscus), we aimed to discover whether their gestural communication is indeed a mutually understood communicative repertoire, in which all individuals can act as both signallers and recipients. While past studies have only examined the expressed repertoire, the set of gesture types that a signaller deploys, we also examined the understood repertoire, the set of gestures to which a recipient reacts in a way that satisfies the signaller. We found that most of the gestural repertoire was both expressed and understood by all age and sex groups, with few exceptions, suggesting that during their lifetimes all individuals may use and understand all gesture types. Indeed, as the number of overall gesture instances increased, so did the proportion of individuals estimated to both express and understand a gesture type. We compared the community repertoire of bonobos to that of chimpanzees, finding an 88 % overlap. Observed differences are consistent with sampling effects generated by the species’ different social systems, and it is thus possible that the repertoire of gesture types available to Pan is determined biologically.     

What is the best strategy for retaining gestures in working memory?

This study aimed to determine whether the recall of gestures in working memory could be enhanced by verbal or gestural strategies. We also attempted to examine whether these strategies could help resist verbal or gestural interference. Fifty-four participants were divided into three groups according to the content of the training session. This included a control group, a verbal strategy group (where gestures were associated with labels) and a gestural strategy group (where participants repeated gestures and were told to imagine reproducing the movements). During the experiment, the participants had to reproduce a series of gestures under three conditions: “no interference”, gestural interference (gestural suppression) and verbal interference (articulatory suppression). The results showed that task performance was enhanced in the verbal strategy group, but there was no significant difference between the gestural strategy and control groups. Moreover, compared to the “no interference” condition, performance decreased in the presence of gestural interference, except within the verbal strategy group. Finally, verbal interference hindered performance in all groups. The discussion focuses on the use of labels to recall gestures and differentiates the induced strategies from self-initiated strategies.     

Gestural communication of the gorilla (Gorilla gorilla): repertoire, intentionality and possible origins

Social groups of gorillas were observed in three captive facilities and one African field site. Cases of potential gesture use, totalling 9,540, were filtered by strict criteria for intentionality, giving a corpus of 5,250 instances of intentional gesture use. This indicated a repertoire of 102 gesture types. Most repertoire differences between individuals and sites were explicable as a consequence of environmental affordances and sampling effects: overall gesture frequency was a good predictor of universality of occurrence. Only one gesture was idiosyncratic to a single individual, and was given only to humans. Indications of cultural learning were few, though not absent. Six gestures appeared to be traditions within single social groups, but overall concordance in repertoires was almost as high between as within social groups. No support was found for the ontogenetic ritualization hypothesis as the chief means of acquisition of gestures. Many gestures whose form ruled out such an origin, i.e. gestures derived from species-typical displays, were used as intentionally and almost as flexibly as gestures whose form was consistent with learning by ritualization. When using both classes of gesture, gorillas paid specific attention to the attentional state of their audience. Thus, it would be unwarranted to divide ape gestural repertoires into ‘innate, species-typical, inflexible reactions’ and ‘individually learned, intentional, flexible communication’. We conclude that gorilla gestural communication is based on a species-typical repertoire, like those of most other mammalian species but very much larger. Gorilla gestures are not, however, inflexible signals but are employed for intentional communication to specific individuals.     

Division of the articulatory loop according to sensory modality using double dissociation

According to Wilson and Fox (2007), working memory for gestures has the same characteristics as the phonological loop. The purpose of our research was to determine whether there is a common articulatory loop for verbal and gestural learning. We carried out two double dissociation experiments. The first involved 84 participants who had to reproduce a series of three gestures under three conditions: control, gestural interference (repeated gestures) and verbal interference (repeated “blah blah”). A significant difference in performance was observed; gestural interference resulted in the weakest performance, while there was no difference between the verbal interference condition and the control group. The second experiment, with 30 participants, involved the memorisation of letters and digits; performance was significantly affected by verbal interference but there was no difference between the gestural interference condition and the control group. The consequences of the dissociations are discussed in relation to Baddeley's (2000) model.