The neural basis of primate social communication

A sophisticated ability both to generate displays of emotion and to respond to expressive behaviors of other individuals has emerged as a specialization in the course of primate evolution. Studies of the social behavior of nonhuman primates, especially those most related to ourselves, indicate that monkeys and apes are able to interpret social signals so as to assess the motivations of others. Studies of brain activity in monkeys looking at pictures of faces, facial expressions, and body movements, reveal regions of apparent specialized responsiveness to visual social stimuli. The existence of a discrete neural system in humans for cognition which generates a psychological model of others is suggested by patterns of deficit seen in certain neurologic syndromes. Empathy has several components and appears to lie on an evolutionary continuum.Based on a presentation given in New Orleans on February 17, 1990, at the AAAS symposium Empathy in Infancy and Later Development. Supported by the Department of Veterans Affairs.     

The neuroethology of primate vocal communication: substrates for the evolution of speech

In this article, we review behavioral and neurobiological studies of the perception and use of species-specific vocalizations by non-human primates. At the behavioral level, primate vocal perception shares many features with speech perception by humans. These features include a left-hemisphere bias towards conspecific vocalizations, the use of temporal features for identifying different calls, and the use of calls to refer to objects and events in the environment. The putative neural bases for some of these behaviors have been revealed by recent studies of the primate auditory and prefrontal cortices. These studies also suggest homologies with the human language circuitry. Thus, a synthesis of cognitive, ethological and neurobiological approaches to primate vocal behavior is likely to yield the richest understanding of the neural bases of speech perception, and might also shed light on the evolutionary precursors to language.     

SLOT: A research platform for investigating multimodal communication

In this article, we present the spatial logistics task (SLOT) platform for investigating multimodal communication between 2 human participants. Presented are the SLOT communication task and the software and hardware that has been developed to run SLOT experiments and record the participants’ multimodal behavior. SLOT offers a high level of flexibility in varying the context of the communication and is particularly useful in studies of the relationship between pen gestures and speech. We illustrate the use of the SLOT platform by discussing the results of some early experiments. The first is an experiment on negotiation with a one-way mirror between the participants, and the second is an exploratory study of automatic recognition of spontaneous pen gestures. The results of these studies demonstrate the usefulness of the SLOT platform for conducting multimodal communication research in both human-human and human-computer interactions.     

Muslims,identity and multimodal communication on the internet

The Internet provides a space and medium within which Muslims can shape the relationship between their religious identity and their social and political affiliations. The subjectivities of Muslims who use online space are in turn shaped by the parameters and possibilities of the Internet’s architecture and language. The multiple linkages of online spaces and the particular vernacular spoken in these spaces, a mix of written text, imagery and sound, privilege new kinds of actors and new forms of expressive and rhetorical activities. In this new space and medium, the question of imagining (or rejecting) a global Muslim identity demonstrates the subtle interplay involved in the formation of religious and media subjectivities. Developing a critical understanding of multimodal representation and communication is an essential component in studying Muslim engagement with the Internet.     

The primate working memory networks

Working memory has long been associated with the prefrontal cortex, since damage to this brain area can critically impair the ability to maintain and update mnemonic information. Anatomical and physiological evidence suggests, however, that the prefrontal cortex is part of a broader network of interconnected brain areas involved in working memory. These include the parietal and temporal association areas of the cerebral cortex, cingulate and limbic areas, and subcortical structures such as the mediodorsal thalamus and the basal ganglia. Neurophysiological studies in primates confirm the involvement of areas beyond the frontal lobe and illustrate that working memory involves parallel, distributed neuronal networks. In this article, we review the current understanding of the anatomical organization of networks mediating working memory and the neural correlates of memory manifested in each of their nodes. The neural mechanisms of memory maintenance and the integrative role of the prefrontal cortex are also discussed.     

The dynamics of multimodal integration: The averaging diffusion model

We combine extant theories of evidence accumulation and multi-modal integration to develop an integrated framework for modeling multimodal integration as a process that unfolds in real time. Many studies have formulated sensory processing as a dynamic process where noisy samples of evidence are accumulated until a decision is made. However, these studies are often limited to a single sensory modality. Studies of multimodal stimulus integration have focused on how best to combine different sources of information to elicit a judgment. These studies are often limited to a single time point, typically after the integration process has occurred. We address these limitations by combining the two approaches. Experimentally, we present data that allow us to study the time course of evidence accumulation within each of the visual and auditory domains as well as in a bimodal condition. Theoretically, we develop a new Averaging Diffusion Model in which the decision variable is the mean rather than the sum of evidence samples and use it as a base for comparing three alternative models of multimodal integration, allowing us to assess the optimality of this integration. The outcome reveals rich individual differences in multimodal integration: while some subjects’ data are consistent with adaptive optimal integration, reweighting sources of evidence as their relative reliability changes during evidence integration, others exhibit patterns inconsistent with optimality.     

The information theoretic entropy function as a total expected participation index for communication network experiments

This paper shows how the concept of an incidence matrix of communications can be used to define the entropy of a finite scheme. The properties of the entropy function are examined and the function is found to be best interpreted as a total expected participation index. Data is presented showing the relationship between structural centrality and the new total expected participation index. In general, as the network becomes more centralized the smaller the value of the participation index and as the network becomes more structurally decentralized the greater the participation index.This research was supported in part by Ford Foundation Grant 1-40055 to the Graduate School of Industrial Administration at Carnegie Institute of Technology for Research in Organizational Behavior.The author wishes to acknowledge the aid of Terry B. Marbach in the preparation of data.     

The effects of aging on the working memory processes of multimodal information

Normal aging is associated with deficits in working memory processes. However, the majority of research has focused on storage or inhibitory processes using unimodal paradigms, without addressing their relationships using different sensory modalities. Hence, we pursued two objectives. First, was to examine the effects of aging on storage and inhibitory processes. Second, was to evaluate aging effects on multisensory integration of visual and auditory stimuli. To this end, young and older participants performed a multimodal task for visual and auditory pairs of stimuli with increasing memory load at encoding and interference during retention. Our results showed an age-related increased vulnerability to interrupting and distracting interference reflecting inhibitory deficits related to the off-line reactivation and on-line suppression of relevant and irrelevant information, respectively. Storage capacity was impaired with increasing task demands in both age groups. Additionally, older adults showed a deficit in multisensory integration, with poorer performance for new visual compared to new auditory information.     

Cognitive function, physical activity, and aging: possible biological links and implications for multimodal interventions

Increasing evidence shows the positive effect of physical activity (PA) on maintaining cognitive function. Both processes seem intrinsically linked to each other. The most likely mechanism is a reciprocal stimulation of neuroplasticity. Based on extensive experimental work on animals and humans, the concept of an enriched environment, including PA and challenging cognitive tasks, has provided the basis to systematically assess possible interventions for successful aging. I will use recent findings on brain mechanisms associated with PA and its effects on higher cognitive function at a systems and molecular level to demonstrate the need to design effective interventions. Such interventions should be designed to take advantage of the presumed compensatory mechanisms of elderly individuals, thereby limiting functional decline in higher cognitive performance in aging people. My review of the most recent relevant publications concerning this topic supports the notion that it is a promising approach to provide cognitive training and PA in conjunction, since the combination may generate synergistic beneficial changes than either one individually. Multimodal training programs should therefore be tested. However, at present there is insufficient evidence to conclude that multimodal interventions are superior to isolated cognitive or physical exercise interventions, since major studies addressing this topic are lacking. These studies are needed to conclusively prove that both strategies will positively interact when used in combined interventions. Finally the use of modern technology for these interventions will be discussed.     

The basis of Stroop interference involving the multimodal correlates of auditory pitch

A pure auditory tone has a range of multimodal qualities that are determined by its pitch. A reaction-time task was used to demonstrate that subjects respond immediately and automatically to these qualities. Subjects were required to press one of two keys depending on which word, from a limited set, appeared on a microcomputer screen. The words were antonyms that represented multimodal stimulus qualities, and they were assigned to alternative responses so that the two words that shared the same response were correlated in the same way with pitch. As an incidental stimulus, either a 50 Hz tone or a 5500 Hz tone accompanied the presentation of each word. Subjects were found to respond more slowly when the multimodal qualities of the tone were incongruent with the qualities represented by the test word. When the stimulus-response mapping rules were changed, however, the Stroop effect did not occur; suggesting that a polarised semantic code of the incidental tone, that embraces its multimodal features, accesses the same semantic register as the equivalent code for the test word itself.     

Form and function in early communication: language and pointing gestures

Pointing gestures of verbally advanced 2-year-olds were contrasted with those of less advanced peers, in order to examine the relationships of gesture to language during the acquisition of each. Hypotheses regarding the replacement of gestural functions by speech as verbal skills improve, regarding developmental correspondences between the two communicative domains, and regarding the independence of language acquisition from nonverbal developments were drawn from evolutionary, structuralist, and nativist viewpoints, respectively. Both formal and functional aspects of each communicative skill were measured, and were shown to be largely unrelated, particularly in the gestural domain. No evidence that language replaced gesture for communication in ontogeny was obtained. Correspondences between gesture and language occurred only between functional aspects of each, and the independence of developing language from gestural advances was suggested by the findings.