Dorsal and ventral streams: a framework for understanding aspects of the functional anatomy of language

Despite intensive work on language-brain relations, and a fairly impressive accumulation of knowledge over the last several decades, there has been little progress in developing large-scale models of the functional anatomy of language that integrate neuropsychological, neuroimaging, and psycholinguistic data. Drawing on relatively recent developments in the cortical organization of vision, and on data from a variety of sources, we propose a new framework for understanding aspects of the functional anatomy of language which moves towards remedying this situation. The framework posits that early cortical stages of speech perception involve auditory fields in the superior temporal gyrus bilaterally (although asymmetrically). This cortical processing system then diverges into two broad processing streams, a ventral stream, which is involved in mapping sound onto meaning, and a dorsal stream, which is involved in mapping sound onto articulatory-based representations. The ventral stream projects ventro-laterally toward inferior posterior temporal cortex (posterior middle temporal gyrus) which serves as an interface between sound-based representations of speech in the superior temporal gyrus (again bilaterally) and widely distributed conceptual representations. The dorsal stream projects dorso-posteriorly involving a region in the posterior Sylvian fissure at the parietal-temporal boundary (area Spt), and ultimately projecting to frontal regions. This network provides a mechanism for the development and maintenance of "parity" between auditory and motor representations of speech. Although the proposed dorsal stream represents a very tight connection between processes involved in speech perception and speech production, it does not appear to be a critical component of the speech perception process under normal (ecologically natural) listening conditions, that is, when speech input is mapped onto a conceptual representation. We also propose some degree of bi-directionality in both the dorsal and ventral pathways. We discuss some recent empirical tests of this framework that utilize a range of methods. We also show how damage to different components of this framework can account for the major symptom clusters of the fluent aphasias, and discuss some recent evidence concerning how sentence-level processing might be integrated into the framework.     

幼儿汉语口语感知特点及神经机制

口语感知是当前心理语言学的研究热点, 然而以往研究大多以婴儿和成人为被试, 缺乏对幼儿口语感知的研究。此外, 现有口语感知模型主要是基于非声调语言研究建立起来的, 对汉语并不完全适用。汉语是一种声调语言, 在语音构成上不同于非声调语言。本项目将立足于汉语口语特点, 以3~5岁幼儿为研究对象, 考察幼儿汉语口语感知特点及神经机制。综合使用眼动方法、ERP方法和LORETA源定位技术探讨以下问题：(1)幼儿在前注意阶段和注意阶段的听觉语音辨别特点; (2)幼儿汉语口语词汇识别过程中音段信息和超音段信息的作用; (3)幼儿汉语口语感知的神经机制。本项目研究结果将揭示幼儿汉语口语感知特点, 为完善现有的口语感知模型提供新的实验证据。     

A Connectionist Model of Phonological Representation in Speech Perception

A number of recent studies have examined the effects of phonological variation on the perception of speech. These studies show that both the lexical representations of words and the mechanisms of lexical access are organized so that natural, systematic variation is tolerated by the perceptual system, while a general intolerance of random deviation is maintained. Lexical abstraction distinguishes between phonetic features that form the invariant core of a word and those that are susceptible to variation. Phonological inference relies on the context of surface changes to retrieve the underlying phonological form. In this article we present a model of these processes in speech perception, based on connectionist learning techniques. A simple recurrent network was trained on the mapping from the variant surface form of speech to the underlying form. Once trained, the network exhibited features of both abstraction and inference in its processing of normal speech, and predicted that similar behavior will be found in the perception of nonsense words. This prediction was confirmed in subsequent research (Gaskell & Marslen-Wilson, 1994).     

Left posterior auditory-related cortices participate both in speech perception and speech production: Neural overlap revealed by fMRI

Recent neuroimaging studies and neuropsychological data suggest that there are regions in posterior auditory cortex that participate both in speech perception and speech production. An outstanding question is whether the same neural regions support both perception and production or whether there exist discrete cortical fields subserving these functions. Previous neurophysiological studies suggest that there is indeed regional overlap between these systems, but those studies used a rehearsal task to assess production. The present study addressed this question in an event-related fMRI experiment in which subjects listened to speech and in separate trials, performed a covert object naming task. Single subject analysis revealed regions of coactivation for speech perception and production in the left posterior superior temporal sulcus (pSTS), left area Spt (a region in the Sylvian fissure at the parietal-temporal boundary), and left inferior frontal gyrus. These results are consistent with lesion data and previous physiological data indicating that posterior auditory cortex plays a role in both reception and expression of speech. We discuss these findings within the context of a neuroanatomical framework that proposes these neural sites are a part of an auditory-motor integration system.     

The neuroethology of primate vocal communication: substrates for the evolution of speech

In this article, we review behavioral and neurobiological studies of the perception and use of species-specific vocalizations by non-human primates. At the behavioral level, primate vocal perception shares many features with speech perception by humans. These features include a left-hemisphere bias towards conspecific vocalizations, the use of temporal features for identifying different calls, and the use of calls to refer to objects and events in the environment. The putative neural bases for some of these behaviors have been revealed by recent studies of the primate auditory and prefrontal cortices. These studies also suggest homologies with the human language circuitry. Thus, a synthesis of cognitive, ethological and neurobiological approaches to primate vocal behavior is likely to yield the richest understanding of the neural bases of speech perception, and might also shed light on the evolutionary precursors to language.     

Multidisciplinary assessment and diagnosis of conversion disorder in a patient with foreign accent syndrome

Multiple reports have described patients with disordered articulation and prosody, often following acute aphasia, dysarthria, or apraxia of speech, which results in the perception by listeners of a foreign-like accent. These features led to the term foreign accent syndrome (FAS), a speech disorder with perceptual features that suggest an indistinct, non-native speaking accent. Also correctly known as psuedoforeign accent, the speech does not typically match a specific foreign accent, but is rather a constellation of speech features that result in the perception of a foreign accent by listeners. The primary etiologies of FAS are cerebrovascular accidents or traumatic brain injuries which affect cortical and subcortical regions critical to expressive speech and language production. Far fewer cases of FAS associated with psychiatric conditions have been reported. We will present the clinical history, neurological examination, neuropsychological assessment, cognitive-behavioral and biofeedback assessments, and motor speech examination of a patient with FAS without a known vascular, traumatic, or infectious precipitant. Repeated multidisciplinary examinations of this patient provided convergent evidence in support of FAS secondary to conversion disorder. We discuss these findings and their implications for evaluation and treatment of rare neurological and psychiatric conditions.     

Perception of synthetic speech produced automatically by rule: Intelligibility of eight text-to-speech systems

We present the results of studies designed to measure the segmental intelligibility of eight text-to-speech systems and a natural speech control, using the Modified Rhyme Test (MRT). Results indicated that the voices tested could be grouped into four categories: natural speech, high-quality synthetic speech, moderate-quality synthetic speech, and low-quality synthetic speech. The overall performance of the best synthesis system, DECtalk-Paul, was equivalent to natural speech only in terms of performance on initial consonants. The findings are discussed in terms of recent work investigating the perception of synthetic speech under more severe conditions. Suggestions for future research on improving the quality of synthetic speech are also considered.     

The fragile nature of the speech-perception deficit in dyslexia: natural vs synthetic speech

A number of studies reported that developmental dyslexics are impaired in speech perception, especially for speech signals consisting of rapid auditory transitions. These studies mostly made use of a categorical-perception task with synthetic-speech samples. In this study, we show that deficits in the perception of synthetic speech do not generalise to the perception of more naturally sounding speech, even if the same experimental paradigm is used. This contrasts with the assumption that dyslexics are impaired in the perception of rapid auditory transitions.     

The supramodal brain: implications for auditory perception

ABSTRACT

The perceptual brain is designed around multisensory input. Areas once thought dedicated to a single sense are now known to work with multiple senses. It has been argued that the multisensory nature of the brain reflects a cortical architecture for which task, rather than sensory system, is the primary design principle. This supramodal thesis is supported by recent research on human echolocation and multisensory speech perception. In this review, we discuss the behavioural implications of a supramodal architecture, especially as they pertain to auditory perception. We suggest that the architecture implies a degree of perceptual parity between the senses and that cross-sensory integration occurs early and completely. We also argue that a supramodal architecture implies that perceptual experience can be shared across modalities and that this sharing should occur even without bimodal experience. We finish by briefly suggesting areas of future research.     

Thinking the voice: neural correlates of voice perception

The human voice is the carrier of speech, but also an "auditory face" that conveys important affective and identity information. Little is known about the neural bases of our abilities to perceive such paralinguistic information in voice. Results from recent neuroimaging studies suggest that the different types of vocal information could be processed in partially dissociated functional pathways, and support a neurocognitive model of voice perception largely similar to that proposed for face perception.     

Towards a functional neuroanatomy of speech perception

The functional neuroanatomy of speech perception has been difficult to characterize. Part of the difficulty, we suggest, stems from the fact that the neural systems supporting 'speech perception' vary as a function of the task. Specifically, the set of cognitive and neural systems involved in performing traditional laboratory speech perception tasks, such as syllable discrimination or identification, only partially overlap those involved in speech perception as it occurs during natural language comprehension. In this review, we argue that cortical fields in the posterior-superior temporal lobe, bilaterally, constitute the primary substrate for constructing sound-based representations of speech, and that these sound-based representations interface with different supramodal systems in a task-dependent manner. Tasks that require access to the mental lexicon (i.e. accessing meaning-based representations) rely on auditory-to-meaning interface systems in the cortex in the vicinity of the left temporal-parietal-occipital junction. Tasks that require explicit access to speech segments rely on auditory-motor interface systems in the left frontal and parietal lobes. This auditory-motor interface system also appears to be recruited in phonological working memory.     

Ambiguity,Competition, and Blending in Spoken Word Recognition

A critical property of the perception of spoken words is the transient ambiguity of the speech signal. In localist models of speech perception this ambiguity is captured by allowing the parallel activation of multiple lexical representations. This paper examines how a distributed model of speech perception can accommodate this property. Statistical analyses of vector spaces show that coactivation of multiple distributed representations is inherently noisy, and depends on parameters such as sparseness and dimensionality. Furthermore, the characteristics of coactivation vary considerably, depending on the organization of distributed representations within the mental lexicon. This view of lexical access is supported by analyses of phonological and semantic word representations, which provide an explanation of a recent set of experiments on coactivation in speech perception (Gaskell & Marslen–Wilson, 1999).     

婴儿对语音和非语音的感知:认知和神经机制

婴儿听觉感知能力的发展对于他们未来的语言学习和社会化都具有重要意义。过去大量的研究主要关注语音感知方面,只有较少的研究将非语音感知纳入考虑之中,但了解非语音感知的特征和机制将有助于增加研究者对听觉加工以及儿童发育的认识。该文分别介绍了婴儿语音感知中的三种偏好——对语音、“婴儿语”和母语的偏好,并尝试着将非语音分为音乐、人类的非言语发声、环境声音三类进行阐述。通过对比这两大类声音的感知得到婴儿可能存在语音感知的左脑偏侧化和音乐感知的右脑偏侧化现象,但这也尚存争议,目前有特定领域模型、特定线索模型和脑网络模型三种理论对偏侧化现象的认知机制进行解释。     

Categorical results do not imply categorical perception

Categorical perception refers to the ability to discriminate between- but not within-category differences along a stimulus continuum. Although categorical perception was thought to be unique to speech, recent studies have yielded similar results with nonspeech continua. The results are usually interpreted in terms of categorical, as opposed to continuous, perception of both speech and nonspeech continua. In contrast, we argue that these continua are perceived continuously, although they are characterized by relatively large increases in discrim-inability near the category boundary. To support this argument, the amplitude rise time of a tone was varied to produce either an increase or a decrease in the intensity during the initial portion of the tone. A bipolar continuum of onset times increasing and decreasing in amplitude yielded traditional categorical results. However, when only half of this continuum was tested, subjects perceived the same sounds continuously. The finding of traditional categorical results along the bipolar continuum, when the sounds were shown to be perceived continuously in another context, argues against the use of traditional categorical results as evidence for categorical perception.     

Seeing to hear better: evidence for early audio-visual interactions in speech identification

Lip reading is the ability to partially understand speech by looking at the speaker's lips. It improves the intelligibility of speech in noise when audio-visual perception is compared with audio-only perception. A recent set of experiments showed that seeing the speaker's lips also enhances sensitivity to acoustic information, decreasing the auditory detection threshold of speech embedded in noise [J. Acoust. Soc. Am. 109 (2001) 2272; J. Acoust. Soc. Am. 108 (2000) 1197]. However, detection is different from comprehension, and it remains to be seen whether improved sensitivity also results in an intelligibility gain in audio-visual speech perception. In this work, we use an original paradigm to show that seeing the speaker's lips enables the listener to hear better and hence to understand better. The audio-visual stimuli used here could not be differentiated by lip reading per se since they contained exactly the same lip gesture matched with different compatible speech sounds. Nevertheless, the noise-masked stimuli were more intelligible in the audio-visual condition than in the audio-only condition due to the contribution of visual information to the extraction of acoustic cues. Replacing the lip gesture by a non-speech visual input with exactly the same time course, providing the same temporal cues for extraction, removed the intelligibility benefit. This early contribution to audio-visual speech identification is discussed in relationships with recent neurophysiological data on audio-visual perception.     

Auditory short-term memory and vowel perception

Memory & Cognition - The distinction between categorical and continuous modes of speech perception has played an important role in recent theoretical accounts of the speech perception process....     

Integrating face and voice in person perception

Integration of information from face and voice plays a central role in our social interactions. It has been mostly studied in the context of audiovisual speech perception: integration of affective or identity information has received comparatively little scientific attention. Here, we review behavioural and neuroimaging studies of face-voice integration in the context of person perception. Clear evidence for interference between facial and vocal information has been observed during affect recognition or identity processing. Integration effects on cerebral activity are apparent both at the level of heteromodal cortical regions of convergence, particularly bilateral posterior superior temporal sulcus (pSTS), and at 'unimodal' levels of sensory processing. Whether the latter reflects feedback mechanisms or direct crosstalk between auditory and visual cortices is as yet unclear.     

What is the relationship between phonological short-term memory and speech processing?

Traditionally, models of speech comprehension and production do not depend on concepts and processes from the phonological short-term memory (pSTM) literature. Likewise, in working memory research, pSTM is considered to be a language-independent system that facilitates language acquisition rather than speech processing per se. We discuss couplings between pSTM, speech perception and speech production, and we propose that pSTM arises from the cycling of information between two phonological buffers, one involved in speech perception and one in speech production. We discuss the specific role of these processes in speech processing, and argue that models of speech perception and production, and our understanding of their neural bases, will benefit from incorporating them.     

La langue et les Lèvres: cross-language influences on bimodal speech perception.

Previous research in speech perception has yielded two sets of findings which are brought together in the present study. First, it has been shown that normal hearing listeners use visible as well as acoustical information when processing speech. Second, it has been shown that there is an effect of specific language experience on speech perception such that adults often have difficulty identifying and discriminating non-native phones. The present investigation was designed to extend and combine these two sets of findings. Two studies were conducted using six consonant-vowel syllables (/ba/, /va/, /alpha a/, /da/, /3a/, and /ga/ five of which occur in French and English, and one (the interdental fricative /alpha a/) which occurs only in English. In Experiment 1, an effect of specific linguistic experience was evident for the auditory identification of the non-native interdental stimulus by French-speakers. In Experiment 2, it was shown that the effect of specific language experience extends to the perception of the visible information in speech. These findings are discussed in terms of their implications for our understanding of cross-language processes in speech perception and for our understanding of the development of bimodal speech perception.     

The Impact and Status of Carol Fowler's Supramodal Theory of Multisensory Speech Perception

ABSTRACT

One important contribution of Carol Fowler's direct approach to speech perception is its account of multisensory perception. This supramodal account proposes a speech function that detects supramodal information available across audition, vision, and touch. This detection allows for the recovery of articulatory primitives that provide the basis of a common currency shared between modalities as well as between perception and production. Common currency allows for perceptual experience to be shared between modalities and supports perceptually guided speaking as well as production-guided perception. In this report, we discuss the contribution and status of the supramodal approach relative to recent research in multisensory speech perception. We argue that the approach has helped motivate a multisensory revolution in perceptual psychology. We then review the new behavioral and neurophysiological research on (a) supramodal information, (b) cross-sensory sharing of experience, and (c) perceptually guided speaking as well as production guided speech perception. We conclude that Fowler's supramodal theory has fared quite well in light of this research.