Dorsal and ventral streams: a framework for understanding aspects of the functional anatomy of language

Despite intensive work on language-brain relations, and a fairly impressive accumulation of knowledge over the last several decades, there has been little progress in developing large-scale models of the functional anatomy of language that integrate neuropsychological, neuroimaging, and psycholinguistic data. Drawing on relatively recent developments in the cortical organization of vision, and on data from a variety of sources, we propose a new framework for understanding aspects of the functional anatomy of language which moves towards remedying this situation. The framework posits that early cortical stages of speech perception involve auditory fields in the superior temporal gyrus bilaterally (although asymmetrically). This cortical processing system then diverges into two broad processing streams, a ventral stream, which is involved in mapping sound onto meaning, and a dorsal stream, which is involved in mapping sound onto articulatory-based representations. The ventral stream projects ventro-laterally toward inferior posterior temporal cortex (posterior middle temporal gyrus) which serves as an interface between sound-based representations of speech in the superior temporal gyrus (again bilaterally) and widely distributed conceptual representations. The dorsal stream projects dorso-posteriorly involving a region in the posterior Sylvian fissure at the parietal-temporal boundary (area Spt), and ultimately projecting to frontal regions. This network provides a mechanism for the development and maintenance of "parity" between auditory and motor representations of speech. Although the proposed dorsal stream represents a very tight connection between processes involved in speech perception and speech production, it does not appear to be a critical component of the speech perception process under normal (ecologically natural) listening conditions, that is, when speech input is mapped onto a conceptual representation. We also propose some degree of bi-directionality in both the dorsal and ventral pathways. We discuss some recent empirical tests of this framework that utilize a range of methods. We also show how damage to different components of this framework can account for the major symptom clusters of the fluent aphasias, and discuss some recent evidence concerning how sentence-level processing might be integrated into the framework.     

Left posterior auditory-related cortices participate both in speech perception and speech production: Neural overlap revealed by fMRI

Recent neuroimaging studies and neuropsychological data suggest that there are regions in posterior auditory cortex that participate both in speech perception and speech production. An outstanding question is whether the same neural regions support both perception and production or whether there exist discrete cortical fields subserving these functions. Previous neurophysiological studies suggest that there is indeed regional overlap between these systems, but those studies used a rehearsal task to assess production. The present study addressed this question in an event-related fMRI experiment in which subjects listened to speech and in separate trials, performed a covert object naming task. Single subject analysis revealed regions of coactivation for speech perception and production in the left posterior superior temporal sulcus (pSTS), left area Spt (a region in the Sylvian fissure at the parietal-temporal boundary), and left inferior frontal gyrus. These results are consistent with lesion data and previous physiological data indicating that posterior auditory cortex plays a role in both reception and expression of speech. We discuss these findings within the context of a neuroanatomical framework that proposes these neural sites are a part of an auditory-motor integration system.     

Functional anatomy of speech perception and speech production: psycholinguistic implications

This paper presents evidence for a new model of the functional anatomy of speech/language (Hickok & Poeppel, 2000) which has, at its core, three central claims: (1) Neural systems supporting the perception of sublexical aspects of speech are essentially bilaterally organized in posterior superior temporal lobe regions; (2) neural systems supporting the production of phonemic aspects of speech comprise a network of predominately left hemisphere systems which includes not only frontal regions, but also superior temporal lobe regions; and (3) the neural systems supporting speech perception and production partially overlap in left superior temporal lobe. This model, which postulates nonidentical but partially overlapping systems involved in the perception and production of speech, explains why psycho- and neurolinguistic evidence is mixed regarding the question of whether input and output phonological systems involve a common network or distinct networks.     

Role of left posterior superior temporal gyrus in phonological processing for speech perception and production

Models of both speech perception and speech production typically postulate a processing level that involves some form of phonological processing. There is disagreement, however, on the question of whether there are separate phonological systems for speech input versus speech output. We review a range of neuroscientific data that indicate that input and output phonological systems partially overlap. An important anatomical site of overlap appears to be the left posterior superior temporal gyrus. We then present the results of a new event-related functional magnetic resonance imaging (fMRI) experiment in which participants were asked to listen to and then (covertly) produce speech. In each participant, we found two regions in the left posterior superior temporal gyrus that responded both to the perception and production components of the task, suggesting that there is overlap in the neural systems that participate in phonological aspects of speech perception and speech production. The implications for neural models of verbal working memory are also discussed in connection with our findings.     

Ambiguity,Competition, and Blending in Spoken Word Recognition

A critical property of the perception of spoken words is the transient ambiguity of the speech signal. In localist models of speech perception this ambiguity is captured by allowing the parallel activation of multiple lexical representations. This paper examines how a distributed model of speech perception can accommodate this property. Statistical analyses of vector spaces show that coactivation of multiple distributed representations is inherently noisy, and depends on parameters such as sparseness and dimensionality. Furthermore, the characteristics of coactivation vary considerably, depending on the organization of distributed representations within the mental lexicon. This view of lexical access is supported by analyses of phonological and semantic word representations, which provide an explanation of a recent set of experiments on coactivation in speech perception (Gaskell & Marslen–Wilson, 1999).     

Homotopic language reorganization in the right hemisphere after early left hemisphere injury

Speech production is inextricably linked to speech perception, yet they are usually investigated in isolation. In this study, we employed a verbal-repetition task to identify the neural substrates of speech processing with two ends active simultaneously using functional MRI. Subjects verbally repeated auditory stimuli containing an ambiguous vowel sound that could be perceived as either a word or a pseudoword depending on the interpretation of the vowel. We found verbal repetition commonly activated the audition-articulation interface bilaterally at Sylvian fissures and superior temporal sulci. Contrasting word-versus-pseudoword trials revealed neural activities unique to word repetition in the left posterior middle temporal areas and activities unique to pseudoword repetition in the left inferior frontal gyrus. These findings imply that the tasks are carried out using different speech codes: an articulation-based code of pseudowords and an acoustic-phonetic code of words. It also supports the dual-stream model and imitative learning of vocabulary.     

What phonological deficit?

We review a series of experiments aimed at understanding the nature of the phonological deficit in developmental dyslexia. These experiments investigate input and output phonological representations, phonological grammar, foreign speech perception and production, and unconscious speech processing and lexical access. Our results converge on the observation that the phonological representations of people with dyslexia may be intact, and that the phonological deficit surfaces only as a function of certain task requirements, notably short-term memory, conscious awareness, and time constraints. In an attempt to reformulate those task requirements more economically, we propose that individuals with dyslexia have a deficit in access to phonological representations. We discuss the explanatory power of this concept and we speculate that a similar notion might also adequately describe the nature of other associated cognitive deficits when present.     

Un modèle neurobiologique de la perception et de l'estimation du temps

Frontal-striatal circuits provide an important neurobiological substrate for timing and time perception as well as for working memory. In this review, we outline recent theoretical and empirical work to suggest that interval timing and working memory rely not only on the same anatomic structures, but also on the same neural representation of a specific stimulus. In the striatal beat-frequency model, cortical neurons fire in an oscillatory fashion to form representations of stimuli, and striatal medium spiny neurons detect those patterns of cortical firing that occur co-incident to important temporal events. Information about stimulus identity can be extracted from the specific cortical networks that are involved in the representation, and information about duration can be extracted from the relative phase of neural firing. The properties derived from these neurobiological models fit well with the psychophysics of timing and time perception as well as with information-processing models that emphasize the importance of temporal coding in a variety of working-memory phenomena.     

Being ourselves and knowing ourselves: an adverbial account of mental representations

This paper takes an evolutionary approach to what we are, namely autopoietic systems with a first person perspective on our surroundings and ourselves. This in contrast with Thomas Metzinger's views in his Being No One. Though perception does involve internal processing and representations, it is argued that perception is direct. We track real features of the world, but fallibly, in a certain way. Moreover, it is claimed that mental representations are quite different from internal neural representations. They are best construed in an adverbial way. What we perceive, the object of perception, is the real world. Internal neural representations are the means by which we perceive the world. And mental representations are the way in which we experience the world, the adverbial content of perception. Finally, what goes for the world goes for ourselves as well: in self-consciousness we track real features of ourselves, but fallibly, in a certain way.     

从脑机制角度看言语知觉的理论争论

言语知觉领域主要的理论争论是听觉理论和动觉理论的对立, 争论的焦点围绕言语知觉是否需要动作表征的中介。言语知觉脑机制的研究有助于澄清事实。脑机制的探讨表明言语知觉主要激活了后部听觉皮层区, 包括颞上皮层的背侧(颞横回和颞平面)和外侧区(颞上回和颞上沟); 而前部和言语产生相关的动作皮层并没有表现出一致的激活模式。言语产生相关的动作表征主要在一些特殊任务情形中以自上而下的反馈机制影响了言语知觉, 可能并非正常言语知觉所必须。     

Cerebral specialization for speech production in persons with Down syndrome.

The study of cerebral specialization in persons with Down syndrome (DS) has revealed an anomalous pattern of organization. Specifically, dichotic listening studies (e.g., Elliott & Weeks, 1993) have suggested a left ear/right hemisphere dominance for speech perception for persons with DS. In the current investigation, the cerebral dominance for speech production was examined using the mouth asymmetry technique. In right-handed, nonhandicapped subjects, mouth asymmetry methodology has shown that during speech, the right side of the mouth opens sooner and to a larger degree then the left side (Graves, Goodglass, & Landis, 1982). The phenomenon of right mouth asymmetry (RMA) is believed to reflect the direct access that the musculature on the right side of the face has to the left hemisphere's speech production systems. This direct access may facilitate the transfer of innervatory patterns to the muscles on the right side of the face. In the present study, the lateralization for speech production was investigated in 10 right-handed participants with DS and 10 nonhandicapped subjects. A RMA at the initiation and end of speech production occurred for subjects in both groups. Surprisingly, the degree of asymmetry between groups did not differ, suggesting that the lateralization of speech production is similar for persons with and persons without DS. These results support the biological dissociation model (Elliott, Weeks, & Elliott, 1987), which holds that persons with DS display a unique dissociation between speech perception (right hemisphere) and speech production (left hemisphere).     

Perception and awareness in phonological processing: the case of the phoneme

The necessity of a “levels-of-processing” approach in the study of mental representations is illustrated by the work on the psychological reality of the phoneme. On the basis of both experimental studies of human behavior and functional imaging data, it is argued that there are unconscious representations of phonemes in addition to conscious ones. These two sets of mental representations are functionally distinct: the former intervene in speech perception and (presumably) production; the latter are developed in the context of learning alphabetic literacy for both reading and writing purposes. Moreover, among phonological units and properties, phonemes may be the only ones to present a neural dissociation at the macro-anatomic level. Finally, it is argued that even if the representations used in speech perception and those used in assembling and in conscious operations are distinct, they may entertain dependency relations.     

A Connectionist Model of Phonological Representation in Speech Perception

A number of recent studies have examined the effects of phonological variation on the perception of speech. These studies show that both the lexical representations of words and the mechanisms of lexical access are organized so that natural, systematic variation is tolerated by the perceptual system, while a general intolerance of random deviation is maintained. Lexical abstraction distinguishes between phonetic features that form the invariant core of a word and those that are susceptible to variation. Phonological inference relies on the context of surface changes to retrieve the underlying phonological form. In this article we present a model of these processes in speech perception, based on connectionist learning techniques. A simple recurrent network was trained on the mapping from the variant surface form of speech to the underlying form. Once trained, the network exhibited features of both abstraction and inference in its processing of normal speech, and predicted that similar behavior will be found in the perception of nonsense words. This prediction was confirmed in subsequent research (Gaskell & Marslen-Wilson, 1994).     

Remembering the past and imagining the future: a neural model of spatial memory and imagery

The authors model the neural mechanisms underlying spatial cognition, integrating neuronal systems and behavioral data, and address the relationships between long-term memory, short-term memory, and imagery, and between egocentric and allocentric and visual and ideothetic representations. Long-term spatial memory is modeled as attractor dynamics within medial-temporal allocentric representations, and short-term memory is modeled as egocentric parietal representations driven by perception, retrieval, and imagery and modulated by directed attention. Both encoding and retrieval/imagery require translation between egocentric and allocentric representations, which are mediated by posterior parietal and retrosplenial areas and the use of head direction representations in Papez's circuit. Thus, the hippocampus effectively indexes information by real or imagined location, whereas Papez's circuit translates to imagery or from perception according to the direction of view. Modulation of this translation by motor efference allows spatial updating of representations, whereas prefrontal simulated motor efference allows mental exploration. The alternating temporal-parietal flows of information are organized by the theta rhythm. Simulations demonstrate the retrieval and updating of familiar spatial scenes, hemispatial neglect in memory, and the effects on hippocampal place cell firing of lesioned head direction representations and of conflicting visual and ideothetic inputs.     

What Are Nervous Systems For?

Abstract

An underlying bias of contemporary cognitive science is that the brain and nervous system are in the business of carrying out computations and building representations. Gibson’s ecological approach, in contrast, is decidedly noncomputational and nonrepresentational. How, then, are we to construe the role of brain and nervous system? We consider this question against the backdrop of evidence for rich achievements in perception and action by agents without brains or nervous systems. If fundamental coordination of perception and action does not require a neural substrate, then what value is added in having one? And if the neural substrate is not in the representational–computational business, then what business is it in? We pursue answers grounded in the constraints of macroscopic, multicellular life and thermodynamics.     

Modeling the categorical perception of speech sounds: A step toward biological plausibility

Our native language has a lifelong effect on how we perceive speech sounds. Behaviorally, this is manifested as categorical perception, but the neural mechanisms underlying this phenomenon are still unknown. Here, we constructed a computational model of categorical perception, following principles consistent with infant speech learning. A self-organizing network was exposed to a statistical distribution of speech input presented as neural activity patterns of the auditory periphery, resembling the way sound arrives to the human brain. In the resulting neural map, categorical perception emerges from most single neurons of the model being maximally activated by prototypical speech sounds, while the largest variability in activity is produced at category boundaries. Consequently, regions in the vicinity of prototypes become perceptually compressed, and regions at category boundaries become expanded. Thus, the present study offers a unifying framework for explaining the neural basis of the warping of perceptual space associated with categorical perception.     

The development of voicing categories: A quantitative review of over 40 years of infant speech perception research

Most research on infant speech categories has relied on measures of discrimination. Such work often employs categorical perception as a linking hypothesis to enable inferences about categorization on the basis of discrimination measures. However, a large number of studies with adults challenge the utility of categorical perception in describing adult speech perception, and this in turn calls into question how to interpret measures of infant speech discrimination. We propose here a parallel channels model of discrimination (built on Pisoni and Tash Perception & Psychophysics, 15(2), 285–290, 1974), which posits that both a noncategorical or veridical encoding of speech cues and category representations can simultaneously contribute to discrimination. This can thus produce categorical perception effects without positing any warping of the acoustic signal, but it also reframes how we think about infant discrimination and development. We test this model by conducting a quantitative review of 20 studies examining infants’ discrimination of voice onset time contrasts. This review suggests that within-category discrimination is surprisingly prevalent even in classic studies and that, averaging across studies, discrimination is related to continuous acoustic distance. It also identifies several methodological factors that may mask our ability to see this. Finally, it suggests that infant discrimination may improve over development, contrary to commonly held notion of perceptual narrowing. These results are discussed in terms of theories of speech development that may require such continuous sensitivity.     

Color perception involves color representations firstly at a semantic level and then at a lexical level

Studies and models have suggested that color perception first involves access to semantic representations of color. This result leads to two questions: (1) is knowledge able to influence the perception of color when associated with a color? and (2) can the perception of color really involve only semantic representations? We developed an experiment where participants have to discriminate the color of a patch (yellow vs. green). The target patch is preceded either by a black-and-white line drawing or by a word representing a natural object associated with the same or a different color (banana vs. frog). We expected a priming effect for pictures because, with a 350-ms SOA, they only involve access to semantic representations of color, whereas words seem only elicit an access to lexical representations. As expected, we found a priming effect for pictures, but also for words. Moreover, we found a general slowdown of response times in the word-prime-condition suggesting the need of an additional processing step to produce priming. In a second experiment, we manipulated the SOA in order to preclude a semantic access in the word-prime-condition that could explain the additional step of processing. We also found a priming effect, suggesting that interaction with perception occurs at a lexical level and the additional step occurs at a color perception level. In the discussion, we develop a new model of color perception assuming that color perception involves access to semantic representations and then access to lexical representations.     

Dissociation of explicit and implicit timing in repetitive tapping and drawing movements

Four experiments explored the hypothesis that temporal processes may be represented and controlled explicitly or implicitly. Tasks hypothesized to require explicit timing were duration discrimination, tapping, and intermittent circle drawing. In contrast, it was hypothesized that timing control during continuous circle drawing does not rely on an explicit temporal representation; rather, temporal control is an emergent property of other control processes (i.e., timing is controlled implicitly). Temporal consistency on the tapping and intermittent drawing tasks was related, and performance on both of these tasks was correlated with temporal acuity on an auditory duration discrimination task. However, timing variability of these 3 tasks was not correlated with timing variability of continuous circle drawing. These results support the hypothesized distinction between explicit and implicit temporal representations.