Dorsal and ventral streams: a framework for understanding aspects of the functional anatomy of language

Despite intensive work on language-brain relations, and a fairly impressive accumulation of knowledge over the last several decades, there has been little progress in developing large-scale models of the functional anatomy of language that integrate neuropsychological, neuroimaging, and psycholinguistic data. Drawing on relatively recent developments in the cortical organization of vision, and on data from a variety of sources, we propose a new framework for understanding aspects of the functional anatomy of language which moves towards remedying this situation. The framework posits that early cortical stages of speech perception involve auditory fields in the superior temporal gyrus bilaterally (although asymmetrically). This cortical processing system then diverges into two broad processing streams, a ventral stream, which is involved in mapping sound onto meaning, and a dorsal stream, which is involved in mapping sound onto articulatory-based representations. The ventral stream projects ventro-laterally toward inferior posterior temporal cortex (posterior middle temporal gyrus) which serves as an interface between sound-based representations of speech in the superior temporal gyrus (again bilaterally) and widely distributed conceptual representations. The dorsal stream projects dorso-posteriorly involving a region in the posterior Sylvian fissure at the parietal-temporal boundary (area Spt), and ultimately projecting to frontal regions. This network provides a mechanism for the development and maintenance of "parity" between auditory and motor representations of speech. Although the proposed dorsal stream represents a very tight connection between processes involved in speech perception and speech production, it does not appear to be a critical component of the speech perception process under normal (ecologically natural) listening conditions, that is, when speech input is mapped onto a conceptual representation. We also propose some degree of bi-directionality in both the dorsal and ventral pathways. We discuss some recent empirical tests of this framework that utilize a range of methods. We also show how damage to different components of this framework can account for the major symptom clusters of the fluent aphasias, and discuss some recent evidence concerning how sentence-level processing might be integrated into the framework.     

A Social Constructivism Decision-Making Approach to Managing Incidental Findings in Neuroimaging Research

Functional magnetic resonance imaging (fMRI) is a powerful tool used in cognitive neuroscientific research. fMRI is noninvasive, safe, and relatively accessible, making it an ideal method to draw inferences about the brain–behavior relationship. When conducting fMRI research, scientists must consider risks associated with brain imaging. In particular, the risk of potentially identifying an abnormal brain finding in an fMRI research scan poses a complex problem that researchers should be prepared to address. This article illustrates how a social constructivism decision-making model can be used as a framework to guide researchers as they develop protocols to address this issue.     

The perceptual root of object-based storage: an interactive model of perception and visual working memory

Mainstream theories of visual perception assume that visual working memory (VWM) is critical for integrating online perceptual information and constructing coherent visual experiences in changing environments. Given the dynamic interaction between online perception and VWM, we propose that how visual information is processed during visual perception can directly determine how the information is going to be selected, consolidated, and maintained in VWM. We demonstrate the validity of this hypothesis by investigating what kinds of perceptual information can be stored as integrated objects in VWM. Three criteria for object-based storage are introduced: (a) automatic selection of task-irrelevant features, (b) synchronous consolidation of multiple features, and (c) stable maintenance of feature conjunctions. The results show that the outputs of parallel perception meet all three criteria, as opposed to the outputs of serial attentive processing, which fail all three criteria. These results indicate that (a) perception and VWM are not two sequential processes, but are dynamically intertwined; (b) there are dissociated mechanisms in VWM for storing information identified at different stages of perception; and (c) the integrated object representations in VWM originate from the "preattentive" or "proto" objects created by parallel perception. These results suggest how visual perception, attention, and VWM can be explained by a unified framework.     

Associative memory and the medial temporal lobes

Associative recognition and recall depend on structures in the medial temporal lobes (MTLs). There is disagreement about whether associative memory is functionally heterogeneous, whether it is functionally distinct from intra-item associative memory and how the MTLs contribute to this kind of memory. Despite an increase in research on associative memory, work has lacked a theoretical framework to guide design and interpretation of studies. One view provides a useful framework by postulating that associative memories differ in the degree to which their informational components converge in MTL structures that create familiarity-supporting or recollection-supporting memory representations. Using this framework, we consider psychological, lesion and functional imaging evidence, highlighting how informational convergence in the brain underlies the associative nature of both memory and perception.     

Distinguishing multi-voxel patterns and mean activation: Why, how, and what does it tell us?

The introduction of multi-voxel pattern analysis (MVPA) to the functional magnetic resonance imaging (fMRI) community has brought a deeper appreciation for the diverse forms of information that can be present within fMRI activity. The conclusions drawn from MVPA investigations are frequently influenced by both the ability to decode information from multi-voxel patterns and mean activation levels. In practice, MVPA studies vary widely in why and how they test for differing overall response levels. In this article, I examine the place of univariate information in MVPA investigations. I first discuss the variety of interpretations given to finding univariate response differences. I go on to discuss some of the analysis approaches used to investigate and compare univariate and multivariate sources of information, which can illuminate their respective contributions. It will be important for the MVPA and general fMRI community to continue to discuss and debate these important issues.     

Spatial and temporal limits in cognitive neuroimaging with fMRI

A large body of research in human perception and cognition has been concerned with the segregation of mental events into their presumed hierarchical processing stages, the temporal aspect of such processing being termed ‘mental chronometry’. Advances in single-event functional magnetic resonance imaging (fMRI) have allowed the extraction of relative timing information between the onset of activity in different neural substrates as well as the duration of cognitive processing during a task, offering new opportunities in the study of human perception and cognition. Single-event fMRI studies have also facilitated increased spatial resolution in fMRI, allowing studies of columnar organization in humans. Important processes such as object recognition, binocular vision and other processes are thought to be organized at the columnar level; thus, these advances in the spatial and temporal capabilities of fMRI allow a new generation of cognitive and basic neuroscience studies to be performed, investigating the temporal and spatial relationships between these cortical sub-units. Such experiments bear a closer resemblance to single-unit or evoked-potential studies than to classical static brain activation maps and might serve as a bridge between primate electrophysiology and human studies. These advances are initially demonstrated only in simple visual and motor system tasks and it is likely to be several years before the techniques we describe are robust enough for general use.     

Perspectival representation and fallacies in metaethics

Abstract

The prevailing theoretical framework for theorising about representation construes all representation as involving objective representational contents. This classic framework has tended to drive philosophers either to claim that evaluative judgements are representations and therefore objective, or else to claim that evaluative judgements are not really representations, because they are not objective. However, a more general, already well-explored framework is available, which will allow theorists to treat evaluative judgements as full-fledged representations (thus doing justice to their representational aspects) while leaving open whether they are objective. Such a more general conception of representational content is exemplified, e.g. by Lewis’s ‘centred contents’ and Gibbard’s framework of ‘contents of judgement’, thus it is not new. I shall start in §1 by introducing the more general framework of perspectival contents and then illustrate in §2 how awareness of it can help expose the fallaciousness of certain widely used forms of argumentation in metaethics.     

Solving the "real" mysteries of visual perception: the world as an outside memory.

Visual science is currently a highly active domain, with much progress being made in fields such as colour vision, stereo vision, perception of brightness and contrast, visual illusions, etc. But the "real" mystery of visual perception remains comparatively unfathomed, or at least relegated to philosophical status: Why it is that we can see so well with what is apparently such a badly constructed visual apparatus? In this paper I will discuss several defects of vision and the classical theories of how they are overcome. I will criticize these theories and suggest an alternative approach, in which the outside world is considered as a kind of external memory store which can be accessed instantaneously by casting one's eyes (or one's attention) to some location. The feeling of the presence and extreme richness of the visual world is, under this view, a kind of illusion, created by the immediate availability of the information in this external store.     

Image Interpretation: Bridging the Gap from Mechanically Produced Image to Representation

There is currently a gap in our understanding of how figures produced by mechanical imaging techniques play evidential roles: several studies based on close examination of scientific practice show that imaging techniques do not yield data whose significance can simply be read off the image. If image-making technology is not a simple matter of nature re-presenting itself to us in a legible way, just how do the images produced provide support for scientific claims? In this article I will first show that there is a distinct question about the semiotics of mechanically produced images that has not yet been answered. I show that my account of visual representations can do so, and I argue that the role of convention involved in my account is compatible with the view that visual representations produced through mechanized imaging techniques can play genuine evidential roles in scientific reasoning.     

视觉表象可视化——视觉表象研究的新途径

功能磁共振技术在表象研究中得到广泛应用是表象研究追求客观化精确性的必然趋势。本文介绍了功能磁共振多变量模式分析方法及其演变历程,探讨了借助该方法实现“视觉表象可视化”的理论依据与亟待解决的关键问题。分析指出“视觉表象可视化”将为表象研究提供全新的研究视角与方法途径。     

Conscious Vision for Action Versus Unconscious Vision for Action?

David Milner and Melvyn Goodale’s dissociation hypothesis is commonly taken to state that there are two functionally specialized cortical streams of visual processing originating in striate (V1) cortex: a dorsal, action‐related “unconscious” stream and a ventral, perception‐related “conscious” stream. As Milner and Goodale acknowledge, findings from blindsight studies suggest a more sophisticated picture that replaces the distinction between unconscious vision for action and conscious vision for perception with a tripartite division between unconscious vision for action, conscious vision for perception, and unconscious vision for perception. The combination excluded by the tripartite division is the possibility of conscious vision for action. But are there good grounds for concluding that there is no conscious vision for action? There is now overwhelming evidence that illusions and perceived size can have a significant effect on action ( Bruno & Franz, 2009 ; Dassonville & Bala, 2004 ; Franz & Gegenfurtner, 2008 ; McIntosh & Lashley, 2008 ). There is also suggestive evidence that any sophisticated visual behavior requires collaboration between the two visual streams at every stage of the process ( Schenk & McIntosh, 2010 ). I nonetheless want to make a case for the tripartite division between unconscious vision for action, conscious vision for perception, and unconscious vision for perception. My aim here is not to refute the evidence showing that conscious vision can affect action but rather to argue (a) that we cannot gain cognitive access to action‐guiding dorsal stream representations, and (b) that these representations do not correlate with phenomenal consciousness. This vindicates the semi‐conservative view that the dissociation hypothesis is best understood as a tripartite division.     

Coordinate transformations in object recognition

A basic problem of visual perception is how human beings recognize objects after spatial transformations. Three central classes of findings have to be accounted for: (a) Recognition performance varies systematically with orientation, size, and position; (b) recognition latencies are sequentially additive, suggesting analogue transformation processes; and (c) orientation and size congruency effects indicate that recognition involves the adjustment of a reference frame. All 3 classes of findings can be explained by a transformational framework of recognition: Recognition is achieved by an analogue transformation of a perceptual coordinate system that aligns memory and input representations. Coordinate transformations can be implemented neurocomputationally by gain (amplitude) modulation and may be regarded as a general processing principle of the visual cortex.     

On the modularity of implicit sequence learning: independent acquisition of spatial, symbolic, and manual sequences

Learning sequential structures is of fundamental importance for a wide variety of human skills. While it has long been debated whether implicit sequence learning is perceptual or response-based, here we propose an alternative framework that cuts across this dichotomy and assumes that sequence learning rests on associative changes that can occur concurrently in distinct processing systems and support the parallel acquisition of multiple uncorrelated sequences. In three experiments we used a serial search task to test critical predictions of this framework. Experiments 1 and 2 showed that participants learnt uncorrelated sequences of auditory letters and manual responses, as well as sequences of visual letters, spatial locations, and manual responses simultaneously, as indicated by a reliable response time (RT) cost incurred by occasional deviants violating either of the sequences. This RT cost was reliable even when participants showing explicit knowledge were excluded. In Experiment 3 learning of spatial and nonspatial sequences was functionally dissociated: whereas a spatio-motor distractor task disrupted learning of location but not of letter sequences, a phonological distractor task had the reverse effect. The distractor tasks thus did not reduce unspecific attentional resources, but selectively disrupted the formation of sequential associations within spatial and nonspatial processing dimensions. These results support the view that implicit sequence learning rests on experience-dependent changes that can occur in parallel in multiple processing systems involved in spatial attention, object recognition, phonological processing, and manual response selection. The resulting dimension-specific sequence representations support independent predictions of what will appear next, where it will appear, and how one will have to respond to it.     

Rationality,perception, and the all-seeing eye

Seeing—perception and vision—is implicitly the fundamental building block of the literature on rationality and cognition. Herbert Simon and Daniel Kahneman’s arguments against the omniscience of economic agents—and the concept of bounded rationality—depend critically on a particular view of the nature of perception and vision. We propose that this framework of rationality merely replaces economic omniscience with perceptual omniscience. We show how the cognitive and social sciences feature a pervasive but problematic meta-assumption that is characterized by an “all-seeing eye.” We raise concerns about this assumption and discuss different ways in which the all-seeing eye manifests itself in existing research on (bounded) rationality. We first consider the centrality of vision and perception in Simon’s pioneering work. We then point to Kahneman’s work—particularly his article “Maps of Bounded Rationality”—to illustrate the pervasiveness of an all-seeing view of perception, as manifested in the extensive use of visual examples and illusions. Similar assumptions about perception can be found across a large literature in the cognitive sciences. The central problem is the present emphasis on inverse optics—the objective nature of objects and environments, e.g., size, contrast, and color. This framework ignores the nature of the organism and perceiver. We argue instead that reality is constructed and expressed, and we discuss the species-specificity of perception, as well as perception as a user interface. We draw on vision science as well as the arts to develop an alternative understanding of rationality in the cognitive and social sciences. We conclude with a discussion of the implications of our arguments for the rationality and decision-making literature in cognitive psychology and behavioral economics, along with suggesting some ways forward.     

Ambiguity,Competition, and Blending in Spoken Word Recognition

A critical property of the perception of spoken words is the transient ambiguity of the speech signal. In localist models of speech perception this ambiguity is captured by allowing the parallel activation of multiple lexical representations. This paper examines how a distributed model of speech perception can accommodate this property. Statistical analyses of vector spaces show that coactivation of multiple distributed representations is inherently noisy, and depends on parameters such as sparseness and dimensionality. Furthermore, the characteristics of coactivation vary considerably, depending on the organization of distributed representations within the mental lexicon. This view of lexical access is supported by analyses of phonological and semantic word representations, which provide an explanation of a recent set of experiments on coactivation in speech perception (Gaskell & Marslen–Wilson, 1999).     

Object Recognition: Insights From Advances in fMRI Methods

ABSTRACT— Recent advancements in imaging methods and analysis approaches have provided important insights about the neural bases of object recognition. We address the potential limitations of standard functional magnetic resonance imaging (fMRI) and discuss methodological advancements, including fMRI-adaptation, pattern analyses, and high-resolution fMRI, that may be more appropriate for studying object and face representations. fMRI-adaptation and high-resolution fMRI measure responses of neural subpopulations within standard fMRI voxels, and pattern analyses examine the information in the distributed activations across voxels, which may differ from the mean response across these voxels. These methods have provided evidence for a multitude of representations across the human ventral stream that provide empirical constraints for cognitive theories of recognition.     

Scaling-up skilled intentionality to linguistic thought

Cognition has traditionally been understood in terms of internal mental representations, and computational operations carried out on internal mental representations. Radical approaches propose to reconceive cognition in terms of agent-environment dynamics. An outstanding challenge for such a philosophical project is how to scale-up from perception and action to cases of what is typically called ‘higher-order’ cognition such as linguistic thought, the case we focus on in this paper. Perception and action are naturally described in terms of agent-environment dynamics, but can a person’s thoughts about absent, abstract or counterfactual states of affairs also be accounted for in such terms? We argue such a question will seem pressing so long as one fails to appreciate how richly resourceful the human ecological niche is in terms of the affordances it provides. The explanatory work that is supposedly done by mental representations in a philosophical analysis of cognition, can instead be done by looking outside of the head to the environment structured by sociomaterial practices, and the affordances it makes available. Once one recognizes how much of the human ecological niche has become structured by activities of talking and writing, this should take away at least some of the motivation for understanding linguistic thinking in terms of content-bearing internal representations. We’ll argue that people can think about absent, abstract or counterfactual because of their skills for engaging with what we will call “enlanguaged affordances”. We make use of the phenomenological analysis of speech in Merleau-Ponty to show how the multiple affordances an individual is ready to engage with in a particular situation will typically include enlanguaged affordances.

     

Levels of processing: a view from functional brain imaging

This paper briefly reviews two central assumptions of the levels-of-processing framework in the light of findings from recent PET and fMRI studies: First, to address the suggestion that memory traces can be seen as records of analyses carried out for the purposes of perception and comprehension, studies on encoding-retrieval overlap in brain activation patterns are considered. Second, to address the suggestion that deeper, more semantic, processing results in more durable traces, studies of how encoding activity relates to processing depth and subsequent memory performance are examined. The results show that some of the sensory regions that are activated during initial perception are subsequently reactivated during retrieval, and activity in frontal and medial-temporal brain regions is related to depth of processing and level of memory performance. Collectively, these results provide support for central components of the levels framework.     

Emergent representations, interaction theory and the cognitive force of metaphor

We focus here on the problem of how new representations emerge through a cognitive agent's interaction with the environment. We address this problem within a framework where representation-building mechanisms operate to create new representations against a backdrop of existing representations, and argue that novel and creative metaphors in any language provide a prime example of this phenomenon. Our approach to modeling the emergence of representation integrates ideas from three different streams of research: (1) the interaction theory of metaphor proposed by Black and others to account for the creativity of metaphors; (2) gestalt theories of perception; and (3) contemporary research supporting a constructivist and action-oriented view of perception and cognition. Combining these insights together, we outline our Gestalt projection model, and discuss three different ways in which new representations might emerge through metaphors. Finally, we propose that metaphor may be viewed as a cognitive force through which a cognitive agent asserts its creative spirit onto the environment.