The supramodal brain: implications for auditory perception

ABSTRACT

The perceptual brain is designed around multisensory input. Areas once thought dedicated to a single sense are now known to work with multiple senses. It has been argued that the multisensory nature of the brain reflects a cortical architecture for which task, rather than sensory system, is the primary design principle. This supramodal thesis is supported by recent research on human echolocation and multisensory speech perception. In this review, we discuss the behavioural implications of a supramodal architecture, especially as they pertain to auditory perception. We suggest that the architecture implies a degree of perceptual parity between the senses and that cross-sensory integration occurs early and completely. We also argue that a supramodal architecture implies that perceptual experience can be shared across modalities and that this sharing should occur even without bimodal experience. We finish by briefly suggesting areas of future research.     

Appearance and reality

There is tension between the appealing idea that visual experience puts us in touch with objective reality and the doctrine of perceptual relativity, the claim that experience varies with such factors as distance, lighting, and angle of view. In this paper I present arguments for relativity, and then go on to propose that in view of relativity, we should believe that the properties presented to us in perception are relational and viewpoint‐dependent. I conclude by sketching a theory of how we nonetheless manage to achieve contact with objective properties. According to the theory, such contact is best explained in terms of perceptual representations acting in concert with cognitive and motoric representations, not in terms of perceptual representations alone.     

Multisensory evidence

It is tempting to think that one's perceptual evidence comprises just what issues from perceiving with each of the respective sensory modalities. However, empirical, rational, and phenomenological considerations show that one's perceptual evidence can outstrip what one possesses due to perceiving with each separate sense. Some novel perceptual evidence stems from the coordinated use of multiple senses. This paper argues that some perceptual evidence in this respect is distinctively multisensory.     

The Senses Considered as One Perceptual System

ABSTRACT

J. J. Gibson (1966) rejected many classical assumptions about perception but retained 1 that dates back to classical antiquity: the assumption of separate senses. We suggest that Gibson's retention of this assumption compromised his novel concept of perceptual systems. We argue that lawful, 1:1 specification of the animal–environment interaction, which is necessary for perception to be direct, cannot exist in individual forms of ambient energy, such as light, or sound. We argue that specification exists exclusively in emergent, higher order patterns that extend across different forms of ambient energy. These emergent, higher order patterns constitute the global array. If specification exists exclusively in the global array, then direct perception cannot be based upon detection of patterns that are confined to individual forms of ambient energy and, therefore, Gibson's argument for the existence of several distinct perceptual systems cannot be correct. We argue that the senses function as a single, irreducible perceptual system that is sensitive exclusively to patterns in the global array. That is, rather than distinct perceptual systems there exists only 1 perceptual system.     

‘The Echo of a Thought in Sight’: Property Perception,Universals and Wittgenstein

Contemporary philosophers of perception, even those with otherwise widely differing beliefs, often hold that universals enter into the content of perceptual experience. This doctrine can even be seen as a trivial inference from the observation that we observe properties – ways that things are – as well as things. I argue that the inference is not trivial but can and should be resisted. Ordinary property perception does not involve awareness of universals. But there are visual (and aural) experiences which do involve determinate universals: following Wittgenstein, I call these ‘aspect experiences’. The common view of perceptual content effectively conflates aspect experiences with mere property perceptions. Wittgenstein’s later writings on the philosophy of psychology provide an alternative way to think about both aspects and properties. It also forms a contrast with Wittgenstein’s own early treatment of perception in the Tractatus, the doctrine of which is much closer to the contemporary norm among philosophers of perception. In seeing how Wittgenstein moved away from his early view, we can see how we might move away from that norm.     

Multisensory Information in the Control of Complex Motor Actions

ABSTRACT— For many of the complex motor actions we perform, perceptual information is available from several different senses including vision, touch, hearing, and the vestibular system. Here I discuss the use of multisensory information for the control of motor action in three particular domains: aviation, sports, and driving. It is shown that performers in these domains use information from multiple senses—frequently with beneficial effects on performance but sometimes with dangerous consequences. Applied psychologists have taken advantage of our natural tendency to integrate sensory information by designing multimodal displays that compensate for situations in which information from one or more of our senses is unreliable or is unattended due to distraction.     

Subliminal action priming modulates the perceived intensity of sensory action consequences

The sense of control over the consequences of one’s actions depends on predictions about these consequences. According to an influential computational model, consistency between predicted and observed action consequences attenuates perceived stimulus intensity, which might provide a marker of agentic control. An important assumption of this model is that these predictions are generated within the motor system. However, previous studies of sensory attenuation have typically confounded motor-specific perceptual modulation with perceptual effects of stimulus predictability that are not specific to motor action. As a result, these studies cannot unambiguously attribute sensory attenuation to a motor locus. We present a psychophysical experiment on auditory attenuation that avoids this pitfall. Subliminal masked priming of motor actions with compatible prime–target pairs has previously been shown to modulate both reaction times and the explicit feeling of control over action consequences. Here, we demonstrate reduced perceived loudness of tones caused by compatibly primed actions. Importantly, this modulation results from a manipulation of motor processing and is not confounded by stimulus predictability. We discuss our results with respect to theoretical models of the mechanisms underlying sensory attenuation and subliminal motor priming.     

Intersensory Redundancy Guides the Development of Selective Attention, Perception, and Cognition in Infancy

Abstract— That the senses provide overlapping information for objects and events is no extravagance of nature. This overlap facilitates attention to critical aspects of sensory stimulation, those that are redundantly specified, and attenuates attention to nonredundantly specified stimulus properties. This selective attention is most pronounced in infancy and gives initial advantage to the perceptual processing of, learning of, and memory for stimulus properties that are redundant, or amodal (e.g., synchrony, rhythm, and intensity), at the expense of modality-specific properties (e.g., color, pitch, and timbre) that can be perceived through only one sense. We review evidence supporting this hypothesis and discuss its implications for theories of perceptual, cognitive, and social development.     

The Structure of Frege's Thoughts

Fregean thoughts (i.e. the senses of assertoric sentences) are structured entities because they are composed of simpler senses that are somehow ordered and interconnected. The constituent senses form a unity because some of them are ‘saturated’ and some ‘unsaturated’. This paper shows that Frege's explanation of the structure of thoughts, which is based on the ‘saturated/unsaturated’ distinction, is by no means sufficient because it permits what I call ‘wild analyses’, which have certain unwelcome consequences. Wild analyses are made possible because any ‘unsaturated’ sense that is a mode of presentation of a concept together with any ‘saturated’ sense forms a thought. The reason is that any concept can be applied to any object (which is presented by a ‘saturated’ sense). This stems from the fact that Frege was willing to admit only total functions. It is also briefly suggested what should be done to block wild analyses.     

Galileo's eye: a new vision of the senses in the work of Galileo Galilei

Reflections on the senses, and particularly on vision, permeate the writings of Galileo Galilei, one of the main protagonists of the scientific revolution. This aspect of his work has received scant attention by historians, in spite of its importance for his achievements in astronomy, and also for the significance in the innovative scientific methodology he fostered. Galileo's vision pursued a different path from the main stream of the then contemporary studies in the field; these were concerned with the dioptrics and anatomy of the eye, as elaborated mainly by Johannes Kepler and Christoph Scheiner. Galileo was more concerned with the phenomenology rather than with the mechanisms of the visual process. His general interest in the senses was psychological and philosophical; it reflected the fallacies and limits of the senses and the ways in which scientific knowledge of the world could be gathered from potentially deceptive appearances. Galileo's innovative conception of the relation between the senses and external reality contrasted with the classical tradition dominated by Aristotle; it paved the way for the modern understanding of sensory processing, culminating two centuries later in Johannes Müller's elaboration of the doctrine of specific nerve energies and in Helmholtz's general theory of perception.     

Late‐ but not early‐onset blindness impairs the development of audio‐haptic multisensory integration

Integrating different senses to reduce sensory uncertainty and increase perceptual precision can have an important compensatory function for individuals with visual impairment and blindness. However, how visual impairment and blindness impact the development of optimal multisensory integration in the remaining senses is currently unknown. Here we first examined how audio‐haptic integration develops and changes across the life span in 92 sighted (blindfolded) individuals between 7 and 70 years of age. We used a child‐friendly task in which participants had to discriminate different object sizes by touching them and/or listening to them. We assessed whether audio‐haptic performance resulted in a reduction of perceptual uncertainty compared to auditory‐only and haptic‐only performance as predicted by maximum‐likelihood estimation model. We then compared how this ability develops in 28 children and adults with different levels of visual experience, focussing on low‐vision individuals and blind individuals that lost their sight at different ages during development. Our results show that in sighted individuals, adult‐like audio‐haptic integration develops around 13–15 years of age, and remains stable until late adulthood. While early‐blind individuals, even at the youngest ages, integrate audio‐haptic information in an optimal fashion, late‐blind individuals do not. Optimal integration in low‐vision individuals follows a similar developmental trajectory as that of sighted individuals. These findings demonstrate that visual experience is not necessary for optimal audio‐haptic integration to emerge, but that consistency of sensory information across development is key for the functional outcome of optimal multisensory integration.     

Common sense about qualities and senses

There has been some recent optimism that addressing the question of how we distinguish sensory modalities will help us consider whether there are limits on a scientific understanding of perceptual states. For example, Block has suggested that the way we distinguish sensory modalities indicates that perceptual states have qualia which at least resist scientific characterization. At another extreme, Keeley argues that our common-sense way of distinguishing the senses in terms of qualitative properties is misguided, and offers a scientific eliminativism about common-sense modalities which avoids appeal to qualitative properties altogether. I’ll argue contrary to Keeley that qualitative properties are necessary for distinguishing senses, and contrary to Block that our common-sense distinction doesn’t indicate that perceptual states have qualia. A non-qualitative characterization of perceptual states isn’t needed to avoid the potential limit on scientific understanding imposed by qualia.

经验基础与知识确证

Whether empirical givenness has the reliability that foundationalists expect is a point about which some philosophers are highly skeptical. Sellars took the doctrine of givenness as a “myth,” denying the existence of immediate perceptual experience. The arguments in contemporary Western epistemology are concentrated on whether sensory experience has conceptual contents, and whether there is any logical relationship between perceptions and beliefs. In fact, once the elements of words and conceptions in empirical perception are affirmed, the logical relationship between perceptual experience and empirical belief is also affirmed. This relationship takes place through perceptual experience acting as evidence for beliefs. The real problem lies in how one should distinguish between the different relationships with perception of singular beliefs and of universal beliefs, and in how singular beliefs can provide justification for universal beliefs. __________ Translated from Zhongguo Shehui Kexue 中国社会科学 (Social Sciences in China), 2007, (1): 65–75     

Liens entre le déclin sensoriel et cognitif dans le vieillissement normal : revue critique de la littérature et apports de l’approche incarnée et située de la cognition

Biological aging impacts many organs including sensory's ones and the brain, and thus, cognition. Research has shown that the sensory and cognitive decline are positively correlated. The main data from this literature are firstly reviewed in the present article. Then, the four main hypotheses that are generally proposed to explain these associations are presented. According to them, sensory decline is supposed to cause cognitive decline, or vice-versa, or these concomitant changes result from the general alteration of the nervous system. However, none of them seems able to account for all of the existing data. Moreover, the precise mechanisms that may drive these associations remain to be clarified. The rest of the article is then dedicated to the embodied and situated cognition approach as it provides a particularly interesting and adequate framework to account for these links. Indeed, according to this approach, cognitive representations are grounded in their sensorimotor properties. In other words, cognitive functioning is not conceived as detached from sensory functioning, but instead directly dependent of it. Representations are thus thought to emerge from the sensorimotor simulation of the properties involved in these representations. Therefore, sensory decline should directly impact cognitive performance. It is then hypothesized that older adults have low-resolution representations decreasing the signal on noise ratio of traces, increasing sensorimotor interferences and thus decreasing cognitive performance. This approach leads to consider low and high levels of sensory and perceptual functioning, which are both impaired in aging. Therefore, the repercussion of the sensory-perceptual decline is not only valid for present processing, but also extend to all past knowledge. Several predictions are then proposed on (1) the link between high-level perceptual functioning and cognitive functioning in older adults; (2) the possible interaction in young adults between sensory and high-level perceptual functioning as a function of the level of interference of the material involved; (3) the possible link between the motor and cognitive functioning in older adults. The clinical consequences in terms of cognitive stimulation of such a perspective will close the article. It is proposed to develop a stimulation program based on pattern separation mechanism to better process sensory interference in order to train older adults to improve cognitive precision and thus performance.     

Conscious thought as simulation of behaviour and perception

A 'simulation' theory of cognitive function can be based on three assumptions about brain function. First, behaviour can be simulated by activating motor structures, as during an overt action but suppressing its execution. Second, perception can be simulated by internal activation of sensory cortex, as during normal perception of external stimuli. Third, both overt and covert actions can elicit perceptual simulation of their normal consequences. A large body of evidence supports these assumptions. It is argued that the simulation approach can explain the relations between motor, sensory and cognitive functions and the appearance of an inner world.     

The human newborn’s umwelt: Unexplored pathways and perspectives

Historically, newborns, and especially premature newborns, were thought to “feel nothing.” However, over the past decades, a growing body of evidence has shown that newborns are aware of their environment, but the extent and the onset of some sensory capacities remain largely unknown. The goal of this review is to update our current knowledge concerning newborns’ perceptual world and how ready they are to cope with an entirely different sensory environment following birth. We aim to establish not only how and when each sensory ability arises during the pre-/postbirth period but also discuss how senses are studied. We conclude that although many studies converge to show that newborns are clearly sentient beings, much is still unknown. Further, we identify a series of internal and external factors that could explain discrepancies between studies, and we propose perspectives for future studies. Finally, through examples from animal studies, we illustrate the importance of this detailed knowledge to pursue the enhancement of newborns’ daily living conditions. Indeed, this is a prerequisite for assessing the effects of the physical environment and routine procedures on newborns’ welfare.     

Gustation: intersensory experience par excellence

On the face of it, basic tactile sensation might seem the only essential sensory requirement for the delivery of foods and beverages to the digestive system. In practice, however, the appropriate delivery of raw materials for the maintenance and repair of the body requires complex sensory and cognitive processes, such that flavour sensation arguably constitutes the pre-eminent example of an integrated multicomponent perceptual experience. To raise the profile of the chemical senses amongst researchers in other perceptual domains, I review here the contribution of various sense modalities to the flavour of foods and beverages. Further, in the light of these multisensory inputs, the physiological and psychophysical research summarised in this paper invites optimism that novel ways will be found to intervene when nutritional status is compromised either by specific dietary restraints, or by taste and smell disorders.     

The development of multisensory processes

To understand the development of sensory processes, it is necessary not only to look at the maturation of each of the sensory systems in isolation, but also to study the development of the nervous systems capacity to integrate information across the different senses. It is through such multisensory integration that a coherent perceptual gestalt of the world comes to be generated. In the adult brain, multisensory convergence and integration take place at a number of brainstem and cortical sites, where individual neurons have been found that respond to multisensory stimuli with patterns of activation that depend on the nature of the stimulus complex and the intrinsic properties of the neuron. Parallels between the responses of these neurons and multisensory behavior and perception suggest that they are the substrates that underlie these cognitive processes. In both cat and monkey models, the development of these multisensory neurons and the appearance of their integrative capacity is a gradual postnatal process. For subcortical structures (i.e., the superior colliculus) this maturational process appears to be gated by the appearance of functional projections from regions of association cortex. The slow postnatal maturation of multisensory processes, coupled with its dependency on functional corticotectal connections, suggested that the development of multisensory integration may be tied to sensory experiences acquired during postnatal life. In support of this, eliminating experience in one sensory modality (i.e., vision) during postnatal development severely compromises the integration of multisensory cues. Research is ongoing to better elucidate the critical development antecedents for the emergence of normal multisensory capacity.Edited by Marie-Hélène Giard and Mark WallaceThis revised version was published in May 2004 with corrections to Fig. 1.     

Automatic auditory disambiguation of visual awareness

Multisensory integration can play a critical role in producing unified and reliable perceptual experience. When sensory information in one modality is degraded or ambiguous, information from other senses can crossmodally resolve perceptual ambiguities. Prior research suggests that auditory information can disambiguate the contents of visual awareness by facilitating perception of intermodally consistent stimuli. However, it is unclear whether these effects are truly due to crossmodal facilitation or are mediated by voluntary selective attention to audiovisually congruent stimuli. Here, we demonstrate that sounds can bias competition in binocular rivalry toward audiovisually congruent percepts, even when participants have no recognition of the congruency. When speech sounds were presented in synchrony with speech-like deformations of rivalling ellipses, ellipses with crossmodally congruent deformations were perceptually dominant over those with incongruent deformations. This effect was observed in participants who could not identify the crossmodal congruency in an open-ended interview (Experiment 1) or detect it in a simple 2AFC task (Experiment 2), suggesting that the effect was not due to voluntary selective attention or response bias. These results suggest that sound can automatically disambiguate the contents of visual awareness by facilitating perception of audiovisually congruent stimuli.     

Fales’s Defense of the Given and Requirements for Being a Reason

Fales defends the doctrine of the given against the Sellarsian dilemma. On his view, sensory experiences, to which one has direct access, can justify basic beliefs. He upholds this view by way of defending an expansive conception of inference, according to which a broadly inferential relation can hold between sensory experiences and perceptual beliefs. The purpose of this paper is to show that Fales’s defense of the given fails. For this purpose, I argue that there are two requirements for being a good reason, and that his conception of direct apprehension faces a serious dilemma with regard to these requirements. In addition, I argue that his expansive conception of inference is unfounded.