Egocentric Spatial Representation in Action and Perception*

Neuropsychological findings used to motivate the “two visual systems” hypothesis have been taken to endanger a pair of widely accepted claims about spatial representation in conscious visual experience. The first is the claim that visual experience represents 3‐D space around the perceiver using an egocentric frame of reference. The second is the claim that there is a constitutive link between the spatial contents of visual experience and the perceiver’s bodily actions. In this paper, I review and assess three main sources of evidence for the two visual systems hypothesis. I argue that the best interpretation of the evidence is in fact consistent with both claims. I conclude with some brief remarks on the relation between visual consciousness and rational agency.     

The Fourier theory of vision.

The historical roots of the Fourier theory of spatial visual perception are traced. The development of the underlying concepts and the psychophysical experiments that led to them, and that they in turn spawned, are examined, as well as their relation to the current knowledge of neural substrates in the retina and primary visual cortex. Allowing nonlinearities or even substituting other types of basis functions does not eliminate the difficulties faced by any theory of visual perception that is built on the notion of fixed spatial filters.     

Space Perception,Visual Dissonance and the Fate of Standard Representationalism

This paper argues that a common form of representationalism has trouble accommodating empirical findings about visual space perception. Vision science tells us that the visual system systematically gives rise to different experiences of the same spatial property. This, combined with a naturalistic account of content, suggests that the same spatial property can have different veridical looks. I use this to argue that a common form of representationalism about spatial experience must be rejected. I conclude by considering alternatives to this view.     

Visual Feeling of Presence

Everyday visual experience constantly confronts us with things we can interact with in the real world. We literally feel the outside presence of physical objects in our environment via visual perceptual experience. The visual feeling of presence is a crucial feature of vision that is largely unexplored in the philosophy of perception, and poorly debated in vision neuroscience. The aim of this article is to investigate the feeling of presence. I suggest that visual feeling of presence depends on the visual representation of a very particular spatial relation with the object we interact with: the visual representation of absolute egocentric depth, which is due to stereoscopic vision.     

Spatial ontology and physical modalities

Conclusion Most relational theories assert both that spatial discourse is reducible to talk about physical objects and their spatial relations, and that the relation of congruence derives from a non-metrical relation which intervals bear or possibly bear to measuring instruments. We have shown that there are serious logical difficulties involved in maintaining both these positions and the thesis of the continuity of space. We have also shown that Grünbaum's motivating argument for the reduction of congruence is unsound, and, moreover, that the prospects of formulating the kind of definition which this reduction requires seem to rest upon a wholly vague notion of physical possibility. Alternatively, if congruence is taken as primitive, it seems plausible that spatial discourse can be reconstructed so that there is no ontological commitment to spatial objects other than physical objects. Whether or not this relational thesis is ultimately defensible requires more elaboration of the relational theory, and investigation of its role in the formalizations of current physical theories. It is odd, considering the wide currency of relational theories, that this work has never been done.Preparation of this paper was supported in part by a grant from the National Science Foundation (Grant No. SOC75-08464) to the first author.     

Distrusting the present

We use the hierarchical nature of Bayesian perceptual inference to explain a fundamental aspect of the temporality of experience, namely the phenomenology of temporal flow. The explanation says that the sense of temporal flow in conscious perception stems from probabilistic inference that the present cannot be trusted. The account begins by describing hierarchical inference under the notion of prediction error minimization, and exemplifies distrust of the present within bistable visual perception and action initiation. Distrust of the present is then discussed in relation to previous research on temporal phenomenology. Finally, we discuss how there may be individual differences in the experience of temporal flow, in particular along the autism spectrum. The resulting view is that the sense of temporal flow in conscious perception results from an internal, inferential process.     

Constitutive strata and the dorsal stream

In his paper, “The Dorsal Stream and the Visual Horizon,” Michael Madary argues that “dorsal stream processing plays a main role in the spatiotemporal limits of visual perception, in what Husserl identified as the visual horizon” (Madary 2011, p. 424). Madary regards himself as thereby providing a theoretical framework “sensitive to basic Husserlian phenomenology” (Madary 2011). In particular, Madary draws connections between perceptual anticipations and the experience of the indeterminate spatial margins, on the one hand, and the Husserlian spatiotemporal visual horizons, on the other. I argue that Madary’s arguments, for a Husserlian view of the two visual systems, are not convincing. When the notion of visual horizon is adequately understood as a constitutive notion, there will be reason to regard the connections between dorsal processing and the Husserlian spatiotemporal horizons as tenuous at best.     

Space perception and William James's metaphysical presuppositions

William James's overtly philosophical work may be more continuous with his psychological work than is sometimes thought. His Essays in Radical Empiricism can be understood as an explicit statement of the absolute presupposition that formed the basis of Jamesian psychology: that direct experience is primary and has to be taken at face value. An examination of James's theory of space perception suggests that, even in his early work, he presupposed the primacy of direct experience, and that later changes in his account of space perception can be understood as making his view more consistent with this presupposition. In his earlier view of space perception, James argued that sensations were directly experienced as spatial, though he accepted that spatial relations between sensations may be constructed by higher order thought. In his later view, however, James argued that spatial relations were just as directly experienced as sensations. The work of T. H. Green may have prompted James to recognize the full consequence of his ideas and to realize that taking experience at face value required that spatial relations be thought of as intrinsic to experience rather than the result of intellectual construction.     

Auditory space perception in left- and right-handers

Several studies have shown that handedness has an impact on visual spatial abilities. Here we investigated the effect of laterality on auditory space perception. Participants (33 right-handers, 20 left-handers) completed two tasks of sound localization. In a dark, anechoic, and sound-proof room, sound stimuli (broadband noise) were presented via 21 loudspeakers mounted horizontally (from 80° on the left to 80° on the right). Participants had to localize the target either by using a swivel hand-pointer or by head-pointing. Individual lateral preferences of eye, ear, hand, and foot were obtained using a questionnaire. With both pointing methods, participants showed a bias in sound localization that was to the side contralateral to the preferred hand, an effect that was unrelated to their overall precision. This partially parallels findings in the visual modality as left-handers typically have a more rightward bias in visual line bisection compared with right-handers. Despite the differences in neural processing of auditory and visual spatial information these findings show similar effects of lateral preference on auditory and visual spatial perception. This suggests that supramodal neural processes are involved in the mechanisms generating laterality in space perception.     

Testing the generality of the zoom-lens model: Evidence for visual-pathway specific effects of attended-region size on perception

There are volumes of information available to process in visual scenes. Visual spatial attention is a critically important selection mechanism that prevents these volumes from overwhelming our visual system’s limited-capacity processing resources. We were interested in understanding the effect of the size of the attended area on visual perception. The prevailing model of attended-region size across cognition, perception, and neuroscience is the zoom-lens model. This model stipulates that the magnitude of perceptual processing enhancement is inversely related to the size of the attended region, such that a narrow attended-region facilitates greater perceptual enhancement than a wider region. Yet visual processing is subserved by two major visual pathways (magnocellular and parvocellular) that operate with a degree of independence in early visual processing and encode contrasting visual information. Historically, testing of the zoom-lens has used measures of spatial acuity ideally suited to parvocellular processing. This, therefore, raises questions about the generality of the zoom-lens model to different aspects of visual perception. We found that while a narrow attended-region facilitated spatial acuity and the perception of high spatial frequency targets, it had no impact on either temporal acuity or the perception of low spatial frequency targets. This pattern also held up when targets were not presented centrally. This supports the notion that visual attended-region size has dissociable effects on magnocellular versus parvocellular mediated visual processing.     

Visual space as physical geometry

The geometrical incongruence between patterns in visual space and structures and patterns of activity in the visual cortex, long known to investigators, serves as a criterion for evaluating physical theories of visual space. The problem of determining the geometry of the visual world (visual geometry) is compared with that of determining the geometry of the physical world (physical geometry). Theories as to the possible physical locus of visual space, whether in the brain or elsewhere, are reviewed, analyzed, and criticized accordingly. It is concluded that on the basis of congruence alone it would be predicted that visual space is not to be found in the brain, even though it is seemingly linked to it causally, as experimental neurology and neurophysiology demonstrate. Alternative theories as to the nature of visual space are considered, but are also found to be inadequate in explaining visual space in terms acceptable to contemporary science.     

数量空间表征的特点:跨视听通道下的SNARC效应

从跨通道的角度入手,采用大小比较任务,对视听单通道及跨通道下数量空间表征的特点及表征过程中的相互影响进行探讨。结果发现,视觉通道和听觉通道均存在SNARC效应;在跨通道任务下,无论启动通道是视觉还是听觉通道,都表现出,当启动通道的数量大小信息与主通道的数量大小一致或无关时,主通道的SNARC效应没有显著变化;但当启动通道的数量大小信息与主通道不一致时,主通道的SNARC效应受到显著影响,表现为降低或消失。这进一步证明了SNARC效应受情境影响的特点,并发现在进行跨通道数量空间表征时,听觉通道的数量信息对视觉通道下的数量空间表征的影响大于视觉通道的数量信息对听觉通道下的数量空间表征的影响。     

The Epistemological Status of Vision and its Implications for Design

Computational theories of vision typically rely on the analysis of two aspects of human visual function: (1) object and shape recognition (2) co-calibration of sensory measurements. Both these approaches are usually based on an inverse-optics model, where visual perception is viewed as a process of inference from a 2D retinal projection to a 3D percept within a Euclidean space schema. This paradigm has had great success in certain areas of vision science, but has been relatively less successful in understanding perceptual representation, namely, the nature of the perceptual encoding. One of the drawbacks of inverse-optics approaches has been the difficulty in defining the constraints needed to make the inference computationally tractable (e.g. regularity assumptions, Bayesian priors, etc.). These constraints, thought to be learned assumptions about the nature of the physical and optical structures of the external world, have to be incorporated into any workable computational model in the inverse-optics paradigm. But inference models that employ an inverse optics plus structural assumptions approach inevitably result in a naïve realist theory of perceptual representation. Another drawback of inference models for theories of perceptual representation is their inability to explain central features of the visual experience. The one most evident in the process and visual understanding of design is the fact that some visual configurations appear, often spontaneously, as perceptually more coherent than others. The epistemological consequences of inferential approaches to vision indicate that they fail to capture enduring aspects of our visual experience. Therefore they may not be suited to a theory of perceptual representation, or useful for an understanding of the role of perception in the design process and product.     

Single-band amplitude demodulation of Müller-Lyer illusion images

The perception of the Müller-Lyer illusion has previously been explained as a result of visual low band-pass spatial filtering, although, in fact, the illusion persists in band-pass and high-pass filtered images without visible low-spatial frequencies. A new theoretical framework suggests that our perceptual experience about the global spatial structure of an image corresponds to the amplitude modulation (AM) component (or its magnitude, also called envelope) of its AM-FM (alternatively, AM-PM) decomposition. Because demodulation is an ill-posed problem with a non-unique solution, two different AM-FM demodulation algorithms were applied here to estimate the envelope of images of Müller-Lyer illusion: the global and exact Daugman and Downing (1995) AMPM algorithm and the local and quasi-invertible Maragos and Bovik (1995) DESA. The images used in our analysis include the classic configuration of illusion in a variety of spatial and spatial frequency content conditions. In all cases, including those of images for which visual low-pass spatial filtering would be ineffective, the envelope estimated by single-band amplitude demodulation has physical distortions in the direction of perceived illusion. It is not plausible that either algorithm could be implemented by the human visual system. It is shown that the proposed second order visual model of pre-attentive segregation of textures (or "back-pocket" model) could recover the image envelope and, thus, explain the perception of this illusion even in Müller-Lyer images lacking low spatial frequencies.     

Role of the parietal cortex in long-term representation of spatial information in the rat

The processing of spatial information in the brain requires a network of structures within which the hippocampus plays a prominent role by elaborating an allocentric representation of space. The parietal cortex has long been suggested to have a complementary function. An overview of lesion and unit recording data in the rat indicates that the parietal cortex is involved in different aspects of spatial information processing including allocentric and egocentric processing. More specifically, the data suggest that the parietal cortex plays a fundamental role in combining visual and motion information, a process that would be important for an egocentric-to-allocentric transformation process. Furthermore, the parietal cortex may also have a role in the long-term storage of representation although this possibility needs further evidence. The data overall show that the parietal cortex occupies a unique position in the brain at the interface of perception and representation.     

Interaction of Perception and Action in Discrete and Continuous Rapid Aiming Tasks

Previously, we have shown that discrete and continuous rapid aiming tasks are governed by distinct visuomotor control mechanisms by assessing the combined visual illusion effects on the perceived and effective index of difficulty (ID). All participants were perceptually biased by the combined visual illusion before they performed the rapid aiming tasks. In the current study, the authors manipulated the order of performing perceptual and motor tasks to examine whether perceptual or motor experience with the illusory visual target would influence the subsequent perceived and effective ID in discrete and continuous tapping tasks. The results supported our hypothesis showing that perceptual experience with the illusory visual target in the discrete condition reduced the effective ID in the subsequent discrete tapping task, and motor experience with the illusory visual target in the continuous condition reduced the illusion effects on the perceived ID in the subsequent perceptual judgment task. The study demonstrates the coinfluence of perception and action, and suggests that perception and action influence one another with different magnitude depending on the spatial frame of reference used to perform the perceptuomotor task.     

Music-reading expertise alters visual spatial resolution for musical notation

Crowding occurs when the perception of a suprathreshold target is impaired by nearby distractors, reflecting a fundamental limitation on visual spatial resolution. It is likely that crowding limits music reading, as each musical note is crowded by adjacent notes and by the five-line staff, similar to word reading, in which letter recognition is reduced by crowding from adjacent letters. Here, we tested the hypothesis that, with extensive experience, music-reading experts have acquired visual skills such that they experience a smaller crowding effect, resulting in higher music-reading fluency. Experts experienced a smaller crowding effect than did novices, but only for musical stimuli, not for control stimuli (Landolt Cs). The magnitude of the crowding effect for musical stimuli could be predicted by individual fluency in music reading. Our results highlight the role of experience in crowding: Visual spatial resolution can be improved specifically for objects associated with perceptual expertise. Music-reading rates are likely limited by crowding, and our results are consistent with the idea that experience alleviates these limitations.     

Visually Controlled Locomotion and Visual Orientation in Animals

A general theory of locomotor behavior in relation to physical objects is presented. Since the controlling stimulation for such behavior is mainly optical, this involves novel assumptions about object perception and about what is called visual kinaesthesis. Evidence for these assumptions is cited. On the basis of this theory it is possible to suppose that animals are visually oriented to the surfaces of their environment, not merely to light as such. In short, it is possible to explain why they seem to have space perception. Implications of this approach for maze-learning are pointed out.     

Odorant quality perception: a metric individual differences approach

Perceptual spaces, in which similar stimuli are located close to each other and dissimilar stimuli are located far apart, have aided in the understanding of the physiological and psychological bases for sensory quality coding. Differences in perception between individuals should be reflected by differences in the spatial relationships between stimuli. If the dimensionality of the perceptual space is small (e.g., color space), individual differences that reflect specific pathologies are readily apparent from visual inspection. On the other hand, if the dimensionality of the perceptual space is large (as is proposed for odor space), visual inspection alone may not reveal individual differences in quality perception. The present work presents an information-theory-based method for quantifying individual differences in quality perception from perceptual confusion matrices. The ability of this method to quantify individual differences in quality perception is shown in a hypothetical example of specific anosmia. Finally, the method is applied to the examination of intrasubject consistency of odorant quality perception.     

Serial dependence in position occurs at the time of perception

Observers perceive objects in the world as stable over space and time, even though the visual experience of those objects is often discontinuous and distorted due to masking, occlusion, camouflage, or noise. How are we able to easily and quickly achieve stable perception in spite of this constantly changing visual input? It was previously shown that observers experience serial dependence in the perception of features and objects, an effect that extends up to 15 seconds back in time. Here, we asked whether the visual system utilizes an object’s prior physical location to inform future position assignments in order to maximize location stability of an object over time. To test this, we presented subjects with small targets at random angular locations relative to central fixation in the peripheral visual field. Subjects reported the perceived location of the target on each trial by adjusting a cursor’s position to match its location. Subjects made consistent errors when reporting the perceived position of the target on the current trial, mislocalizing it toward the position of the target in the preceding two trials (Experiment 1). This pull in position perception occurred even when a response was not required on the previous trial (Experiment 2). In addition, we show that serial dependence in perceived position occurs immediately after stimulus presentation, and it is a fast stabilization mechanism that does not require a delay (Experiment 3). This indicates that serial dependence occurs for position representations and facilitates the stable perception of objects in space. Taken together with previous work, our results show that serial dependence occurs at many stages of visual processing, from initial position assignment to object categorization.