Perceiving binocular depth with reference to a common surface

A common surface is a spatial regularity of our terrestrial environment. For instance, we walk on the common ground surface, lay a variety of objects on the table top, and display our favorite paintings on the wall. It has been proposed that the visual system utilizes this regularity as a reference frame for coding objects' distances. Presumably, by treating the common surface as such--i.e. an anticipated constant--the visual system can reduce its coding redundancy, and divert its resources to representing other information. For intermediate-distance space perception, it has been found that absolute distance judgment is most accurate when a common ground surface is available. Here we explored if the common surface also serves as the reference frame for the processing of binocular-disparity information, which is a predominant cue for near-distance space perception. We capitalized on an established observation where the perceived slant of a surface with linear binocular-disparity gradient is underestimated. Clearly, if the visual system utilizes this incorrectly represented slant surface as a reference frame for coding the objects' locations, the perceived depth separation between the objects will be adversely affected. Our results confirm this, by showing that the depth judgment of objects (two laterally separated vertical lines) on, or in the vicinity of, the surface is underestimated. Furthermore, we show that the impact of the common surface on perceived depth separation most likely occurs at the surface-representation level where the visual surface has been explicitly delineated, rather than at the earlier disparity-processing level.     

The origins of causal perception: evidence from postdictive processing in infancy

The currency of our visual experience consists not only of visual features such as color and motion, but also seemingly higher-level features such as causality--as when we see two billiard balls collide, with one causing the other to move. One of the most important and controversial questions about causal perception involves its origin: do we learn to see causality, or does this ability derive in part from innately specified aspects of our cognitive architecture? Such questions are difficult to answer, but can be indirectly addressed via experiments with infants. Here we explore causal perception in 7-month-old infants, using a different approach from previous work. Recent work in adult visual cognition has demonstrated a postdictive aspect to causal perception: in certain situations, we can perceive a collision between two objects in an ambiguous display even after the moment of potential 'impact' has already passed. This illustrates one way in which our conscious perception of the world is not an instantaneous moment-by-moment construction, but rather is formed by integrating information over short temporal windows. Here we demonstrate analogous postdictive processing in infants' causal perception. This result demonstrates that even infants' visual systems process information in temporally extended chunks. Moreover, this work provides a new way of demonstrating causal perception in infants that differs from previous strategies, and is immune to some previous types of critiques.     

A comparison of natural-image-based models of simple-cell coding

Models such as that of Olshausen and Field (O&F, 1997 Vision Research 37 3311-3325) and principal components analysis (PCA) have been used to model simple-cell receptive fields, and to try to elucidate the statistical principles underlying visual coding in area V1. They connect the statistical structure of natural images with the statistical structure of the coding used in V1. The O&F model has created particular interest because the basis functions it produces resemble the receptive fields of simple cells. We evaluate these models in terms of their sparseness and dispersal, both of which have been suggested as desirable for efficient visual coding. However, both attributes have been defined ambiguously in the literature, and we have been obliged to formulate specific definitions in order to allow any comparison between models at all. We find that both attributes are strongly affected by any preprocessing (e.g. spectral pseudo-whitening or a logarithmic transformation) which is often applied to images before they are analysed by PCA or the O&F model. We also find that measures of sparseness are affected by the size of the filters--PCA filters with small receptive fields appear sparser than PCA filters with larger spatial extent. Finally, normalisation of the means and variances of filters influences measures of dispersal. It is necessary to control for all of these factors before making any comparisons between different models. Having taken these factors into account, we find that the code produced by the O&F model is somewhat sparser than the code produced by PCA. However, the difference is rather smaller than might have been expected, and a measure of dispersal is required to distinguish clearly between the two models.     

Perceiving causality after the fact: postdiction in the temporal dynamics of causal perception

In simple dynamic events we can easily perceive not only motion, but also higher-level properties such as causality, as when we see one object collide with another. Several researchers have suggested that such causal perception is an automatic and stimulus-driven process, sensitive only to particular sorts of visual information, and a major research project has been to uncover the nature of these visual cues. Here, rather than investigating what information affects causal perception, we instead explore the temporal dynamics of when certain types of information are used. Surprisingly, we find that certain visual events can determine whether we perceive a collision in an ambiguous situation even when those events occur after the moment of potential 'impact' in the putative collision has already passed. This illustrates a type of postdictive perception: our conscious perception of the world is not an instantaneous moment-by-moment construction, but rather is formed by integrating information presented within short temporal windows, so that new information which is obtained can influence the immediate past in our conscious awareness. Such effects have been previously demonstrated for low-level motion phenomena, but the present results demonstrate that postdictive processes can influence higher-level event perception. These findings help to characterize not only the 'rules' of causal perception, but also the temporal dynamics of how and when those rules operate.     

视时距知觉适应后效的空间选择性

时距知觉适应后效是指长时间适应于某一特定时距会导致个体对后续时距产生知觉偏差。其中对视时距知觉适应后效空间选择性的探讨存在争议,有研究支持位置不变性,也有研究支持位置特异性。这类研究能有效揭示时距编码的认知神经机制,位置不变性可能意味着时距编码位于较高级的脑区,而位置特异性则可能意味着时距编码位于初级视觉皮层。未来还可以探究时距知觉适应后效的视觉坐标表征方式,开展多通道研究以及相应的神经基础研究。     

Seeing the disappearance of unseen objects

Because of the massive amount of incoming visual information, perception is fundamentally selective. We are aware of only a small subset of our visual input at any given moment, and a great deal of activity can occur right in front of our eyes without reaching awareness. While previous work has shown that even salient visual objects can go unseen, here we demonstrate the opposite pattern, wherein observers perceive stimuli which are not physically present. In particular, we show in two motion-induced blindness experiments that unseen objects can momentarily reenter awareness when they physically disappear: in some situations, you can see the disappearance of something you can't see. Moreover, when a stimulus changes outside of awareness in this situation and then physically disappears, observers momentarily see the altered version--thus perceiving properties of an object that they had never seen before, after that object is already gone. This phenomenon of 'perceptual reentry' yields new insights into the relationship between visual memory and conscious awareness.     

Space coding for sensorimotor transformations can emerge through unsupervised learning

The posterior parietal cortex (PPC) is fundamental for sensorimotor transformations because it combines multiple sensory inputs and posture signals into different spatial reference frames that drive motor programming. Here, we present a computational model mimicking the sensorimotor transformations occurring in the PPC. A recurrent neural network with one layer of hidden neurons (restricted Boltzmann machine) learned a stochastic generative model of the sensory data without supervision. After the unsupervised learning phase, the activity of the hidden neurons was used to compute a motor program (a population code on a bidimensional map) through a simple linear projection and delta rule learning. The average motor error, calculated as the difference between the expected and the computed output, was less than 3°. Importantly, analyses of the hidden neurons revealed gain-modulated visual receptive fields, thereby showing that space coding for sensorimotor transformations similar to that observed in the PPC can emerge through unsupervised learning. These results suggest that gain modulation is an efficient coding strategy to integrate visual and postural information toward the generation of motor commands.     

Space coding for sensorimotor transformations can emerge through unsupervised learning

The posterior parietal cortex (PPC) is fundamental for sensorimotor transformations because it combines multiple sensory inputs and posture signals into different spatial reference frames that drive motor programming. Here, we present a computational model mimicking the sensorimotor transformations occurring in the PPC. A recurrent neural network with one layer of hidden neurons (restricted Boltzmann machine) learned a stochastic generative model of the sensory data without supervision. After the unsupervised learning phase, the activity of the hidden neurons was used to compute a motor program (a population code on a bidimensional map) through a simple linear projection and delta rule learning. The average motor error, calculated as the difference between the expected and the computed output, was less than 3°. Importantly, analyses of the hidden neurons revealed gain-modulated visual receptive fields, thereby showing that space coding for sensorimotor transformations similar to that observed in the PPC can emerge through unsupervised learning. These results suggest that gain modulation is an efficient coding strategy to integrate visual and postural information toward the generation of motor commands.     

Spontaneous number representation in mosquitofish

While there is convincing evidence that preverbal human infants and non-human primates can spontaneously represent number, considerable debate surrounds the possibility that such capacity is also present in other animals. Fish show a remarkable ability to discriminate between different numbers of social companions. Previous work has demonstrated that in fish the same set of signature limits that characterize non-verbal numerical systems in primates is present but yet to provide any demonstration that fish can really represent number rather than basing their discrimination on continuous attributes that co-vary with number. In the present work, using the method of ‘item by item’ presentation, we provide the first evidence that fish are capable of selecting the larger group of social companions relying exclusively on numerical information. In our tests subjects could choose between one large and one small group of companions when permitted to see only one fish at a time. Fish were successful when both small (3 vs. 2) and large numbers (8 vs. 4) were involved and their performance was not affected by the density of the fish or by the overall space occupied by the group.     

Foster mother state of mind and treatment use: Different challenges for different people

On the basis of our research findings from the last 10 years, we have developed a training program for foster parents that targets three critical needs for infants in foster care. This treatment program has now been manualized to allow dissemination to other sites. With our intervention as well as any other, we see it as critical to tailor services to the needs of individual clients rather than adhering in doctrinaire fashion to the manual. In particular, we suggest that attending to foster parents' state of mind with regard to attachment is vital to providing an effective treatment. In this article, we present examples of foster parents with different states of mind and describe how treatment is tailored to meet their individual needs.     

Storage of features, conjunctions and objects in visual working memory

Working memory can be divided into separate subsystems for verbal and visual information. Although the verbal system has been well characterized, the storage capacity of visual working memory has not yet been established for simple features or for conjunctions of features. The authors demonstrate that it is possible to retain information about only 3-4 colors or orientations in visual working memory at one time. Observers are also able to retain both the color and the orientation of 3-4 objects, indicating that visual working memory stores integrated objects rather than individual features. Indeed, objects defined by a conjunction of four features can be retained in working memory just as well as single-feature objects, allowing many individual features to be retained when distributed across a small number of objects. Thus, the capacity of visual working memory must be understood in terms of integrated objects rather than individual features.     

Visual attention as a multilevel selection process

Natural visual scenes are cluttered and contain many different objects that cannot all be processed simultaneously. Therefore, attentional mechanisms are needed to select relevant and to filter out irrelevant information. Evidence from functional brain imaging reveals that attention operates at various processing levels within the visual system and beyond. First, the lateral geniculate nucleus appears to be the first stage in the processing of visual information that is modulated by attention, consistent with the idea that it may play an important role as an early gatekeeper in controlling neural gain. Second, areas at intermediate cortical-processing levels, such as V4 and TEO, appear to be important sites at which attention filters out unwanted information by means of receptive field mechanisms. Third, the attention mechanisms that operate in the visual system appear to be controlled by a distributed network of higher order areas in the frontal and parietal cortex, which generate top-down signals that are transmitted via feedback connections to the visual system. And fourth, the pulvinar of the thalamus may operate by integrating and coordinating attentional functions in concert with the fronto-parietal network, although much needs to be learned about its functional properties. The overall view that emerges from the studies reviewed in this article is that neural mechanisms of selective attention operate at multiple stages in the visual system and beyond and are determined by the visual processing capabilities of each stage. In this respect, attention can be considered in terms of a multilevel selection process.     

Visual image analysis by square wavelets: empirical evidence supporting a theoretical agreement between wavelet analysis and receptive field organization of visual cortical neurons

It was proposed that the human visual system analyzes images into square wavelets. To test this view, comparisons were made between the perceived similarity-dissimilarity of alphabet letters and the wavelet analyses of those same letters. For the proposal to be considered tenable, the coefficients of the wavelet analysis of similar letters must be similar, and the coefficients of the wavelet analysis of dissimilar letters must be dissimilar. From a selection of 12 letters, four pairs of letters had been reported by Van der Heijden, Mathas, and Van den Roovaart as very similar, and four other pairs of letters dissimilar. Each of the 12 letters was separately depicted in 8 x 8 matrices, and the signal represented by each of the matrices was analyzed into square wavelets using a new and original procedure which yielded a single set of coefficients for each matrix. Correlations between sets of coefficients were high (r ranged from .88 to .58) for those letter pairs judged high in similarity; correlations were low (r ranged from -.02 to .29) for those letter pairs judged low in similarity. When the correlations between the coefficients of wavelets of all eight-letter pairs were compared with the judged similarity-dissimilarity of all eight-letter pairs, the linear agreement was statistically significant. Agreement was found between the neurophysiological mapping of receptive fields of visual cortical neurons and the vectors or the pattern of pluses and minuses which characterized the wavelet analysis. Furthermore, regeneration of the visual image, or the pattern of neural activity representing the image, could be described by a tree-like flow of information among visual cortical neurons which received response data from visual receptive fields, the response data being wavelet coefficients. Results indicate the analysis accurately produces reliable transformations of visual patterns and may be a process used by the visual system.     

Sounds exaggerate visual shape

While perceiving speech, people see mouth shapes that are systematically associated with sounds. In particular, a vertically stretched mouth produces a /woo/ sound, whereas a horizontally stretched mouth produces a /wee/ sound. We demonstrate that hearing these speech sounds alters how we see aspect ratio, a basic visual feature that contributes to perception of 3D space, objects and faces. Hearing a /woo/ sound increases the apparent vertical elongation of a shape, whereas hearing a /wee/ sound increases the apparent horizontal elongation. We further demonstrate that these sounds influence aspect ratio coding. Viewing and adapting to a tall (or flat) shape makes a subsequently presented symmetric shape appear flat (or tall). These aspect ratio aftereffects are enhanced when associated speech sounds are presented during the adaptation period, suggesting that the sounds influence visual population coding of aspect ratio. Taken together, these results extend previous demonstrations that visual information constrains auditory perception by showing the converse - speech sounds influence visual perception of a basic geometric feature.     

On Becoming Redundant or What Computers Shouldn't Do

I argue here that the development of machines that provide for us what we could previously provide for ourselves may sometimes be a dubious blessing. For the value of many goods is not independent of the way in which they are produced and in particular of the human contribution to their production. With a large range of goods it may matter to us both that people rather than machines contribute to their production and that we ourselves make some such contribution. We have a need to be constructively engaged in the service of our own and one another's ends. We also have an interest both in the extent to which the society in which we live includes all its members in such engagement and the extent to which the goods we enjoy are the fruits of such inclusive human endeavour. A significant and shared human contribution to the meeting of our needs is itself one of our deepest needs. These thoughts are developed primarily with reference to the values found in art, conversation and work.     

The reference frame for encoding and retention of motion depends on stimulus set size

The goal of this study was to investigate the reference frames used in perceptual encoding and storage of visual motion information. In our experiments, observers viewed multiple moving objects and reported the direction of motion of a randomly selected item. Using a vector-decomposition technique, we computed performance during smooth pursuit with respect to a spatiotopic (nonretinotopic) and to a retinotopic component and compared them with performance during fixation, which served as the baseline. For the stimulus encoding stage, which precedes memory, we found that the reference frame depends on the stimulus set size. For a single moving target, the spatiotopic reference frame had the most significant contribution with some additional contribution from the retinotopic reference frame. When the number of items increased (Set Sizes 3 to 7), the spatiotopic reference frame was able to account for the performance. Finally, when the number of items became larger than 7, the distinction between reference frames vanished. We interpret this finding as a switch to a more abstract nonmetric encoding of motion direction. We found that the retinotopic reference frame was not used in memory. Taken together with other studies, our results suggest that, whereas a retinotopic reference frame may be employed for controlling eye movements, perception and memory use primarily nonretinotopic reference frames. Furthermore, the use of nonretinotopic reference frames appears to be capacity limited. In the case of complex stimuli, the visual system may use perceptual grouping in order to simplify the complexity of stimuli or resort to a nonmetric abstract coding of motion information.     

Cooperative representation of visual borders.

Recently it has been reported that the visual cortical cells which are engaged in cooperative coding of global stimulus features, display synchrony in their firing rates when both are stimulated. Alternative models identify global stimulus features with the course spatial scales of the image. Versions of the Munsterberg or Café Wall illusions which differ in their low spatial frequency content were used to show that in all cases it was the high spatial frequencies in the image which determined the strength and direction of these illusions. Since cells responsive to high spatial frequencies have small receptive fields, cooperative coding must be involved in the representation of long borders in the image.     

Measuring the time course of selection during visual search

In visual search tasks, observers can guide their attention towards items in the visual field that share features with the target item. In this series of studies, we examined the time course of guidance toward a subset of items that have the same color as the target item. Landolt Cs were placed on 16 colored disks. Fifteen distractor Cs had gaps facing up or down while one target C had a gap facing left or right. Observers searched for the target C and reported which side contained the gap as quickly as possible. In the absence of other information, observers must search at random through the Cs. However, during the trial, the disks changed colors. Twelve disks were now of one color and four disks were of another color. Observers knew that the target C would always be in the smaller color set. The experimental question was how quickly observers could guide their attention to the smaller color set. Results indicate that observers could not make instantaneous use of color information to guide the search, even when they knew which two colors would be appearing on every trial. In each study, it took participants 200–300 ms to fully utilize the color information once presented. Control studies replicated the finding with more saturated colors and with colored C stimuli (rather than Cs on colored disks). We conclude that segregation of a display by color for the purposes of guidance takes 200–300 ms to fully develop.     

The visual system discounts emotional deviants when extracting average expression

There has been a recent surge in the study of ensemble coding, the idea that the visual system represents a set of similar items using summary statistics (Alvarez & Oliva, 2008; Ariely, 2001; Chong & Treisman, 2003; Parkes, Lund, Angelucci, Solomon, & Morgan, 2001). We previously demonstrated that this ability extends to faces and thus requires a high level of object processing (Haberman & Whitney, 2007, 2009). Recent debate has centered on the nature of the summary representation of size (e.g., Myczek & Simons, 2008) and whether the perceived average simply reflects the sampling of a very small subset of the items in a set. In the present study, we explored this further in the context of faces, asking observers to judge the average expressions of sets of faces containing emotional outliers. Our results suggest that the visual system implicitly and unintentionally discounts the emotional outliers, thereby computing a summary representation that encompasses the vast majority of the information present. Additional computational modeling and behavioral results reveal that an intentional cognitive sampling strategy does not accurately capture observer performance. Observers derive precise ensemble information given a 250-msec exposure, suggesting a rapid and flexible system not bound by the limits of serial attention.