Electrophysiological evidence for a shared representational medium for visual images and visual percepts

Does mental imagery involve the activation of representations in the visual system? Systematic effects of imagery on visual signal detection performance have been used to argue that imagery and the perceptual processing of stimuli interact at some common locus of activity (Farah, 1985). However, such a result is neutral with respect to the question of whether the interaction occurs during modality-specific visual processing of the stimulus. If imagery affects stimulus processing at early, modality-specific stages of stimulus representation, this implies that the shared stimulus representations are visual, whereas if imagery affects stimulus processing only at later, amodal stages of stimulus representation, this implies that imagery involves more abstract, postvisual stimulus representations. To distinguish between these two possibilities, we repeated the earlier imagery-perception interaction experiment while recording event-related potentials (ERPs) to stimuli from 16 scalp electrodes. By observing the time course and scalp distribution of the effect of imagery on the ERP to stimuli, we can put constraints on the locus of the shared representations for imagery and perception. An effect of imagery was seen within 200 ms following stimulus presentation, at the latency of the first negative component of the visual ERP, localized at the occipital and posterior temporal regions of the scalp, that is, directly over visual cortex. This finding provides support for the claim that mental images interact with percepts in the visual system proper and hence that mental images are themselves visual representations.     

Using Neuroimaging to Measure Mental Representations: Finding Color-OpponentNeurons in Visual Cortex

To date, most neuroimaging studies have tried to localize large regions in the brain that are responsible for specific behaviors or mental operations. Psychological theory, however, is more concerned with the nature of mental functions than with the locations of their neural substrates. This article reviews work that moves beyond functional anatomy to measure how color is represented by neurons in visual cortex. The general approach has three basic steps: First, the areas under investigation are localized in the brain. Second, responses of these areas are measured to sets of systematically varying stimuli, designed to uncover the nature of cortical representations. Third, the neuroimaging data are compared with behavioral data measured with the same stimuli. Results obtained using this approach support the hypothesis that primary visual cortex contains color-opponent neurons that support perception.     

Graspability and object processing in infants

This review pursues the idea that a dual visual system approach is fruitful for interpreting work on infant cognition. We provide examples from the visual perception and cognition literature demonstrating that the potential graspability of stimuli typically used in infant studies influence how these stimuli are processed by the infant brain. Specifically, we argue that small, local, familiar and moving stimuli are more likely to be processed by the dorsal (how or action) stream of visual processing. In contrast, larger, stationary objects are likely to be processed by the ventral (what or perception) stream. This analysis clarifies apparently conflicting results in the literature.     

Testing the efficiency of Markov chain Monte Carlo with People using facial affect categories

Exploring how people represent natural categories is a key step toward developing a better understanding of how people learn, form memories, and make decisions. Much research on categorization has focused on artificial categories that are created in the laboratory, since studying natural categories defined on high-dimensional stimuli such as images is methodologically challenging. Recent work has produced methods for identifying these representations from observed behavior, such as reverse correlation (RC). We compare RC against an alternative method for inferring the structure of natural categories called Markov chain Monte Carlo with People (MCMCP). Based on an algorithm used in computer science and statistics, MCMCP provides a way to sample from the set of stimuli associated with a natural category. We apply MCMCP and RC to the problem of recovering natural categories that correspond to two kinds of facial affect (happy and sad) from realistic images of faces. Our results show that MCMCP requires fewer trials to obtain a higher quality estimate of people's mental representations of these two categories.     

心理表征的可视化途径：基于噪音的反向相关图像分类技术

社会心理学对图像的心理表征研究一直难以将心理活动的内容准确刻画出来。近10年来出现了一种新心理物理学方法——“反向相关图像分类技术”, 该技术假定观察者的反应与视觉噪音存在相关关系, 且反应是依照观察者的社会判断标准进行而非随机做出; 通过对其做出反应的相应噪音模式的足够次数的权重计算与视觉代码显现, 从而将观察者内在的评估特点可视化。该技术已在特质研究、种族和群际偏见等领域取得了一些成果, 但是未来仍需解决实验次数过多, 分离混杂的噪音以及被试的表现等问题, 才能获得更为真实的心理表征。     

Perceiving imitatible stimuli: consequences of isomorphism between input and output.

For more than a century, psychologists have been intrigued by the idea that mental representations of perceived human actions are closely connected with mental representations of performing those same actions. In this article, connections between input and output representations are considered in terms of the potential for imitation. A broad range of evidence suggests that, for imitatible stimuli, input and output representations are isomorphic to one another, allowing mutual influence between perception and motoric planning that is rapid, effortless, and possibly obligatory. Thus, the cognitive consequences of imitatibility may underlie such diverse phenomena as phoneme perception; imitation in neonates; echoic memory; stimulus-response compatibility; conduction aphasia; maintenance rehearsal; and a variety of developmental and social activities such as language acquisition, social learning, empathy, and monitoring one's own behavior.     

Visual mental rotation of possible and impossible objects

Participants were tested on two visual mental rotation tasks using three-dimensional "possible" and "impossible" shapes. Both types of stimuli can be easily encoded by their parts and how they are spatially organized. However, while possible shapes can also be easily encoded as a global image, it is more difficult to encode impossible shapes in such a way. Participants visually rotated both types of stimuli at comparable rates, reflecting that local representations were used in the process of visual mental rotation.     

Examining the interactivity of lexical orthographic and phonological processing.

The number and type of connections involving different levels of orthographic and phonological representations differentiate between several models of spoken and visual word recognition. At the sublexical level of processing, Borowsky, Owen, and Fonos (1999) demonstrated evidence for direct processing connections from grapheme representations to phoneme representations (i.e., a sensitivity effect) over and above any bias effects, but not in the reverse direction. Neural network models of visual word recognition implement an orthography to phonology processing route that involves the same connections for processing sublexical and lexical information, and thus a similar pattern of cross-modal effects for lexical stimuli are expected by models that implement this single type of connection (i.e., orthographic lexical processing should directly affect phonological lexical processing, but not in the reverse direction). Furthermore, several models of spoken word perception predict that there should be no direct connections between orthographic representations and phonological representations, regardless of whether the connections are sublexical or lexical. The present experiments examined these predictions by measuring the influence of a cross-modal word context on word target discrimination. The results provide constraints on the types of connections that can exist between orthographic lexical representations and phonological lexical representations.     

Perceptual load influences auditory space perception in the ventriloquist aftereffect

A period of exposure to trains of simultaneous but spatially offset auditory and visual stimuli can induce a temporary shift in the perception of sound location. This phenomenon, known as the ‘ventriloquist aftereffect’, reflects a realignment of auditory and visual spatial representations such that they approach perceptual alignment despite their physical spatial discordance. Such dynamic changes to sensory representations are likely to underlie the brain’s ability to accommodate inter-sensory discordance produced by sensory errors (particularly in sound localization) and variability in sensory transduction. It is currently unknown, however, whether these plastic changes induced by adaptation to spatially disparate inputs occurs automatically or whether they are dependent on selectively attending to the visual or auditory stimuli. Here, we demonstrate that robust auditory spatial aftereffects can be induced even in the presence of a competing visual stimulus. Importantly, we found that when attention is directed to the competing stimuli, the pattern of aftereffects is altered. These results indicate that attention can modulate the ventriloquist aftereffect.     

Visual body-size adaptation and estimation of tactile distance

Estimating the size of bodies is crucial for interactions with physical and social environments. Body-size perception is malleable and can be altered using visual adaptation paradigms. However, it is unclear whether such visual adaptation effects also transfer to other modalities and influence, for example, the perception of tactile distances. In this study, we employed a visual adaptation paradigm. Participants were exposed to images of expanded or contracted versions of self- or other-identity bodies. Before and after this adaptation, they were asked to manipulate the width of body stimuli to appear as ‘normal’ as possible. We replicated an effect of visual adaptation such that the body-size selected as most ‘normal’ was larger after exposure to expanded and thinner after exposure to contracted adaptation stimuli. In contrast, we did not find evidence that this adaptation effect transfers to distance estimates for paired tactile stimuli delivered to the abdomen. A Bayesian analysis showed that our data provide moderate evidence that there is no effect of visual body-size adaptation on the estimation of spatial parameters in a tactile task. This suggests that visual body-size adaptation effects do not transfer to somatosensory body-size representations.     

Attention to memory: orienting attention to sound object representations

Despite a growing acceptance that attention and memory interact, and that attention can be focused on an active internal mental representation (i.e., reflective attention), there has been a paucity of work focusing on reflective attention to ‘sound objects’ (i.e., mental representations of actual sound sources in the environment). Further research on the dynamic interactions between auditory attention and memory, as well as its degree of neuroplasticity, is important for understanding how sound objects are represented, maintained, and accessed in the brain. This knowledge can then guide the development of training programs to help individuals with attention and memory problems. This review article focuses on attention to memory with an emphasis on behavioral and neuroimaging studies that have begun to explore the mechanisms that mediate reflective attentional orienting in vision and more recently, in audition. Reflective attention refers to situations in which attention is oriented toward internal representations rather than focused on external stimuli. We propose four general principles underlying attention to short-term memory. Furthermore, we suggest that mechanisms involved in orienting attention to visual object representations may also apply for orienting attention to sound object representations.     

Where perception meets memory: A review of repetition priming in visual search tasks

What we have recently seen and attended to strongly influences how we subsequently allocate visual attention. A clear example is how repeated presentation of an object’s features or location in visual search tasks facilitates subsequent detection or identification of that item, a phenomenon known as priming. Here, we review a large body of results from priming studies that suggest that a short-term implicit memory system guides our attention to recently viewed items. The nature of this memory system and the processing level at which visual priming occurs are still debated. Priming might be due to activity modulations of low-level areas coding simple stimulus characteristics or to higher level episodic memory representations of whole objects or visual scenes. Indeed, recent evidence indicates that only minor changes to the stimuli used in priming studies may alter the processing level at which priming occurs. We also review recent behavioral, neuropsychological, and neurophysiological evidence that indicates that the priming patterns are reflected in activity modulations at multiple sites along the visual pathways. We furthermore suggest that studies of priming in visual search may potentially shed important light on the nature of cortical visual representations. Our conclusion is that priming occurs at many different levels of the perceptual hierarchy, reflecting activity modulations ranging from lower to higher levels, depending on the stimulus, task, and context—in fact, the neural loci that are involved in the analysis of the stimuli for which priming effects are seen.     

MENTAL IMAGERY AND THE VARIETIES OF AMODAL PERCEPTION

The problem of amodal perception is the problem of how we represent features of perceived objects that are occluded or otherwise hidden from us. Bence Nanay (2010 ) has recently proposed that we amodally perceive an object's occluded features by imaginatively projecting them into the relevant regions of visual egocentric space. In this paper, I argue that amodal perception is not a single, unitary capacity. Drawing appropriate distinctions reveals amodal perception to be characterized not only by mental imagery, as Nanay suggests, but also by genuinely visual representations as well as beliefs. I conclude with some brief remarks on the role of object‐directed bodily action in conferring a sense of unseen presence on an object's occluded features.     

Data‐driven Methods for Modeling Social Perception

How do we model the complexity of social perception? A major methodological problem is that the space of possible variables driving social perceptions is infinitely large, thus posing an insurmountable hurdle for conventional approaches. Here, we describe a set of data‐driven methods whose objective is to identify quantitative relationships between high‐dimensional variables (e.g., visual images) and behaviors (e.g., perceptual decisions) with as little bias as possible. We focus on social perception of faces, although the methods could be applied to other visual and nonvisual categories. We review two reverse correlation approaches: (a) psychophysical methods based on judgments of images altered with randomly generated noise, where the analysis relates the random variations of the images to judgments; and (b) methods based on judgments of randomly generated faces from a statistical, multidimensional face space model, where the analysis relates the dimensions of the face model to judgments.     

视觉表象可视化——视觉表象研究的新途径

功能磁共振技术在表象研究中得到广泛应用是表象研究追求客观化精确性的必然趋势。本文介绍了功能磁共振多变量模式分析方法及其演变历程,探讨了借助该方法实现“视觉表象可视化”的理论依据与亟待解决的关键问题。分析指出“视觉表象可视化”将为表象研究提供全新的研究视角与方法途径。     

Mental Representation of Familiar Others: The Impact of Occupation,Sex, and Race

The current study used a double primed semantic decision task to investigate the role of social group information in mental representations of familiar others. Extrapolating from social role theory, we predicted that social role information would facilitate responding to familiar targets regardless of the specific task at hand. The names of celebrities were used as stimuli, as people know them because of their social role (i.e., their occupation). Primes and targets were matched (or mismatched) on all combinations of sex, race, and occupation. Participants were randomly assigned to task condition: indicating whether the prime and target were (a) of the same sex or not, (b) of the same race or not, or (c) of the same occupation or not. Results indicated that participants responded faster to celebrity prime-target pairs that were matched on occupation regardless of task condition. In addition, participants responded faster to celebrity prime-target pairs that were matched on sex—but only when sex was relevant to the task. Similar findings were not found for race. The implications of these findings for understanding mental representations of familiar others and for person perception are discussed.     

A functional-cognitive framework for attitude research

In attitude research, behaviours are often used as proxies for attitudes and attitudinal processes. This practice is problematic because it conflates the behaviours that need to be explained (explanandum) with the mental constructs that are used to explain these behaviours (explanans). In the current chapter we propose a meta-theoretical framework that resolves this problem by distinguishing between two levels of analysis. According to the proposed framework, attitude research can be conceptualised as the scientific study of evaluation. Evaluation is defined not in terms of mental constructs but in terms of elements in the environment, more specifically, as the effect of stimuli on evaluative responses. From this perspective, attitude research provides answers to two questions: (1) Which elements in the environment moderate evaluation? (2) What mental processes and representations mediate evaluation? Research on the first question provides explanations of evaluative responses in terms of elements in the environment (functional level of analysis); research on the second question offers explanations of evaluation in terms of mental processes and representations (cognitive level of analysis). These two levels of analysis are mutually supportive, in that better explanations at one level lead to better explanations at the other level. However, their mutually supportive relation requires a clear distinction between the concepts of their explanans and explanandum, which are conflated if behaviours are treated as proxies for mental constructs. The value of this functional-cognitive framework is illustrated by applying it to four central questions of attitude research.     

Why do animals differ in their susceptibility to geometrical illusions?

In humans, geometrical illusions are thought to reflect mechanisms that are usually helpful for seeing the world in a predictable manner. These mechanisms deceive us given the right set of circumstances, correcting visual input where a correction is not necessary. Investigations of non-human animals’ susceptibility to geometrical illusions have yielded contradictory results, suggesting that the underlying mechanisms with which animals see the world may differ across species. In this review, we first collate studies showing that different species are susceptible to specific illusions in the same or reverse direction as humans. Based on a careful assessment of these findings, we then propose several ecological and anatomical factors that may affect how a species perceives illusory stimuli. We also consider the usefulness of this information for determining whether sight in different species might be more similar to human sight, being influenced by contextual information, or to how machines process and transmit information as programmed. Future testing in animals could provide new theoretical insights by focusing on establishing dissociations between stimuli that may or may not alter perception in a particular species. This information could improve our understanding of the mechanisms behind illusions, but also provide insight into how sight is subjectively experienced by different animals, and the degree to which vision is innate versus acquired, which is difficult to examine in humans.     

Image representation,scaling and cognitive model of object perception

Neuropsychological investigations of visual imagery and representations have led to a deeper understanding of the spatial perception, representation and memory. But how each individual perceives object’s geometrical properties and how they differ from person to person, both under event-related memory and normal recollecting memory in the presence or in the absence of direct sensory stimulation is still unclear. Spatial knowledge is diverse, complex, and multi-modal, as are the situations in which it is used. All seem to agree that a cognitive map is a mental representation of an external environment. The image scaling is important in understanding the psychological dysfunctions of patients suffering from spatial cognition problems. The scaling becomes self-evident in art forms, when people are asked to draw image of objects they see actively or from their short or long term memory. In this paper we develop a comprehensive model of this scaling factor and its implications in spatial image representation and memory. We also extend its notion in understanding the perception of objects whose representations are normally not possible (like the perception of universal scales, infinities and parallel lines) but are well comprehended by the human brains. Here we give a scaling factor which is variable depending on the situations for a person based on his visual memory and drawing capabilities. And then extend it to analyse his cognitive strengths, disorders and any imperfections. This model also helps in formalizing the architectural cognitive maps needed to change the scaling factor, depending on the types of visual works one performs.     

初级视觉皮层在表象表征方式研究中的作用

表象的信息表征方式一直是心理学研究的热点问题,脑成像技术在该问题的研究中发挥了巨大作用。本文以初级视觉皮层(V1)在表象表征方式研究中的作用为主线,系统梳理了基于脑成像技术开展的表象实质争论的核心问题,归纳分析了相关争论问题演绎发展的内在逻辑脉络,在此基础上指出了表象研究中需要进一步解决的关键问题,以期能够促进相关研究问题的进一步开展。