Eye movements disrupt episodic future thinking

Remembering the past and imagining the future both rely on complex mental imagery. We considered the possibility that constructing a future scene might tap a component of mental imagery that is not as critical for remembering past scenes. Whereas visual imagery plays an important role in remembering the past, we predicted that spatial imagery plays a crucial role in imagining the future. For the purpose of teasing apart the different components underpinning scene construction in the two experiences of recalling episodic memories and shaping novel future events, we used a paradigm that might selectively affect one of these components (i.e., the spatial). Participants performed concurrent eye movements while remembering the past and imagining the future. These concurrent eye movements selectively interfere with spatial imagery, while sparing visual imagery. Eye movements prevented participants from imagining complex and detailed future scenes, but had no comparable effect on the recollection of past scenes. Similarities between remembering the past and imagining the future are coupled with some differences. The present findings uncover another fundamental divergence between the two processes.     

Neuronal correlates of full and partial visual conscious perception

Stimuli may induce only partial consciousness—an intermediate between null and full consciousness—where the presence but not identity of an object can be reported. The differences in the neuronal basis of full and partial consciousness are poorly understood. We investigated if evoked and oscillatory activity could dissociate full from partial conscious perception. We recorded human cortical activity with magnetoencephalography (MEG) during a visual perception task in which stimulus could be either partially or fully perceived. Partial consciousness was associated with an early increase in evoked activity and theta/low-alpha-band oscillations while full consciousness was also associated with late evoked activity and beta-band oscillations. Full from partial consciousness was dissociated by stronger evoked activity and late increase in theta oscillations that were localized to higher-order visual regions and posterior parietal and prefrontal cortices. Our results reveal both evoked activity and theta oscillations dissociate partial and full consciousness.     

任务信息通达对视觉表象眼动的影响

通过两个实验探讨了视觉表象任务信息的通达对表象加工眼动的影响。结果表明, 在低通达条件下, 表象任务的眼动复制了知觉任务的眼动; 随着表象任务信息通达水平的提高, 眼动的注视点平均持续时间、平均眼跳距离和平均眼跳时间会发生规律性变化; 眼动控制与任务信息通达水平对表象眼动的影响存在不同的机制。实验结果佐证了眼动在视觉表象中起机能性作用的观点。     

Interference effects demonstrate distinct roles for visual and motor imagery during the mental representation of human action

Four experiments were completed to characterize the utilization of visual imagery and motor imagery during the mental representation of human action. In Experiment 1, movement time functions for a motor imagery human locomotion task conformed to a speed-accuracy trade-off similar to Fitts' Law, whereas those for a visual imagery object motion task did not. However, modality-specific interference effects in Experiment 2 demonstrate visual and motor imagery as cooperative processes when the action represented is tied to visual coordinates in space. Biomechanic-specific motor interference effects found in Experiment 3 suggest one basis for separation of processing channels within motor imagery. Finally, in Experiment 4 representations of motor actions were found to be generated using only visual imagery under certain circumstances: namely, when the imaginer represented the motor action of another individual while placed at an opposing viewpoint. These results suggest that the modality of representation recruited to generate images of human action is dependent on the dynamic relationship between the individual, movement, and environment.     

Common pathways in mental imagery and pain perception: an fMRI study of a subject with an amputated arm

The present paper reviews data from two previous studies in our laboratory, as well as some additional new data, on the neuronal representation of movement and pain imagery in a subject with an amputated right arm. The subject imagined painful and non-painful finger movements in the amputated stump while being in a MRI scanner, acquiring EPI-images for fMRI analysis. In Study I (Ersland et al., 1996) the Subject alternated tapping with his intact left hand fingers and imagining "tapping" with the fingers of his amputated right arm. The results showed increased neuronal activation in the right motor cortex (precentral gyrus) when tapping with the fingers of the left hand, and a corresponding activation in the left motor cortex when imagining tapping with the fingers of the amputated right arm. Finger tappings of the intact left hand fingers also resulted in a larger activated precentral area than imagery "finger tapping" of the amputated right arm fingers. In Study II (Rosen et al., 2001 in press) the same subject imagining painful and pleasurable finger movements, and still positions of the fingers of the amputated arm. The results showed larger activations over the motor cortex for movement imagining versus imagining the hand being in a still position, and larger activations over the sensory cortex when imagining painful experiences. It can therefore be concluded that not only does imagery activate the same motor areas as real finger movements, but also that adding instructions of pain together with imaging moving the fingers intensified the activation compared with adding instructions about non-painful experiences. From these studies, it is clear that areas activated during actual motor execution to a large extent also are activated during mental imagery of the same motor commands. In this respect the present studies add to studies of visual imagery that have shown a similar correspondence in activation between actual object perception and imagery of the same object.     

Eye scanpaths during visual imagery reenact those of perception of the same visual scene

Eye movements during mental imagery are not epiphenomenal but assist the process of image generation. Commands to the eyes for each fixation are stored along with the visual representation and are used as spatial index in a motor‐based coordinate system for the proper arrangement of parts of an image. In two experiments, subjects viewed an irregular checkerboard or color pictures of fish and were subsequently asked to form mental images of these stimuli while keeping their eyes open. During the perceptual phase, a group of subjects was requested to maintain fixation onto the screen's center, whereas another group was free to inspect the stimuli. During the imagery phase, all of these subjects were free to move their eyes. A third group of subjects (in Experiment 2) was free to explore the pattern but was requested to maintain central fixation during imagery. For subjects free to explore the pattern, the percentage of time spent fixating a specific location during perception was highly correlated with the time spent on the same (empty) locations during imagery. The order of scanning of these locations during imagery was correlated to the original order during perception. The strength of relatedness of these scanpaths and the vividness of each image predicted performance accuracy. Subjects who fixed their gaze centrally during perception did the same spontaneously during imagery. Subjects free to explore during perception, but maintaining central fixation during imagery, showed decreased ability to recall the pattern. We conclude that the eye scanpaths during visual imagery reenact those of perception of the same visual scene and that they play a functional role.     

The many directions of time

Event duration perception is fundamental to cognitive functioning. Recent research has shown that localized sensory adaptation compresses perceived duration of brief visual events in the adapted location; however, there is disagreement on whether the source of these temporal distortions is cortical or pre-cortical. The current study reveals that spatially localized duration compression can also be direction contingent, in that duration compression is induced when adapting and test stimuli move in the same direction but not when they move in opposite directions. Because of its direction-contingent nature, the induced duration compression reported here is likely to be cortical in origin. A second experiment shows that the adaptation processes driving duration compression can occur at or beyond human cortical area MT+, a specialized motion center located upstream from primary visual cortex. The direction-specificity of these temporal mechanisms, in conjunction with earlier reports of pre-cortical temporal mechanisms driving duration perception, suggests that our encoding of subsecond event duration is driven by activity at multiple levels of processing.     

Visual working memory contaminates perception

Indirect evidence suggests that the contents of visual working memory may be maintained within sensory areas early in the visual hierarchy. We tested this possibility using a well-studied motion repulsion phenomenon in which perception of one direction of motion is distorted when another direction of motion is viewed simultaneously. We found that observers misperceived the actual direction of motion of a single motion stimulus if, while viewing that stimulus, they were holding a different motion direction in visual working memory. Control experiments showed that none of a variety of alternative explanations could account for this repulsion effect induced by working memory. Our findings provide compelling evidence that visual working memory representations directly interact with the same neural mechanisms as those involved in processing basic sensory events.     

The differential contributions of visual imagery constructs on autobiographical thinking

There is a growing theoretical and empirical consensus on the central role of visual imagery in autobiographical memory. However, findings from studies that explore how individual differences in visual imagery are reflected on autobiographical thinking do not present a coherent story. One reason for the mixed findings was suggested to be the treatment of visual imagery as an undifferentiated construct while evidence shows that there is more than one type of visual imagery. The present study investigates the relative contributions of different imagery constructs; namely, object and spatial imagery, on autobiographical memory processes. Additionally, it explores whether a similar relation extends to imagining the future. The results indicate that while object imagery was significantly correlated with several phenomenological characteristics, such as the level of sensory and perceptual details for past events – but not for future events – spatial imagery predicted the level of episodic specificity for both past and future events. We interpret these findings as object imagery being recruited in tasks of autobiographical memory that employ reflective processes while spatial imagery is engaged during direct retrieval of event details. Implications for the role of visual imagery in autobiographical thinking processes are discussed.     

Simulating the role of visual selective attention during the development of perceptual completion

We recently proposed a multi‐channel, image‐filtering model for simulating the development of visual selective attention in young infants (Schlesinger, Amso & Johnson, 2007). The model not only captures the performance of 3‐month‐olds on a visual search task, but also implicates two cortical regions that may play a role in the development of visual selective attention. In the current simulation study, we used the same model to simulate 3‐month‐olds’ performance on a second measure, the perceptual unity task. Two parameters in the model – corresponding to areas in the occipital and parietal cortices – were systematically varied while the gaze patterns produced by the model were recorded and subsequently analyzed. Three key findings emerged from the simulation study. First, the model successfully replicated the performance of 3‐month‐olds on the unity perception task. Second, the model also helps to explain the improved performance of 2‐month‐olds when the size of the occluder in the unity perception task is reduced. Third, in contrast to our previous simulation results, variation in only one of the two cortical regions simulated (i.e. recurrent activity in posterior parietal cortex) resulted in a performance pattern that matched 3‐month‐olds. These findings provide additional support for our hypothesis that the development of perceptual completion in early infancy is promoted by progressive improvements in visual selective attention and oculomotor skill.     

Opposite effects of visual versus imagined presentation of faces on subsequent sex perception

Viewing faces of one sex changes the perception of subsequently seen ambiguous faces. Here we investigate if the mechanisms responsible for this sex aftereffect are also activated during mental imagery of faces. Participants categorized the sex of ambiguous faces after either viewing images of male or female actors' faces or imagining these same faces. As in previous studies, the ambiguous images were categorized as female more often after viewing male faces than after viewing female faces. The opposite effect was found for imagined faces, however; the ambiguous images were categorized as female more often after imagining female faces than after imagining male faces. Although our results are inconsistent with findings that imagined faces cause either no aftereffects or similar aftereffects to visually presented faces, our results are consistent with recent evidence that visual and imagined presentation of faces cause opposite adaptation effects on an early electrophysiological response associated with face processing.     

Testing the dorsal stream attention hypothesis: Electrophysiological correlates and the effects of ventral stream damage

The roles of dorsal and ventral processing streams in visual orienting and conscious perception were examined in two experiments. The first employed high density EEG with source localization. The second comprised a neuropsychological case study. Visual orienting was assessed with an attention procedure, where peripheral letters cued participants towards a target location. In the perception procedure participants responded to the same letters by performing an explicit conscious discrimination. In Experiment 1, the peripheral letters elicited rapid dorsal stream activation in the attention procedure, and this activation preceded top-down enhancement of target processing in occipital cortex. In the perception procedure early ventral stream activation was seen. In addition, peripheral letters elicited an “early directing attention negativity” (EDAN) over parietal recording sites in the attention procedure, but not in the perception procedure. In Experiment 2, a patient with a bilateral ventral stream lesion but preserved dorsal stream function showed clear disruption to performance in the perception procedure, whilst exhibiting a normal visual orienting effect in the attention procedure. Taken together these findings (1) highlight the distinct roles of the dorsal and ventral streams in attention and perception, and (2) suggest how these streams might interact, via reentrant effects of attention on perceptual processing.     

Tracking the processes behind conscious perception: a review of event-related potential correlates of visual consciousness

Event-related potential (ERP) studies have attempted to discover the processes that underlie conscious visual perception by contrasting ERPs produced by stimuli that are consciously perceived with those that are not. Variability of the proposed ERP correlates of consciousness is considerable: the earliest proposed ERP correlate of consciousness (P1) coincides with sensory processes and the last one (P3) marks postperceptual processes. A negative difference wave called visual awareness negativity (VAN), typically observed around 200 ms after stimulus onset in occipitotemporal sites, gains strong support for reflecting the processes that correlate with, and possibly enable, aware visual perception. Research suggests that the early parts of conscious processing can proceed independently of top-down attention, although top-down attention may modulate visual processing even before consciousness. Evidence implies that the contents of consciousness are provided by interactions in the ventral stream, but indispensable contributions from dorsal regions influence already low-level visual responses.     

Sensory features of mental images in the framework of human actions

What determines the sensory impression of a self-generated motor image? Motor imagery is a process in which subjects imagine executing a body movement with a strong kinesthetic and/or visual component from a first-person perspective. Both sensory modalities can be combined flexibly to form a motor image. 90 participants of varying ages had to freely generate motor images from a large set of movements. They were asked to rate their kinesthetic as well as their visual impression, the perceived vividness, and their personal experience with the imagined movement. Data were subjected to correlational analyses, linear regressions, and representation similarity analyses. Results showed that both action characteristics and experience drove the sensory impression of motor images with a strong individual component. We conclude that imagining actions that impose varying demands can be considered as reexperiencing actions by using one’s own sensorimotor representations that represent not only individual experience but also action demands.     

Early perception of written syllables in French: An event-related potential study

The present study examined whether written syllable units are perceived in first steps of letter string processing. An illusory conjunction experiment was conducted while event-related potentials were recorded. Colored pseudowords were presented such that there was a match or mismatch between the syllable boundaries and the color boundaries. The results showed that congruent stimuli for which the syllable and color boundaries coincided produced a greater positive-going waveform than incongruent stimuli for which the syllable and color boundaries did not coincide. This syllable–color congruency effect was observed to occur both prior to 100 ms, and in a time window commencing at 150 ms. This finding suggests that syllable units are perceived in the initial steps of French visual word processing. Moreover, this study underlines the specificity of the illusory conjunction paradigm as a relevant and powerful tool of investigation of sublexical units that are automatically evoked from word perception.     

初级视觉皮层在表象表征方式研究中的作用

表象的信息表征方式一直是心理学研究的热点问题,脑成像技术在该问题的研究中发挥了巨大作用。本文以初级视觉皮层(V1)在表象表征方式研究中的作用为主线,系统梳理了基于脑成像技术开展的表象实质争论的核心问题,归纳分析了相关争论问题演绎发展的内在逻辑脉络,在此基础上指出了表象研究中需要进一步解决的关键问题,以期能够促进相关研究问题的进一步开展。     

Flexibility of working memory encoding in a sentence–picture–sound verification task

Dual-process accounts of working memory have suggested distinct encoding processes for verbal and visual information in working memory, but encoding for nonspeech sounds (e.g., tones) is not well understood. This experiment modified the sentence–picture verification task to include nonspeech sounds with a complete factorial examination of all possible stimulus pairings. Participants studied simple stimuli–pictures, sentences, or sounds–and encoded the stimuli verbally, as visual images, or as auditory images. Participants then compared their encoded representations to verification stimuli–again pictures, sentences, or sounds–in a two-choice reaction time task. With some caveats, the encoding strategy appeared to be as important or more important than the external format of the initial stimulus in determining the speed of verification decisions. Findings suggested that: (1) auditory imagery may be distinct from verbal and visuospatial processing in working memory; (2) visual perception but not visual imagery may automatically activate concurrent verbal codes; and (3) the effects of hearing a sound may linger for some time despite recoding in working memory. We discuss the role of auditory imagery in dual-process theories of working memory.     

Contribution of acetylcholine to visual cortex plasticity

Acetylcholine is involved in a variety of brain functions. In the visual cortex, the pattern of cholinergic innervation varies considerably across different mammalian species and across different cortical layers within the same species. The physiological effects of acetylcholine in the visual cortex display complex responses, which are likely due to cholinergic receptor subtype composition in cytoplasm membrane as well as interaction with other transmitter systems within the local neural circuitry. The functional role of acetylcholine in visual cortex is believed to improve the signal-to-noise ratio of cortical neurons during visual information processing. Available evidence suggests that acetylcholine is also involved in experience-dependent visual cortex plasticity. At the level of synaptic transmission, activation of muscarinic receptors has been shown to play a permissive role in visual cortex plasticity. Among the muscarinic receptor subtypes, the M(1) receptor seems to make a predominant contribution towards modifications of neural circuitry. The signal transduction cascade of the cholinergic pathway may act synergistically with that of the NMDA receptor pathway, whose activation is a prerequisite for cortical plasticity.     

The aftermath of memory retrieval for recycling visual working memory representations

We examined the aftermath of accessing and retrieving a subset of information stored in visual working memory (VWM)—namely, whether detection of a mismatch between memory and perception can impair the original memory of an item while triggering recognition-induced forgetting for the remaining, untested items. For this purpose, we devised a consecutive-change detection task wherein two successive testing probes were displayed after a single set of memory items. Across two experiments utilizing different memory-testing methods (whole vs. single probe), we observed a reliable pattern of poor performance in change detection for the second test when the first test had exhibited a color change. The impairment after a color change was evident even when the same memory item was repeatedly probed; this suggests that an attention-driven, salient visual change made it difficult to reinstate the previously remembered item. The second change detection, for memory items untested during the first change detection, was also found to be inaccurate, indicating that recognition-induced forgetting had occurred for the unprobed items in VWM. In a third experiment, we conducted a task that involved change detection plus continuous recall, wherein a memory recall task was presented after the change detection task. The analyses of the distributions of recall errors with a probabilistic mixture model revealed that the memory impairments from both visual changes and recognition-induced forgetting are explained better by the stochastic loss of memory items than by their degraded resolution. These results indicate that attention-driven visual change and recognition-induced forgetting jointly influence the “recycling” of VWM representations.     

大脑视知觉调控的神经机制

大脑的知觉加工并非单纯由外部刺激驱动,而是存在自上而下的知觉调控。尽管这一现象被大量实验研究证实,但其神经机制仍然是认知神经科学研究的重要问题。本研究系统介绍了知觉调控的神经基础、实现形式、研究范式,及其理论模型,分析指出了当前研究面临的主要问题,并对未来的研究进行了展望,以期促进该问题研究的进一步开展。