Integrating visual images and visual percepts across time and space

Recent studies suggest that visual information can be integrated over a relatively long delay (> 1500 ms) to form a more complete representation (image-percept integration). The current studies investigated whether this process can occur between stimuli that differ in their spatial properties. Participants viewed two dot arrays that filled all but one space in a square or rectangular grid when combined, and reported the missing space. The arrays differed either in size or orientation. Performance reached a comparable level as when spatial properties were matched. However, such performance depends on at least two processes. We suggest an early encoding process and a later image formation/spatial attention reallocation process are required. The flexibility of the image-percept integration process suggests a strong mechanism to form more complete or detailed representations over time, even when the retinal size and orientation of the scene may change between successive views.     

Temporal integration of spatially filtered visual images.

Factors which govern the temporal integration of spatial information were examined in a group of five experiments. A series of high-pass and low-pass spatially filtered versions of a visual scene were generated. Observers' ratings of these filtered versions of the scene for perceived image quality indicated that quality was determined both by the bandwidth of spatial information and the presence of high-spatial-frequency edge information. When sequences of three different versions of the scene were presented over an interval of 120 ms the perceived quality of the resulting composite image was determined both from the ratings of the individual components of that sequence and from the order in which these components were presented. When the order of spatial information in a sequence moved from coarse to fine detail the perceived quality of the composite image was significantly better than when the order moved from fine to coarse. This evidence of a coarse-to-fine bias in pattern integration was further investigated with a detection paradigm. The pattern of errors once again indicated that temporal integration of spatial information was superior when a coarse-to-fine mode of information delivery was employed. Taken together the data indicate that the pattern-integration mechanism has an inherent order bias and does not accumulate spatial information so efficiently when the 'natural' coarse-to-fine order is violated.     

Electrophysiological evidence for a shared representational medium for visual images and visual percepts

Does mental imagery involve the activation of representations in the visual system? Systematic effects of imagery on visual signal detection performance have been used to argue that imagery and the perceptual processing of stimuli interact at some common locus of activity (Farah, 1985). However, such a result is neutral with respect to the question of whether the interaction occurs during modality-specific visual processing of the stimulus. If imagery affects stimulus processing at early, modality-specific stages of stimulus representation, this implies that the shared stimulus representations are visual, whereas if imagery affects stimulus processing only at later, amodal stages of stimulus representation, this implies that imagery involves more abstract, postvisual stimulus representations. To distinguish between these two possibilities, we repeated the earlier imagery-perception interaction experiment while recording event-related potentials (ERPs) to stimuli from 16 scalp electrodes. By observing the time course and scalp distribution of the effect of imagery on the ERP to stimuli, we can put constraints on the locus of the shared representations for imagery and perception. An effect of imagery was seen within 200 ms following stimulus presentation, at the latency of the first negative component of the visual ERP, localized at the occipital and posterior temporal regions of the scalp, that is, directly over visual cortex. This finding provides support for the claim that mental images interact with percepts in the visual system proper and hence that mental images are themselves visual representations.     

The locus of spatial attention during the temporal integration of visual memories and visual percepts

Temporal integration is a process by which two serially presented visual stimuli are mentally integrated to form a composite representation. In the present research, we explored how spatial selective attention is used during the delay separating stimuli, in order to determine the contents of spatial working memory in this task. A two-task situation was created. On the primary task, two dot arrays were serially presented within a grid, leaving one space empty, which subjects identified. On the secondary task, instead of the second array, a discrimination probe was presented. Integration accuracy increased through delays of 1,500 msec, revealing an estimate of the time required to form an optimal memory trace for integration. Once the memory trace was formed (but not before), response time to the probe was faster if it was presented in a location previously occupied by a dot from Array 1. This indicates that during the delay separating the arrays, the subjects assigned spatial attention to the locations occupied by the first array and actively maintained the leading array in working memory. Implications for theories of visual processing and memory are discussed.     

Temporal and spatial factors in the fading of diffuse visual images

Pattern vision breaks down after a period of inspecting a diffusely contoured dark figure upon a ground of uniform brightness and texture. When the field is illuminated with flickering light it is found that the persistence of pattern vision is inversely related to flicker frequency up to the region of fusion. At low flicker frequencies complex figures persist longer than less complex ones of equivalent area, but the reverse applies at frequencies above fusion. These findings are consistent with data obtained from experiments using stabilized retinal images     

Inducing synesthesia in non-synesthetes: Short-term visual deprivation facilitates auditory-evoked visual percepts

Sounds can modulate activity in visual cortex, facilitating the detection of visual targets. However, these sound-driven modulations are not thought to evoke conscious visual percepts in the general population. In individuals with synesthesia, however, multisensory interactions do lead to qualitatively different experiences such as sounds evoking flashes of light. Why, if multisensory interactions are present in all individuals, do only synesthetes experience abnormal qualia? Competing models differ in the time required for synesthetic experiences to emerge. The cross-activation model suggests synesthesia arises over months or years from the development of abnormal neural connections. Here we demonstrate that after ∼5 min of visual deprivation, sounds can evoke synesthesia-like percepts (vivid colors and Klüver form-constants) in ∼50% of non-synesthetes. These results challenge aspects of the cross-activation model and suggest that synesthesia exists as a latent feature in all individuals, manifesting when the balance of activity across the senses has been altered.     

Temporal integration differences between crossed and uncrossed stereoscopic mechanisms

In this study, we investigated temporal integration of disparity information for crossed and uncrossed stereopsis. Across three experiments, exposure duration thresholds were measured for stereoscopic stimuli created from dynamic random-dot stereograms. In Experiment 1, an investigation of disparity detection showed that detection thresholds were equal for the crossed and uncrossed directions. In Experiment 2, an examination of duration limits for depth perception showed that critical durations were lower, and depth more veridical, for crossed depth than for uncrossed depth. In Experiment 3, an investigation of depth discrimination revealed that discrimination thresholds were lower for crossed depth than for uncrossed depth. These results suggest that crossed and uncrossed mechanisms differ in terms of their temporal integration properties at processing levels involving the computation and discrimination of depth.     

Temporal integration and segregation of brief visual stimuli: Patterns of correlation in time

Two brief sequential displays separated by a brief interstimulus interval (ISI) are often perceived as a temporally integrated unitary configuration. The probability of temporal integration can be decreased by increasing the ISI or (counterintuitively) by increasing stimulus duration. We tested three hypotheses of the relative contributions of stimulus duration and ISI to the breakdown of temporal integration (the storage, processing, and temporal correlation hypotheses). In the first of two experiments, stimulus duration and ISI were varied factorially, and estimates of temporal integration were obtained with a form-part integration task. The second experiment was a replication of the first at two levels of stimulus intensity. The outcomes were inconsistent with the storage and processing options, but confirmed predictions from the temporal correlation hypothesis. Whether two sequential stimuli are perceived as temporally integrated or disjoint depends not on the availability of visible persistence, but on the emergence of a neural code that is based on the temporal correlation between the two visual responses.     

Temporal integration and perception duration

Abstract.— A mathematical model is proposed for the relationship between the intensity of a stimulus as function of time and the intensity of the corresponding respcnse in the form of a conscious experience. The model is based on the assumption that the psychophysical power law is valid for the energy contained in a sliding time-window. The width and shape of this time-window will determine both the region of temporal integration and the perception duration. The consequences of this model are compared with some measurements in hearing and vision, found in the literature.     

Dissociation between the mental rotation of visual images and motor images in unilateral brain-damaged patients

Deficits in the mental rotation of body parts and of external objects can be doubly dissociated (Rumiati, Tomasino, Vorano, Umiltà, & De Luce, 2001; Sirigu & Duhamel, 2001; Tomasino, Rumiati, & Umità, in press). The aim of this study was to replicate this finding and to then investigate the relevance of the specific hemispheres in these deficits. Nine patients with unilateral lesions (five in the Left Hemisphere and four in the Right) and 20 control subjects, performed a single task requiring mental rotation of hands, and two tasks requiring mental rotation of external objects. RH patients were impaired in the rotation of external objects, but showed intact performance on the rotation of hands; the opposite pattern was found for LH patients. These results support the view that the LH contributes to the mental rotation of hands, recruiting processes specific to motor preparation, while the RH is specialized for mental rotation of external objects.     

Temporal integration of monocular images separated in time: stereopsis, stereoacuity, and binocular luster

We evaluated stereopsis and binocular luster using electronically controlled shutter glasses with alternating monocular stimulation. In Experiment 1, we used the standard method for testing stereoacuity to obtain a gradual measure of stereopsis. Stereo thresholds decreased with increasing alternating frequency of two monocular half-images without a delay between them. Increasing delays led to increasing thresholds. In Experiment 2, we compared stereopsis resulting from two monocular half-images of a random-dot stereogram and binocular luster with respect to the minimum alternating frequency of the two half-images and the maximum interocular delay that were tolerated without a breakdown of the impression. Below 3 Hz, no stereopsis occurred. Binocular luster was observed only above 10 Hz. The mean threshold of interocular delay for detecting the global figure in a random-dot stereogram was about 51 msec, but for binocular luster it was about 20 msec. Overall, temporal integration was better for stereopsis than for binocular luster.     

Temporal relationships between eye fixations and manual reactions in visual search

Observers freely searched for, and manually responded to, the presence of a target in multistimulus displays. The stimuli were presented on a cinema screen such that each display subtended a large visual angle to encourage the use of eye movements. Times taken to initially fixate the target (T1Fs) were compared to manual response times (MRTs). The results of two experiments were qualitatively similar, despite different levels of difficulty between them. MRTs were a linear function of T1Fs, but only when fixations did not occur very early after the onset of the stimulus display. When fixations were made very soon after the onset of the display, T1Fs were independent of MRTs. The findings were described within the framework of a one-way synchronization model which was modified to accommodate attention effects in visual search. Finally, the methodology provides a novel means of quantifying the contributions of eye movements to manual acknowledgements in real-world vision-guided tasks.     

Long-term memory for elementary visual percepts: memory psychophysics of context and acquisition effects

In the first phase of each of two experiments, participants learned to associate a set of labels (i.e., consonant-vowel-consonant [CVC]) with a set of line lengths by using a paired-associate learning procedure. In the second phase of each experiment, these learned labels were used as memorial standards in the method of constant stimuli. Psychometric functions and the associated indices of discriminative performance (i.e., Weber fractions [WFs], just noticeable difference, and point of subjective equality) were then obtained for the remembered standards. In Experiment 1, WFs (i.e., the indices of memory precision) obtained with remembered standards were found to be higher (i.e., had poorer discriminability) than were WFs obtained with perceptual standards. In addition, WFs obtained with the remembered standards exhibited serial position effects (i.e., poorer discriminability for central items in the memory ensemble) and systematically varied with set size (i.e., the number of standards in the memory set), but WFs obtained with perceptual standards did not depend on serial position or set size. In Experiment 2, increasing the number of acquisition trials reduced WFs and diminished serial position effects. In addition, WFs did not vary systematically with the "physical" spacing between the standards in memory, but they did with the ordinal spacing. The results are consistent with a noisy analogue representation of remembered magnitudes, whereby central items in a memory ensemble are subject to lateral inhibition and thus reduced discriminability. Finally, presentation order effects, as defined by the classic time-order error, were observed with purely perceptual comparisons but not with comparisons involving a remembered standard. This latter finding is inconsistent with a strong form of the functional equivalence view of perception and memory.     

Flicker-light induced visual phenomena: frequency dependence and specificity of whole percepts and percept features

Flickering light induces visual hallucinations in human observers. Despite a long history of the phenomenon, little is known about the dependence of flicker-induced subjective impressions on the flicker frequency. We investigate this question using Ganzfeld stimulation and an experimental paradigm combining a continuous frequency scan (1–50 Hz) with a focus on re-occurring, whole percepts. On the single-subject level, we find a high degree of frequency stability of percepts. To generalize across subjects, we apply two rating systems, (1) a set of complex percept classes derived from subjects’ reports and (2) an enumeration of elementary percept features, and determine distributions of occurrences over flicker frequency. We observe a stronger frequency specificity for complex percept classes than elementary percept features. Comparing the similarity relations among percept categories to those among frequency profiles, we observe that though percepts are preferentially induced by particular frequencies, the frequency does not unambiguously determine the experienced percept.     

Temporal integration and vibrotactile backward masking

Subjects were presented with vibrotactile target patterns to their left index fingertips. The target patterns varied in the number of line segments that they contained and were presented in the presence or absence of a backward-masking stimulus. The stimulus onset asynchrony (SOA) between the target and masker was varied. In an identification task, subjects' errors indicated that the effect of the masker at brief SOAs was to increase the perceived number of line segments in the target. This effect diminished with increasing SOA, and at the longest SOAs subjects confused targets with patterns containing the same number of line segments but varying in how the line segments were related. In an estimation task, the effect of the masker was to increase the number of line segments estimated to be contained in the target pattern. The effect of the masker at brief SOAs is discussed in terms of an integration theory of vibrotactile backward masking. At longer SOAs, the results suggest that the masker may interfere with the extraction of relational information in the target pattern.     

Reexamining the possible benefits of visual crowding: dissociating crowding from ensemble percepts

Peripheral objects and their features become indistinct when closely surrounding but nonoverlapping objects are present. Most models suggest that this phenomenon, called crowding, reflects limitations of visual processing, but an intriguing idea is that it may be, in part, adaptive. Specifically, the mechanism generating crowding may simultaneously facilitate ensemble representations of features, leaving meaningful information about clusters of objects. In two experiments, we tested whether visual crowding and the perception of ensemble features share a common mechanism. Observers judged the orientation of a crowded bar, or the ensemble orientation of all bars in the upper and lower visual fields. While crowding was predictably stronger in the upper relative to the lower visual field, the ensemble percept did not vary between the visual fields. Featural averaging within the crowded region does not always scale with the resolution limit defined by crowding, suggesting that dissociable processes contribute to visual crowding and ensemble percepts.