Vision of a pictorial hand modulates visual-tactile interactions

The participants in this study discriminated the position of tactile target stimuli presented at the tip or the base of the forefinger of one of the participants’ hands, while ignoring visual distractor stimuli. The visual distractor stimuli were presented from two circles on a display aligned with the tactile targets in Experiment 1 or orthogonal to them in Experiment 2. Tactile discrimination performance was slower and less accurate when the visual distractor stimuli were presented from incongruent locations relative to the tactile target stimuli (e.g., tactile target at the base of the finger with top visual distractor) highlighting a cross-modal congruency effect. We examined whether the presence and orientation of a simple line drawing of a hand, which was superimposed on the visual distractor stimuli, would modulate the cross-modal congruency effects. When the tactile targets and the visual distractors were spatially aligned, the modulatory effects of the hand picture were small (Experiment 1). However, when they were spatially misaligned, the effects were much larger, and the direction of the cross-modal congruency effects changed in accordance with the orientation of the picture of the hand, as if the hand picture corresponded to the participants’ own stimulated hand (Experiment 2). The results suggest that the two-dimensional picture of a hand can modulate processes maintaining our internal body representation. We also observed that the cross-modal congruency effects were influenced by the postures of the stimulated and the responding hands. These results reveal the complex nature of spatial interactions among vision, touch, and proprioception.     

Tool-use changes multimodal spatial interactions between vision and touch in normal humans

In a visual-tactile interference paradigm, subjects judged whether tactile vibrations arose on a finger or thumb (upper vs. lower locations), while ignoring distant visual distractor lights that also appeared in upper or lower locations. Incongruent visual distractors (e.g. a lower light combined with upper touch) disrupt such tactile judgements, particularly when appearing near the tactile stimulus (e.g. on the same side of space as the stimulated hand). Here we show that actively wielding tools can change this pattern of crossmodal interference. When such tools were held in crossed positions (connecting the left hand to the right visual field, and vice-versa), the spatial constraints on crossmodal interference reversed, so that visual distractors in the other visual field now disrupted tactile judgements most for a particular hand. This phenomenon depended on active tool-use, developing with increased experience in using the tool. We relate these results to recent physiological and neuropsychological findings.     

Mislocalizations of touch to a fake hand

Observers can mislocalize a tactile target delivered to an unseen hand if a visible rubber glove is positioned next to a pair of distractor lights that flash in correlation with the tactile target (Pavani, Spence, & Driver, 2000). In the present study, we explored visual, tactile, and postural factors that influence this fake hand effect. Comparison with baseline conditions revealed that the fake hand effect was larger than a general spatial congruity effect but weaker than the effect obtained when tactile and visual stimuli were actually in the same locations (Experiment 1). Surprisingly, the effect did not depend on direct vision of the fake hand (Experiments 1 and 2), nor was it enhanced by congruent tactile information (Experiment 3). However, the fake hand effect was sensitive to the postural compatibility of the real and the fake hands (Experiment 4). These findings indicate that the available sensory information is used flexibly to incorporate the rubber glove into the body schema.     

Spatial constraints on visual-tactile cross-modal distractor congruency effects

Across three experiments, participants made speeded elevation discrimination responses to vibrotactile targets presented to the thumb (held in a lower position) or the index finger (upper position) of either hand, while simultaneously trying to ignore visual distractors presented independently from either the same or a different elevation. Performance on the vibrotactile elevation discrimination task was slower and less accurate when the visual distractor was incongruent with the elevation of the vibrotactile target (e.g., a lower light during the presentation of an upper vibrotactile target to the index finger) than when they were congruent, showing that people cannot completely ignore vision when selectively attending to vibrotactile information. We investigated the attentional, temporal, and spatial modulation of these cross-modal congruency effects by manipulating the direction of endogenous tactile spatial attention, the stimulus onset asynchrony between target and distractor, and the spatial separation between the vibrotactile target, any visual distractors, and the participant’s two hands within and across hemifields. Our results provide new insights into the spatiotemporal modulation of crossmodal congruency effects and highlight the utility of this paradigm for investigating the contributions of visual, tactile, and proprioceptive inputs to the multisensory representation of peripersonal space.     

On the inability to ignore touch when responding to vision in the crossmodal congruency task

We investigated the extent to which people can selectively ignore distracting vibrotactile information when performing a visual task. In Experiment 1, participants made speeded elevation discrimination responses (up vs. down) to a series of visual targets presented from one of two eccentricities on either side of central fixation, while simultaneously trying to ignore task-irrelevant vibrotactile distractors presented independently to the finger (up) vs. thumb (down) of either hand. Participants responded significantly more slowly, and somewhat less accurately, when the elevation of the vibrotactile distractor was incongruent with that of the visual target than when they were presented from the same (i.e., congruent) elevation. This crossmodal congruency effect was significantly larger when the visual and tactile stimuli appeared on the same side of space than when they appeared on different sides, although the relative eccentricity of the two stimuli within the hemifield (i.e., same vs. different) had little effect on performance. In Experiment 2, participants who crossed their hands over the midline showed a very different pattern of crossmodal congruency effects to participants who adopted an uncrossed hands posture. Our results suggest that both the relative external location and the initial hemispheric projection of the target and distractor stimuli contribute jointly to determining the magnitude of the crossmodal congruency effect when participants have to respond to vision and ignore touch.     

What aspects of vision facilitate haptic processing?

We investigate how vision affects haptic performance when task-relevant visual cues are reduced or excluded. The task was to remember the spatial location of six landmarks that were explored by touch in a tactile map. Here, we use specially designed spectacles that simulate residual peripheral vision, tunnel vision, diffuse light perception, and total blindness. Results for target locations differed, suggesting additional effects from adjacent touch cues. These are discussed. Touch with full vision was most accurate, as expected. Peripheral and tunnel vision, which reduce visuo-spatial cues, differed in error pattern. Both were less accurate than full vision, and significantly more accurate than touch with diffuse light perception, and touch alone. The important finding was that touch with diffuse light perception, which excludes spatial cues, did not differ from touch without vision in performance accuracy, nor in location error pattern. The contrast between spatially relevant versus spatially irrelevant vision provides new, rather decisive, evidence against the hypothesis that vision affects haptic processing even if it does not add task-relevant information. The results support optimal integration theories, and suggest that spatial and non-spatial aspects of vision need explicit distinction in bimodal studies and theories of spatial integration.     

Attentional control settings prevent abrupt onsets from capturing visual spatial attention

When a visual distractor appears earlier than a visual target in a target-detection task, response time is faster if the distractor appears at the same location as the target. When a visual distractor appears concurrently with a visual target in a target-detection task, response time is slowed relative to when no distractor is presented. Both effects have been taken as evidence of the capture of visual spatial attention, yet capture by early distractors is contingent on top-down attentional control settings (ACSs), and capture by concurrent distractors is not. The present study evaluated whether this incongruity is attributable to the timing of distractors (earlier than vs. concurrently with the target), or to the employed comparisons (same location/different location vs. distractor/no distractor). Using a task that presented both early and concurrent distractors, we observed that, regardless of timing, capture was contingent on ACSs when assessed by the same-location/different-location comparison. This result suggests that, although irrelevant stimuli cause nonspatial purely stimulus-driven effects, the capture of visual spatial attention is contingent on ACSs.     

The effect of visual threat on spatial attention to touch

Viewing a threatening stimulus can bias visual attention toward that location. Such effects have typically been investigated only in the visual modality, despite the fact that many threatening stimuli are most dangerous when close to or in contact with the body. Recent multisensory research indicates that a neutral visual stimulus, such as a light flash, can lead to a tactile attention shift towards a nearby body part. Here, we investigated whether the threat value of a visual stimulus modulates its effect on attention to touch. Participants made speeded discrimination responses about tactile stimuli presented to one or other hand, preceded by a picture cue (snake, spider, flower or mushroom) presented close to the same or the opposite hand. Pictures of snakes led to a significantly greater tactile attentional facilitation effect than did non-threatening pictures of flowers and mushrooms. Furthermore, there was a correlation between self-reported fear of snakes and spiders and the magnitude of early facilitation following cues of that type. These findings demonstrate that the attentional bias towards threat extends to the tactile modality and indicate that perceived threat value can modulate the cross-modal effect that a visual cue has on attention to touch.     

Gestalt grouping effects on tactile information processing: when touching hands override spatial proximity

Using a tactile variant of the negative-priming paradigm, we analyzed the influence of Gestalt grouping on the ability of participants to ignore distracting tactile information. The distance between participants’ hands, to which the target and distractor stimuli were simultaneously delivered, was varied (near/touching hands vs. hands far apart). In addition, the influence of touching hands was controlled, as participants wore gloves and their hands were blocked from vision by a cover. The magnitude of the tactile negative-priming effect was modulated by the interaction between hand separation and whether or not gloves were worn. When the hands were touching, negative priming emerged only while wearing gloves that prevented direct skin-to-skin contact. In contrast, when the separation between the participants’ hands was larger, negative priming emerged only when gloves were not worn. This pattern of results is interpreted in terms of the competing influences of two interacting Gestalt principles—namely, connectedness and proximity—on the processing of tactile distractors.     

Tactile selective attention and body posture: assessing the multisensory contributions of vision and proprioception

This study addressed the role of proprioceptive and visual cues to body posture during the deployment of tactile spatial attention. Participants made speeded elevation judgments (up vs. down) to vibrotactile targets presented to the finger or thumb of either hand, while attempting to ignore vibrotactile distractors presented to the opposite hand. The first two experiments established the validity of this paradigm and showed that congruency effects were stronger when the target hand was uncertain (Experiment 1) than when it was certain (Experiment 2). Varying the orientation of the hands revealed that these congruency effects were determined by the position of the target and distractor in external space, and not by the particular skin sites stimulated (Experiment 3). Congruency effects increased as the hands were brought closer together in the dark (Experiment 4), demonstrating the role of proprioceptive input in modulating tactile selective attention. This spatial modulation was also demonstrated when a mirror was used to alter the visually perceived separation between the hands (Experiment 5). These results suggest that tactile, spatially selective attention can operate according to an abstract spatial frame of reference, which is significantly modulated by multisensory contributions from both proprioception and vision.     

Tactile localization performance in children with developmental coordination disorder (DCD) corresponds to their motor skill and not their cognitive ability

When localizing touches to the hands, typically developing children and adults show a “crossed hands effect” whereby identifying which hand received a tactile stimulus is less accurate when the hands are crossed than uncrossed. This demonstrates the use of an external frame of reference for locating touches to one’s own body. Given that studies indicate that developmental vision plays a role in the emergence of external representations of touch, and reliance on vision for representing the body during action is atypical in developmental coordination disorder (DCD), we investigated external spatial representations of touch in children with DCD using the “crossed hands effect”. Nineteen children with DCD aged 7–11 years completed a tactile localization task in which posture (uncrossed, crossed) and view (hands seen, unseen) were varied systematically. Their performance was compared to that of 35 typically developing controls (19 of a similar age and cognitive ability, and 16 of a younger age but similar fine motor ability). Like controls, the DCD group exhibited a crossed hands effect, whilst their overall tactile localization performance was weaker than their peers of similar age and cognitive ability, but in line with younger controls of similar motor ability. For children with movement difficulties, these findings indicate tactile localization impairments in relation to age expectations, but apparently typical use of an external reference frame for localizing touch.     

A cross-modal interference effect in grasping objects

The aim of the present paper was to investigate how the kinematics of a hand reaching toward a visual target would be influenced by haptic and proprioceptive input from an unseen distractor actively grasped in the other, nonreaching hand. The main results were that the amplitude of maximum grip aperture was smaller and the time to maximum grip aperture was earlier when the distractor was smaller than the target. The interference effect from the distractor was similar for both hands as they reached. Furthermore, results from a vibrating-distractor condition for passive tactile input revealed that the interference effects were evident only when the distractor was actively grasped. We suggest that neural processing of proprioceptive and tactile information relevant to distractor size produced the observed interference effects. We also emphasize the importance of active manipulation of the distractor stimulus in eliciting such interference effects.     

Within-hemifield posture changes affect tactile–visual exogenous spatial cueing without spatial precision,especially in the dark

We investigated the effects of seen and unseen within-hemifield posture changes on crossmodal visual–tactile links in covert spatial attention. In all experiments, a spatially nonpredictive tactile cue was presented to the left or the right hand, with the two hands placed symmetrically across the midline. Shortly after a tactile cue, a visual target appeared at one of two eccentricities within either of the hemifields. For half of the trial blocks, the hands were aligned with the inner visual target locations, and for the remainder, the hands were aligned with the outer target locations. In Experiments 1 and 2, the inner and outer eccentricities were 17.5º and 52.5º, respectively. In Experiment 1, the arms were completely covered, and visual up–down judgments were better when on the same side as the preceding tactile cue. Cueing effects were not significantly affected by hand or target alignment. In Experiment 2, the arms were in view, and now some target responses were affected by cue alignment: Cueing for outer targets was only significant when the hands were aligned with them. In Experiment 3, we tested whether any unseen posture changes could alter the cueing effects, by widely separating the inner and outer target eccentricities (now 10º and 86º). In this case, hand alignment did affect some of the cueing effects: Cueing for outer targets was now only significant when the hands were in the outer position. Although these results confirm that proprioception can, in some cases, influence tactile–visual links in exogenous spatial attention, they also show that spatial precision is severely limited, especially when posture is unseen.     

Integration of hand and finger location in external spatial coordinates for tactile localization

Tactile stimulus location is automatically transformed from somatotopic into external spatial coordinates, rendering information about the location of touch in three-dimensional space. This process is referred to as tactile remapping. Whereas remapping seems to occur automatically for the hands and feet, the fingers may constitute an exception in that some studies have implied purely somatotopic coding of touch to the fingers. When participants judge the order of two tactile stimuli, they often err when the stimulated body parts (usually the two hands) are crossed, presumably because somatotopic and external coordinates are in conflict in crossed postures. Using this task, we investigated, first, whether the fingers are unlike other limbs with regard to spatial coding, by testing whether crossing effects, indicative of external coding, were observable when stimulating two fingers, either on the same or on different hands. Second, we investigated the interaction of hand and finger posture in tactile localization of finger stimuli. Crossing effects emerged when fingers and hands were crossed, suggesting external coding for all body parts. Crossing effects were larger when both hand and finger were located in the hemifield opposite to their body side, and smaller when only hand or finger lay in the opposite hemifield. We suggest that tactile location is estimated by integrating the external location of all relevant body parts, here of a finger and its belonging hand, and that such integrative coding may represent a general principle for body part processing as well as for tool use.     

Sensory temporal processing in adults with early hearing loss

This study examined tactile and visual temporal processing in adults with early loss of hearing. The tactile task consisted of punctate stimulations that were delivered to one or both hands by a mechanical tactile stimulator. Pairs of light emitting diodes were presented on a display for visual stimulation. Responses consisted of YES or NO judgments as to whether the onset of the pairs of stimuli was perceived simultaneously or non-simultaneously. Tactile and visual temporal thresholds were significantly higher for the deaf group when compared to controls. In contrast to controls, tactile and visual temporal thresholds for the deaf group did not differ when presentation locations were examined. Overall findings of this study support the notion that temporal processing is compromised following early deafness regardless of the spatial location in which the stimuli are presented.     

Cross-modal links in exogenous covert spatial orienting between touch, audition, and vision

Three experiments investigated cross-modal links between touch, audition, and vision in the control of covert exogenous orienting. In the first two experiments, participants made speeded discrimination responses (continuous vs. pulsed) for tactile targets presented randomly to the index finger of either hand. Targets were preceded at a variable stimulus onset asynchrony (150,200, or 300 msec) by a spatially uninformative cue that was either auditory (Experiment 1) or visual (Experiment 2) on the same or opposite side as the tactile target. Tactile discriminations were more rapid and accurate when cue and target occurred on the same side, revealing cross-modal covert orienting. In Experiment 3, spatially uninformative tactile cues were presented prior to randomly intermingled auditory and visual targets requiring an elevation discrimination response (up vs. down). Responses were significantly faster for targets in both modalities when presented ipsilateral to the tactile cue. These findings demonstrate that the peripheral presentation of spatially uninforrnative auditory and visual cues produces cross-modal orienting that affects touch, and that tactile cues can also produce cross-modal covert orienting that affects audition and vision.     

Crossmodal change blindness between vision and touch

Change blindness is the name given to people's inability to detect changes introduced between two consecutively-presented scenes when they are separated by a distractor that masks the transients that are typically associated with change. Change blindness has been reported within vision, audition, and touch, but has never before been investigated when successive patterns are presented to different sensory modalities. In the study reported here, we investigated change detection performance when the two to-be-compared stimulus patterns were presented in the same sensory modality (i.e., both visual or both tactile) and when one stimulus pattern was tactile while the other was presented visually or vice versa. The two to-be-compared patterns were presented consecutively, separated by an empty interval, or else separated by a masked interval. In the latter case, the masked interval could either be tactile or visual. The first experiment investigated visual-tactile and tactile-visual change detection performance. The results showed that in the absence of masking, participants detected changes in position accurately, despite the fact that the two to-be-compared displays were presented in different sensory modalities. Furthermore, when a mask was presented between the two to-be-compared displays, crossmodal change blindness was elicited no matter whether the mask was visual or tactile. The results of two further experiments showed that performance was better overall in the unimodal (visual or tactile) conditions than in the crossmodal conditions. These results suggest that certain of the processes underlying change blindness are multisensory in nature. We discuss these findings in relation to recent claims regarding the crossmodal nature of spatial attention.     

特征和位置信息在价值驱动注意捕获中的作用

本研究采用训练−测试范式考察颜色和位置信息在价值驱动注意捕获中的作用。实验1考察是否存在基于具体位置的价值驱动的注意捕获效应。在训练阶段, 被试对8个位置中两个位置出现的红色目标反应伴随着高奖励反馈, 而对另外两个位置出现的红色目标反应伴随着低奖励反馈, 其它4个位置为中性位置, 没有目标出现。在测试阶段, 一半试次中红色刺激作为分心物出现。结果发现, 只有当分心刺激出现在高奖励位置和两个高奖励位置之间的中性位置时才能够捕获注意; 实验2考察颜色和位置信息在价值驱动注意捕获中的交互作用。在训练阶段, 将颜色特征和位置信息联合起来进行学习。在测试阶段, 和高、低奖励相联结的颜色刺激各在1/3试次中作为分心物出现。结果发现, 只有当高奖励颜色出现在高奖励位置或出现在高奖励位置之间的中性位置时才能够捕获注意。研究结果表明：(1)位置联结的价值驱动的注意捕获效应能够泛化到特定邻近位置上; (2)个体在训练阶段将颜色和位置的联合特征与奖励建立联结, 训练阶段建立的联结不能泛化到部分特征上。价值驱动注意捕获效应的泛化具有选择性。     

Cross-modal attention modulates tactile subitizing but not tactile numerosity estimation

Debate remains about whether the same attentional mechanism subserves subitizing (with number of items less than or equal to 4) and numerosity estimation (with number of items equal to or larger than 5), and evidence is scarce from the tactile modality. Here, we examined tactile numerosity perception. Using tactile Braille displays, participants completed the following three main tasks: (1) Unisensory task with focused attention: Participants reported the number (1~12) of the tactile pins. (2) Unisensory task with divided attention: Participants compared the numbers of pins across the upper and lower area of their left index fingers, in addition to reporting the number of tactile pins on their right index fingers. (3) Cross-modal task with divided attention: Participants reported the number of tactile pins and compared the numbers of visual dots across the upper and lower part of a (illusory) rectangle that overlaid the tactile stimuli. We found that performance of subitizing rather than estimation was interfered with in dual tasks, regardless of whether distractor events were from the same modality (tactile modality) or from a different modality (visual modality). Moreover, a further test of visual/tactile working memory capacity revealed that the precision of tactile subitizing, in the presence of a visual distractor, was correlated with the capacity of visual working memory, not of tactile working memory. Overall, our study revealed that tactile numerosity perception is accounted for by amodal attentional modulation yet by differential attentional mechanisms in terms of subitizing and estimation.     

Effects of posture on tactile localization by 4 years of age are modulated by sight of the hands: evidence for an early acquired external spatial frame of reference for touch

Adults show a deficit in their ability to localize tactile stimuli to their hands when their arms are in the less familiar, crossed posture. It is thought that this ‘crossed‐hands deficit’ arises due to a conflict between the anatomical and external spatial frames of reference within which touches can be encoded. The ability to localize a single tactile stimulus applied to one of the two hands across uncrossed‐hands and crossed‐hands postures was investigated in typically developing children (aged 4 to 6 years). The effect of posture was also compared across conditions in which children did, or did not, have visual information about current hand posture. All children, including the 4‐year‐olds, demonstrated the crossed‐hands deficit when they did not have sight of hand posture, suggesting that touch is located in an external reference frame by this age. In this youngest age group, when visual information about current hand posture was available, tactile localization performance was impaired specifically when the children's hands were uncrossed. We propose that this may be due to an early difficulty with integrating visual representations of the hand within the body schema.