Influence of vestibular signals on bodily self-consciousness: Different sensory weighting strategies based on visual dependency

Previous studies showed that the vestibular system is crucial for multisensory integration, however, its contribution to bodily self-consciousness more specifically on full-body illusions is not well understood. Thus, the current study examined the role of visuo-vestibular conflict on a full-body illusion (FBI) experiment that was induced during a supine body position. In a mixed design experiment, 56 participants underwent through a full-body illusion protocol. During the experiment, half of the participants received synchronous visuo-tactile stimulation, and the other half received asynchronous visuo-tactile stimulation, while their physical body was lying in a supine position, but the virtual body was standing. Additionally, the contribution of individual sensory weighting strategies was investigated via the Rod and Frame task (RFT), which was applied both before (pre-FBI standing and pre-FBI supine) and after the full-body illusion (post-FBI supine) protocol. Subjective reports of the participants confirmed previous findings suggesting that there was a significant increase in ownership over a virtual body during synchronous visuo-tactile stimulation. Additionally, further categorization of participants based on their visual dependency (by RFT) showed that those participants who rely more on visual information (visual field dependents) perceived the full-body illusion more strongly than non-visual field dependents during the synchronous visuo-tactile stimulation condition. Further analysis provided not only a quantitative demonstration of full-body illusion but also revealed changes in perceived self-orientation based on their field dependency. Altogether, findings of the current study make further contributions to our understanding of the vestibular system and brought new insight for individual sensory weighting strategies during a full-body illusion.     

Electrophysiological correlates of crossmodal visual distractor congruency effects: Evidence for response conflict

To investigate the basis of crossmodal visual distractor congruency effects, we recorded event-related brain potentials (ERP) while participants performed a tactile location-discrimination task. Participants made speeded tactile location-discrimination responses to tactile targets presented to the index fingers or thumbs while ignoring simultaneously presented task-irrelevant visual distractor stimuli at either the same (congruent) or a different (incongruent) location. Behavioural results were in line with previous studies, showing slowed response times and increased error rates on incongruent compared with congruent visual distractor trials. To clarify the effect of visual distractors on tactile processing, concurrently recorded ERPs were analyzed for poststimulus, preresponse, and postresponse modulations. An enhanced negativity was found in the time range of the N2 component on incongruent compared with congruent visual distractor trials prior to correct responses. In addition, postresponse ERPs showed the presence of error-related negativity components on incorrect-response trials and enhanced negativity for congruent-incorrect compared with incongruent-incorrect trials. This pattern of ERP results has previously been related to response conflict (Yeung, Botvinick, & Cohen, 2004). Importantly, no modulation of early somatosensory ERPs was present prior to the N2 time range, which may have suggested the contribution of other perceptual or postperceptual processes to crossmodal congruency effects. Taken together, our results suggest that crossmodal visual distractor effects are largely due to response conflict.     

橡胶手错觉:拥有感研究的实验范式及其应用

橡胶手错觉是一种健康个体将非肉体的假手视为自己真实身体的一部分的体验, 这种错觉可以通过同时轻刷被试面前可见的橡胶手及其不可见的真手而产生.橡胶手错觉已成为一种研究身体拥有感的重要范式, 其产生机制可以分为“自下而上的认知匹配”与“自上而下的认知匹配”两种加工方式.前者涉及视觉与触觉刺激的同步性; 而后者涉及被试心中预存的身体意象与身体图式(包括真假手之间模态特征,位置空间的相似性).综合上述证据, 身体模型假说与个人边缘空间理论进一步为拥有感产生的复杂机制提供了整合两种加工方式的解释.橡胶手错觉范式已经被用于探索精神分裂症患者等特殊被试病理分析,错觉产生和心理特质之间的关系, 以及神经外科和术后恢复上.未来的研究应该更加重视范式本身的拓展, 应用虚拟现实技术来提高身体模态的仿真效果, 利用橡胶手拥有感的易感性作为筛选与预测身体意象障碍疾病的指标.     

On the inability to ignore touch when responding to vision in the crossmodal congruency task

We investigated the extent to which people can selectively ignore distracting vibrotactile information when performing a visual task. In Experiment 1, participants made speeded elevation discrimination responses (up vs. down) to a series of visual targets presented from one of two eccentricities on either side of central fixation, while simultaneously trying to ignore task-irrelevant vibrotactile distractors presented independently to the finger (up) vs. thumb (down) of either hand. Participants responded significantly more slowly, and somewhat less accurately, when the elevation of the vibrotactile distractor was incongruent with that of the visual target than when they were presented from the same (i.e., congruent) elevation. This crossmodal congruency effect was significantly larger when the visual and tactile stimuli appeared on the same side of space than when they appeared on different sides, although the relative eccentricity of the two stimuli within the hemifield (i.e., same vs. different) had little effect on performance. In Experiment 2, participants who crossed their hands over the midline showed a very different pattern of crossmodal congruency effects to participants who adopted an uncrossed hands posture. Our results suggest that both the relative external location and the initial hemispheric projection of the target and distractor stimuli contribute jointly to determining the magnitude of the crossmodal congruency effect when participants have to respond to vision and ignore touch.     

Multisensory integration and age-dependent sensitivity to body representation modification induced by the rubber hand illusion

The aim of the present study was to compare junior and senior healthy participants for their multimodal integration capability. The instrument used for the investigation was the rubber hand illusion (RHI) where synchronous and asynchronous multimodal stimulation were applied. The study focused on assessing the rate of integration for visuo-tactile and visuo-proprioceptive stimuli. Methods: From a large sample of right-handed volunteers, 50 senior employees (containing younger and middle age adults) and 51 senior retired (senior adults) participants were recruited. Results: The between-subject analyses revealed that individuals both the junior and the senior participants responded to induction of RHI with vivid ownership and disownership experiences and a higher mislocalization error in the synchronous condition. However, the between-group analysis showed that participants in the senior group reported less vivid ownership and less vivid total RHI experiences scores compared to members of the junior group, but no mislocalization error differences were found between the groups. Conclusion: The results indicated that when visuo-tactile stimuli synchronously presented, the gain in multisensory integration decreased in seniors group. In contrast, in the case of visuo-proprioceptive synchronous presentation, the efficiency of multisensory integration remained unchanged across the lifespan.     

Beyond differences between the body schema and the body image: insights from body hallucinations

The distinction between the body schema and the body image has become the stock in trade of much recent work in cognitive neuroscience and philosophy. Yet little is known about the interactions between these two types of body representations. We need to account not only for their dissociations in rare cases, but also for their convergence most of the time. Indeed in our everyday life the body we perceive does not conflict with the body we act with. Are the body image and the body schema then somehow reshaping each other or are they relatively independent and do they only happen to be congruent? On the basis of the study of bodily hallucinations, we consider which model can best account for the body schema/body image interactions.     

Co-speech iconic gestures and visuo-spatial working memory

Three experiments tested the role of verbal versus visuo-spatial working memory in the comprehension of co-speech iconic gestures. In Experiment 1, participants viewed congruent discourse primes in which the speaker's gestures matched the information conveyed by his speech, and incongruent ones in which the semantic content of the speaker's gestures diverged from that in his speech. Discourse primes were followed by picture probes that participants judged as being either related or unrelated to the preceding clip. Performance on this picture probe classification task was faster and more accurate after congruent than incongruent discourse primes. The effect of discourse congruency on response times was linearly related to measures of visuo-spatial, but not verbal, working memory capacity, as participants with greater visuo-spatial WM capacity benefited more from congruent gestures. In Experiments 2 and 3, participants performed the same picture probe classification task under conditions of high and low loads on concurrent visuo-spatial (Experiment 2) and verbal (Experiment 3) memory tasks. Effects of discourse congruency and verbal WM load were additive, while effects of discourse congruency and visuo-spatial WM load were interactive. Results suggest that congruent co-speech gestures facilitate multi-modal language comprehension, and indicate an important role for visuo-spatial WM in these speech–gesture integration processes.     

Numerosity judgments for tactile stimuli distributed over the body surface

A large body of research now supports the claim that two different and dissociable processes are involved in making numerosity judgments regarding visual stimuli: subitising (fast and nearly errorless) for up to 4 stimuli, and counting (slow and error-prone) when more than 4 stimuli are presented. We studied tactile numerosity judgments for combinations of 1-7 vibrotactile stimuli presented simultaneously over the body surface. In experiment 1, the stimuli were presented once, while in experiment 2 conditions of single presentation and repeated presentation of the stimulus were compared. Neither experiment provided any evidence for a discontinuity in the slope of either the RT or error data suggesting that subitisation does not occur for tactile stimuli. By systematically varying the intensity of the vibrotactile stimuli in experiment 3, we were able to demonstrate that participants were not simply using the 'global intensity' of the whole tactile display to make their tactile numerosity judgments, but were, instead, using information concerning the number of tactors activated. The results of the three experiments reported here are discussed in relation to current theories of counting and subitising, and potential implications for the design of tactile user interfaces are highlighted.     

Active control as evidence in favor of sense of ownership in the moving Virtual Hand Illusion

The sense of ownership, the feeling that our body belongs to ourselves, relies on multiple sources of sensory information. Among these sources, the contribution of visuomotor information is still debated. We tested the effect of active control in the sense of ownership in the moving Virtual Hand Illusion. Participants reported sense of ownership and sense of agency over a virtual arm in which we manipulated the morphological congruence of the hand and the visuomotor information. We found that congruent active control enhanced and maintained the reported sense of ownership over a hand that appeared detached from the body, but not in a morphological congruent limb. Also, incongruent active control, achieved by adding noise to the trajectory of the movement, decreased both reported sense of agency and ownership. Overall, our results are consistent with a framework in which active control acts as evidence for eliciting a sense of ownership.     

The rubber hand illusion: sensitivity and reference frame for body ownership

When subjects view stimulation of a rubber hand while feeling congruent stimulation of their own hand, they may come to feel that the rubber hand is part of their own body. This illusion of body ownership is termed 'Rubber Hand Illusion' (RHI). We investigated sensitivity of RHI to spatial mismatches between visual and somatic experience. We compared the effects of spatial mismatch between the stimulation of the two hands, and equivalent mismatches between the postures of the two hands. We created the mismatch either by adjusting stimulation or posture of the subject's hand, or, in a separate group of subjects, by adjusting stimulation or posture of the rubber hand. The matching processes underlying body ownership were asymmetrical. The illusion survived small changes in the subject's hand posture, but disappeared when the same posture transformations were applied to the rubber hand. Mismatch between the stimulation delivered to the subject's hand and the rubber hand abolished the illusion. The combination of these two situations is of particular interest. When the subject's hand posture was slightly different from the rubber hand posture, the RHI remained as long as stimulation of the two hands was congruent in a hand-centred spatial reference frame, even though the altered posture of the subject's hand meant that stimulation was incongruent in external space. Conversely, the RHI was reduced when the stimulation was incongruent in hand-centred space but congruent in external space. We conclude that the visual-tactile correlation that causes the RHI is computed within a hand-centred frame of reference, which is updated with changes in body posture. Current sensory evidence about what is 'me' is interpreted with respect to a prior mental body representation.     

身体拥有感错觉对疼痛的影响及其作用机制

身体拥有感错觉以多通道的感官整合为基础,包括视觉-触觉、视觉-本体感觉、触觉-本体感觉、视觉-内感受的整合等。来自实验室急性痛以及慢性痛的大量结果表明拥有感错觉可产生镇痛作用。身体部位尺寸大小、肢体透明度等相关影响因素也逐渐得到揭示。拥有感错觉影响疼痛的机制与身体表征的改变有关,该过程可能涉及“身体网络”与“疼痛网络”间的连接,其中后顶叶皮层可能有重要作用。未来研究应更为深入地探索拥有感错觉范式以考察不同因素对疼痛的影响;在现有多感官整合的范式中重视内感受的作用;考察拥有感错觉改变疼痛的认知神经机制;区分急、慢性痛条件下以及不同慢性痛类型中拥有感错觉对疼痛的不同影响。     

Sad or fearful? The influence of body posture on adults’ and children’s perception of facial displays of emotion

The current research investigated the influence of body posture on adults' and children's perception of facial displays of emotion. In each of two experiments, participants categorized facial expressions that were presented on a body posture that was congruent (e.g., a sad face on a body posing sadness) or incongruent (e.g., a sad face on a body posing fear). Adults and 8-year-olds made more errors and had longer reaction times on incongruent trials than on congruent trials when judging sad versus fearful facial expressions, an effect that was larger in 8-year-olds. The congruency effect was reduced when faces and bodies were misaligned, providing some evidence for holistic processing. Neither adults nor 8-year-olds were affected by congruency when judging sad versus happy expressions. Evidence that congruency effects vary with age and with similarity of emotional expressions is consistent with dimensional theories and "emotional seed" models of emotion perception.     

Mislocalizations of touch to a fake hand

Observers can mislocalize a tactile target delivered to an unseen hand if a visible rubber glove is positioned next to a pair of distractor lights that flash in correlation with the tactile target (Pavani, Spence, & Driver, 2000). In the present study, we explored visual, tactile, and postural factors that influence this fake hand effect. Comparison with baseline conditions revealed that the fake hand effect was larger than a general spatial congruity effect but weaker than the effect obtained when tactile and visual stimuli were actually in the same locations (Experiment 1). Surprisingly, the effect did not depend on direct vision of the fake hand (Experiments 1 and 2), nor was it enhanced by congruent tactile information (Experiment 3). However, the fake hand effect was sensitive to the postural compatibility of the real and the fake hands (Experiment 4). These findings indicate that the available sensory information is used flexibly to incorporate the rubber glove into the body schema.     

Automatic auditory disambiguation of visual awareness

Multisensory integration can play a critical role in producing unified and reliable perceptual experience. When sensory information in one modality is degraded or ambiguous, information from other senses can crossmodally resolve perceptual ambiguities. Prior research suggests that auditory information can disambiguate the contents of visual awareness by facilitating perception of intermodally consistent stimuli. However, it is unclear whether these effects are truly due to crossmodal facilitation or are mediated by voluntary selective attention to audiovisually congruent stimuli. Here, we demonstrate that sounds can bias competition in binocular rivalry toward audiovisually congruent percepts, even when participants have no recognition of the congruency. When speech sounds were presented in synchrony with speech-like deformations of rivalling ellipses, ellipses with crossmodally congruent deformations were perceptually dominant over those with incongruent deformations. This effect was observed in participants who could not identify the crossmodal congruency in an open-ended interview (Experiment 1) or detect it in a simple 2AFC task (Experiment 2), suggesting that the effect was not due to voluntary selective attention or response bias. These results suggest that sound can automatically disambiguate the contents of visual awareness by facilitating perception of audiovisually congruent stimuli.     

Working memory capacity predicts conflict-task performance

The relationship between the ability to maintain task goals and working memory capacity (WMC) is firmly established, but evidence for WMC-related differences in conflict processing is mixed. We investigated whether WMC (measured using two complex-span tasks) mediates differences in adjustments of cognitive control in response to conflict. Participants performed a Simon task in which congruent and incongruent trials were equiprobable, but in which the proportion of congruency repetitions (congruent trials followed by congruent trials or incongruent trials followed by incongruent trials) and thus the need for trial-by-trial adjustments in cognitive control varied by block. The overall Simon effect did not depend on WMC capacity. However, for the low-WMC participants the Simon effect decreased as the proportion of congruency repetitions decreased, whereas for the high- and average-WMC participants it was relatively constant across conditions. Distribution analysis of the Simon effect showed more evidence for the inhibition of stimulus location in the low- than in the high-WMC participants, especially when the proportion of congruency repetitions was low. We hypothesize that low-WMC individuals exhibit more interference from task-irrelevant information due to weaker preparatory control prior to stimulus presentation and, thus, stronger reliance on reactive recruitment of cognitive control.     

Physical attractiveness stereotype and memory

Three experiments examined explicit and implicit memory for information that is congruent with the physical attractiveness stereotype (i.e. attractive-positive and unattractive-negative) and information that is incongruent with the physical attractiveness stereotype (i.e. attractive-negative and unattractive-positive). Measures of explicit recognition sensitivity and implicit discriminability revealed a memorial advantage for congruent compared to incongruent information, as evident from hit and false alarm rates and reaction times, respectively. Measures of explicit memory showed a recognition bias toward congruent compared to incongruent information, where participants tended to call congruent information old, independently of whether the information had been shown previously or not. This recognition bias was unrelated to reports of subjective confidence in retrieval. The present findings shed light on the cognitive mechanisms that might mediate discriminatory behavior towards physically attractive and physically unattractive individuals.     

Processing emotion information from both the face and body: An eye-movement study

This study examined the perception of emotional expressions, focusing on the face and the body. Photographs of four actors expressing happiness, sadness, anger, and fear were presented in congruent (e.g., happy face with happy body) and incongruent (e.g., happy face with fearful body) combinations. Participants selected an emotional label using a four-option categorisation task. Reaction times and accuracy for the categorisation judgement, and eye movements were the dependent variables. Two regions of interest were examined: face and body. Results showed better accuracy and faster reaction times for congruent images compared to incongruent images. Eye movements showed an interaction in which there were more fixations and longer dwell times to the face and fewer fixations and shorter dwell times to the body with incongruent images. Thus, conflicting information produced a marked effect on information processing in which participants focused to a greater extent on the face compared to the body.     

A sad thumbs up: incongruent gestures and disrupted sensorimotor activity both slow processing of facial expressions

Recognising a facial expression is more difficult when the expresser's body conveys incongruent affect. Existing research has documented such interference for universally recognisable bodily expressions. However, it remains unknown whether learned, conventional gestures can interfere with facial expression processing. Study 1 participants (N?=?62) viewed videos of people simultaneously producing facial expressions and hand gestures and reported the valence of either the face or hand. Responses were slower and less accurate when the face-hand pairing was incongruent compared to congruent. We hypothesised that hand gestures might exert an even stronger influence on facial expression processing when other routes to understanding the meaning of a facial expression, such as with sensorimotor simulation, are disrupted. Participants in Study 2 (N?=?127) completed the same task, but the facial mobility of some participants was restricted, which disrupted face processing in prior work. The hand-face congruency effect from Study 1 was replicated. The facial mobility manipulation affected males only, and it did not moderate the congruency effect. The present work suggests the affective meaning of conventional gestures is processed automatically and can interfere with face perception, but does not suggest that perceivers rely more on gestures when sensorimotor face processing is disrupted.     

The relation between body semantics and spatial body representations

The present study addressed the relation between body semantics (i.e. semantic knowledge about the human body) and spatial body representations, by presenting participants with word pairs, one below the other, referring to body parts. The spatial position of the word pairs could be congruent (e.g. EYE / MOUTH) or incongruent (MOUTH / EYE) with respect to the spatial position of the words’ referents. In addition, the spatial distance between the words’ referents was varied, resulting in word pairs referring to body parts that are close (e.g. EYE / MOUTH) or far in space (e.g. EYE / FOOT). A spatial congruency effect was observed when subjects made an iconicity judgment (Experiments 2 and 3) but not when making a semantic relatedness judgment (Experiment 1). In addition, when making a semantic relatedness judgment (Experiment 1) reaction times increased with increased distance between the body parts but when making an iconicity judgment (Experiments 2 and 3) reaction times decreased with increased distance. These findings suggest that the processing of body-semantics results in the activation of a detailed visuo-spatial body representation that is modulated by the specific task requirements. We discuss these new data with respect to theories of embodied cognition and body semantics.     

Spatial constraints on visual-tactile cross-modal distractor congruency effects

Across three experiments, participants made speeded elevation discrimination responses to vibrotactile targets presented to the thumb (held in a lower position) or the index finger (upper position) of either hand, while simultaneously trying to ignore visual distractors presented independently from either the same or a different elevation. Performance on the vibrotactile elevation discrimination task was slower and less accurate when the visual distractor was incongruent with the elevation of the vibrotactile target (e.g., a lower light during the presentation of an upper vibrotactile target to the index finger) than when they were congruent, showing that people cannot completely ignore vision when selectively attending to vibrotactile information. We investigated the attentional, temporal, and spatial modulation of these cross-modal congruency effects by manipulating the direction of endogenous tactile spatial attention, the stimulus onset asynchrony between target and distractor, and the spatial separation between the vibrotactile target, any visual distractors, and the participant’s two hands within and across hemifields. Our results provide new insights into the spatiotemporal modulation of crossmodal congruency effects and highlight the utility of this paradigm for investigating the contributions of visual, tactile, and proprioceptive inputs to the multisensory representation of peripersonal space.