多感觉整合中的声音诱发闪光错觉效应

声音诱发闪光错觉效应是典型的视听整合错觉现象, 是指当视觉闪光刺激与间隔100 ms内的听觉声音刺激不等数量呈现时, 被试知觉视觉闪光的数量与听觉声音的数量相等。声音诱发闪光错觉的影响因素既包括自下而上和自上而下的被试内差异因素, 也包括视听刺激依赖程度、视听整合的发展和视听刺激知觉敏感性等被试间差异因素。该效应的产生在时程上主要体现在早期加工阶段, 在脑区上主要涉及多处皮层及皮层下相关脑区。未来研究应进一步考察注意、奖赏和视听整合方式等认知加工对声音诱发闪光错觉的影响, 同时也应该关注声音诱发闪光错觉对记忆和学习的影响以及结合计算模型和神经科学的手段进一步探讨其认知神经机制。     

Children can optimally integrate multisensory information after a short action‐like mini game training

Combining information across different sensory modalities is of critical importance for the animal's survival and a core feature of human's everyday life. In adulthood, sensory information is often integrated in a statistically optimal fashion, so that the combined estimates of two or more senses are more reliable than the best single one. Several studies have shown that young children use one sense to calibrate the others, which results in unisensory dominance and undermines their optimal multisensory integration abilities. In this study we trained children aged 4–5 years with action‐like mini games, to determine whether it could improve their multisensory as well as their visuo‐spatial skills. Multisensory integration abilities were assessed using a visuo‐haptic size discrimination task, while visuo‐spatial attention skills were investigated using a multiple object tracking task (MOT). We found that 2‐weeks training were sufficient to observe both optimal multisensory integration and visuo‐spatial enhancements selectively in the group trained with action‐like mini games. This plastic change persisted up to 3 months, as assessed in a follow‐up. Our novel findings reveal that abilities that are commonly known to emerge in late childhood can be promoted in younger children through action‐like mini games and have long‐lasting effects. Our data have clinical implications, in that they suggest that specific trainings could potentially help children with multisensory integration deficits.     

外源性注意与多感觉整合的交互关系

外源性注意与多感觉整合的交互关系是一个复杂且具有争议的研究领域, 一直以来备受研究者们关注。为了解释两者间的交互作用机制, 本文基于已有研究成果从两方面综述了外源性注意与多感觉整合的交互关系：(1)外源性注意可以通过自下而上的方式调节多感觉整合, 包括空间不确定性、感知觉敏感度和感觉通道间信号强度差异三种理论假说; (2)多感觉整合可以调节外源性注意。一方面, 来自多感觉通道的刺激能够以自下而上的方式自动整合, 整合后的多感觉通道刺激比单通道刺激具有更大的凸显性从而有效地吸引注意。另一方面, 整合后的多感觉通道刺激能够作为多感觉信号模板存储于大脑之中, 从而在任务中实现自上而下地调节注意捕获。     

多感觉整合范式中潜在的跨通道转换效应

大脑可以对来自不同感觉通道的信息进行处理与整合。与单一感觉通道相比, 个体对同时呈现在不同感觉通道的目标信号的响应会更快。对于这种现象的一种主要理论解释是共同激活模型, 该模型认为来自不同通道的刺激在特定的脑区汇聚整合, 比如顶叶内沟、颞上沟和前额叶皮层区域。整合后的信号强度更大, 可以更快地触发反应, 但是信号的整合发生在认知加工的哪一阶段目前尚未有明确结论。当个体对出现在不同感觉通道之间的任务转换进行加工时, 产生与感觉通道相关的任务转换的损失小于跨感觉通道转换损失与任务转换损失的总和, 这为与感觉通道相关的转换代价来源于任务设置的惯性和干扰提供了证据。而在单通道和多通道之间发生转换时, 跨通道转换代价会减小甚至消失, 这是由于同时发生的多感觉整合抵消了一部分损失, 这种现象支持了共同激活模型理论。然而, 多感觉信号整合对任务转换的神经加工过程产生怎样的影响并不清楚, 在未来的研究中可以把多感觉整合范式同经典的任务转换范式结合改进, 进而确定跨通道转换的加工机制和多感觉信号整合的发生阶段。     

集中和分散注意对多感觉整合中听觉主导效应的影响

听觉主导效应是指多感觉通道信息整合过程中,听觉通道中的信息得到优先加工,从而主导其他感觉通道的信息。研究采用经典的声音诱发闪光错觉的范式,通过两个实验操纵了注意资源的分配方式以及实验任务难度,考察了主动注意听觉通道的声音刺激对声音诱发闪光错觉产生的影响,以及任务难度对声音诱发闪光错觉的影响。结果发现:(1)裂变错觉会受到注意资源分配程度的影响,但是融合错觉则不然;(2)任务难度既不会影响裂变错觉,也不会影响融合错觉。说明了分散注意能够影响听觉主导效应中的裂变错觉,并且这种主导效应与任务难度无关。     

我是谁?——人际间多感觉刺激下的识脸错觉

识脸错觉是在人际间同步多感觉刺激下将他人面孔感知为自我面孔的一种主观体验。继Tsakiris报告了识脸错觉现象之后,研究者通过选取不同的刺激呈现方式、不同的被试,重点对被试的主观体验、行为反应两个方面进行了考察,并得到了大量新的研究成果;发现年龄、性别及内部敏感性是影响识脸错觉强度的重要因素,右侧颞顶联合区、顶内沟和枕下回的神经活动与被试主观报告的错觉体验强度相关。未来识脸错觉研究应侧重于研究策略的多样化并为生物特征识别中新模态的选择提供理论支持。此外,同步多感觉刺激技术的开发与应用将会对自我新面孔认同训练具有重要意义。     

Late‐ but not early‐onset blindness impairs the development of audio‐haptic multisensory integration

Integrating different senses to reduce sensory uncertainty and increase perceptual precision can have an important compensatory function for individuals with visual impairment and blindness. However, how visual impairment and blindness impact the development of optimal multisensory integration in the remaining senses is currently unknown. Here we first examined how audio‐haptic integration develops and changes across the life span in 92 sighted (blindfolded) individuals between 7 and 70 years of age. We used a child‐friendly task in which participants had to discriminate different object sizes by touching them and/or listening to them. We assessed whether audio‐haptic performance resulted in a reduction of perceptual uncertainty compared to auditory‐only and haptic‐only performance as predicted by maximum‐likelihood estimation model. We then compared how this ability develops in 28 children and adults with different levels of visual experience, focussing on low‐vision individuals and blind individuals that lost their sight at different ages during development. Our results show that in sighted individuals, adult‐like audio‐haptic integration develops around 13–15 years of age, and remains stable until late adulthood. While early‐blind individuals, even at the youngest ages, integrate audio‐haptic information in an optimal fashion, late‐blind individuals do not. Optimal integration in low‐vision individuals follows a similar developmental trajectory as that of sighted individuals. These findings demonstrate that visual experience is not necessary for optimal audio‐haptic integration to emerge, but that consistency of sensory information across development is key for the functional outcome of optimal multisensory integration.     

内外源性空间注意对多感觉整合的影响

本文采用内-外源性空间线索靶子范式, 操控内源性线索有效性(有效线索、无效线索)、外源性线索有效性(有效线索、无效线索)、目标刺激类型(视觉刺激、听觉刺激、视听觉刺激)三个自变量。通过两个不同任务难度的实验(实验1: 简单定位任务; 实验2: 复杂辨别任务)来考察内外源性空间注意对多感觉整合的影响。两个实验结果均发现外源性空间注意显著减弱了多感觉整合效应, 内源性空间注意没有显著增强多感觉整合效应; 实验2中还发现了内源性空间注意会对外源性空间注意减弱多感觉整合效应产生影响。结果表明, 与内源性空间注意不同, 外源性空间注意对多感觉整合的影响不易受任务难度的调控; 当任务较难时内源性空间注意会影响外源性空间注意减弱多感觉整合效应的过程。由此推测, 内外源性空间注意对多感觉整合的调节并非彼此独立、而是相互影响的。     

Dimensional interactions in vibrotactile information processing

The effect of irrelevant dimensional variation on the processing of vibrotactile stimuli was measured. Six observers performed a speeded classification task with stimuli varying along the dimensions of pitch and loudness. Choice reaction times were obtained for stimuli differing on one dimension alone, on two correlated dimensions, or on two orthogonally varied dimensions. Compared to one-dimension performance, reaction times were faster in the correlated condition and slower in the orthogonal condition. In general, these findings agreed with similar experiments in other modalities, with the exception that the effects in this study tended to be stronger for cases in which loudness, rather than pitch, was the relevant dimension. The results are explained in terms of the integrality of pitch and loudness and of the relative discriminability of dimensions.     

情绪信息的多通道整合

自然环境中人类情绪信息的交流是依靠多个感觉通道实现的,多通道整合是情绪加工的基础。近年来的行为学、电生理学与神经成像的研究表明,情绪信息的加工具有跨通道自动整合的特点,它发生在认知加工的早期阶段,与颞上回、颞中回、海马旁回和丘脑等脑区密切相关。不同情绪的整合既有共同的神经基础,又有各自独特的加工区域。情绪信息的整合机制还可能与加工类型及注意资源有关。在未来研究中,实验的标准化、动态化、自然化有助于提高研究的准确性和研究间的可比性,而对特殊群体的研究,以及综合研究情绪加工与注意等其他认知过程则有助于我们进一步探索多通道整合的神经机制。     

Perspectives on multisensory experience and cognitive development in infants with cochlear implants

Infants learn about their environment through sensory exploration, acquiring knowledge that is important for cognitive development. However, little is known about the sensory exploration of infants with profound hearing loss before or after they receive cochlear implants. This paper reviews aspects of sensory perception and cognitive development in hearing infants, discusses the implications of delayed access to auditory information for multisensory perception and cognitive development in infants who use cochlear implants, and suggests several new directions for research addressing multisensory exploration and cognitive development in infants with cochlear implants.     

The effects of aging on the working memory processes of multimodal information

Normal aging is associated with deficits in working memory processes. However, the majority of research has focused on storage or inhibitory processes using unimodal paradigms, without addressing their relationships using different sensory modalities. Hence, we pursued two objectives. First, was to examine the effects of aging on storage and inhibitory processes. Second, was to evaluate aging effects on multisensory integration of visual and auditory stimuli. To this end, young and older participants performed a multimodal task for visual and auditory pairs of stimuli with increasing memory load at encoding and interference during retention. Our results showed an age-related increased vulnerability to interrupting and distracting interference reflecting inhibitory deficits related to the off-line reactivation and on-line suppression of relevant and irrelevant information, respectively. Storage capacity was impaired with increasing task demands in both age groups. Additionally, older adults showed a deficit in multisensory integration, with poorer performance for new visual compared to new auditory information.     

Serial order processing in human infants and the role of multisensory redundancy

Perception, production, and understanding of sequences is fundamental to human behavior and depends, in large part, on the ability to detect serial order. Despite the importance of this issue across many domains of human functioning, the development of serial order skills has been neglected in developmental studies. The current article reviews evidence that the basic temporal and spatiotemporal skills that are necessary for the development of serial order skills emerge early in human development. The article then presents recent evidence from the authors laboratory showing that serial order perceptual skills emerge at the same time and improve rapidly. Consistent with a multisensory redundancy view of perception, when serial order perceptual abilities first emerge in infancy, they depend critically on the redundant specification of sequences in both the auditory and visual modalities. The findings suggest that infants ability to perceive the surface serial order characteristics of sequentially organized events provides the necessary antecedents to the development of more complex serial order skills that ultimately enable us to extract meanings from sequentially organized events and perform complex sequential actions.Edited by: Marie-Hélène Giard and Mark Wallace     

Capacity limitations in human information processing

The nature of processing demands during a letter-match task was investigated in an extension of the Posner and Boies (1971) paradigm. In Experiment I, a visual probe was employed in addition to an auditory probe in two different experimental conditions. The shape of the auditory probe reaction time (RT) function was similar to that found by Posner and Boies. However, in contrast to their findings, RT was greatly increased shortly after presentation of the first letter for the visual probe function. It was concluded that perceptual as well as postperceptual limitations on processing capacity exist. A second experiment provided further support for this hypothesis.     

通道估计可靠性先验知识在早期的知觉加工阶段影响多感觉信息整合

统计最优化理论认为, 在多感觉信息整合的过程中, 大脑是以加权平均的方式将多个感觉通道的信息整合为统一的感觉信息, 通道信息的利用权重根据各通道信息的可靠性来确定。近期的几个行为研究结果则显示, 有关通道估计可靠性的先验知识同样能够影响通道信息在整合过程中的利用权重。然而, 这些研究结果不能确定先验知识对多感觉信息整合的影响是发生在认知加工的感知觉阶段还是决策阶段。当前研究致力于对此问题进行探讨。实验通过赋予两种颜色字母不同的视、听一致概率(高概率、低概率), 测量并分析被试在各概率水平下对视听一致刺激的反应时。实验数据显示不同的视听一致概率能够调制对视听一致刺激的反应时。该结果揭示通道估计可靠性先验知识在早期的知觉加工阶段影响多感觉信息整合。