情绪和时间辨别任务中视听整合与预先准备效应

本研究分别在时间和情绪认知维度上考察预先准备效应对情绪视听整合的影响。时间辨别任务(实验1)发现视觉引导显著慢于听觉引导,并且整合效应量为负值。情绪辨别任务(实验2)发现整合效应量为正值;在负性情绪整合中,听觉引导显著大于视觉引导;在正性情绪整合中,视觉引导显著大于听觉引导。研究表明,情绪视听整合基于情绪认知加工,而时间辨别会抑制整合;此外,跨通道预先准备效应和情绪预先准备效应都与引导通道有关。

The audio-visual integration and pre-preparation effects in emotion and timing discrimination tasks

Emotional audio-visual integration is an effective way to transmit and decode emotional information, which is of great significance to human survival and evolution. In multisensory integration, it is generally understood that one sensory signal promotes or alters the perception of other sensory signals. Studies have shown that the cross-sensory pre-preparation effect between visual and auditory is a key influencing factor of emotional audio-visual integration. However, it has not received much attention in the pre-preparation effect, particularly with regard to the emotional dimension. And previous research did not distinguish the emotional pre-preparation effect from the cross-sensory pre-preparation effect. Thus the present two studies started from the perspective of these two pre-preparation effects to gain insight into the emotional audio-visual integration processing mechanism. Experiment 1 adopted the time order judgment and simultaneous judgment tasks. Participants were asked to determine the chronological sequence of the visual and auditory stimuli, or whether they were presented at the same time. Experiment 2 adopted the emotion discrimination tasks. Participants were required to determine the emotional valence of visual, auditory, or audio-visual stimuli. The purpose of these two studies was to examine the influence of pre-preparation effects on emotional audio-visual integration either in the time or emotional cognitive dimensions. In Experiment 1, the response time of the visual-leading condition was significantly slower than the auditory-leading condition, and the accuracy was significantly lower than the auditory-leading condition. However, the amount of relative multisensory response enhancement （rMRE） for each condition of audiovisual integration was negative. It meant that audio-visual integration did not have many advantages compared with the single visual or auditory sensory. In Experiment 2, the rMRE of the negative emotion in auditory-leading condition was significantly greater than that in the visual-leading condition; the rMRE of the positive emotion in visual-leading condition was significantly greater than auditory-leading condition; however, the difference between the visual- and the auditory-leading condition on the rMRE of the neutral emotion was not significant. These research results showed that in the temporal dimension, the mechanism of the audio-visual integration was manifested as the feature matching processing cross the visual and auditory, and a better pre-preparation effect for the presentation time was in auditory than visual sensory. In the emotional cognitive dimension, the integration of the emotional information among the visual and auditory sensory had occurred with the positive rMRE amount, and the better emotional preparation effect had been discovered in the visual sensory than the auditory. Moreover, auditory channels played a greater role when integrating negative emotions, while visual channels dominate when integrating positive emotions. In summary, present experimental evidence showed that the enhancement of the emotional audiovisual integration was based on the cognitive processing of emotional information, and the temporal discrimination inhibited the emotional audiovisual integration. Either the pre-preparation effect of cross-sensory matching or emotional integration promoted the processing of visual and auditory information respectively and depended on the different neural circuits under visual- or auditory-leading independently. At last, the advantage of audio-visual integration showed that the cross-sensory and emotional pre-preparation effect was mediated by the tasks, demonstrating a separation between them.