Contextual influences of emotional speech prosody on face processing: How much is enough?

The influence of emotional prosody on the evaluation of emotional facial expressions was investigated in an event-related brain potential (ERP) study using a priming paradigm, the facial affective decision task. Emotional prosodic fragments of short (200-msec) and medium (400-msec) duration were presented as primes, followed by an emotionally related or unrelated facial expression (or facial grimace, which does not resemble an emotion). Participants judged whether or not the facial expression represented an emotion. ERP results revealed an N400-like differentiation for emotionally related prime-target pairs when compared with unrelated prime-target pairs. Faces preceded by prosodic primes of medium length led to a normal priming effect (larger negativity for unrelated than for related prime-target pairs), but the reverse ERP pattern (larger negativity for related than for unrelated prime-target pairs) was observed for faces preceded by short prosodic primes. These results demonstrate that brief exposure to prosodic cues can establish a meaningful emotional context that influences related facial processing; however, this context does not always lead to a processing advantage when prosodic information is very short in duration.     

新生儿情绪性语音加工的正性偏向——来自事件相关电位的证据

准确解码语音中的情绪信息能让个体更好地适应社会环境, 此能力对新生儿和婴儿尤其重要, 因为人类刚出生时听觉系统远比视觉系统发育得完善。虽然已有研究表明5~7月龄的婴儿能分辨不同情绪种类的语音, 但目前对新生儿的研究还非常少。人类是否在出生时即具有分辨不同种类情绪性语音的能力?新生儿对情绪的加工是否存在正性或负性偏向?本文选用odd-ball范式考察高兴、恐惧、愤怒三种韵律性语音在1~6天龄新生儿大脑中诱发的事件相关电位。实验1直接对比三种情绪性条件, 发现新生儿大脑的额区(F3和F4电极点)可以区分情绪性语音的正负性, 正性(高兴)语音诱发的“失匹配反应”幅度明显大于负性(愤怒和恐惧)语音。实验2采用偏差和标准刺激反转的odd-ball范式, 证实了实验1的结果并非源于三种情绪语音物理属性的差异。本文的结果提示, 新生儿大脑可自动辨别正性与负性情绪语音, 但尚不能将愤怒和恐惧两种负性语音区分开来。更重要的是, 高兴语音比两种负性语音诱发了更大的失匹配反应, 这一结果首次从神经学层面(电生理指标)为新生儿情绪性语音加工的正性偏向提供了证据。     

语境情绪韵律对新异词汇学习过程的影响

模拟自然学习语境,探讨情绪韵律对具体和抽象词学习的调节作用及其产生原因。被试在高兴、愤怒、中性语境情绪韵律条件下学习抽象和具体词,记录学习过程及测试过程中的行为与脑电反应。结果发现在愤怒韵律背景下学习的抽象词加工的正确率更低、反应时更长;并且诱发了更为显著的脑电成分。表明愤怒韵律对词汇学习尤其是抽象词学习有显著的消极作用。此外,情绪韵律对词汇学习的调节作用是在学习过程中对词汇语义提取及晚期语义整合产生影响而形成的。     

Crossmodal integration of emotional information from face and voice in the infant brain

We examined 7-month-old infants' processing of emotionally congruent and incongruent face-voice pairs using ERP measures. Infants watched facial expressions (happy or angry) and, after a delay of 400 ms, heard a word spoken with a prosody that was either emotionally congruent or incongruent with the face being presented. The ERP data revealed that the amplitude of a negative component and a subsequent positive component in infants' ERPs varied as a function of crossmodal emotional congruity. An emotionally incongruent prosody elicited a larger negative component in infants' ERPs than did an emotionally congruent prosody. Conversely, the amplitude of infants' positive component was larger to emotionally congruent than to incongruent prosody. Previous work has shown that an attenuation of the negative component and an enhancement of the later positive component in infants' ERPs reflect the recognition of an item. Thus, the current findings suggest that 7-month-olds integrate emotional information across modalities and recognize common affect in the face and voice.     

语言和音乐中短语边界的认知加工

综述了近年来语言和音乐中有关短语边界加工的一些研究成果。首先,语言中使用事件相关电位技术（ERP）进行的众多研究表明,口语中的语调短语边界和阅读中的逗号都能诱发中止正漂移（CPS）这种反映语调短语中止的ERP成分。然后介绍了近期关于音乐短语结构的一系列ERP和事件相关磁场（ERF）研究,结果发现在各种实验条件下音乐CPS都能够稳定产生。最后,对今后汉语韵律层级结构边界的认知加工研究做出展望     

An electrophysiological investigation into the automaticity of emotional face processing in high versus low trait anxious individuals

To examine the extent of automaticity of emotional face processing in high versus low trait anxious participants, event-related potentials (ERPs) were recorded to emotional (fearful, happy) and neutral faces under varying task demands (low load, high load). Results showed that perceptual encoding of emotional faces, as reflected in P1 and early posterior negativity components, was unaffected by the availability of processing resources. In contrast, the postperceptual registration and storage of emotion-related information, as reflected in the late positive potential component at frontal locations, was influenced by the availability of processing resources, and this effect was further modulated by level of trait anxiety. Specifically, frontal ERP augmentations to emotional faces were eliminated in the more demanding task for low trait anxious participants, whereas ERP enhancements to emotional faces were unaffected by task load in high trait anxious participants. This result suggests greater automaticity in processing affective information in high trait anxious participants.     

Understanding speaker attitudes from prosody by adults with Parkinson's disease

The ability to interpret vocal (prosodic) cues during social interactions can be disrupted by Parkinson's disease, with notable effects on how emotions are understood from speech. This study investigated whether PD patients who have emotional prosody deficits exhibit further difficulties decoding the attitude of a speaker from prosody. Vocally inflected but semantically nonsensical ‘pseudo‐utterances’ were presented to listener groups with and without PD in two separate rating tasks. Task 1 required participants to rate how confident a speaker sounded from their voice and Task 2 required listeners to rate how polite the speaker sounded for a comparable set of pseudo‐utterances. The results showed that PD patients were significantly less able than HC participants to use prosodic cues to differentiate intended levels of speaker confidence in speech, although the patients could accurately detect the polite/impolite attitude of the speaker from prosody in most cases. Our data suggest that many PD patients fail to use vocal cues to effectively infer a speaker's emotions as well as certain attitudes in speech such as confidence, consistent with the idea that the basal ganglia play a role in the meaningful processing of prosodic sequences in spoken language ( Pell & Leonard, 2003 ).     

An event-related brain potential analysis of visual word priming effects

Two experiments are reported that provide evidence on task-induced effects during visual lexical processing in a prime-target semantic priming paradigm. The research focuses on target expectancy effects by manipulating the proportion of semantically related and unrelated word pairs. In Experiment 1, a lexical decision task was used and reaction times (RTs) and event-related brain potentials (ERPs) were obtained. In Experiment 2, subjects silently read the stimuli, without any additional task demands, and ERPs were recorded. The RT and ERP results of Experiment 1 demonstrate that an expectancy mechanism contributed to the priming effect when a high proportion of related word pairs was presented. The ERP results of Experiment 2 show that in the absence of extraneous task requirements, an expectancy mechanism is not active. However, a standard ERP semantic priming effect was obtained in Experiment 2. The combined results show that priming effects due to relatedness proportion are induced by task demands and are not a standard aspect of online lexical processing.     

外显和内隐情绪韵律加工的脑机制：近红外成像研究

准确识别言语中的情绪韵律信息对社会交往非常重要。本研究采用功能近红外成像技术, 探索外显和内隐情绪加工条件下愤怒、恐惧、快乐三种情绪韵律加工过程中的大脑皮层神经活动。结果表明, 对愤怒、恐惧、快乐韵律进行特异性加工的脑区分别为左侧额极/眶额叶、右侧缘上回、左侧额下回, 其中右侧缘上回脑区同时受到情绪和任务的调控。此外, 右侧颞中回、颞下回和颞极在情绪外显任务中的激活明显强于内隐任务。本研究的结果部分支持了情绪韵律的层次模型, 也对该模型的第三层次, 即“额区对语音情绪信息的精细加工需要外显性情绪加工任务参与”提出了质疑。     

Examining the evidence for an independent semantic analyzer: an ERP study in Spanish

Recent ERP findings challenge the widespread assumption that syntactic and semantic processes are tightly coupled. Syntactically well-formed sentences that are semantically anomalous due to thematic mismatches elicit a P600, the component standardly associated with syntactic anomaly. This ‘thematic P600’ effect has been attributed to detection of semantically plausible thematic relations that conflict with the surface syntactic structure of the sentence, implying a processing architecture with an independent semantic analyzer. A key finding is that the P600 is selectively sensitive to the presence of plausible verb-argument relations, and that otherwise an N400 is elicited (The hearty meal was devouring … vs. The dusty tabletop was devouring …: Kim & Osterhout, 2005). The current study investigates in Spanish whether the evidence for an independent semantic analyzer is better explained by a standard architecture that rapidly integrates multiple sources of lexical, syntactic, and semantic information. The study manipulated the presence of plausible thematic relations, and varied the choice of auxiliary between passive-biased fue and active-progressive biased estaba. Results show a late positivity that appeared as soon as comprehenders detected an improbable combination of subject animacy, auxiliary bias, or verb voice morphology. This effect appeared at the lexical verb in the fue conditions and at the auxiliary in the estaba conditions. The late positivity elicited by surface thematic anomalies was the same, regardless of the presence of a plausible non-surface interpretation, and no N400 effects were elicited. These findings do not implicate an independent semantic analyzer, and are compatible with standard language processing architectures.     

Stem allomorphy in the Spanish mental lexicon: evidence from behavioral and ERP experiments

This study presents results from a nonce-word elicited production task and a reading experiment using event-related brain potentials (ERPs) investigating finite forms of Spanish verbs which consist of marked stems and regular person and number agreement suffixes. The first experiment showed that unmarked stems are productively extended to nonce words, whereas marked stems generalize more restrictively to nonce words, based on lexical similarity to existing stem forms. The second experiment yielded a lexical ERP signature for stem violations and an ERP pattern signaling morpho-syntactic (rule-based) processing for suffix violations. We argue that stem allomorphy is lexically represented in the Spanish mental lexicon, with marked stems forming subnodes of structured lexical entries.     

Lexical integration: Sequential effects of syntactic and semantic information

Both semantic and syntactic context constraints can influence word processing at the level of lexical integration. In event-related brain potentials (ERPs), semantic integration is reflected by a negativity around 400 msec (N400), whereas phrase structure assignment and syntactic integration are assumed to be reflected by an early left anterior negativity and a late positivity (P600), respectively. An ERP study is presented in which participants read different types of sentences whose terminal verb was either congruent with the preceding context or incongruent due to a phrase structure violation, a semantic violation, or both. The main finding was that only the pure semantic violation condition, but not the combined semantic and syntactic violation condition, elicited a large N400. The two conditions containing phrase structure violations were predominantly characterized by a P600. Both semantic violation conditions, moreover, displayed a late negativity around 700 msec that overlapped with the P600 in the double violation condition. The absence of an N400 effect for elements that are syntactically as well as semantically incongruent with prior context suggests an early influence of phrase structure information on processes of lexical-semantic integration. The present data are discussed in comparison to previous ERP findings, and a new view of lexical integration processes is proposed.     

Interplay of emotional valence and concreteness in word processing: An event-related potential study with verbs

The functional locus of emotional valence in word processing remains an open question. In event-related potentials, emotion has been found to elicit an early posterior negativity (EPN), which is assumed to reflect attention catching by the words’ meaning. Previously, the EPN was modulated by word category with verbs exhibiting longer EPN latencies compared with other word categories. Here we examined whether concreteness, a semantic variable, influences emotion processing. Within a lexical decision task for verbs, emotional valence (positive, negative, and neutral) was orthogonally combined with concreteness (concrete and abstract). EPN onset was found already at 250 ms post-stimulus for concrete verbs, whereas it started 50 ms later for abstract verbs. Concreteness effects occurred after the start of main effects of emotion. Thus, the elicitation of the EPN seems to be based on semantic processes, with emotional valence being accessed before other semantic aspects such as concreteness of verbs.     

Brain mechanisms involved in angry prosody change detection in school-age children and adults,revealed by electrophysiology

Voices transmit social signals through speech and/or prosody. Emotional prosody conveys key information about the emotional state of a speaker and is thus a crucial cue that one has to detect in order to develop efficient social communication. Previous studies in adults reported different brain responses to emotional than to neutral prosodic deviancy. The aim of this study was to characterize such specific emotional deviancy effects in school-age children. The mismatch negativity (MMN) and P3a evoked potentials, reflecting automatic change detection and automatic attention orienting, respectively, were obtained for neutral and emotional angry deviants in both school-age children (n = 26) and adults (n = 14). Shorter latencies were found for emotional than for neutral preattentional responses in both groups. However, whereas this effect was observed on the MMN in adults, it appeared in an early discriminative negativity preceding the MMN in children. A smaller P3a amplitude was observed for the emotional than for the neutral deviants at all ages. Overall, the brain responses involved in specific emotional change processing are already present during childhood, but responses have not yet reached an adult pattern. We suggest that these processing differences might contribute to the known improvement of emotional prosody perception between childhood and adulthood.     

The role of spatial attention in the processing of facial expression: An ERP study of rapid brain responses to six basic emotions

To investigate the time course of emotional expression processing, we recorded ERP responses to stimulus arrays containing neutral versus angry, disgusted, fearful, happy, sad, or surprised faces. In one half of the experiment, the task was to discriminate emotional and neutral facial expressions. Here, an enhanced early frontocentral positivity was elicited in response to emotional as opposed to neutral faces, followed by a broadly distributed positivity and an enhanced negativity at lateral posterior sites. These emotional expression effects were very similar for all six basic emotional expressions. In the other half of the experiment, attention was directed away from the faces toward a demanding perceptual discrimination task. Under these conditions, emotional expression effects were completely eliminated, demonstrating that brain processes involved in the detection and analysis of facial expression require focal attention. The face-specific N170 component was unaffected by any emotional expression, supporting the hypothesis that structural encoding and expression analysis are independent processes.     

情绪词加工过程及其情绪效应特点：ERP的证据

语言是传递情绪信息的重要媒介, 探讨情绪语言的加工过程一直是情绪研究的重点。文章聚焦于情绪词加工的ERP研究, 介绍了情绪词加工中的ERP成分、效价与唤醒度的作用关系, 以及情绪词情绪效应的特点。情绪词加工的ERP成分包括非常早期的P1-N1成分, 早期的EPN成分和晚期的LPP成分; 效价和唤醒度均在情绪词加工中具有重要作用, 但两者的关系还未得到一致结论。相比情绪图片, 情绪词加工所引发的情绪效应更弱, 并且存在积极效价偏向。今后的研究还需进一步探讨影响情绪词情绪效应产生的相关因素; 进一步区分情绪标注词和情绪负载词, 探讨两类情绪词加工的异同。此外, 结合眼动追踪技术探讨自然语境中的情绪词加工机制也是未来研究的一个重要方向。     

情绪面孔的意识神经相关物及其无意识自动加工：来自事件相关电位的证据

意识的神经相关物尚有争议, 且个体能否无意识自动检测视觉环境变化尚不清楚。本研究采用非注意视盲范式操控视觉意识, 并引入具有社会信息的情绪面孔, 探讨意识的神经相关物以及视觉意识与自动检测变化机制的关系。在A阶段, 部分被试对任务无关的情绪面孔处于无意识水平; 在B阶段, 所有被试对任务无关的情绪面孔处于意识水平; 在C阶段, 所有被试对任务相关的情绪面孔处于意识水平。结果显示, 任务无关的情绪面孔的意识过程诱发视觉意识负波(visual awareness negativity, VAN)、晚期正成分(late positivity, LP)和晚期枕区正成分(late occipital positivity, LOP)。此外, 无意识的情绪面孔能诱发视觉失匹配负波(visual mismatch negativity, vMMN), 且其幅值不受意识影响, 但是受任务相关性调制。这些结果提示对情绪面孔的视觉意识在不同的时间进程上有不同的ERP指标——VAN反映早期知觉经验, 而LP和LOP反映晚期意识过程, 而且面孔情绪信息的自动加工独立于视觉意识, 但是受视觉注意调制。     

Cross-modal integration of emotional face and voice in congruous and incongruous pairs: The P2 ERP effect

The study investigates cross-modal simultaneous processing of emotional tone of voice and emotional facial expression by event-related potentials (ERPs), using a wide range of different emotions (happiness, sadness, fear, anger, surprise, and disgust). Auditory emotional stimuli (a neutral word pronounced in an affective tone) and visual patterns (emotional facial expressions) were matched in congruous (the same emotion in face and voice) and incongruous (different emotions) pairs. Subjects (N=31) were required to watch and listen to the stimuli in order to comprehend them. Repeated measures ANOVAs showed a positive ERP deflection (P2), more posterior distributed. This P2 effect may represent a marker of cross-modal integration, modulated as a function of congruous/incongruous condition. Indeed, it shows an ampler peak in response to congruous stimuli than incongruous ones. It is suggested P2 can be a cognitive marker of multisensory processing, independently from the emotional content.     

Unattended emotional intonations modulate linguistic prosody processing

Prosody or speech melody subserves linguistic (e.g., question intonation) and emotional functions in speech communication. Findings from lesion studies and imaging experiments suggest that, depending on function or acoustic stimulus structure, prosodic speech components are differentially processed in the right and left hemispheres. This direct current (DC) potential study investigated the linguistic processing of digitally manipulated pitch contours of sentences that carried an emotional or neutral intonation. Discrimination of linguistic prosody was better for neutral stimuli as compared to happily as well as fearfully spoken sentences. Brain activation was increased during the processing of happy sentences as compared to neutral utterances. Neither neutral nor emotional stimuli evoked lateralized processing in the left or right hemisphere, indicating bilateral mechanisms of linguistic processing for pitch direction. Acoustic stimulus analysis suggested that prosodic components related to emotional intonation, such as pitch variability, interfered with linguistic processing of pitch course direction.