Communicating emotion: linking affective prosody and word meaning

The present study investigated the role of emotional tone of voice in the perception of spoken words. Listeners were presented with words that had either a happy, sad, or neutral meaning. Each word was spoken in a tone of voice (happy, sad, or neutral) that was congruent, incongruent, or neutral with respect to affective meaning, and naming latencies were collected. Across experiments, tone of voice was either blocked or mixed with respect to emotional meaning. The results suggest that emotional tone of voice facilitated linguistic processing of emotional words in an emotion-congruent fashion. These findings suggest that information about emotional tone is used in the processing of linguistic content influencing the recognition and naming of spoken words in an emotion-congruent manner.     

Interference produced by emotional conflict associated with anterior cingulate activation

The anterior cingulate cortex (ACC) is involved in cognition and emotion. In the classic Stroop task, presentation of stimuli that are in response conflict with one another produces activation in the caudal ACC. In the emotional Stroop task, presentation of emotionally salient stimuli produces activation in the rostral ACC. Presentation of stimuli that are emotionally conflicting should activate the caudal ACC; stimuli that are emotionally salient should activate the rostral ACC. We tested this prediction using functional magnetic resonance imaging while subjects made emotional valence judgments of words overlaid on emotional faces (word-face Stroop task). Emotionally incongruent pairs were responded to more slowly than emotionally congruent pairs. Emotionally incongruent trials were associated with increased activation within the caudal ACC, whereas no ACC activation was found in response to emotional saliency. These results support the conflict-monitoring model of caudal ACC and extend this function to conflict within the domain of emotional stimuli.     

Explicit vs. implicit emotional processing: The interaction between processing type and executive control

The current study examined whether the interaction between emotion and executive control (EC) is modulated by the processing type of the emotional information. Namely, whether the emotional information is explicitly processed, implicitly processed or passively viewed. In each trial, a negative or neutral picture preceded an arrow-flanker stimulus that was congruent or incongruent. Incongruent stimuli are known to recruit EC. Explicit processing of the pictures (Experiment 1a), which required responding to their emotional content, resulted in emotional interference for congruent but not for incongruent stimuli. Similar effects were shown for the passive viewing condition (Experiment 2). In contrast, implicit processing (Experiment 1b), which required responding to non-emotional content, resulted in emotional interference for both congruent and incongruent stimuli. Thus, our findings indicate that implicit emotional processing affects performance independently of EC recruitment. In contrast, explicit emotional processing and passive viewing of emotional pictures lead to reduced emotional interference when EC is recruited.     

Perceiving verbal and vocal emotions in a second language

Emotional inferences from speech require the integration of verbal and vocal emotional expressions. We asked whether this integration is comparable when listeners are exposed to their native language and when they listen to a language learned later in life. To this end, we presented native and non-native listeners with positive, neutral and negative words that were spoken with a happy, neutral or sad tone of voice. In two separate tasks, participants judged word valence and ignored tone of voice or judged emotional tone of voice and ignored word valence. While native listeners outperformed non-native listeners in the word valence task, performance was comparable in the voice task. More importantly, both native and non-native listeners responded faster and more accurately when verbal and vocal emotional expressions were congruent as compared to when they were incongruent. Given that the size of the latter effect did not differ as a function of language proficiency, one can conclude that the integration of verbal and vocal emotional expressions occurs as readily in one's second language as it does in one's native language.     

Perceiving verbal and vocal emotions in a second language

Emotional inferences from speech require the integration of verbal and vocal emotional expressions. We asked whether this integration is comparable when listeners are exposed to their native language and when they listen to a language learned later in life. To this end, we presented native and non-native listeners with positive, neutral and negative words that were spoken with a happy, neutral or sad tone of voice. In two separate tasks, participants judged word valence and ignored tone of voice or judged emotional tone of voice and ignored word valence. While native listeners outperformed non-native listeners in the word valence task, performance was comparable in the voice task. More importantly, both native and non-native listeners responded faster and more accurately when verbal and vocal emotional expressions were congruent as compared to when they were incongruent. Given that the size of the latter effect did not differ as a function of language proficiency, one can conclude that the integration of verbal and vocal emotional expressions occurs as readily in one's second language as it does in one's native language.     

Preattentive processing of audio-visual emotional signals

Previous research has shown that redundant information in faces and voices leads to faster emotional categorization compared to incongruent emotional information even when attending to only one modality. The aim of the present study was to test whether these crossmodal effects are predominantly due to a response conflict rather than interference at earlier, e.g. perceptual processing stages. In Experiment 1, participants had to categorize the valence and rate the intensity of happy, sad, angry and neutral unimodal or bimodal face–voice stimuli. They were asked to rate either the facial or vocal expression and ignore the emotion expressed in the other modality. Participants responded faster and more precisely to emotionally congruent compared to incongruent face–voice pairs in both the Attend Face and in the Attend Voice condition. Moreover, when attending to faces, emotionally congruent bimodal stimuli were more efficiently processed than unimodal visual stimuli. To study the role of a possible response conflict, Experiment 2 used a modified paradigm in which emotional and response conflicts were disentangled. Incongruency effects were significant even in the absence of response conflicts. The results suggest that emotional signals available through different sensory channels are automatically combined prior to response selection.     

Depth of facial expression processing depends on stimulus visibility: Behavioral and electrophysiological evidence of priming effects

Participants performed a priming task during which emotional faces served as prime stimuli and emotional words served as targets. Prime-target pairs were congruent or incongruent, and two levels of prime visibility were obtained by varying the duration of the masked primes. To probe a neural signature of the impact of the masked primes, lateralized readiness potentials (LRPs) were recorded over motor cortex. In the high-visibility condition, responses to word targets were faster when the prime-target pairs were congruent than when they were incongruent, providing evidence of priming effects. In line with the behavioral results, the electrophysiological data showed that high-visibility face primes resulted in LRP differences between congruent and incongruent trials, suggesting that prime stimuli initiated motor preparation. Contrary to the above pattern, no evidence for reaction time or LRP differences was observed in the low-visibility condition, revealing that the depth of facial expression processing is dependent on stimulus visibility.     

面孔表情和声音情绪信息整合对返回抑制的影响

返回抑制(inhibition of return, IOR)与情绪刺激都具有引导注意偏向、提高搜索效率的特点, 但二者间是否存在一定的交互作用迄今为止尚不明确。研究采用“线索-目标”范式并在视听双通道呈现情绪刺激来考察情绪刺激的加工与IOR的交互作用。实验1中情绪刺激以单通道视觉面孔或一致的视听双通道呈现, 实验2通过在视听通道呈现不一致的情绪刺激进一步考察视听双通道情绪一致刺激对IOR的影响是否是由听觉通道一致的情绪刺激导致的, 即是否对听觉通道的情绪刺激进行了加工。结果发现, 视听双通道情绪一致刺激能够削弱IOR, 但情绪不一致刺激与IOR之间不存在交互作用, 并且单双通道的IOR不存在显著差异。研究结果表明仅在视听双通道呈现情绪一致刺激时, 才会影响同一阶段的IOR, 这进一步支持了IOR的知觉抑制理论。     

Proactive and reactive control depends on emotional valence: a Stroop study with emotional expressions and words

We examined proactive and reactive control effects in the context of task-relevant happy, sad, and angry facial expressions on a face-word Stroop task. Participants identified the emotion expressed by a face that contained a congruent or incongruent emotional word (happy/sad/angry). Proactive control effects were measured in terms of the reduction in Stroop interference (difference between incongruent and congruent trials) as a function of previous trial emotion and previous trial congruence. Reactive control effects were measured in terms of the reduction in Stroop interference as a function of current trial emotion and previous trial congruence. Previous trial negative emotions exert greater influence on proactive control than the positive emotion. Sad faces in the previous trial resulted in greater reduction in the Stroop interference for happy faces in the current trial. However, current trial angry faces showed stronger adaptation effects compared to happy faces. Thus, both proactive and reactive control mechanisms are dependent on emotional valence of task-relevant stimuli.     

Observing repetitive finger movements modulates response times of auditorily cued finger movements

Our motor and perceptual representations of actions seem to be intimately linked and the human mirror neuron system (MNS) has been proposed as the mediator. In two experiments, we presented biological or non-biological movement stimuli that were either congruent or incongruent to a required response prompted by a tone. When the tone occurred with the onset of the last movement in a series, i.e., it was perceived during the movement presentation, congruent biological stimuli resulted in faster reaction times than congruent non-biological stimuli. The opposite was observed for incongruent stimuli. When the tone was presented after visual movement stimulation, however, no such interaction was present. This implies that biological movement stimuli only affect motor behaviour during visual processing but not thereafter. These data suggest that the MNS is an "online" system; longstanding repetitive visual stimulation (Experiment 1) has no benefit in comparison to only one or two repetitions (Experiment 2).     

情绪韵律调节情绪词识别的ERP研究

采用韵律/词汇干扰范式和延迟匹配任务, 通过两个ERP实验, 考察了汉语口语中情绪韵律能否、以及如何调节情绪词的识别。实验一中, 不同类型的情绪韵律分组呈现, ERP结果显示, 同与情绪韵律效价一致的情绪词相比, 与情绪韵律效价不一致的情绪词诱发了走向更负的P200、N300和N400成分; 实验二中, 不同类型的情绪韵律随机呈现, 上述效价一致性效应依然存在。实验结果表明, 情绪韵律能够调节情绪词识别, 主要表现在对情绪词的音韵编码和语义加工的双重易化上。     

Congruency effects in part-list cuing inhibition.

In demonstrations of part-list cuing inhibition, subjects who are shown a subset of studied list words recall fewer noncue words than subjects not shown such part-list cues. We propose that part-list inhibition is governed in part by an incongruency principle: Inhibition occurs to the extent that part-list cues induce a retrieval framework different from that used to encode list items. In Experiment 1, word lists were studied followed by a test of free recall either without cues, or with cue words serially organized to be either congruent or incongruent with the order of studied items. In Experiments 2-4, cues consisted of every second study item in the original presentation order (congruent ones) or reordered to form famous names or familiar idioms that had been hidden in the study list (incongruent cues). More part-list inhibition was observed with incongruent cues than congruent cues in all 4 experiments.     

Processing negative valence of word pairs that include a positive word

Previous research has suggested that cognitive performance is interrupted by negative relative to neutral or positive stimuli. We examined whether negative valence affects performance at the word or phrase level. Participants performed a semantic decision task on word pairs that included either a negative or a positive target word. In Experiment 1, the valence of the target word was congruent with the overall valence conveyed by the word pair (e.g., fat kid). As expected, response times were slower in the negative condition relative to the positive condition. Experiment 2 included target words that were incongruent with the overall valence of the word pair (e.g., fat salary). Response times were longer for word pairs whose overall valence was negative relative to positive, even though these word pairs included a positive word. Our findings support the Cognitive Primacy Hypothesis, according to which emotional valence is extracted after conceptual processing is complete.     

Acquired equivalence and distinctiveness of cues: I. Exploring a neural network approach

Rats were placed in 4 contexts (A, B, C, D) where they received 2 auditory stimuli (X, Y); in A and B, presentations of X were paired with food and those of Y were not, and in C and D, Y was paired with food and X was not. Rats then received combinations of contexts that had provided congruent (AB, CD) or incongruent (AD, CB) information about X and Y's relationship to food. Responding was more variable during congruent than incongruent trials (Experiment 1) and was systematically increased and decreased during congruent (relative to incongruent) trials by the presentation of food or no food, respectively (Experiment 2). These results support a connectionist approach to acquired changes in stimulus distinctiveness.     

Positive and negative emotions modulate attention allocation in color-flanker task processing: Evidence from event related potentials

The purpose of this study was to explore the effects of emotions in conflict processing. Behavioral and event related potential (ERP) data were acquired from twenty participants while they were performing a color-flanker task, in which the stimuli were emotional and neutral words. Through temporal principal component analysis (PCA) of ERP, three PCA components were extracted with their time windows mapped to three ERP components: N1, N2 and P3. Further analysis revealed, when the stimuli were the positive words, N1 was marginally greater for the congruent condition than for the incongruent condition (p = .06); while the stimuli were the negative words, N1 was greater for the incongruent condition than for the congruent condition (p = .02); however, no significant interaction effect involving the valence of words (positive vs. negative) and the color congruency (congruent vs. incongruent) was found on N2, P3 and the behavior data. The result suggested that negative emotion might lead to more attention allocation in the incongruent processing, while positive emotion might lead to more attention allocation in the congruent processing. Furthermore, ERP source analysis in the present study confirmed anterior cingulate cortex was involved in the conflict monitoring mechanism.     

Evidence for task conflict in the Stroop effect

C. M. MacLeod and P. A. MacDonald (2000) suggested that congruent and incongruent Stroop stimuli cause more task conflict than neutral stimuli because the anterior cingulate cortex is more activated with these stimuli. This study investigated behavioral expression for this pattern. Experiment 1 reduced task conflict control by increasing the proportion of nonword neutrals. Additionally, half the trials had conflict or neutral cues. The control reduction revealed the task conflict. For noncued trials, response time was longer for congruent stimuli than for neutral stimuli (reverse facilitation effect). In addition, response time for congruent stimuli was longer when stimuli were uncued vs. cued. Experiment 2 increased task conflict control by changing the neutral stimuli to noncolor words. Consequently, the task conflict expression disappeared.     

I know how you feel: Task-irrelevant facial expressions are spontaneously processed at a semantic level

Previous studies have demonstrated that emotions are automatically processed. Even with subliminal presentations, subjects involuntarily mimic specific facial expressions, are influenced by the valence of a preceding emotion during judgments, and exhibit slowed responses to personally meaningful emotions; these effects are due to reflexive mimicry, unconscious carryover of valence, and attentional capture, respectively. However, perception-action effects indicate that rapid processing should involve deep, semantic-level representations of emotion (e.g., “fear”), even in the absence of a clinical emotion disorder. To test this hypothesis, we developed an emotional Stroop task (Emostroop) in which subjects responded nonverbally to emotion words superimposed over task-irrelevant images of faces displaying congruent or incongruent emotional expressions. Subjects reliably responded more slowly to incongruent than to congruent stimuli, and this interference was related to trait measures of emotionality. Rapid processing of facial emotions spontaneously activates semantic, content-rich representations at the level of the specific emotion.     

Priming of emotion recognition

Four experiments investigated priming of emotion recognition using a range of emotional stimuli, including facial expressions, words, pictures, and nonverbal sounds. In each experiment, a prime-target paradigm was used with related, neutral, and unrelated pairs. In Experiment 1, facial expression primes preceded word targets in an emotion classification task. A pattern of priming of emotional word targets by related primes with no inhibition of unrelated primes was found. Experiment 2 reversed these primes and targets and found the same pattern of results, demonstrating bidirectional priming between facial expressions and words. Experiment 2 also found priming of facial expression targets by picture primes. Experiment 3 demonstrated that priming occurs not just between pairs of stimuli that have a high co-occurrence in the environment (for example, nonverbal sounds and facial expressions), but with stimuli that co-occur less frequently and are linked mainly by their emotional category (for example, nonverbal sounds and printed words). This shows the importance of the prime and target sharing a common emotional category, rather than their previous co-occurrence. Experiment 4 extended the findings by showing that there are category-based effects as well as valence effects in emotional priming, supporting a categorical view of emotion recognition.     

Attentional control adjustments in Eriksen and Stroop task performance can be independent of response conflict

In the Eriksen flanker and colour-word Stroop tasks, the response time (RT) difference between incongruent and congruent trials is smaller following incongruent trials than following congruent trials: the “Gratton effect” (Gratton, Coles, & Donchin, 1992). According to the prevailing conflict-monitoring theory (Botvinick, Braver, Barch, Carter, & Cohen, 2001), the Gratton effect reflects attentional control adjustment following response conflict on incongruent trials. However, because previous studies compared incongruent and congruent trials, it remains unclear to what extent the Gratton effect is driven by incongruent rather than congruent trials. To resolve this issue, we included neutral trials in addition to incongruent and congruent trials in the Eriksen (Experiment 1) and Stroop (Experiment 2) tasks. Participants responded manually and vocally in both tasks. Moreover, we assessed responding to Stroop stimuli that were preceded by neutral cues or by incongruent- or congruent-predicting cues (Experiment 3). In all three experiments, the RT difference between incongruent and congruent trials was larger for postcongruent trials than for postincongruent and postneutral trials. These findings suggest that control adjustments can be independent of response conflict, challenging conflict-monitoring theory.