基于人类注意机制的微表情检测方法

微表情是一种持续时间极短、不易被察觉的面部动作, 揭示了个体的真实情绪, 可以被广泛地应用于谎言识别等领域。而微表情检测的研究受到小样本问题的限制。针对该问题, 本文结合计算机视觉技术与认知心理学实验方法进行探索。首先, 结合眼动技术和呈现-判断范式与阈下情绪启动效应的行为实验范式, 考察微表情识别中选择注意分配的认知机制, 细化人类识别微表情时的特征兴趣区域。其次, 结合人类注意机制, 提出基于自监督学习的多模态微表情检测方法。通过理论和关键技术的突破, 为真实场景下微表情检测的应用奠定基础。     

微表情研究及其应用

微表情是一种持续时间仅为1/25秒至1/5秒的非常快速的表情, 表达了人试图压抑与隐藏的真正情感。本文系统梳理已公开发表的微表情实证研究报告, 对注重于测量微表情识别能力的早期研究、目前基于微表情训练工具(METT)的微表情识别应用研究以及刚刚萌芽的微表情表达研究进行总结分析, 明确指出以往研究中存在的问题, 建议未来研究应探讨METT在不同文化中的有效性, 研究微表情表达的基本特点和主要影响因素, 并注重发展自动化的微表情识别工具。本文提出, 基于微表情的自动谎言识别系统将是微表情研究未来的应用方向之一。     

微表情面部肌电跨模态分析及标注算法

长久以来,微表情的小样本问题始终制约着微表情分析的发展,而小样本问题归根到底是因为微表情的数据标注十分困难。本研究希望借助面部肌电作为技术手段,从微表情数据自动标注、半自动标注和无标注三个方面各提出一套解决方案。对于自动标注,提出基于面部远端肌电的微表情自动标注方案;对于半自动标注,提出基于单帧标注的微表情起止帧自动标注;对于无标注,提出了基于肌电信号的跨模态自监督学习算法。同时,本研究还希望借助肌电模态,对微表情的呈现时间和幅度等机理特征进行拓展研究。     

微表情在欺骗检测中的应用

众多研究表明微表情和欺骗密切相关,考察如何利用微表情进行欺骗检测是微表情研究的一个重要领域。通过对欺骗与欺骗检测的基本概念、微表情作为欺骗检测线索的理论依据及相关应用等的较全面阐述,论证了微表情作为欺骗检测的情绪线索具有可行性和一定的有效性,且相比其他生理、认知的欺骗检测线索具有更多的优势,在国家安全、临床医学等领域的欺骗检测中已有较多的应用。针对微表情在欺骗检测中的应用存在的争议,指出未来应进一步研究欺骗是否一定伴随着微表情、微表情训练能否促进欺骗检测能力、如何提高微表情实验室研究的外部效度、个体差异对微表情利用的影响等核心问题,并指出了可能的解决途径。     

抑郁症患者的面部表情识别研究

抑郁症患者的情绪信息加工已成为前沿科学问题和研究热点,其特异模式或许可以作为检测抑郁的有效指标。研究发现,与正常人相比,抑郁症患者对正性情绪信息的反应水平以及对情绪表情(尤其是悲伤表情)的辨别能力都有所降低,不仅抑制负性情绪的能力受损,而且在社会情境下精确判断面部表情微弱改变的能力也受损。本文系统梳理了有关抑郁症患者面部表情识别研究及其主要发现,并借鉴对精神分裂症患者微表情识别训练的研究结果提出有必要开展抑郁症患者微表情识别及训练的研究,最后分析该领域目前存在的问题并提出未来研究展望。     

微表情的特征、识别、训练和影响因素

微表情是人压抑的真实情绪的泄露, 时间约为1/25~1/2 s, 常出现在说谎情境, 可能源于椎体束运动系统控制的自主面部运动和椎体束外运动系统控制的非自主面部运动的“拔河比赛”。已有研究构建了微表情数据库, 开发了BART和JACBART等微表情识别测验以及METT训练, 并探讨了微表情识别及训练与认知、人格、文化、职业等的关系。但这些研究考察的中性表情背景下微表情只是镶嵌在各表情背景下真实微表情的特例, 生态程度不高; 少量研究探索了表情背景对微表情识别的影响但未建立标准测验。未来应开发生态化微表情识别标准测验和训练, 尝试跨越微表情发生和识别的界限, 并采用生理−心理、电生理和脑成像技术探索其机制。     

短暂呈现的背景情绪词对面部表情识别的影响

语言常常伴随表情出现,情绪知觉一定程度上是通过语言建构的,语言对情绪表情的识别具有重要作用。但情绪词对短暂呈现的面部表情(即人工微表情)的识别是否也有影响并不清楚。采用短暂呈现的情绪词和面部表情,使用类Stroop干扰任务与情感启动任务,考察背景情绪词对面部表情识别的影响。结果发现,情绪词效价与面部表情效价的匹配关系的主效应显著,这表明背景情绪词影响短暂呈现的面部表情的识别。研究结果深化了对微表情识别机制的理解。     

学习障碍儿童情绪理解与同伴接受性*及社会行为的关系

该研究以90名学习障碍儿童为研究对象,探讨学习障碍儿童情绪理解与同伴接受性和社会行为的关系,比较情绪理解不同成分对同伴接受性和社会行为的影响作用。研究发现情绪理解不同成分对学习障碍儿童同伴接受性和社会行为具有不同程度的影响,具体表现为:1)基本情绪识别和自我意识情绪识别是学习障碍儿童同伴接受性的最佳预测变量;2)自我意识情绪识别和表情识别是学习障碍儿童问题行为的最佳预测变量;3)自我意识情绪识别、情绪内部原因理解、情绪隐藏理解、情绪改变理解和情绪外部原因理解是学习障碍儿童社会能力的最佳预测变量。     

生态化微表情识别测验EMERT的建立——对JACBART微表情识别测验的改进与发展

经典JACBART微表情识别测验只考察平静表情背景下微表情识别,生态效度不高。本研究创建生态化微表情识别测验,考察所有7种基本表情背景下的6种微表情识别特征。结果发现:(1)该测验具有良好的重测信度、校标效度和生态效度,能够稳定有效地测量生态化微表情识别。(2)信效度检验揭示了生态化微表情识别特征。某些生态化微表情识别存在训练效应。生态化微表情与经典微表情或普通表情普遍相关。恐惧、悲伤、厌恶、愤怒微表情背景主效应显著;惊讶和愉快微表情背景主效应不显著,成对比较发现各背景下惊讶/愉快微表情差异不显著,但是与普通表情有广泛的显著差异。用不同背景下微表情识别正确率的标准差定义生态化微表情识别波动,发现生态化微表情识别具有稳定的波动性。     

面部反馈在微表情识别过程中的作用

本研究采用JACBART范式,通过考察增强面部反馈对高强度微表情识别的影响（实验1）和增强面部反馈对低强度微表情识别的影响（实验2）来探究面部反馈在微表情识别过程中的作用。结果发现：（1）增强面部反馈不能提高被试对高强度微表情识别的准确率;（2）增强面部反馈降低了低强度微表情的识别准确率。研究提示面部反馈参与了微表情的识别过程,但它是一种对微表情识别“有害”的抑制性线索,且这一线索的作用受微表情强度的调节。     

Facial expressions and the ability to recognize emotions from the eyes or mouth: A comparison among old adults,young adults,and children

The aim of the present study was to contribute to the literature on the ability to recognize anger, happiness, fear, surprise, sadness, disgust, and neutral emotions from facial information (whole face, eye region, mouth region). More specifically, the aim was to investigate older adults' performance in emotions recognition using the same tool used in the previous studies on children and adults’ performance and verify if the pattern of emotions recognition show differences compared with the other two groups. Results showed that happiness is among the easiest emotions to recognize while the disgust is always among the most difficult emotions to recognize for older adults. The findings seem to indicate that is more easily recognizing emotions when pictures represent the whole face; compared with the specific region (eye and mouth regions), older participants seems to recognize more easily emotions when the mouth region is presented. In general, the results of the study did not detect a decay in the ability to recognize emotions from the face, eyes, or mouth. The performance of the old adults is statistically worse than the other two groups in only a few cases: in anger and disgust recognition from the whole face; in anger recognition from the eye region; and in disgust, fear, and neutral emotion recognition from mouth region.     

High Hostility Among Smokers Predicts Slower Recognition of Positive Facial Emotion

High levels of trait hostility are associated with wide-ranging interpersonal deficits and heightened physiological response to social stressors. These deficits may be attributable in part to individual differences in the perception of social cues. The present study evaluated the ability to recognize facial emotion among 48 high hostile (HH) and 48 low hostile (LH) smokers and whether experimentally-manipulated acute nicotine deprivation moderated relations between hostility and facial emotion recognition. A computer program presented series of pictures of faces that morphed from a neutral emotion into increasing intensities of happiness, sadness, fear, or anger, and participants were asked to identify the emotion displayed as quickly as possible. Results indicated that HH smokers, relative to LH smokers, required a significantly greater intensity of emotion expression to recognize happiness. No differences were found for other emotions across HH and LH individuals, nor did nicotine deprivation moderate relations between hostility and emotion recognition. This is the first study to show that HH individuals are slower to recognize happy facial expressions and that this occurs regardless of recent tobacco abstinence. Difficulty recognizing happiness in others may impact the degree to which HH individuals are able to identify social approach signals and to receive social reinforcement.     

The influence of emotion on face processing

According to the broaden-and-build theory, positive emotions broaden one's thought-action repertoire, which may manifest as a widened attentional scope in cognitive processing. The present study directly tests this hypothesis by examining the influences of induced emotions (positive, neutral and negative) on holistic processing of face (Experiment 1) and face discrimination (Experiment 2). In both experiments, emotions induced with images from the International Affective Picture System significantly interacted with face processing. That is, positive emotions engendered greater holistic face encoding in a composite-face task in Experiment 1 and more accurate face discrimination in Experiment 2, relative to the neutral condition. In contrast, negative emotions impaired holistic face encoding in the composite-face task and reduced face discrimination accuracy. Taken together, these results provide further support for the attentional broadening effect of positive affect by demonstrating that induced positive emotions facilitate holistic/configural processing.     

Personality traits modulate skin conductance response to emotional pictures: An investigation with Cloninger’s model of personality

Several studies found strong relationships between main personality dimensions and emotions, and more particularly between neuroticism and negative emotions, as well as between extraversion and positive ones. In these studies, personality was mainly assessed with respect to the big five model, and emotions were evaluated with subjective rating scales. To extend the exploration of the associations between personality and emotion, the aim of the present study was to investigate the relationships between Cloninger’s dimensions of personality and emotional reactivity assessed by skin conductance response (SCR). In 55 normal subjects, SCR was recorded while they were presented pictures selected as positive, negative and neutral from the International Affective Picture System. The results showed no influence of personality on response magnitude, but subjects with higher harm avoidance scores showed longer half-recovery times for negative relative to neutral pictures. This result is consistent with the dispositional bias of harm avoidance to respond intensely to signals of aversive stimuli.     

Going virtual in the early years: 30-month-old toddlers recognize commonly used emojis

People nowadays do not only display facial expressions in face to face communication but also in on-line communication using graphical symbols named emojis. The present study explored 30-month-old toddlers’ ability to recognize emojis that represented six basic human emotions. In the study, 38 toddlers first saw scenarios that elicited different emotions in an actor and were asked to visually identify the matching emoji in the presence of a distractor. Eye-tracking results showed that toddlers could correctly identify the emoji that represented the emotion in each scenario. Toddlers then heard different emotion words and were again found to be able to identify the matching emoji. These findings provide preliminary evidence that the ability to recognize and understand the emotional value of emotion symbols in the virtual world emerges early in development.     

前瞻性情绪作为社会风险的信息源假说：公共场景下风险决策的情绪及文化机制探讨

本研究从梳理决策的情绪理论入手, 在综合几种主要理论的基础上提出了“前瞻性情绪作为社会风险信息源”的假说。前瞻性情绪是在决策过程中产生的, 由于对决策选择的预期而感受到的, 进而影响决策行为的情绪。社会环境下的风险事件具有突发和稍纵即逝的特征, 如果一个公民对这类风险事件做出反应, 其结果的概率难于量化。在这种情况下, 前瞻性情绪能够为决策者快速提供有关信息, 并形成对决策预期结果严重性和可能性的综合表征, 从而成为公众场景下风险决策的有效线索。不同前瞻性情绪的组合在面对社会性风险事件时具有针对性的作用。近年来在文化比较中对于面子、荣誉和尊严文化的划分, 也为我们研究文化的情绪特征和行为效应提供了理论框架。本研究拟通过四项研究和多个实验探讨前瞻性情绪如何影响公众场景下人们的风险应对行为, 并对三种文化的情绪特征进行比较, 探索情绪性决策中个人因素、群体因素、及文化因素如何共同决定公共场景下风险决策(如亲社会行为或反社会行为)的发生与发展, 同时也期望为公共政策的制定及风险管理提供科学依据。     

How social context moderates the self-evaluative emotions experienced due to health risk behaviour

When people are confronted with the potential negative physical outcomes of their own health risk behaviour, they experience a self-threat. This threat is felt as negative self-evaluative emotions. We hypothesise that the threat will lead to more private self-evaluative emotions (e.g. regret) in a private social context, whereas more public self-evaluative emotions (e.g. embarrassment) will be felt in a public social context with negative norms. Consistent with our hypotheses, we show that participants anticipate feeling more private self-evaluative emotions when confronted with the negative consequences of their unhealthy behaviour when alone, and more public self-evaluative emotions when in a group (Study 1). They further anticipate more public self-evaluative emotions in response to a health self-threat when the group norm is negative, and more private self-evaluative emotions when the group norm is lenient (Study 2). Finally, in a cross-sectional study amongst smokers, we show that private but not public negative self-evaluative emotions concerning their own smoking habits are positively correlated with the intent to quit smoking (Study 3). These studies show that a distinction needs to be made between public and private self-evaluative emotions, in terms of their antecedents and effects. Theoretical implications and further lines of research are discussed.