Categorical and dimensional perceptions in decoding emotional facial expressions

We investigated whether categorical perception and dimensional perception can co-occur while decoding emotional facial expressions. In Experiment 1, facial continua with endpoints consisting of four basic emotions (i.e., happiness–fear and anger–disgust) were created by a morphing technique. Participants rated each facial stimulus using a categorical strategy and a dimensional strategy. The results show that the happiness–fear continuum was divided into two clusters based on valence, even when using the dimensional strategy. Moreover, the faces were arrayed in order of the physical changes within each cluster. In Experiment 2, we found a category boundary within other continua (i.e., surprise–sadness and excitement–disgust) with regard to the arousal and valence dimensions. These findings indicate that categorical perception and dimensional perception co-occurred when emotional facial expressions were rated using a dimensional strategy, suggesting a hybrid theory of categorical and dimensional accounts.     

Categorical and dimensional perceptions in decoding emotional facial expressions

We investigated whether categorical perception and dimensional perception can co-occur while decoding emotional facial expressions. In Experiment 1, facial continua with endpoints consisting of four basic emotions (i.e., happiness-fear and anger-disgust) were created by a morphing technique. Participants rated each facial stimulus using a categorical strategy and a dimensional strategy. The results show that the happiness-fear continuum was divided into two clusters based on valence, even when using the dimensional strategy. Moreover, the faces were arrayed in order of the physical changes within each cluster. In Experiment 2, we found a category boundary within other continua (i.e., surprise-sadness and excitement-disgust) with regard to the arousal and valence dimensions. These findings indicate that categorical perception and dimensional perception co-occurred when emotional facial expressions were rated using a dimensional strategy, suggesting a hybrid theory of categorical and dimensional accounts.     

Encoding and decoding of nonverbal behavior through facial expressions

Ten males and ten females served as both senders and receivers of nonverbal expressions in an experiment designed to examine various kinds of sending-receiving relationships. While the overall sending-receiving relationship for five types of expressions combined was positive and nearly statistically significant (.10 > p > .05), the category-specific sending-receiving relationships were near zero in magnitude or slightly negative. Sending-receiving relationships that were category specific and involved same-sex communication attempts only were found to be more negative with some being statistically significant. Females were found to be significantly better receivers but not significantly better senders than males. The results were discussed in terms of recent theoretical notions concerning sending and receiving processes.     

Caricaturing facial expressions

The physical differences between facial expressions (e.g. fear) and a reference norm (e.g. a neutral expression) were altered to produce photographic-quality caricatures. In Experiment 1, participants rated caricatures of fear, happiness and sadness for their intensity of these three emotions; a second group of participants rated how 'face-like' the caricatures appeared. With increasing levels of exaggeration the caricatures were rated as more emotionally intense, but less 'face-like'. Experiment 2 demonstrated a similar relationship between emotional intensity and level of caricature for six different facial expressions. Experiments 3 and 4 compared intensity ratings of facial expression caricatures prepared relative to a selection of reference norms - a neutral expression, an average expression, or a different facial expression (e.g. anger caricatured relative to fear). Each norm produced a linear relationship between caricature and rated intensity of emotion; this finding is inconsistent with two-dimensional models of the perceptual representation of facial expression. An exemplar-based multidimensional model is proposed as an alternative account.     

Perceiving facial expressions

Three experiments investigated the perception of facial displays of emotions. Using a morphing technique, Experiment 1 (identification task) and Experiment 2 (ABX discrimination task) evaluated the merits of categorical and dimensional models of the representation of these stimuli. We argue that basic emotions—as they are usually defined verbally—do not correspond to primary perceptual categories emerging from the visual analysis of facial expressions. Instead, the results are compatible with the hypothesis that facial expressions are coded in a continuous anisotropic space structured by valence axes. Experiment 3 (identification task) introduces a new technique for generating chimeras to address the debate between feature-based and holistic models of the processing of facial expressions. Contrary to the pure holistic hypothesis, the results suggest that an independent assessment of discrimination features is possible, and may be sufficient for identifying expressions even when the global facial configuration is ambiguous. However, they also suggest that top-down processing may improve identification accuracy by assessing the coherence of local features.     

Infant attention to facial expressions and facial motion

Three-month-old infants were shown moving faces and still faces on videotape in a paired-comparison situation. Motion type was clearly specified, and facial expression and motion were separately varied. Infants saw a still face, internal motion on the face (i.e., motion of the internal features), and whole object (i.e., side-to-side) motion, each with happy and neutral expressions. Infants showed preference for expressions when the face was still and when it showed internal motion. Facial expression and facial motion were equally preferred, and both appeared to be salient dimensions of the face for three-month-old infants.     

Repeated short presentations of morphed facial expressions change recognition and evaluation of facial expressions

This study investigated whether sensitivity to and evaluation of facial expressions varied with repeated exposure to non-prototypical facial expressions for a short presentation time. A morphed facial expression was presented for 500 ms repeatedly, and participants were required to indicate whether each facial expression was happy or angry. We manipulated the distribution of presentations of the morphed facial expressions for each facial stimulus. Some of the individuals depicted in the facial stimuli expressed anger frequently (i.e., anger-prone individuals), while the others expressed happiness frequently (i.e., happiness-prone individuals). After being exposed to the faces of anger-prone individuals, the participants became less sensitive to those individuals’ angry faces. Further, after being exposed to the faces of happiness-prone individuals, the participants became less sensitive to those individuals’ happy faces. We also found a relative increase in the social desirability of happiness-prone individuals after exposure to the facial stimuli.     

Rapid facial reactions to emotional facial expressions

This study explored how rapidly emotion specific facial muscle reactions were elicited when subjects were exposed to pictures of angry and happy facial expressions. In three separate experiments, it was found that distinctive facial electromyographic reactions, i.e., greater Zygomaticus major muscle activity in response to happy than to angry stimuli and greater Corrugator supercilii muscle activity in response to angry stimuli, were detectable after only 300–400 ms of exposure. These findings demonstrate that facial reactions are quickly elicited, indicating that expressive emotional reactions can be very rapidly manifested and are perhaps controlled by fast operating facial affect programs.     

Unconscious facial reactions to emotional facial expressions

Studies reveal that when people are exposed to emotional facial expressions, they spontaneously react with distinct facial electromyographic (EMG) reactions in emotion-relevant facial muscles. These reactions reflect, in part, a tendency to mimic the facial stimuli. We investigated whether corresponding facial reactions can be elicited when people are unconsciously exposed to happy and angry facial expressions. Through use of the backward-masking technique, the subjects were prevented from consciously perceiving 30-ms exposures of happy, neutral, and angry target faces, which immediately were followed and masked by neutral faces. Despite the fact that exposure to happy and angry faces was unconscious, the subjects reacted with distinct facial muscle reactions that corresponded to the happy and angry stimulus faces. Our results show that both positive and negative emotional reactions can be unconsciously evoked, and particularly that important aspects of emotional face-to-face communication can occur on an unconscious level.     

Perception of facial expressions

Three studies examined the nature of the contributions of each hemisphere to the processing of facial expressions and facial identity. A pair of faces, the members of which differed in either expression or identity, were presented to the right or left field. Subjects were required to compare the members of the pair to each other (experiments 1 and 2) or to a previously presented sample (experiment 3). The results revealed that both face and expression perception show an LVF superiority although the two tasks could be differentiated in terms of overall processing time and the interaction of laterality differences with sex. No clear-cut differences in laterality emerged for processing of positive and negative expressions.     

Perceptual skill in decoding facial affect

The perceptual skill in identifying the briefly exposed photographs of facial affect was examined with 75 men and 75 women who were asked to judge six cross-culturally recognizable slides, each expressing one of the six emotions exposed at three different time intervals. Each subject was required to give six judgments and the number of correct responses was considered his score. Analysis by a 3 X 2 factorial design indicated the longer the exposure of the photograph the better the recognition. Although there was no over-all sex difference, analysis by emotions confirmed significantly better accuracy by the women judging 'sadness' than by men, and men were more accurate for 'anger' than were women.     

Culture and facial expressions of emotion

With over a century of theoretical developments and empirical investigation in broad fields (e.g., anthropology, psychology, evolutionary biology), the universality of facial expressions of emotion remains a central debate in psychology. How near or far, then, is this debate from being resolved? Here, I will address this question by highlighting and synthesizing the significant advances in the field that have elevated knowledge of facial expression recognition across cultures. Specifically, I will discuss the impact of early major theoretical and empirical contributions in parallel fields and their later integration in modern research. With illustrative examples, I will show that the debate on the universality of facial expressions has arrived at a new juncture and faces a new generation of exciting questions.     

Enhancing images of facial expressions

Facial images can be enhanced by application of an algorithm--the caricature algorithm--that systematically manipulates their distinctiveness (Benson & Perrett, 1991c; Brennan, 1985). In this study, we first produced a composite facial image from natural images of the six facial expressions of fear, sadness, surprise, happiness, disgust, and anger shown on a number of different individual faces (Ekman & Friesen, 1975). We then caricatured the composite images with respect to a neutral (resting) expression. Experiment 1 showed that rated strength of the target expression was directly related to the degree of enhancement for all the expressions. Experiment 2, which used a free rating procedure, found that, although caricature enhanced the strength of the target expression (more extreme ratings), it did not necessarily enhance its purity, inasmuch as the attributes of nontarget expressions were also enhanced. Naming of prototypes, of original exemplar images, and of caricatures was explored in Experiment 3 and followed the pattern suggested by the free rating conditions of Experiment 2, with no overall naming advantage to caricatures under these conditions. Overall, the experiments suggested that computational methods of compositing and caricature can be usefully applied to facial images of expression. Their utility in enhancing the distinctiveness of the expression depends on the purity of expression in the source image.     

Dynamic facial expressions are processed holistically,but not more holistically than static facial expressions

There is evidence that facial expressions are perceived holistically and featurally. The composite task is a direct measure of holistic processing (although the absence of a composite effect implies the use of other types of processing). Most composite task studies have used static images, despite the fact that movement is an important aspect of facial expressions and there is some evidence that movement may facilitate recognition. We created static and dynamic composites, in which emotions were reliably identified from each half of the face. The magnitude of the composite effect was similar for static and dynamic expressions identified from the top half (anger, sadness and surprise) but was reduced in dynamic as compared to static expressions identified from the bottom half (fear, disgust and joy). Thus, any advantage in recognising dynamic over static expressions is not likely to stem from enhanced holistic processing, rather motion may emphasise or disambiguate diagnostic featural information.     

Spontaneous facial mimicry in response to dynamic facial expressions

Based on previous neuroscientific evidence indicating activation of the mirror neuron system in response to dynamic facial actions, we hypothesized that facial mimicry would occur while subjects viewed dynamic facial expressions. To test this hypothesis, dynamic/static facial expressions of anger/happiness were presented using computer-morphing (Experiment 1) and videos (Experiment 2). The subjects' facial actions were unobtrusively videotaped and blindly coded using Facial Action Coding System [FACS; Ekman, P., & Friesen, W. V. (1978). Facial action coding system. Palo Alto, CA: Consulting Psychologist]. In the dynamic presentations common to both experiments, brow lowering, a prototypical action in angry expressions, occurred more frequently in response to angry expressions than to happy expressions. The pulling of lip corners, a prototypical action in happy expressions, occurred more frequently in response to happy expressions than to angry expressions in dynamic presentations. Additionally, the mean latency of these actions was less than 900 ms after the onset of dynamic changes in facial expression. Naive raters recognized the subjects' facial reactions as emotional expressions, with the valence corresponding to the dynamic facial expressions that the subjects were viewing. These results indicate that dynamic facial expressions elicit spontaneous and rapid facial mimicry, which functions both as a form of intra-individual processing and as inter-individual communication.     

Conscious and nonconscious discrimination of facial expressions

Abrupt discontinuities in recognizing categories of emotion are found for the labelling of consciously perceived facial expressions. This has been taken to imply that, at a conscious level, we perceive facial expressions categorically. We investigated whether the abrupt discontinuities found in categorization for conscious recognition would be replaced by a graded transition for subthreshold stimuli. Fifteen volunteers participated in two experiments, in which participants viewed faces morphed from 100% fear to 100% disgust along seven increments. In Experiment A, target faces were presented for 30 ms, in Experiment B for 170 ms. Participants made two-alternative forced-choice decisions between fear and disgust. Results for the 30 ms presentation time indicated a significant linear trend between degree of morphing and classification of the images. Results for 170 ms presentation time followed the higher order function found in studies of categorical perception. These results provide preliminary evidence for separate processes underlying conscious and nonconscious perception of facial expressions of emotion.     

Processes underlying congruent and incongruent facial reactions to emotional facial expressions

The present electromyographic study is a first step toward shedding light on the involvement of affective processes in congruent and incongruent facial reactions to facial expressions. Further, empathy was investigated as a potential mediator underlying the modulation of facial reactions to emotional faces in a competitive, a cooperative, and a neutral setting. Results revealed less congruent reactions to happy expressions and even incongruent reactions to sad and angry expressions in the competition condition, whereas virtually no differences between the neutral and the cooperation condition occurred. Effects on congruent reactions were found to be mediated by cognitive empathy, indicating that the state of empathy plays an important role in the situational modulation of congruent reactions. Further, incongruent reactions to sad and angry faces in a competition setting were mediated by the emotional reaction of joy, supporting the assumption that incongruent facial reactions are mainly based on affective processes. Additionally, strategic processes (specifically, the goal to create and maintain a smooth, harmonious interaction) were found to influence facial reactions while being in a cooperative mindset. Now, further studies are needed to test for the generalizability of these effects.     

Detection of emotional facial expressions and anti-expressions

Detection of emotional facial expressions has been shown to be more efficient than detection of neutral expressions. However, it remains unclear whether this effect is attributable to visual or emotional factors. To investigate this issue, we conducted two experiments using the visual search paradigm with photographic stimuli. We included a single target facial expression of anger or happiness in presentations of crowds of neutral facial expressions. The anti-expressions of anger and happiness were also presented. Although anti-expressions produced changes in visual features comparable to those of the emotional facial expressions, they expressed relatively neutral emotions. The results consistently showed that reaction times (RTs) for detecting emotional facial expressions (both anger and happiness) were shorter than those for detecting anti-expressions. The RTs for detecting the expressions were negatively related to experienced emotional arousal. These results suggest that efficient detection of emotional facial expressions is not attributable to their visual characteristics but rather to their emotional significance.     

Categorical perception of facial expressions.

People universally recognize facial expressions of happiness, sadness, fear, anger, disgust, and perhaps, surprise, suggesting a perceptual mechanism tuned to the facial configuration displaying each emotion. Sets of drawings were generated by computer, each consisting of a series of faces differing by constant physical amounts, running from one emotional expression to another (or from one emotional expression to a neutral face). Subjects discriminated pairs of faces, then, in a separate task, categorized the emotion displayed by each. Faces within a category were discriminated more poorly than faces in different categories that differed by an equal physical amount. Thus emotional expressions, like colors and speech sounds, are perceived categorically, not as a direct reflection of their continuous physical properties.