Perceptual and affective mechanisms in facial expression recognition: An integrative review

Facial expressions of emotion involve a physical component of morphological changes in a face and an affective component conveying information about the expresser’s internal feelings. It remains unresolved how much recognition and discrimination of expressions rely on the perception of morphological patterns or the processing of affective content. This review of research on the role of visual and emotional factors in expression recognition reached three major conclusions. First, behavioral, neurophysiological, and computational measures indicate that basic expressions are reliably recognized and discriminated from one another, albeit the effect may be inflated by the use of prototypical expression stimuli and forced-choice responses. Second, affective content along the dimensions of valence and arousal is extracted early from facial expressions, although this coarse affective representation contributes minimally to categorical recognition of specific expressions. Third, the physical configuration and visual saliency of facial features contribute significantly to expression recognition, with “emotionless” computational models being able to reproduce some of the basic phenomena demonstrated in human observers. We conclude that facial expression recognition, as it has been investigated in conventional laboratory tasks, depends to a greater extent on perceptual than affective information and mechanisms.     

Reflexive Attention Modulates Processing of Visual Stimuli in Human Extrastriate Cortex

Attention can be oriented reflexively to a location in space by an abrupt change in the visual scene. In the present study, we investigated the consequences of reflexive attention on the neural processing of visual stimuli. The findings show that reflexively oriented attention produces modulations in early sensory analysis at the same extrastriate neural locus as the earliest effects of voluntarily focused attention. In addition, stimulus processing was found to be enhanced at later stages of analysis, which reflect stimulus relevance. As is the case with behavioral measures of reflexive attention, these physiological enhancement effects are rapidly engaged but short-lived. As time passes between the initial attention-capturing event and subsequent stimuli, the extrastriate effect reverses, and the enhancement of higher order processing subsides. These findings indicate that reflexive attention is able to affect perceptions of the visual world by modulating neural processing as early as extrastriate visual cortex.     

群体面孔情绪的整体编码及其影响因素

当面孔以群体形式出现,认知神经系统会自动整合情绪信息提取平均情绪,此过程被称为群体面孔情绪的整体编码。探讨其与低水平整体表征的分离,与个体表征的关系及神经活动特点是揭示其加工机制的关键,但目前尚未形成系统性模型。未来应综合利用眼动、神经电生理和脑成像技术,结合注意、记忆及社会线索进一步拓展对其认知神经机制和影响因素的研究,同时关注具有认知情感障碍的特殊人群,并从毕生发展的角度探索其发展轨迹。     

Parallel processing in visual perception and memory: What goes where and when?

This article begins with reviews of parallel processing models in the areas of visual perception and memory, pointing out kinds of information purported to be processed in each, and the overlap in the physiological substrates involved. Next, some pertinent literature having to do with the linkage between perception and memory is reviewed (e.g., visual memory for what or where), concluding that there exists a serious lack of research and knowledge of how different perceptual processes may lead to facilitated, distorted or impaired memory in different forms of storage. Some possible scenarios are presented concerning how perceptual information might be interfaced with memorial mechanisms, and some working hypotheses are considered. Finally, a new paradigm is outlined that examines the linkage between local and global perceptual processing and explicit and implicit learning. This paradigm combines the global precedence paradigm of Navon (1977; 1981) and the sequence learning paradigm of Nissen and Bullemer (1987). Convincing arguments indicate that global stimuli are mediated more quickly via one perceptual stream (the M-cell pathway), but can be processed more slowly by another (the P-cell system). Local aspects of the stimuli are exclusively mediated by the P-cell system. The results of two experiments employing iterations of stimulus sequence, in which sequence learning is possible and measurable in terms of reaction time changes over trials are presented. The second experiment indicates that information thought to be mediated by the M-cell pathway results in incidental sequential learning, while other information thought to be mediated by the P-cell pathway does not. Spatial filtering of the visual stimuli reveals that low spatial frequencies are necessary for sequence learning to occur. The issue of whether this learning is implicit or explicit is also discussed. Ideas for future research, exploring this new area of interest, are proposed. Current knowledge of perceptual and memorial deficits in special populations are considered in an attempt to identify new areas of investigation.     

Automatic attention lateral asymmetry in visual discrimination tasks

There is evidence that automatic visual attention favors the right side. This study investigated whether this lateral asymmetry interacts with the right hemisphere dominance for visual location processing and left hemisphere dominance for visual shape processing. Volunteers were tested in a location discrimination task and a shape discrimination task. The target stimuli (S2) could occur in the left or right hemifield. They were preceded by an ipsilateral, contralateral or bilateral prime stimulus (S1). The attentional effect produced by the right S1 was larger than that produced by the left S1. This lateral asymmetry was similar between the two tasks suggesting that the hemispheric asymmetries of visual mechanisms do not contribute to it. The finding that it was basically due to a longer reaction time to the left S2 than to the right S2 for the contralateral S1 condition suggests that the inhibitory component of attention is laterally asymmetric.     

Tracking the influence of reflexive attention on sensory and cognitive processing

Previously, we demonstrated that reflexive attention facilitates early visual processing during form discrimination (Hopfinger & Mangun, 1998). In the present study, we tested whether reflexive facilitation of early visual processing will be generated when task load is low (simple luminance detection). Target stimuli that were preceded at short cue-to-target intervals by irrelevant visual events (cues) elicited an enhanced sensory (P1) event-related potential (ERP) component as well as an enhanced longer latency, cognitive ERP component (P300). At long cue-to-target intervals, facilitation in these ERP components was no longer observed, and, although inhibition of return (IOR) was observed in reaction times, the ERPs did not show an inhibition of sensory processing. These results provide converging evidence that reflexive attention transiently facilitates neural processing of visual inputs at multiple stages of analysis (i.e., sensory processing and higher order cognitive processing) but question the view that IOR is manifest at the earliest visual cortical stages of analysis.     

Visually induced gains in pitch discrimination: Linking audio-visual processing with auditory abilities

Perception is fundamentally a multisensory experience. The principle of inverse effectiveness (PoIE) states how the multisensory gain is maximal when responses to the unisensory constituents of the stimuli are weak. It is one of the basic principles underlying multisensory processing of spatiotemporally corresponding crossmodal stimuli that are well established at behavioral as well as neural levels. It is not yet clear, however, how modality-specific stimulus features influence discrimination of subtle changes in a crossmodally corresponding feature belonging to another modality. Here, we tested the hypothesis that reliance on visual cues to pitch discrimination follow the PoIE at the interindividual level (i.e., varies with varying levels of auditory-only pitch discrimination abilities). Using an oddball pitch discrimination task, we measured the effect of varying visually perceived vertical position in participants exhibiting a wide range of pitch discrimination abilities (i.e., musicians and nonmusicians). Visual cues significantly enhanced pitch discrimination as measured by the sensitivity index d’, and more so in the crossmodally congruent than incongruent condition. The magnitude of gain caused by compatible visual cues was associated with individual pitch discrimination thresholds, as predicted by the PoIE. This was not the case for the magnitude of the congruence effect, which was unrelated to individual pitch discrimination thresholds, indicating that the pitch-height association is robust to variations in auditory skills. Our findings shed light on individual differences in multisensory processing by suggesting that relevant multisensory information that crucially aids some perceivers’ performance may be of less importance to others, depending on their unisensory abilities.     

Multimodal signal detection: Independent decisions vs. integration

The pooling of information from simultaneous, spatially congruent auditory and visual stimuli is examined in a signal detection task. The paradigm used permits discrimination among a number of models of the decision mechanisms involved in processing multiple component stimuli. Parameter-free predictions are presented for the weighted integration model and for three versions of the independent decisions model. The data support an independent decisions model of the bimodal detection process in which attention is shared equally between modalities.     

HEMISPHERIC DIFFERENCES IN AUDITORY PERCEPTION ARE SIMILAR TO THOSE FOUND IN VISUAL PERCEPTION

Abstract— In a pitch discrimination task, subjects were faster and more accurate in judging low-frequency sounds when these stimuli were presented to the left ear, compared with the right ear. In contrast, a right-ear advantage was found with high-frequency sounds. The effect was in terms of relative frequency and not absolute frequency, suggesting that the effect arisen from pastsensory mechanisms. A simitar laterality effect has been reported in visual perception with stimuli varying in spatial frequency. These multimodal laterality effects may reflect a general computational difference between the two cerebral hemispheres, with the left hemisphere biased for processing high-frequency information and the right hemisphere biased for processing low-frequency information.     

Integration of information in visual search

Two experiments studied how information about the nontarget items in a visual search task is used for the control of the search. The first experiment used the detection of “hurdle” stimuli to demonstrate that efficient memory representations of the context items can be established within each particular trial. This finding is explained by a model for the short-term integration of context information. The second experiment which varied the complexity of the local but not the global context provided some information about the nature of the integration operations involved. In its final version the model postulates two stages of processing with independent mechanisms of integration. Spatial integration at the first stage deletes repetitions within small samples. Temporal integration at the second stage stores and primes the memory representations of the context items over larger intervals. It is assumed that transient temporal integration within trials is mediated by the same mechanism that underlies permanent temporal integration between trials.     

Visual binding of English and Chinese word parts is limited to low temporal frequencies

Some perceptual mechanisms manifest high temporal precision, allowing reports of visual information even when that information is restricted to windows smaller than 50 ms. Other visual judgments are limited to much coarser time scales. What about visual information extracted at late processing stages, for which we nonetheless have perceptual expertise, such as words? Here, the temporal limits on binding together visual word parts were investigated. In one trial, either the word 'ball' was alternated with 'deck', or 'dell' was alternated with 'back', with all stimuli presented at fixation. These stimuli restrict the time scale of the rod identities because the two sets of alternating words form the same image at high alternation frequencies. Observers made a forced choice between the two alternatives. Resulting 75% thresholds are restricted to 5 Hz or less for words and nonword letter strings. A similar result was obtained in an analogous experiment with Chinese participants viewing alternating Chinese characters. These results support the theory that explicit perceptual access to visual information extracted at late stages is limited to coarse time scales.     

Attention-Deficit Hyperactivity Disorder and Event-Related Potentials: Evidence for Deficits in Allocating Attentional Resources to Relevant Stimuli

Current research on the nature of attentional deficits in attention-deficit hyperactivity disorder (ADHD) is reviewed with a focus on studies using event-related potentials (ERPs). A robust effect is the smaller amplitude of the P3 wave to both auditory and visual stimuli for individuals with ADHD than for normal controls. This effect is indicative of deficits in the allocation of attentional resources to task-relevant information. The Nd wave is also smaller for individuals with ADHD, an effect that suggests impairments in the discrimination and preferential processing of task-relevant information, rather than distractibility by irrelevant stimuli. Studies of the effects of methylphenidate on ERPs in ADHD indicate that this stimulant enhances the quality of stimulus evaluation and speeds response time. These specific effects support the view that ADHD involves deficits in energetic processes that are required for processing task-relevant stimuli.     

Objects that induce face pareidolia are prioritized by the visual system

The human visual system has evolved specialized neural mechanisms to rapidly detect faces. Its broad tuning for facial features is thought to underlie the illusory perception of faces in inanimate objects, a phenomenon called face pareidolia. Recent studies on face pareidolia suggest that the mechanisms underlying face processing, at least at the early stages of visual encoding, may treat objects that resemble faces as real faces; prioritizing their detection. In our study, we used breaking continuous flash suppression (b-CFS) to examine whether the human visual system prioritizes the detection of objects that induce face pareidolia over stimuli matched for object content. Similar to previous b-CFS results using real face stimuli, we found that participants detected the objects with pareidolia faces faster than object-matched control stimuli. Given that face pareidolia has been more frequently reported amongst individuals prone to hallucinations, we also explored whether this rapid prioritization is intact in individuals with schizophrenia, and found evidence suggesting that it was. Our findings suggest that face pareidolia engages a broadly tuned mechanism that facilitates rapid face detection. This may involve the proposed fast subcortical pathway that operates outside of visual awareness.     

Neural correlates of acquired color category effects

Category training can induce category effects, whereby color discrimination of stimuli spanning a newly learned category boundary is enhanced relative to equivalently spaced stimuli from within the newly learned category (e.g., categorical perception). However, the underlying mechanisms of these acquired category effects are not fully understood. In the current study, Event-Related Potentials (ERPs) were recorded during a visual oddball task where standard and deviant colored stimuli from the same or different novel categories were presented. ERPs were recorded for a test group who were trained on these novel categories, and for an untrained control group. Category effects were only found for the test group on the trained region of color space, and only occurred during post-perceptual stages of processing. These findings provide new evidence for the involvement of cognitive mechanisms in acquired category effects and suggest that category effects of this kind can exist independent of early perceptual processes.     

Executive working memory load induces inattentional blindness

When attention is engaged in a task, unexpected events in the visual scene may go undetected, a phenomenon known as inattentional blindness (IB). At what stage of information processing must attention be engaged for IB to occur? Although manipulations that tax visuospatial attention can induce IB, the evidence is more equivocal for tasks that engage attention at late, central stages of information processing. Here, we tested whether IB can be specifically induced by central executive processes. An unexpected visual stimulus was presented during the retention interval of a working memory task that involved either simply maintaining verbal material or rearranging the material into alphabetical order. The unexpected stimulus was more likely to be missed during manipulation than during simple maintenance of the verbal information. Thus, the engagement of executive processes impairs the ability to detect unexpected, task-irrelevant stimuli, suggesting that IB can result from central, amodal stages of processing.     

The feeling of fluent perception: a single experience from multiple asynchronous sources

Zeki and co-workers recently proposed that perception can best be described as locally distributed, asynchronous processes that each create a kind of microconsciousness, which condense into an experienced percept. The present article is aimed at extending this theory to metacognitive feelings. We present evidence that perceptual fluency-the subjective feeling of ease during perceptual processing-is based on speed of processing at different stages of the perceptual process. Specifically, detection of briefly presented stimuli was influenced by figure-ground contrast, but not by symmetry (Experiment 1) or the font (Experiment 2) of the stimuli. Conversely, discrimination of these stimuli was influenced by whether they were symmetric (Experiment 1) and by the font they were presented in (Experiment 2), but not by figure-ground contrast. Both tasks however were related with the subjective experience of fluency (Experiments 1 and 2). We conclude that subjective fluency is the conscious phenomenal correlate of different processing stages in visual perception.     

Plasticity of the callosal system

A series of experiments examined the potential plasticity of the callosal system in both epileptic patients and in kittens submitted to corpus callosotomy at various ages. The patients were tested for unilateral discrimination and interhemispheric transfer of tactile information. The youngest patient was also required to perform additional inter- and intrahemispheric comparisons of visual and tactile stimuli. The animals were tested for interhemispheric transfer of visual discriminations. The results suggest that in both animals and humans there exists a critical period before which callosal section does not disrupt interhemispheric communication. The results also indicate that the compensatory mechanisms used to achieve interhemispheric transfer in the absence of the corpus callosum may vary according to the sensory modality involved. The possible physiological and/or functional mechanisms responsible for callosal plasticity are discussed.     

HIGH-FIDELITY PERCEPTUAL LONG-TERM MEMORY

Abstract— Visual long-term memory for spatial information was assessed by measuring discrimination thresholds for the spatial frequency of sinusoidal gratings in a delayed discrimination task, with various time intervals separating test and reference stimuli The results demonstrate perfect preservation of information across J-s to 50-hr retention tervals in the sense that the spatial frequency discrimination thresholds retained at the level defined by the spatial solution of the sensory analysis This high-fidelity spatial long-term memory may represent an elementary memory mechanism for precategorical storage of visual features     

Novelty responding and behavioral development in young,developmentally delayed children

This study examined whether or not a measure of information processing ability based on the discrimination of novel and familiar stimuli was related to behavioral development among developmentally-delayed infants. Two samples of handicapped infants were administered multiple measures of visual novelty discrimination and a battery of assessments which were representative of available measures of development in infancy. The results indicated that, as a group, the developmentally delayed infants were capable of discriminating novel and familiar stimuli. Also, correlational analyses indicated that responding to novelty was related to developmental accessment performance in both samples. This finding is consistent with previous data which indicates that novelty response measures are associated with important individual differences in young children.     

视听跨通道信息的整合与冲突控制

多通道信息交互是指来自某个感觉通道的信息与另一感觉通道的信息相互作用、相互影响的一系列加工过程。主要包括两个方面：一是不同感觉通道的输入如何整合;二是跨通道信息的冲突控制。本文综述了视听跨通道信息整合与冲突控制的行为心理机制和神经机制,探讨了注意对视听信息整合与冲突控制的影响。未来需探究视听跨通道信息加工的脑网络机制,考察特殊群体的跨通道整合和冲突控制以帮助揭示其认知和社会功能障碍的机制。