Dimensions of Speech Perception: Semantic Associations in the Affective Lexicon

The affective lexicon has been explained in terms of three underlying dimensions: Evaluation, Activity, and Potency. We assessed the importance of these dimensions during online speech perception. Participants made speeded lexical decisions about emotion words that were heard in a tone of voice that was either congruent or incongruent with the word's meaning. The denotative semantic category from which words were chosen was significantly related to lexical decision times (P < 0.001). Tone of voice did not influence decision times, nor did it interact with semantic category. Regression analyses showed that lexical decision times were significantly predicted by dimension weights on Potency, and by the three-way interaction between dimension weights on Evaluation, Activity, and Potency (both P s < 0.001). The implications of this study for models of knowledge representation and perception are discussed.     

Analyzing the aesthetic impressions of alexithymic Japanese students

We investigated the association between alexithymic tendencies as measured by the 20-item Toronto Alexithymia Scale and the characteristics of aesthetic impressions for words and drawings. Impressions were measured using the semantic differential technique in three groups of Japanese students (High scores n = 31, Moderate scores n = 40, and a Control group n = 182) who were presented only words (Word condition), only drawings (Drawing condition), or pairs comprising one word plus one drawing that represented the same emotional categories (Drawing/Word condition). In a factor analysis on the data from participants, three factors (Evaluation, Activity, and Potency) were extracted. Based on the factor scores, the distances among the three conditions were calculated for each of the groups. For Potency, significant group differences were found between the Drawing/Word and other conditions. In the High-scoring alexithymia group, in particular, the Potency impressions based on tactile sensation, e.g., soft-hard, blunt-sharp, were amplified regardless of stimulus condition. These results are discussed in the context of somatosensory amplification associated with alexithymia and the difficulty of distinguishing between emotion and somatic sensations.     

Integration of faces and voices, but not faces and names, in person recognition

Recent studies on cross-modal recognition suggest that face and voice information are linked for the purpose of person identification. We tested whether congruent associations between familiarized faces and voices facilitated subsequent person recognition relative to incongruent associations. Furthermore, we investigated whether congruent face and name associations would similarly benefit person identification relative to incongruent face and name associations. Participants were familiarized with a set of talking video-images of actors, their names, and their voices. They were then tested on their recognition of either the face, voice, or name of each actor from bimodal stimuli which were either congruent or novel (incongruent) associations between the familiarized face and voice or face and name. We found that response times to familiarity decisions based on congruent face and voice stimuli were facilitated relative to incongruent associations. In contrast, we failed to find a benefit for congruent face and name pairs. Our findings suggest that faces and voices, but not faces and names, are integrated in memory for the purpose of person recognition. These findings have important implications for current models of face perception and support growing evidence for multisensory effects in face perception areas of the brain for the purpose of person recognition.     

Applying the attractor field model to social cognition: Perceptual discrimination is facilitated, but memory is impaired for faces displaying evaluatively congruent expressions

A new model of mental representation is applied to social cognition: the attractor field model. Using the model, the authors predicted and found a perceptual advantage but a memory disadvantage for faces displaying evaluatively congruent expressions. In Experiment 1, participants completed a same/different perceptual discrimination task involving morphed pairs of angry-to-happy Black and White faces. Pairs of faces displaying evaluatively incongruent expressions (i.e., happy Black, angry White) were more likely to be labeled as similar and were less likely to be accurately discriminated from one another than faces displaying evaluatively congruent expressions (i.e., angry Black, happy White). Experiment 2 replicated this finding and showed that objective discriminability of stimuli moderated the impact of attractor field effects on perceptual discrimination accuracy. In Experiment 3, participants completed a recognition task for angry and happy Black and White faces. Consistent with the attractor field model, memory accuracy was better for faces displaying evaluatively incongruent expressions. Theoretical and practical implications of these findings are discussed.     

The comparability of verbal and nonverbal semantic differential scales in a joint semantic space

As an alternative to the verbal Semantic Differential (SD) measure of affective meaning systems, a nonverbal instrument-the Graphic Differential (GD)-has been developed for subject populations who have difficulty in verbal ratings. However, the factor structures of the pictographic scales have never been examined in a joint-verbal and nonverbal-semantic space as to whether their structures are equivalent to the affective systems defined by the verbal markers. The present study assessed this issue by reanalyzing the French (1977) data. The results indicated that when verbal SD and nonverbal GD scales were factor-analyzed separately, the three-dimensional structures of Evaluation (E), Potency (P), and Activity (A) were recaptured in both domains. However, when they were analyzed simultaneously in a joint semantic space, only the E and A dimensions were congruent. The GD P scales carried the meanings of denotative Heaviness, rather than the affective Potency. For future investigation, suggestion was made to systematically study both the affective and denotative semantic features within the contexts of nonverbal communciations.The authors made equal contributions toward the completion of the present research.     

Does matching of internal and external facial features depend on orientation and viewpoint?

Although it is recognized that external (hair, head and face outline, ears) and internal (eyes, eyebrows, nose, mouth) features contribute differently to face recognition it is unclear whether both feature classes predominately stimulate different sensory pathways. We employed a sequential speed-matching task to study face perception with internal and external features in the context of intact faces, and at two levels of contextual congruency. Both internal and external features were matched faster and more accurately in the context of totally congruent/incongruent facial stimuli compared to just featurally congruent/incongruent faces. Matching of totally congruent/incongruent faces was not affected by the matching criteria, but was strongly modulated by orientation and viewpoint. On the contrary, matching of just featurally congruent/incongruent faces was found to depend on the feature class to be attended, with strong effects of orientation and viewpoint only for matching of internal features, but not of external features. The data support the notion that different processing mechanisms are involved for both feature types, with internal features being handled by configuration sensitive mechanisms whereas featural processing modes dominate when external features are the focus.     

Probing Representations of Gymnastics Movements: A Visual Priming Study

In this study, we designed a visual short‐term priming paradigm to investigate the mechanisms underlying the priming of movements and to probe movement representations in motor experts and matched controls. We employed static visual stimuli that implied or not human whole‐body movements, that is, gymnastics movements and static positions. Twelve elite female gymnasts and twelve matched controls performed a speeded two‐choice response time task. The participants were presented with congruent and incongruent prime‐target pairs and had to decide whether the target stimulus represented a gymnastics movement or a static position. First, a visual priming effect was observed in the two groups. Second, a stimulus–response rote association could not easily account for our results. Novel primes never presented as targets could also prime the targets. Third, by manipulating three levels of prime‐target relations in moving congruent pairs, we demonstrated that the more similar prime‐target pairs, the greater the facilitation in target. Lastly, gymnastics motor expertise impacted on priming effects.     

Phonology is fundamental in skilled reading: Evidence from ERPs

Research consistently indicates the importance of phonological processing in early reading development, yet the role of phonology in skilled reading is still not well understood. Two event-related potential (ERP) experiments investigated the nature and time course of phonological processing during skilled visual word recognition using a masked priming paradigm. Phonological syllable priming was examined by presenting prime-target pairs either with the same first syllable, or with one letter more or fewer. In this visually matched design, items like po## -PONY and pon### -PONDER appeared in the congruent condition. Conversely, pon# -PONY and po#### -PONDER appeared in the incongruent condition. In both experiments, the magnitude of the first negative peak (N1) was reduced in the phonologically congruent condition as compared to the incongruent condition. This syllable congruency effect is the first neurophysiological evidence for phonological syllable activation in the initial moments of visual word recognition. The early time course of this activation indicates that suprasegmental phonological processing is fundamental to skilled reading.     

Contextual mood priming following left and right hemisphere damage

This research examined the influence of mood-congruent and mood-incongruent contexts on recognizing affective prosody after brain damage. Predictions stemmed from an associative network theory of learning and memory. Thirty-three male subjects, 11 each in right hemisphere damaged (RHD), left hemisphere damaged (LHD), and normal control groups judged moods from the prosody of semantically neutral phrases. In one task, the prosodic stimulus phrases were judged in isolation. In another task, the phrases were preceded by short paragraphs which were either congruent or incongruent in emotional tone with the prosodic stimuli. These paragraphs were designed to prime specific mood choices. As anticipated, LHD subjects' prosodic mood recognition was more accurate when given congruent rather than incongruent affective contexts. Congruent contexts facilitated, and incongruent contexts disrupted, their prosodic mood judgments to the same extent as normals. RHD subjects showed a partial context decrement. They were less accurate than normal or LHD subjects in the congruent condition, and were unaffected by incongruent contexts. When given congruent biasing paragraphs, however, RHD subjects did experience facilitation on a par with that found for the other groups, indicating spared sensitivity to certain contextual factors. The distinction between automatic and effortful processes is offered as a potential explanation for the RHD group's pattern of performance.     

Effects of horizontal viewing angle on visual and audiovisual speech recognition.

The authors investigated the effects of changes in horizontal viewing angle on visual and audiovisual speech recognition in 4 experiments, using a talker's face viewed full face, three quarters, and in profile. When only experimental items were shown (Experiments 1 and 2), identification of unimodal visual speech and visual speech influences on congruent and incongruent auditory speech were unaffected by viewing angle changes. However, when experimental items were intermingled with distractor items (Experiments 3 and 4), identification of unimodal visual speech decreased with profile views, whereas visual speech influences on congruent and incongruent auditory speech remained unaffected by viewing angle changes. These findings indicate that audiovisual speech recognition withstands substantial changes in horizontal viewing angle, but explicit identification of visual speech is less robust. Implications of this distinction for understanding the processes underlying visual and audiovisual speech recognition are discussed.     

Crossmodal integration of emotional information from face and voice in the infant brain

We examined 7-month-old infants' processing of emotionally congruent and incongruent face-voice pairs using ERP measures. Infants watched facial expressions (happy or angry) and, after a delay of 400 ms, heard a word spoken with a prosody that was either emotionally congruent or incongruent with the face being presented. The ERP data revealed that the amplitude of a negative component and a subsequent positive component in infants' ERPs varied as a function of crossmodal emotional congruity. An emotionally incongruent prosody elicited a larger negative component in infants' ERPs than did an emotionally congruent prosody. Conversely, the amplitude of infants' positive component was larger to emotionally congruent than to incongruent prosody. Previous work has shown that an attenuation of the negative component and an enhancement of the later positive component in infants' ERPs reflect the recognition of an item. Thus, the current findings suggest that 7-month-olds integrate emotional information across modalities and recognize common affect in the face and voice.     

A Study of the affective responses elicited by occupational stimuli

In this study the semantic differential was used to assess the properties of affect elicited by occupational stimuli. Vocationally committed men studying medicine (N = 119), business (N = 120), and engineering (N = 83) responded to a semantic differential containing occupational concepts representing the fields of medicine, business, engineering, the clergy, and law enforcement. A factor analysis of the data yielded a semantic space for all three groups composed of three orthogonal dimensions of affect called Evaluation, Activity, and Potency, in that order of salience. Tests of factorial similarity indicated that the semantic spaces of the three groups were virtually identical. It was concluded that occupational stimuli elicit Evaluative, Activity, and Potency responses, and that all three affective dimensions should be used when predicting occupational behavior.     

Visual and audiovisual speech perception with color and gray-scale facial images

Research has shown that auditory speech recognition is influenced by the appearance of a talker's face, but the actual nature of this visual information has yet to be established. Here, we report three experiments that investigated visual and audiovisual speech recognition using color, gray-scale, and point-light talking faces (which allowed comparison with the influence of isolated kinematic information). Auditory and visual forms of the syllables /ba/, /bi/, /ga/, /gi/, /va/, and /vi/ were used to produce auditory, visual, congruent, and incongruent audiovisual speech stimuli. Visual speech identification and visual influences on identifying the auditory components of congruent and incongruent audiovisual speech were identical for color and gray-scale faces and were much greater than for point-light faces. These results indicate that luminance, rather than color, underlies visual and audiovisual speech perception and that this information is more than the kinematic information provided by point-light faces. Implications for processing visual and audiovisual speech are discussed.     

Using art for comparison and distraction: Effects on negative emotions and judgements of satisfaction

Abstract

Two studies examined whether emotional comparison and distraction with emotion congruent and incongruent art would improve the well-being of dysphoric undergraduates. In both studies, subjects: (1) imagined a sad event; (2) compared their mood to that expressed by incongruent art (upward comparison) or congruent art (downward comparison); or focused on technical features of incongruent art (incongruent distraction) or congruent art (congruent distraction); and (3) rated their emotions and life satisfaction. The incongruent distraction group reported feeling more positive and more satisfied, and the downward comparison group reported feeling more satisfied, than the upward comparison or congruent distraction groups. Thus, comparison and distraction can improve well-being when directed towards emotion congruent and incongruent art, respectively.     

Stimulus-outcome interactions during instrumental discrimination learning by rats and humans

The associative structure mediating goal-directed action was investigated using congruent and incongruent conditional discriminations. The stimulus was the same as the outcome in each component of the congruent discriminations, whereas the stimulus of one component of the incongruent discriminations was the same as the outcome of the other component. Humans, but not rats, learned the congruent discrimination more rapidly than the incongruent discrimination, a difference that the authors attribute to the fact that outcome-response associations caused response conflict in the incongruent discrimination. Moreover, responding was resistant to outcome devaluation following incongruent, but not congruent, training, suggesting that both humans and rats adopted a stimulus-response strategy to resolve the incongruent discrimination.     

The impact of perceptual, cognitive and motor factors on bimanual coordination

Bimanual coordination is governed by constraints that permit congruent movements to be performed more easily than incongruent movements. Theories concerning the origin of these constraints range from low level motor-muscle explanations to high level perceptual–cognitive ones. To elucidate the processes underlying coordinative constraints, we asked subjects to use a pair of left–right joysticks to acquire corresponding pairs of congruent and incongruent targets presented on a video monitor under task conditions designed to systematically modulate the impact of several perceptual–cognitive processes commonly required for bimanual task performance. These processes included decoding symbolic cues, detecting goal targets, conceptualizing movements in terms of goal target configuration, planning movement trajectories, producing saccades and perceiving visual feedback. Results demonstrate that constraints arise from target detection and trajectory planning processes that can occur prior to movement initiation as well as from inherent muscle properties that emerge during movement execution, and that the manifestation of these constraints can be significantly altered by the ability to visually monitor movement progress.     

School children's responses on a semantic differential over a 10-year span

A semantic differential scale was administered to 208 school children when they were in the second, fourth, sixth, eighth, tenth, and twelfth grades. Their perceptions towards two concepts were measured, Education (going to school) and Work (having a job). Each semantic differential scale had 15 adjective pairs and reflected the three underlying factors of Evaluative, Potency, and Activity. Because the study was conducted for 10 years (ages seven to 18), the changing cognitive developmental stages of the children were expected to influence factor analytic and reliability results. Confirmatory factor analysis, which forced the data into three factors, did not clearly identify the expected three factors, although more items loaded on the three factors with age. An exploratory factor analysis identified a trend across grades from six to four factors over time. Reliability also improved across age groups. Caution should be exercised when using the semantic differential with young children in investigations of abstract concepts.