Perceiving speech from inverted faces

We examined whether the orientation of the face influences speech perception in face-to-face communication. Participants identified auditory syllables, visible syllables, and bimodal syllables presented in an expanded factorial design. The syllables were /ba/, /va/, /δa/, or /da/. The auditory syllables were taken from natural speech whereas the visible syllables were produced by computer animation of a realistic talking face. The animated face was presented either as viewed in normal upright orientation or inverted orientation (180° frontal rotation). The central intent of the study was to determine if an inverted view of the face would change the nature of processing bimodal speech or simply influence the information available in visible speech. The results with both the upright and inverted face views were adequately described by the fuzzy logical model of perception (FLMP). The observed differences in the FLMP’s parameter values corresponding to the visual information indicate that inverting the view of the face influences the amount of visible information but does not change the nature of the information processing in bimodal speech perception     

La langue et les Lèvres: cross-language influences on bimodal speech perception.

Previous research in speech perception has yielded two sets of findings which are brought together in the present study. First, it has been shown that normal hearing listeners use visible as well as acoustical information when processing speech. Second, it has been shown that there is an effect of specific language experience on speech perception such that adults often have difficulty identifying and discriminating non-native phones. The present investigation was designed to extend and combine these two sets of findings. Two studies were conducted using six consonant-vowel syllables (/ba/, /va/, /alpha a/, /da/, /3a/, and /ga/ five of which occur in French and English, and one (the interdental fricative /alpha a/) which occurs only in English. In Experiment 1, an effect of specific linguistic experience was evident for the auditory identification of the non-native interdental stimulus by French-speakers. In Experiment 2, it was shown that the effect of specific language experience extends to the perception of the visible information in speech. These findings are discussed in terms of their implications for our understanding of cross-language processes in speech perception and for our understanding of the development of bimodal speech perception.     

Processing steps in perception of speech sounds]

The aim of this study is to investigate whether speech sounds--as is stated by the widely accepted theory of categorical perception of speech--can be perceived only as instances of phonetic categories, or whether physical differences between speech sounds lead to perceptual differences regardless of their phonetic categorization. Subjects listened to pairs of synthetically generated speech sounds that correspond to realizations of the syllables "ba" and "pa" in natural German, and they were instructed to decide as fast as possible whether they perceived them as belonging to the same or to different phonetic categories. For 'same'-responses reaction times become longer when the physical distance between the speech sounds is increased; for 'different'-responses reaction times become shorter with growing physical distance between the stimuli. The results show that subjects can judge speech sounds on the basis of perceptual continua, which is inconsistent with the theory of categorical perception. A mathematical model is presented that attempts to explain the results by postulating two interacting stages of processing, a psychoacoustical and a phonetic one. The model is not entirely confirmed by the data, but it seems to deserve further consideration.     

Potential perceptual bases for successful use of a vibrotactile speech perception aid

This paper examines the possibility that perception of vibrotactile speech stimuli is enhanced in adults with early and life-long use of hearing aids. We present evidence that vibrotactile aid benefit in adults is directly related to the age at which the hearing aid was fitted and the duration of its use. The stimulus mechanism responsible for this effect is hypothesized to be long-term vibrotactile stimulation by high powered hearing aids. We speculate on possible mechanisms for enhanced vibrotactile speech perception as the result of hearing aid use: (1) long-term experience receiving degraded or impoverished speech stimuli results in a speech processing system that is more effective for novel stimuli, independent of perceptual modality; and/or (2) long-term sensory/perceptual experience causes neural changes that result in more effective delivery of speech information via somatosensory pathways.     

自闭症者对音乐和言语音高的加工

自闭症是一种神经发展障碍,主要表现为社会互动障碍,语言交流困难以及刻板行为等症状。已有研究表明,在音乐方面,自闭症者不仅表现出较强的音乐音高知觉能力,而且在音乐表演方面也体现出某些优势。然而,在言语方面,自闭症者除了在言语音高轮廓分辨任务中得分较高外,对言语语调的知觉能力明显比正常人更差,同时,他们对言语语调的产生也存在障碍。本研究不仅可以推进音乐和言语对比研究,而且也为自闭症者言语康复提供借鉴。     

Perceptual awareness and categorical representation of faces: evidence from masked priming

How internal categories influence how we perceive the world is a fundamental question in cognitive sciences. Yet, the relation between perceptual awareness and perceptual categorization has remained largely uncovered so far. Here, we addressed this question by focusing on face perception during subliminal and conscious perception. We used morphed continua between two face identities and we assessed, through a masked priming paradigm, the perceptual processing of these morphed faces under subliminal and supraliminal conditions. We found that priming from subliminal faces followed linearly the information present in the primes, while priming from visible faces revealed a non-linear profile, indicating a categorical processing of face identities. Our results thus point to a special relation between perceptual awareness and categorical processing of faces, and support the dissociation between two modes of information processing: a subliminal mode involving analog treatment of stimuli information, and a supraliminal mode relying on discrete representation.     

Bad things come easier to the mind but harder to the body: Evidence from brain oscillations

An intriguing finding of research on emotional processing is a discrepancy between perception and behavior. Perceptually, a robust finding is that negative stimuli are processed faster and more efficiently than positive stimuli. Behaviorally, a similarly robust finding is that response times are slower for negative than for positive stimuli. We proposed and tested a novel account to explain this still unexplained discrepancy, on the basis of the assumption that negative valence narrows perceptual processes to the benefit of speeded perception, but broadens motor processes at the cost of slowed responding. Participants performed a valence judgment task in which they responded with their left or right hand to negative and positive stimuli that were presented on the left or right, and we measured the activation of relevant/deactivation of irrelevant perceptual and motor processes, as revealed by the lateralization of electroencephalographic brain oscillations. Stimulus-related lateralization of alpha activity (8–12 Hz) over perceptual areas was increased for negative stimuli, indicating more efficient perceptual processing. By contrast, response-related lateralization of beta activity (20–25 Hz) over motor areas was decreased for negative stimuli, indicating less efficient response activation. Consistent with our predictions, more detailed analyses showed that both lateralization effects were caused by dynamics at the level of inhibiting irrelevant processes. For negative as compared to positive stimuli, the inhibition of irrelevant perceptual processes was increased, but the inhibition of irrelevant motor processes was decreased. These findings indicate that the discrepancy between perception and behavior in emotional processing may stem from asymmetrical effects of emotional valence on the breadth of cortical activations in perceptual and motor networks.     

Action-dependent perceptual invariants: from ecological to sensorimotor approaches

Ecological and sensorimotor theories of perception build on the notion of action-dependent invariants as the basic structures underlying perceptual capacities. In this paper we contrast the assumptions these theories make on the nature of perceptual information modulated by action. By focusing on the question, how movement specifies perceptual information, we show that ecological and sensorimotor theories endorse substantially different views about the role of action in perception. In particular we argue that ecological invariants are characterized with reference to transformations produced in the sensory array by movement: such invariants are transformation-specific but do not imply motor-specificity. In contrast, sensorimotor theories assume that perceptual invariants are intrinsically tied to specific movements. We show that this difference leads to different empirical predictions and we submit that the distinction between motor equivalence and motor-specificity needs further clarification in order to provide a more constrained account of action/perception relations.     

Developmental course of auditory processing interactions: Garner interference and Simon interference.

Previous research suggests that with increasing age children become more efficient in inhibiting conflicting responses and in resisting interference from irrelevant information. We assessed the abilities of 100 children (ages 3-16 years) and 20 adults to resist interference during the processing of 2 auditory dimensions of speech, namely the speaker's gender and spatial location. The degree of interference from irrelevant variability in either dimension did not vary with age. Apparently, young children do not have more difficulty in resisting interference when the nontarget and the target are both perceptual attributes. We also assessed the participants' abilities to inhibit conflicting task-irrelevant information from spatial location and to resist interference from spatial variability in the context of conflict. In the presence of conflicting task-irrelevant information, both interference effects declined significantly with age. Developmental change in auditory processing seems to vary as a function of (1) the nature of the target-nontarget combination and (2) the presence/absence of conflicting task-irrelevant information.     

Load theory of selective attention and cognitive control

A load theory of attention in which distractor rejection depends on the level and type of load involved in current processing was tested. A series of experiments demonstrates that whereas high perceptual load reduces distractor interference, working memory load or dual-task coordination load increases distractor interference. These findings suggest 2 selective attention mechanisms: a perceptual selection mechanism serving to reduce distractor perception in situations of high perceptual load that exhaust perceptual capacity in processing relevant stimuli and a cognitive control mechanism that reduces interference from perceived distractors as long as cognitive control functions are available to maintain current priorities (low cognitive load). This theory resolves the long-standing early versus late selection debate and clarifies the role of cognitive control in selective attention.     

动允直接知觉的共鸣原理

传统认知理论认为我们只能间接知觉动允,而生态认知心理学则认为对动允的知觉是个体与环境信息直接共鸣的过程。生态认知心理学首先判定动允并不是存在于我们头脑中的认识,而是实际存在于个体-环境系统中的事实,对动允的直接知觉就是对动允的检测而非推测,是对动允信息的拾取而非对记忆内容的提取。这种直接知觉表现为身体的相关动作机制（动作神经元、肌肉组织等）与环境中动允信息的共鸣。进化与学习在共鸣的形成与发展中具有重要作用。     

Perceptual narrowing in face- and speech-perception domains in infancy: A longitudinal approach

During the first year of life, infants undergo a process known as perceptual narrowing, which reduces their sensitivity to classes of stimuli which the infants do not encounter in their environment. It has been proposed that perceptual narrowing for faces and speech may be driven by shared domain-general processes. To investigate this theory, our study longitudinally tested 50 German Caucasian infants with respect to these domains first at 6 months of age followed by a second testing at 9 months of age. We used an infant-controlled habituation-dishabituation paradigm to test the infants’ ability to discriminate among other-race Asian faces and non-native Cantonese speech tones, as well as same-race Caucasian faces as a control. We found that while at 6 months of age infants could discriminate among all stimuli, by 9 months of age they could no longer discriminate among other-race faces or non-native tones. However, infants could discriminate among same-race stimuli both at 6 and at 9 months of age. These results demonstrate that the same infants undergo perceptual narrowing for both other-race faces and non-native speech tones between the ages of 6 and 9 months. This parallel development of perceptual narrowing occurring in both the face and speech perception modalities over the same period of time lends support to the domain-general theory of perceptual narrowing in face and speech perception.     

Dilution: atheoretical burden or just load? A reply to Tsal and Benoni (2010)

Load theory of attention proposes that distractor processing is reduced in tasks with high perceptual load that exhaust attentional capacity within task-relevant processing. In contrast, tasks of low perceptual load leave spare capacity that spills over, resulting in the perception of task-irrelevant, potentially distracting stimuli. Tsal and Benoni (2010) find that distractor response competition effects can be reduced under conditions with a high search set size but low perceptual load (due to a singleton color target). They claim that the usual effect of search set size on distractor processing is not due to attentional load but instead attribute this to lower level visual interference. Here, we propose an account for their findings within load theory. We argue that in tasks of low perceptual load but high set size, an irrelevant distractor competes with the search nontargets for remaining capacity. Thus, distractor processing is reduced under conditions in which the search nontargets receive the spillover of capacity instead of the irrelevant distractor. We report a new experiment testing this prediction. Our new results demonstrate that, when peripheral distractor processing is reduced, it is the search nontargets nearest to the target that are perceived instead. Our findings provide new evidence for the spare capacity spillover hypothesis made by load theory and rule out accounts in terms of lower level visual interference (or mere "dilution") for cases of reduced distractor processing under low load in displays of high set size. We also discuss additional evidence that discounts the viability of Tsal and Benoni's dilution account as an alternative to perceptual load.     

Levels of processing in speech perception

Four experiments are reported investigating previous findings that speech perception interferes with concurrent verbal memory but difficult nonverbal perceptual tasks do not, to any great degree. The forgetting produced by processing noisy speech could not be attributed to task difficulty, since equally difficult nonspeech tasks did not produce forgetting, and the extent of forgetting produced by speech could be manipulated independently of task difficulty. The forgetting could not be attributed to similarity between memory material and speech stimuli, since clear speech, analyzed in a simple and probably acoustically mediated discrimination task, produced little forgetting. The forgetting could not be attributed to a combination of similarity and difficutly since a very easy speech task involving clear speech produced as much forgetting as noisy speech tasks, as long as overt reproduction of the stimuli was required. By assuming that noisy speech and overtly reproduced speech are processed at a phonetic level but that clear, repetitive speech can be processed at a purely acoustic level, the forgetting produced by speech perception could be entirely attributed to the level at which the speech was processed. In a final experiment, results were obtained which suggest that if prior set induces processing of noisy and clear speech at comparable levels, the difference between the effects of noisy speech processing and clear speech processing on concurrent memory is completely eliminated.     

Perceptual equivalence of acoustic cues in speech and nonspeech perception

Trading relations show that diverse acoustic consequences of minimal contrasts in speech are equivalent in perception of phonetic categories. This perceptual equivalence received stronger support from a recent finding that discrimination was differentially affected by the phonetic cooperation or conflict between two cues for the /slIt/-/splIt/contrast. Experiment 1 extended the trading relations and perceptual equivalence findings to the /sei/-/stei/contrast. With a more sensitive discrimination test, Experiment 2 found that cue equivalence is a characteristic of perceptual sensitivity to phonetic information. Using “sine-wave analogues” of the /sei/-/stei/stimuli, Experiment 3 showed that perceptual integration of the cues was phonetic, not psychoacoustic, in origin. Only subjects who perceived the sine-wave stimuli as “say” and “stay” showed a trading relation and perceptual equivalence; subjects who perceived them as nonspeech failed to integrate the two dimensions perceptually. Moreover, the pattern of differences between obtained and predicted discrimination was quite similar across the first two experiments and the “say”-“stay” group of Experiment 3, and suggested that phonetic perception was responsible even for better-than-predicted performance by these groups. Trading relations between speech cues, and the perceptual equivalence that underlies them, thus appear to derive specifically from perception of phonetic information.     

Characterizing the influence of native language experience on adult speech perception.

Previous cross-language research has indicated that some speech contrasts present greater perceptual difficulty for adult non-native listeners than others do. It has been hypothesized that phonemic, phonetic, and acoustic factors contribute to this variability. Two experiments were conducted to evaluate systematically the role of phonemic status and phonetic familiarity in the perception of non-native speech contrasts and to test predictions derived from a model proposed by Best, McRoberts, and Sithole (1988). Experiment 1 showed that perception of an unfamiliar phonetic contrast was not less difficult for subjects who had experience with an analogous phonemic distinction in their native language than for subjects without such analogous experience. These results suggest that substantive phonetic experience influences the perception of non-native contrasts, and thus should contribute to a conceptualization of native language-processing skills. In Experiment 2, English listeners' perception of two related nonphonemic place contrasts was not consistently different as had been expected on the basis of phonetic familiarity. A clear order effect in the perceptual data suggests that interactions between different perceptual assimilation patterns or acoustic properties of the two contrasts, or interactions involving both of these factors, underlie the perception of the two contrasts in this experiment. It was concluded that both phonetic familiarity and acoustic factors are potentially important to the explanation of variability in perception of nonphonemic contrasts. The explanation of how linguistic experience shapes speech perception will require characterizing the relative contribution of these factors, as well as other factors, including individual differences and variables that influence a listener's orientation to speech stimuli.     

非言语声音影响汉语听者言语声音的知觉

采用启动范式, 以汉语听者为被试, 考察了非言语声音是否影响言语声音的知觉。实验1考察了纯音对辅音范畴连续体知觉的影响, 结果发现纯音影响到辅音范畴连续体的知觉, 表现出频谱对比效应。实验2考察了纯音和复合音对元音知觉的影响, 结果发现与元音共振峰频率一致的纯音或复合音加快了元音的识别, 表现出启动效应。两个实验一致发现非言语声音能够影响言语声音的知觉, 表明言语声音知觉也需要一个前言语的频谱特征分析阶段, 这与言语知觉听觉理论的观点一致。     

Visual contribution to the multistable perception of speech

The multistable perception of speech, or verbal transformation effect, refers to perceptual changes experienced while listening to a speech form that is repeated rapidly and continuously. In order to test whether visual information from the speaker's articulatory gestures may modify the emergence and stability of verbal auditory percepts, subjects were instructed to report any perceptual changes during unimodal, audiovisual, and incongruent audiovisual presentations of distinct repeated syllables. In a first experiment, the perceptual stability of reported auditory percepts was significantly modulated by the modality of presentation. In a second experiment, when audiovisual stimuli consisting of a stable audio track dubbed with a video track that alternated between congruent and incongruent stimuli were presented, a strong correlation between the timing of perceptual transitions and the timing of video switches was found. Finally, a third experiment showed that the vocal tract opening onset event provided by the visual input could play the role of a bootstrap mechanism in the search for transformations. Altogether, these results demonstrate the capacity of visual information to control the multistable perception of speech in its phonetic content and temporal course. The verbal transformation effect thus provides a useful experimental paradigm to explore audiovisual interactions in speech perception.     

The Impact and Status of Carol Fowler's Supramodal Theory of Multisensory Speech Perception

ABSTRACT

One important contribution of Carol Fowler's direct approach to speech perception is its account of multisensory perception. This supramodal account proposes a speech function that detects supramodal information available across audition, vision, and touch. This detection allows for the recovery of articulatory primitives that provide the basis of a common currency shared between modalities as well as between perception and production. Common currency allows for perceptual experience to be shared between modalities and supports perceptually guided speaking as well as production-guided perception. In this report, we discuss the contribution and status of the supramodal approach relative to recent research in multisensory speech perception. We argue that the approach has helped motivate a multisensory revolution in perceptual psychology. We then review the new behavioral and neurophysiological research on (a) supramodal information, (b) cross-sensory sharing of experience, and (c) perceptually guided speaking as well as production guided speech perception. We conclude that Fowler's supramodal theory has fared quite well in light of this research.     

Perceptual interference at encoding enhances item-specific encoding and disrupts relational encoding: Evidence from multiple recall tests

Interfering with perception during encoding can enhance later memory, a phenomenon known as the perceptual interference effect. This effect is investigated in the context of the item-specific-relational framework (e.g., Hunt & McDaniel, 1993), which suggests that the perceptual interference enhances item-specific encoding and impedes relational encoding. Two experiments performed with multiple recall tests support this view. Prior research indicates that item-specific processing increases item gains across tests, whereas relational processing protects against item losses (e.g., Burns, 1993; Klein, Loftus, Kihlstrom, & Aseron, 1989). Consistent with the item-specific-relational framework, perceptual interference produced significant increases in both item gains and losses relative to a control condition.