Acquisition of auditory–visual intermodal matching-to-sample by a chimpanzee (Pan troglodytes): comparison with visual–visual intramodal matching

A chimpanzee acquired an auditory–visual intermodal matching-to-sample (AVMTS) task, in which, following the presentation of a sample sound, the subject had to select from two alternatives a photograph that corresponded to the sample. The acquired AVMTS performance might shed light on chimpanzee intermodal cognition, which is one of the least understood aspects in chimpanzee cognition. The first aim of this paper was to describe the training process of the task. The second aim was to describe through a series of experiments the features of the chimpanzee AVMTS performance in comparison with results obtained in a visual intramodal matching task, in which a visual stimulus alone served as the sample. The results show that the acquisition of AVMTS was facilitated by the alternation of auditory presentation and audio-visual presentation (i.e., the sample sound together with a visual presentation of the object producing the particular sample sound). Once AVMTS performance was established for the limited number of stimulus sets, the subject showed rapid transfer of the performance to novel sets. However, the subject showed a steep decay of matching performance as a function of the delay interval between the sample and the choice alternative presentations when the sound alone, but not the visual stimulus alone, served as the sample. This might suggest a cognitive limitation for the chimpanzee in auditory-related tasks. Accepted after revision: 11 September 2001 Electronic Publication     

Perceptual adaptation to non-native speech

This study investigated talker-dependent and talker-independent perceptual adaptation to foreign-accent English. Experiment 1 investigated talker-dependent adaptation by comparing native English listeners' recognition accuracy for Chinese-accented English across single and multiple talker presentation conditions. Results showed that the native listeners adapted to the foreign-accented speech over the course of the single talker presentation condition with some variation in the rate and extent of this adaptation depending on the baseline sentence intelligibility of the foreign-accented talker. Experiment 2 investigated talker-independent perceptual adaptation to Chinese-accented English by exposing native English listeners to Chinese-accented English and then testing their perception of English produced by a novel Chinese-accented talker. Results showed that, if exposed to multiple talkers of Chinese-accented English during training, native English listeners could achieve talker-independent adaptation to Chinese-accented English. Taken together, these findings provide evidence for highly flexible speech perception processes that can adapt to speech that deviates substantially from the pronunciation norms in the native talker community along multiple acoustic-phonetic dimensions.     

汉语母语者视听言语知觉发展趋势研究

6~12岁儿童、13~18岁青少年和20~30岁成人被试各30名,运用McGurk效应研究范式对汉语母语者视听言语知觉的发展趋势进行探讨。所有被试需要接受纯听和视听两种条件下的测试,其任务是出声报告自己听到的刺激。结果发现：（1）三个年龄阶段汉语母语者被试在安静听力环境下的单音节加工中都受到了视觉线索的影响,表现出了McGurk效应;（2）三个年龄阶段汉语母语者被试McGurk效应的强度存在显著差异,其受视觉言语影响的程度表现出随年龄增长而增强的发展趋势;（3）13岁以后汉语被试在视听一致下对视觉线索的依赖没有显著增强,但是在视听冲突下视觉言语的影响仍然在逐渐增强。     

Changes in auditory frequency guide visual-spatial attention

How do the characteristics of sounds influence the allocation of visual–spatial attention? Natural sounds typically change in frequency. Here we demonstrate that the direction of frequency change guides visual–spatial attention more strongly than the average or ending frequency, and provide evidence suggesting that this cross-modal effect may be mediated by perceptual experience. We used a Go/No-Go color-matching task to avoid response compatibility confounds. Participants performed the task either with their heads upright or tilted by 90°, misaligning the head-centered and environmental axes. The first of two colored circles was presented at fixation and the second was presented in one of four surrounding positions in a cardinal or diagonal direction. Either an ascending or descending auditory-frequency sweep was presented coincident with the first circle. Participants were instructed to respond to the color match between the two circles and to ignore the uninformative sounds. Ascending frequency sweeps facilitated performance (response time and/or sensitivity) when the second circle was presented at the cardinal top position and descending sweeps facilitated performance when the second circle was presented at the cardinal bottom position; there were no effects of the average or ending frequency. The sweeps had no effects when circles were presented at diagonal locations, and head tilt entirely eliminated the effect. Thus, visual–spatial cueing by pitch change is narrowly tuned to vertical directions and dominates any effect of average or ending frequency. Because this cross-modal cueing is dependent on the alignment of head-centered and environmental axes, it may develop through associative learning during waking upright experience.     

The link between speech perception and production is phonological and abstract: evidence from the shadowing task

This study reports a shadowing experiment, in which one has to repeat a speech stimulus as fast as possible. We tested claims about a direct link between perception and production based on speech gestures, and obtained two types of counterevidence. First, shadowing is not slowed down by a gestural mismatch between stimulus and response. Second, phonetic detail is more likely to be imitated in a shadowing task if it is phonologically relevant. This is consistent with the idea that speech perception and speech production are only loosely coupled, on an abstract phonological level.     

Sing that tune: Infants’ perception of melody and lyrics and the facilitation of phonetic recognition in songs

To better understand how infants process complex auditory input, this study investigated whether 11-month-old infants perceive the pitch (melodic) or the phonetic (lyric) components within songs as more salient, and whether melody facilitates phonetic recognition. Using a preferential looking paradigm, uni-dimensional and multi-dimensional songs were tested; either the pitch or syllable order of the stimuli varied. As a group, infants detected a change in pitch order in a 4-note sequence when the syllables were redundant (experiment 1), but did not detect the identical pitch change with variegated syllables (experiment 2). Infants were better able to detect a change in syllable order in a sung sequence (experiment 2) than the identical syllable change in a spoken sequence (experiment 1). These results suggest that by 11 months, infants cannot “ignore” phonetic information in the context of perceptually salient pitch variation. Moreover, the increased phonetic recognition in song contexts mirrors findings that demonstrate advantages of infant-directed speech. Findings are discussed in terms of how stimulus complexity interacts with the perception of sung speech in infancy.     

The influence of infant-directed speech on 12-month-olds’ intersensory perception of fluent speech

The present study examined whether infant-directed (ID) speech facilitates intersensory matching of audio–visual fluent speech in 12-month-old infants. German-learning infants’ audio–visual matching ability of German and French fluent speech was assessed by using a variant of the intermodal matching procedure, with auditory and visual speech information presented sequentially. In Experiment 1, the sentences were spoken in an adult-directed (AD) manner. Results showed that 12-month-old infants did not exhibit a matching performance for the native, nor for the non-native language. However, Experiment 2 revealed that when ID speech stimuli were used, infants did perceive the relation between auditory and visual speech attributes, but only in response to their native language. Thus, the findings suggest that ID speech might have an influence on the intersensory perception of fluent speech and shed further light on multisensory perceptual narrowing.     

语言加工过程中的视听跨通道整合

日常生活中, 语言的使用往往出现在某个视觉情境里。大量认知科学研究表明, 视觉信息与语言信息加工模块并不是独立工作, 而是存在复杂的交互作用。本文以视觉信息对语言加工的影响为主线, 首先对视觉信息影响言语理解, 言语产生以及言语交流的相关研究进展进行了综述。其次, 重点对视觉信息影响语言加工的机制进行了探讨。最后介绍了关于视觉信息影响语言加工的计算模型, 并对未来的研究方向提出了展望。     

Variable Perception of White Noise in Ambiguous Phonetic Contexts: The Case of /p/ and /f/

The roles of spectro-temporal coherence, lexical status, and word position in the perception of speech in acoustic signals containing a mixture of speech and nonspeech sounds were investigated. Stimuli consisted of nine (non)words in which either white noise was inserted only into the silent interval preceding and/or following the onset of vocalic transitions ambiguous between /p/ and /f/, or in which white noise overlaid the entire utterance. Ten listeners perceived 85% /f/s when noise was inserted only into the silent interval signaling a stop closure, 47% /f/s when noise overlaid the entire (non)words, and 1% in the control condition that contained no noise. Effects of spectro-temporal coherence seemed to have dominated perceptual outcomes, although the lexical status and position of the critical phoneme also appeared to affect responses. The results are explained more adequately by the theory of Auditory Scene Analysis than by the Motor Theory of Speech Perception.     

Do temporal processes underlie left hemisphere dominance in speech perception?

It is not unusual to find it stated as a fact that the left hemisphere is specialized for the processing of rapid, or temporal aspects of sound, and that the dominance of the left hemisphere in the perception of speech can be a consequence of this specialization. In this review we explore the history of this claim and assess the weight of this assumption. We will demonstrate that instead of a supposed sensitivity of the left temporal lobe for the acoustic properties of speech, it is the right temporal lobe which shows a marked preference for certain properties of sounds, for example longer durations, or variations in pitch. We finish by outlining some alternative factors that contribute to the left lateralization of speech perception.     

Different kinds of bilinguals: different kinds of brains: the neural organisation of two languages in one brain

The aim of this study was to determine whether the type of bilingualism affects neural organisation. We performed identification experiments and mismatch negativity (MMN) registrations in Finnish and Swedish language settings to see, whether behavioural identification and neurophysiological discrimination of vowels depend on the linguistic context, and whether there is a difference between two kinds of bilinguals. The stimuli were two vowels, which differentiate meaning in Finnish, but not in Swedish. The results indicate that Balanced Bilinguals are inconsistent in identification performance, and they have a longer MMN latency. Moreover, their MMN amplitude is context-independent, while Dominant Bilinguals show a larger MMN in the Finnish context. These results indicate that Dominant Bilinguals inhibit the preattentive discrimination of native contrast in a context where the distinction is non-phonemic, but this is not possible for Balanced Bilinguals. This implies that Dominant Bilinguals have separate systems, while Balanced Bilinguals have one inseparable system.