The vocal repertoire of the domesticated zebra finch: a data-driven approach to decipher the information-bearing acoustic features of communication signals

Although a universal code for the acoustic features of animal vocal communication calls may not exist, the thorough analysis of the distinctive acoustical features of vocalization categories is important not only to decipher the acoustical code for a specific species but also to understand the evolution of communication signals and the mechanisms used to produce and understand them. Here, we recorded more than 8000 examples of almost all the vocalizations of the domesticated zebra finch, Taeniopygia guttata: vocalizations produced to establish contact, to form and maintain pair bonds, to sound an alarm, to communicate distress or to advertise hunger or aggressive intents. We characterized each vocalization type using complete representations that avoided any a priori assumptions on the acoustic code, as well as classical bioacoustics measures that could provide more intuitive interpretations. We then used these acoustical features to rigorously determine the potential information-bearing acoustical features for each vocalization type using both a novel regularized classifier and an unsupervised clustering algorithm. Vocalization categories are discriminated by the shape of their frequency spectrum and by their pitch saliency (noisy to tonal vocalizations) but not particularly by their fundamental frequency. Notably, the spectral shape of zebra finch vocalizations contains peaks or formants that vary systematically across categories and that would be generated by active control of both the vocal organ (source) and the upper vocal tract (filter).     

Vocal learning of a communicative signal in captive chimpanzees, Pan troglodytes

We hypothesized that chimpanzees could learn to produce attention-getting (AG) sounds via positive reinforcement. We conducted a vocal assessment in 76 captive chimpanzees for their use of AG sounds to acquire the attention of an otherwise inattentive human. Fourteen individuals that did not produce AG sounds during the vocal assessment were evaluated for their ability to acquire the use of an AG sound through operant conditioning and to employ these sounds in an attention-getting context. Nine of the 14 chimpanzees were successfully shaped using positive reinforcement to produce an AG sound. In a post-training vocal assessment, eight of the nine individuals that were successfully trained to produce AG sounds generalized the use of these newly acquired signals to communicatively relevant situations. Chimpanzees possess the ability to acquire the use of a communicative signal via operant conditioning and can generalize the use of this newly acquired signal to appropriate communicative contexts.     

Processing emotions in sounds: cross-domain aftereffects of vocal utterances and musical sounds

Nonlinguistic signals in the voice and musical instruments play a critical role in communicating emotion. Although previous research suggests a common mechanism for emotion processing in music and speech, the precise relationship between the two domains is unclear due to the paucity of direct evidence. By applying the adaptation paradigm developed by Bestelmeyer, Rouger, DeBruine, and Belin [2010. Auditory adaptation in vocal affect perception. Cognition, 117(2), 217–223. doi:10.1016/j.cognition.2010.08.008], this study shows cross-domain aftereffects from vocal to musical sounds. Participants heard an angry or fearful sound four times, followed by a test sound and judged whether the test sound was angry or fearful. Results show cross-domain aftereffects in one direction – vocal utterances to musical sounds, not vice-versa. This effect occurred primarily for angry vocal sounds. It is argued that there is a unidirectional relationship between vocal and musical sounds where emotion processing of vocal sounds encompasses musical sounds but not vice-versa.     

Acoustic interaction in animal groups: signaling in noisy and social contexts

It has long been known that individuals of many species vocally communicate with one another in noisy environments and in rich contexts of social interaction. It has recently become clear that researchers interested in understanding acoustic communication in animal groups must study vocal signaling in these noisy and socially complex settings. Furthermore, recent methodological advances have made it increasingly clear that the authors can tackle these more complex questions effectively. The articles in this Special Issue stem from a Symposium held at the June 2006 meeting of the Acoustical Society of America, and illustrate some of the taxonomic and methodological diversity in studies aimed at understanding how acoustic communication functions in social grouping. This introduction to the Special Issue provides a brief overview of the articles and key ideas in this field of inquiry, and suggests some future directions to take the field to help us understand how social pressures in animal groups may influence, and be influenced by, acoustic signals.     

Demystifying infant vocal imitation: The roles of mouth looking and speaker’s gaze

Vocal imitation plays a fundamental role in human language acquisition from infancy. Little is known, however, about how infants imitate other's sounds. We focused on three factors: (a) whether infants receive information from upright faces, (b) the infant's observation of the speaker's mouth and (c) the speaker directing their gaze towards the infant. We recorded the eye movements of 6‐month‐olds who participated in experiments watching videos of a speaker producing vowel sounds. We found that an infants’ tendency to vocally imitate such videos increased as a function of (a) seeing upright rather than inverted faces, (b) their increased looking towards the speaker's mouth and (c) whether the speaker directed their gaze towards, rather than away from infants. These latter findings are consistent with theories of motor resonance and natural pedagogy respectively. New light has been shed on the cues and underlying mechanisms linking infant speech perception and production.     

Vocal signals only impact speakers’ own emotions when they are self-attributed

Emotions are often accompanied by vocalizations whose acoustic features provide information about the physiological state of the speaker. Here, we ask if perceiving these affective signals in one’s own voice has an impact on one’s own emotional state, and if it is necessary to identify these signals as self-originated for the emotional effect to occur. Participants had to deliberate out loud about how they would feel in various familiar emotional scenarios, while we covertly manipulated their voices in order to make them sound happy or sad. Perceiving the artificial affective signals in their own voice altered participants’ judgements about how they would feel in these situations. Crucially, this effect disappeared when participants detected the vocal manipulation, either explicitly or implicitly. The original valence of the scenarios also modulated the vocal feedback effect. These results highlight the role of the exteroception of self-attributed affective signals in the emergence of emotional feelings.     

Preschoolers’ appreciation of speaker vocal affect as a cue to referential intent

An eye tracking methodology was used to evaluate 3- and 4-year-old children’s sensitivity to speaker affect when resolving referential ambiguity. Children were presented with pictures of three objects on a screen (including two referents of the same kind, e.g., an intact doll and a broken doll, and one distracter item), paired with a prerecorded referentially ambiguous instruction (e.g., “Look at the doll”). The intonation of the instruction varied in terms of the speaker’s vocal affect: positive-sounding, negative-sounding, or neutral. Analyses of eye gaze patterns indicated that 4-year-olds, but not 3-year-olds, were more likely to look to the referent whose state matched the speaker’s vocal affect as the noun was heard (e.g., looked more often to the broken doll referent in the negative affect condition). These findings indicate that 4-year-olds can use speaker affect to help identify referential mappings during on-line comprehension.     

辅音和元音知觉中的启动效应差异

采用启动范式,在两个实验中分别考察了辅音范畴和元音范畴知觉中的启动效应。启动音是纯音和目标范畴本身,目标音是辅音范畴和元音范畴连续体。结果发现辅音范畴连续体知觉的范畴反应百分比受到纯音和言语启动音影响,辅音范畴知觉的反应时只受言语启动音影响;元音范畴连续体知觉的范畴反应百分比不受两种启动音影响,但元音范畴知觉的反应时受到言语启动音影响。实验结果表明辅音范畴和元音范畴知觉中的启动效应存在差异,这为辅音和元音范畴内在加工机制的差异提供了新证据。     

Similarity and categorization of environmental sounds

Four experiments investigated the acoustical correlates of similarity and categorization judgments of environmental sounds. In Experiment 1, similarity ratings were obtained from pairwise comparisons of recordings of 50 environmental sounds. A three-dimensional multidimensional scaling (MDS) solution showed three distinct clusterings of the sounds, which included harmonic sounds, discrete impact sounds, and continuous sounds. Furthermore, sounds from similar sources tended to be in close proximity to each other in the MDS space. The orderings of the sounds on the individual dimensions of the solution were well predicted by linear combinations of acoustic variables, such as harmonicity, amount of silence, and modulation depth. The orderings of sounds also correlated significantly with MDS solutions for similarity ratings of imagined sounds and for imagined sources of sounds, obtained in Experiments 2 and 3--as was the case for free categorization of the 50 sounds (Experiment 4)--although the categorization data were less well predicted by acoustic features than were the similarity data.     

Ear differences in the recall of fricatives and vowels

Two experiments on the free recall of dichotically presented synthetic speech sounds are reported. The first shows that the right ear advantage for initial fricative consonants is not simply a function of the recognition response class, but that it is also a function of the particular acoustic cues used to achieve that response. This is true both for the whole response, and for the constituent phonetic features. The second experiment shows that when both the response class and the particular stimuli presented on certain trials are held constant, the right ear advantage for the constant stimuli can be influenced by the range of other stimuli occurring in the experiment. Vowels show a right ear advantage when, within the experiment, there is uncertainty as to vocal tract size, but they show no ear advantage when all the vowels in the experiment are from the same vocal tract. These results are interpreted as demonstrating that there are differences between the ears, and probably between the hemispheres, at some stage between the acoustic analysis of the signal and its identification as a phonetic category.     

Generalization in perceptual learning for speech

Lexical context strongly influences listeners’ identification of ambiguous sounds. For example, a sound midway between /f/ and /s/ is reported as /f/ in “sheri_’” but as /s/ in “Pari_.” Norris, McQueen, and Cutler (2003) have demonstrated that after hearing such lexically determined phonemes, listeners expand their phonemic categories to include more ambiguous tokens than before. We tested whether listeners adjust their phonemic categories for a specific speaker: Do listeners learn a particular speaker’s “accent”? Similarly, we examined whether perceptual learning is specific to the particular ambiguous phonemes that listeners hear, or whether the adjustments generalize to related sounds. Participants heard ambiguous /d/ or /t/ phonemes during a lexical decision task. They then categorized sounds on /d/-/t/ and /b/-/p/ continua, either in the same voice that they had heard for lexical decision, or in a different voice. Perceptual learning generalized across both speaker and test continua: Changes in perceptual representations are robust and broadly tuned.     

人声加工的神经机制

人声是人类听觉环境中最熟知和重要的声音, 传递着大量社会相关信息。与视觉人脸加工类似, 大脑对人声也有着特异性加工。研究者使用电生理、脑成像等手段找到了对人声有特异性反应的脑区, 即颞叶人声加工区(TVA), 并发现非人类动物也有类似的特异性加工区域。人声加工主要涉及言语、情绪和身份信息的加工, 分别对应于三条既相互独立又相互作用的神经通路。研究者提出了双通路模型、多阶段模型和整合模型分别对人声的言语、情绪和身份加工进行解释。未来研究需要进一步讨论人声加工的特异性能否由特定声学特征的选择性加工来解释, 并深入探究特殊人群(如自闭症和精神分裂症患者)的人声加工的神经机制。     

The Aesthetic Preference for Nature Sounds Depends on Sound Object Recognition

People across the world seek out beautiful sounds in nature, such as a babbling brook or a nightingale song, for positive human experiences. However, it is unclear whether this positive aesthetic response is driven by a preference for the perceptual features typical of nature sounds versus a higher‐order association of nature with beauty. To test these hypotheses, participants provided aesthetic judgments for nature and urban soundscapes that varied on ease of recognition. Results demonstrated that the aesthetic preference for nature soundscapes was eliminated for the sounds hardest to recognize, and moreover the relationship between aesthetic ratings and several measured acoustic features significantly changed as a function of recognition. In a follow‐up experiment, requiring participants to classify these difficult‐to‐identify sounds into nature or urban categories resulted in a robust preference for nature sounds and a relationship between aesthetic ratings and our measured acoustic features that was more typical of easy‐to‐identify sounds. This pattern of results was replicated with computer‐generated artificial noises, which acoustically shared properties with the nature and urban soundscapes but by definition did not come from these environments. Taken together, these results support the conclusion that the recognition of a sound as either natural or urban dynamically organizes the relationship between aesthetic preference and perceptual features and that these preferences are not inherent to the acoustic features. Implications for nature's role in cognitive and affective restoration are discussed.     

Interference effects in an auditory Stroop task: congruence and correspondence

Two auditory Stroop experiments are reported in which subjects responded vocally to the gender of the speaker of stimulus words. The interference effects found were broadly similar to those of a manual-response auditory Stroop paradigm although response-set effects appeared to operate in the data. In all experimental conditions responses were slower to stimulus words which named potential responses than to those words which did not. Such effects have not been found in previous manual-response experiments and it is suggested that the relatively high degree of S-R and of ideomotor compatibility in the vocal response task, compared with the manual task, may be responsible for this difference between the paradigms.The second experiment enabled a comparison to be made between the effects of congruence between relevant and irrelevant elements of the Stroop stimulus and the effects of correspondence between the response and the irrelevant aspect of the stimulus. A predominant influence of congruence was found when the subject responded using words semantically related to speaker gender, but an effect of correspondence was obtained when an arbitrary pairing of response word and speaker gender was required.     

Developmental changes in sensitivity to vocal paralanguage

Developmental changes in children’s sensitivity to the role of acoustic variation in the speech stream in conveying speaker affect (vocal paralanguage) were examined. Four‐, 7‐ and 10‐year‐olds heard utterances in three formats: low‐pass filtered, reiterant, and normal speech. The availability of lexical and paralinguistic information varied across these three formats in a way that required children to base their judgments of speaker affect on different configurations of cues in each format. Across ages, the best performance was obtained when a rich array of acoustic cues was present and when there was no competing lexical information. Four‐year‐olds performed at chance when judgments had to be based solely on speech prosody in the filtered format and they were unable to selectively attend to paralanguage when discrepant lexical cues were present in normal speech. Seven‐year‐olds were significantly more sensitive to the paralinguistic role of speech prosody in filtered speech than were 4‐year‐olds and there was a trend toward greater attention to paralanguage when lexical and paralinguistic cues were inconsistent in normal speech. An integration of the ability to utilize prosodic cues to speaker affect with attention to paralanguage in cases of lexical/paralinguistic discrepancy was observed for 10‐year‐olds. The results are discussed in terms of the development of a perceptual bias emerging out of selective attention to language.     

A vocal response time system for use with sentence verification tasks

A software system for the reliable detection of vocal response onset is described. The system was designed specifically for the measurement of vocal response times to speech stimuli presented aurally in a sound field in the presence of some background noise. The response time extraction method described here is robust to masking noise and extraneous sounds that may be included in the subject’s recorded response. In addition, the response words do not have to be limited to a small set because the system is able to differentiate the onset of any speech sounds, including low-energy fricatives. The method described here may be implemented with any computer sound system because it relies only on the sound conversion clock for timing accuracy and uses postprocessing of the signal after acquisition for response extraction. The response time extraction technique as currently implemented does not recognize subject’s responses but could be incorporated into an automatic speech recognition system.     

From sensory to long-term memory: evidence from auditory memory reactivation studies

Everyday experience tells us that some types of auditory sensory information are retained for long periods of time. For example, we are able to recognize friends by their voice alone or identify the source of familiar noises even years after we last heard the sounds. It is thus somewhat surprising that the results of most studies of auditory sensory memory show that acoustic details, such as the pitch of a tone, fade from memory in ca. 10-15 s. One should, therefore, ask (1) what types of acoustic information can be retained for a longer term, (2) what circumstances allow or help the formation of durable memory records for acoustic details, and (3) how such memory records can be accessed. The present review discusses the results of experiments that used a model of auditory recognition, the auditory memory reactivation paradigm. Results obtained with this paradigm suggest that the brain stores features of individual sounds embedded within representations of acoustic regularities that have been detected for the sound patterns and sequences in which the sounds appeared. Thus, sounds closely linked with their auditory context are more likely to be remembered. The representations of acoustic regularities are automatically activated by matching sounds, enabling object recognition.     

Learning how to learn from social feedback: The origins of early vocal development

Infants’ prelinguistic vocalizations reliably organize vocal turn-taking with social partners, creating opportunities for learning to produce the sound patterns of the ambient language. This social feedback loop supporting early vocal learning is well-documented, but its developmental origins have yet to be addressed. When do infants learn that their non-cry vocalizations influence others? To test developmental changes in infant vocal learning, we assessed the vocalizations of 2- and 5-month-old infants in a still-face interaction with an unfamiliar adult. During the still-face, infants who have learned the social efficacy of vocalizing increase their babbling rate. In addition, to assess the expectations for social responsiveness that infants build from their everyday experience, we recorded caregiver responsiveness to their infants’ vocalizations during unstructured play. During the still-face, only 5-month-old infants showed an increase in vocalizing (a vocal extinction burst) indicating that they had learned to expect adult responses to their vocalizations. Caregiver responsiveness predicted the magnitude of the vocal extinction burst for 5-month-olds. Because 5-month-olds show a vocal extinction burst with unfamiliar adults, they must have generalized the social efficacy of their vocalizations beyond their familiar caregiver. Caregiver responsiveness to infant vocalizations during unstructured play was similar for 2- and 5-month-olds. Infants thus learn the social efficacy of their vocalizations between 2 and 5 months of age. During this time, infants build associations between their own non-cry sounds and the reactions of adults, which allows learning of the instrumental value of vocalizing.     

The relevance of temporal cues in a fish sound: a first experimental investigation using modified signals in cichlids

Playback experiments have been a useful tool for studying the function of sounds and the relevance of different sound characteristics in signal recognition in many different species of vertebrates. However, successful playback experiments in sound-producing fish remain rare, and few studies have investigated the role of particular sound features in the encoding of information. In this study, we set-up an apparatus in order to test the relevance of acoustic signals in males of the cichlid Metriaclima zebra. We found that territorial males responded more to playbacks by increasing their territorial activity and approaching the loudspeaker during and after playbacks. If sounds are used to indicate the presence of a competitor, we modified two sound characteristics, that is, the pulse period and the number of pulses, in order to investigate whether the observed behavioural response was modulated by the temporal structure of sounds recorded during aggressive interactions. Modified sounds yielded little or no effect on the behavioural response they elicited in territorial males, suggesting a high tolerance for variations in pulse period and number of pulses. The biological function of sounds in M. zebra and the lack of responsiveness to our temporal modifications are discussed.