‘Please sort these voice recordings into 2 identities’: Effects of task instructions on performance in voice sorting studies

We investigated the effects of two types of task instructions on performance on a voice sorting task by listeners who were either familiar or unfamiliar with the voices. Listeners were asked to sort 15 naturally varying stimuli from two voice identities into perceived identities. Half of the listeners sorted the recordings freely into as many identities as they perceived; the other half were forced to sort stimuli into two identities only. As reported in previous studies, unfamiliar listeners formed more clusters than familiar listeners. Listeners therefore perceived different naturally varying stimuli from the same identity as coming from different identities, while being highly accurate at telling apart the stimuli from different voices. We further show that a change in task instructions – forcing listeners to sort stimuli into two identities only – helped unfamiliar listeners to overcome this selective failure at ‘telling people together’. This improvement, however, came at the cost of an increase in errors in telling people apart. For familiar listeners, similar non-significant trends were apparent. Therefore, even when informed about correct number of identities, listeners may fail to accurately perceive identity further highlighting that voice identity perception in the context of natural within-person variability is a challenging task. We discuss our results in terms of similarities and differences to findings in the face perception literature and their importance in applied settings, such as forensic voice identification.     

How does familiarity with a voice affect trait judgements?

From only a single spoken word, listeners can form a wealth of first impressions of a person’s character traits and personality based on their voice. However, due to the substantial within-person variability in voices, these trait judgements are likely to be highly stimulus-dependent for unfamiliar voices: The same person may sound very trustworthy in one recording but less trustworthy in another. How trait judgements differ when listeners are familiar with a voice is unclear: Are listeners who are familiar with the voices as susceptible to the effects of within-person variability? Does the semantic knowledge listeners have about a familiar person influence their judgements? In the current study, we tested the effect of familiarity on listeners’ trait judgements from variable voices across 3 experiments. Using a between-subjects design, we contrasted trait judgements by listeners who were familiar with a set of voices – either through laboratory-based training or through watching a TV show – with listeners who were unfamiliar with the voices. We predicted that familiarity with the voices would reduce variability in trait judgements for variable voice recordings from the same identity (cf. Mileva, Kramer & Burton, Perception, 48, 471 and 2019, for faces). However, across the 3 studies and two types of measures to assess variability, we found no compelling evidence to suggest that trait impressions were systematically affected by familiarity.     

How many voices did you hear? Natural variability disrupts identity perception from unfamiliar voices

Our voices sound different depending on the context (laughing vs. talking to a child vs. giving a speech), making within‐person variability an inherent feature of human voices. When perceiving speaker identities, listeners therefore need to not only ‘tell people apart’ (perceiving exemplars from two different speakers as separate identities) but also ‘tell people together’ (perceiving different exemplars from the same speaker as a single identity). In the current study, we investigated how such natural within‐person variability affects voice identity perception. Using voices from a popular TV show, listeners, who were either familiar or unfamiliar with this show, sorted naturally varying voice clips from two speakers into clusters to represent perceived identities. Across three independent participant samples, unfamiliar listeners perceived more identities than familiar listeners and frequently mistook exemplars from the same speaker to be different identities. These findings point towards a selective failure in ‘telling people together’. Our study highlights within‐person variability as a key feature of voices that has striking effects on (unfamiliar) voice identity perception. Our findings not only open up a new line of enquiry in the field of voice perception but also call for a re‐evaluation of theoretical models to account for natural variability during identity perception.     

Commonsense beliefs and the identification of familiar voices

Two experiments are reported in which participants attempted to reject the tape‐recorded voice of a stranger and identify by name the voices of three personal associates who differed in their level of familiarity. In Experiment 1 listeners were asked to identify speakers as soon as possible, but were not allowed to change their responses once made. In Experiment 2 listeners were permitted to change their responses over successive presentations of increasing durations of voice segments. Also, in Experiment 2 half of the listeners attempted to identify speakers who spoke in normal‐tone voices, and the remainder attempted to identify the same speakers who spoke in whispers. Separate groups of undergraduate students attempted to predict the performance of the listeners in both experiments. Accuracy of performance depended on the familiarity of speakers and tone of speech. A between‐subjects analysis of rated confidence was diagnostic of accuracy for high familiar and low familiar speakers (Experiment 1), and for moderate familiar and unfamiliar normal‐tone speakers (Experiment 2). A modified between‐subjects analysis assessed across the four levels of familiarity yielded reliable accuracy‐confidence correlations in both experiments. Beliefs about the accuracy of voice identification were inflated relative to the significantly lower actual performance for most of the normal‐tone and whispered‐speech conditions. Forensic significance and generalizations are addressed. Copyright © 2001 John Wiley & Sons, Ltd.     

The effects of talker familiarity on spoken word identification in younger and older listeners

The present study was designed to examine age differences in the ability to use voice information acquired intentionally (Experiment 1) or incidentally (Experiment 2) as an aid to spoken word identification. Following both implicit and explicit voice learning, participants were asked to identify novel words spoken either by familiar talkers (ones they had been exposed to in the training phase) or by 4 unfamiliar voices. In both experiments, explicit memory for talkers' voices was significantly lower in older than in young listeners. Despite this age-related decline in voice recognition, however, older adults exhibited equivalent, and in some cases greater, benefit than young listeners from having words spoken by familiar talkers. Implications of the findings for age-related changes in explicit versus implicit memory systems are discussed.     

The role of language familiarity in voice identification

Four experiments examined the effects of language characteristics on voice identification. In Experiment 1, monolingual English listeners identified bilinguals' voices much better when they spoke English than when they spoke German. The opposite outcome was found in Experiment 2, in which the listeners were monolingual in German. In Experiment 3, monolingual English listeners also showed better voice identification when bilinguals spoke a familiar language (English) than when they spoke an unfamiliar one (Spanish). However, English-Spanish bilinguals hearing the same voices showed a different pattern, with the English-Spanish difference being statistically eliminated. Finally, Experiment 4 demonstrated that, for English-dominant listeners, voice recognition deteriorates systematically as the passage being spoken is made less similar to English by rearranging words, rearranging syllables, and reversing normal text. Taken together, the four experiments confirm that language familiarity plays an important role in voice identification.     

Familiarity influences judgments of sex: the case of voice recognition

Two experiments are reported in which subjects made judgments about the sex or the familiarity of a voice. In experiment 1, subjects were fans of the BBC-radio soap opera, The Archers, and familiar voice clips were taken from this programme. Subjects showed a large reduction in response times when making sex judgments to familiar voices, despite the fact that sex judgments are generally much faster than familiarity judgments. In experiment 2, the same familiar clips were played to subjects unfamiliar with the soap opera, and no difference was observed in times to make sex judgments to Archers or non-Archers voices. We conclude that, unlike the case of face recognition, sex and identity processing of voices are not independent. The findings constrain models of person recognition across multiple modalities.     

Talker-specific learning in speech perception

The effects of perceptual learning of talker identity on the recognition of spoken words and sentences were investigated in three experiments. In each experiment, listeners were trained to learn a set of 10 talkers’ voices and were then given an intelligibility test to assess the influence of learning the voices on the processing of the linguistic content of speech. In the first experiment, listeners learned voices from isolated words and were then tested with novel isolated words mixed in noise. The results showed that listeners who were given words produced by familiar talkers at test showed better identification performance than did listeners who were given words produced by unfamiliar talkers. In the second experiment, listeners learned novel voices from sentence-length utterances and were then presented with isolated words. The results showed that learning a talker’s voice from sentences did not generalize well to identification of novel isolated words. In the third experiment, listeners learned voices from sentence-length utterances and were then given sentence-length utterances produced by familiar and unfamiliar talkers at test. We found that perceptual learning of novel voices from sentence-length utterances improved speech intelligibility for words in sentences. Generalization and transfer from voice learning to linguistic processing was found to be sensitive to the talker-specific information available during learning and test. These findings demonstrate that increased sensitivity to talker-specific information affects the perception of the linguistic properties of speech in isolated words and sentences.     

Increased pitch increases accuracy of voice identification

Previous work suggested that greater accuracy rates in identifying voices that have been increased in frequency over those that have been decreased in frequency may be due to complex vocal characteristics and specific memory for familiar voices. Here we asked 17 men and 21 women between the ages of 18 and 21 to learn a simple vowel exemplar produced by an unfamiliar target speaker and measured the proportion of times the frequency-shifted exemplar was identified as the originally encoded target speaker. Analysis showed that exemplars when increased in frequency were perceived as belonging to the target speaker significantly more often than exemplars which were decreased in frequency. These findings suggest that the greater accuracy in identifying speakers with increased frequency voice samples does not require previous familiarity with the vocalizations of a particular speaker or complex memory schemata for familiar voices.     

Listen to your mother! The role of talker familiarity in infant streaming

Little is known about the acoustic cues infants might use to selectively attend to one talker in the presence of background noise. This study examined the role of talker familiarity as a possible cue. Infants either heard their own mothers (maternal-voice condition) or a different infant's mother (novel-voice condition) repeating isolated words while a female distracter voice spoke fluently in the background. Subsequently, infants heard passages produced by the target voice containing either the familiarized, target words or novel words. Infants in the maternal-voice condition listened significantly longer to the passages containing familiar words; infants in the novel-voice condition showed no preference. These results suggest that infants are able to separate the simultaneous speech of two women when one of the voices is highly familiar to them. However, infants seem to find separating the simultaneous speech of two unfamiliar women extremely difficult.     

Audiovisual speech facilitates voice learning

In this research, we investigated the effects of voice and face information on the perceptual learning of talkers and on long-term memory for spoken words. In the first phase, listeners were trained over several days to identify voices from words presented auditorily or audiovisually. The training data showed that visual information about speakers enhanced voice learning, revealing cross-modal connections in talker processing akin to those observed in speech processing. In the second phase, the listeners completed an auditory or audiovisual word recognition memory test in which equal numbers of words were spoken by familiar and unfamiliar talkers. The data showed that words presented by familiar talkers were more likely to be retrieved from episodic memory, regardless of modality. Together, these findings provide new information about the representational code underlying familiar talker recognition and the role of stimulus familiarity in episodic word recognition.     

Forensic voice discrimination by lay listeners: The effect of speech type and background noise on performance

In forensic settings, lay (nonexpert) listeners may be required to compare voice samples for identity. In two experiments we investigated the effect of background noise and variations in speaking style on performance. In each trial, participants heard two recordings, responded whether the voices belonged to the same person, and provided a confidence rating. In Experiment 1, the first recording featured read speech and the second featured read or spontaneous speech. Both recordings were presented in quiet, or with background noise. Accuracy was highest when recordings featured the same speaking style. In Experiment 2, background noise either occurred in the first or second recording. Accuracy was higher when it occurred in the second. The overall results reveal that both speaking style and background noise can disrupt accuracy. Although there is a relationship between confidence and accuracy in all conditions, it is variable. The forensic implications of these findings are discussed.     

The role of audiovisual asynchrony in person recognition

Audiovisual integration (AVI) has been demonstrated to play a major role in speech comprehension. Previous research suggests that AVI in speech comprehension tolerates a temporal window of audiovisual asynchrony. However, few studies have employed audiovisual presentation to investigate AVI in person recognition. Here, participants completed an audiovisual voice familiarity task in which the synchrony of the auditory and visual stimuli was manipulated, and in which visual speaker identity could be corresponding or noncorresponding to the voice. Recognition of personally familiar voices systematically improved when corresponding visual speakers were presented near synchrony or with slight auditory lag. Moreover, when faces of different familiarity were presented with a voice, recognition accuracy suffered at near synchrony to slight auditory lag only. These results provide the first evidence for a temporal window for AVI in person recognition between approximately 100 ms auditory lead and 300 ms auditory lag.     

Hearing facial identities

While audiovisual integration is well known in speech perception, faces and speech are also informative with respect to speaker recognition. To date, audiovisual integration in the recognition of familiar people has never been demonstrated. Here we show systematic benefits and costs for the recognition of familiar voices when these are combined with time-synchronized articulating faces, of corresponding or noncorresponding speaker identity, respectively. While these effects were strong for familiar voices, they were smaller or nonsignificant for unfamiliar voices, suggesting that the effects depend on the previous creation of a multimodal representation of a person's identity. Moreover, the effects were reduced or eliminated when voices were combined with the same faces presented as static pictures, demonstrating that the effects do not simply reflect the use of facial identity as a “cue” for voice recognition. This is the first direct evidence for audiovisual integration in person recognition.     

Hearing facial identities

While audiovisual integration is well known in speech perception, faces and speech are also informative with respect to speaker recognition. To date, audiovisual integration in the recognition of familiar people has never been demonstrated. Here we show systematic benefits and costs for the recognition of familiar voices when these are combined with time-synchronized articulating faces, of corresponding or noncorresponding speaker identity, respectively. While these effects were strong for familiar voices, they were smaller or nonsignificant for unfamiliar voices, suggesting that the effects depend on the previous creation of a multimodal representation of a person's identity. Moreover, the effects were reduced or eliminated when voices were combined with the same faces presented as static pictures, demonstrating that the effects do not simply reflect the use of facial identity as a “cue” for voice recognition. This is the first direct evidence for audiovisual integration in person recognition.     

The effects of the presence of a face and direct eye gaze on voice identity learning

We rarely become familiar with the voice of another person in isolation but usually also have access to visual identity information, thus learning to recognize their voice and face in parallel. There are conflicting findings as to whether learning to recognize voices in audiovisual vs audio-only settings is advantageous or detrimental to learning. One prominent finding shows that the presence of a face overshadows the voice, hindering voice identity learning by capturing listeners' attention (Face Overshadowing Effect; FOE). In the current study, we tested the proposal that the effect of audiovisual training on voice identity learning is driven by attentional processes. Participants learned to recognize voices through either audio-only training (Audio-Only) or through three versions of audiovisual training, where a face was presented alongside the voices. During audiovisual training, the faces were either looking at the camera (Direct Gaze), were looking to the side (Averted Gaze) or had closed eyes (No Gaze). We found a graded effect of gaze on voice identity learning: Voice identity recognition was most accurate after audio-only training and least accurate after audiovisual training including direct gaze, constituting a FOE. While effect sizes were overall small, the magnitude of FOE was halved for the Averted and No Gaze conditions. With direct gaze being associated with increased attention capture compared to averted or no gaze, the current findings suggest that incidental attention capture at least partially underpins the FOE. We discuss these findings in light of visual dominance effects and the relative informativeness of faces vs voices for identity perception.     

Audiovisual identity perception from naturally-varying stimuli is driven by visual information

Identity perception often takes place in multimodal settings, where perceivers have access to both visual (face) and auditory (voice) information. Despite this, identity perception is usually studied in unimodal contexts, where face and voice identity perception are modelled independently from one another. In this study, we asked whether and how much auditory and visual information contribute to audiovisual identity perception from naturally-varying stimuli. In a between-subjects design, participants completed an identity sorting task with either dynamic video-only, audio-only or dynamic audiovisual stimuli. In this task, participants were asked to sort multiple, naturally-varying stimuli from three different people by perceived identity. We found that identity perception was more accurate for video-only and audiovisual stimuli compared with audio-only stimuli. Interestingly, there was no difference in accuracy between video-only and audiovisual stimuli. Auditory information nonetheless played a role alongside visual information as audiovisual identity judgements per stimulus could be predicted from both auditory and visual identity judgements, respectively. While the relationship was stronger for visual information and audiovisual information, auditory information still uniquely explained a significant portion of the variance in audiovisual identity judgements. Our findings thus align with previous theoretical and empirical work that proposes that, compared with faces, voices are an important but relatively less salient and a weaker cue to identity perception. We expand on this work to show that, at least in the context of this study, having access to voices in addition to faces does not result in better identity perception accuracy.     

Cross-modal recognition of human individuals in domestic horses (Equus caballus)

This study has shown that domestic horses are capable of cross-modal recognition of familiar humans. It was demonstrated that horses are able to discriminate between the voices of a familiar and an unfamiliar human without seeing or smelling them at the same moment. Conversely, they were able to discriminate the same persons when only exposed to their visual and olfactory cues, without being stimulated by their voices. A cross-modal expectancy violation setup was employed; subjects were exposed both to trials with incongruent auditory and visual/olfactory identity cues and trials with congruent cues. It was found that subjects responded more quickly, longer and more often in incongruent trials, exhibiting heightened interest in unmatched cues of identity. This suggests that the equine brain is able to integrate multisensory identity cues from a familiar human into a person representation that allows the brain, when deprived of one or two senses, to maintain recognition of this person.     

What's in a Forename? Cue Familiarity and Stereotypical Thinking

Recent research has characterized categorical thinking as an essential component of the person perception process. Yet relatively little is known about the myriad factors that moderate the accessibility of this mode of thought. With regard to this we hypothesized that the subjective familiarity of a person's forename may play an important role in triggering categorical thinking. Specifically, category-based knowledge may be more accessible when triggered by familiar than unfamiliar forenames. We report the results of three experiments that supported this prediction. Relative to unfamiliar names, participants required less time to verify the gender of familiar forenames (Experiment 1) and semantic priming was more pronounced when stereotype-related material followed the presentation of familiar than unfamiliar items (Experiment 2). Also, familiar forenames attracted more extreme gender-based evaluations than their unfamiliar counterparts (Experiment 3). We consider the theoretical and methodological implications of these findings for a variety of issues in person perception.