Variability and recognition memory: Are there analogous indexical effects in music and speech?

Indexical effects refer to the influence of surface variability of the to-be-remembered items, such as different voices speaking the same words or different timbres (musical instruments) playing the same melodies, on recognition memory performance. The nature of timbre effects in melody recognition was investigated in two experiments. Experiment 1 showed that melodies that remained in the same timbre from study to test were discriminated better than melodies presented in a previously studied but different, or unstudied timbre at test. Timbre effects are attributed solely to instance-specific matching, rather than timbre-specific familiarity. In Experiment 2, when a previously unstudied timbre was similar to the original timbre and it played the melodies at test, performance was comparable to the condition when the exact same timbre was repeated at test. The use of a similar timbre at test enabled the listener to discriminate old from new melodies reliably. Overall, our data suggest that timbre-specific information is encoded and stored in long-term memory. Analogous indexical effects arising from timbre (nonmusical) and voice (nonlexical) attributes in music and speech processing respectively are implied and discussed.     

Characterizing the influence of native language experience on adult speech perception.

Previous cross-language research has indicated that some speech contrasts present greater perceptual difficulty for adult non-native listeners than others do. It has been hypothesized that phonemic, phonetic, and acoustic factors contribute to this variability. Two experiments were conducted to evaluate systematically the role of phonemic status and phonetic familiarity in the perception of non-native speech contrasts and to test predictions derived from a model proposed by Best, McRoberts, and Sithole (1988). Experiment 1 showed that perception of an unfamiliar phonetic contrast was not less difficult for subjects who had experience with an analogous phonemic distinction in their native language than for subjects without such analogous experience. These results suggest that substantive phonetic experience influences the perception of non-native contrasts, and thus should contribute to a conceptualization of native language-processing skills. In Experiment 2, English listeners' perception of two related nonphonemic place contrasts was not consistently different as had been expected on the basis of phonetic familiarity. A clear order effect in the perceptual data suggests that interactions between different perceptual assimilation patterns or acoustic properties of the two contrasts, or interactions involving both of these factors, underlie the perception of the two contrasts in this experiment. It was concluded that both phonetic familiarity and acoustic factors are potentially important to the explanation of variability in perception of nonphonemic contrasts. The explanation of how linguistic experience shapes speech perception will require characterizing the relative contribution of these factors, as well as other factors, including individual differences and variables that influence a listener's orientation to speech stimuli.     

Perceptual equivalence of acoustic cues in speech and nonspeech perception

Trading relations show that diverse acoustic consequences of minimal contrasts in speech are equivalent in perception of phonetic categories. This perceptual equivalence received stronger support from a recent finding that discrimination was differentially affected by the phonetic cooperation or conflict between two cues for the /slIt/-/splIt/contrast. Experiment 1 extended the trading relations and perceptual equivalence findings to the /sei/-/stei/contrast. With a more sensitive discrimination test, Experiment 2 found that cue equivalence is a characteristic of perceptual sensitivity to phonetic information. Using “sine-wave analogues” of the /sei/-/stei/stimuli, Experiment 3 showed that perceptual integration of the cues was phonetic, not psychoacoustic, in origin. Only subjects who perceived the sine-wave stimuli as “say” and “stay” showed a trading relation and perceptual equivalence; subjects who perceived them as nonspeech failed to integrate the two dimensions perceptually. Moreover, the pattern of differences between obtained and predicted discrimination was quite similar across the first two experiments and the “say”-“stay” group of Experiment 3, and suggested that phonetic perception was responsible even for better-than-predicted performance by these groups. Trading relations between speech cues, and the perceptual equivalence that underlies them, thus appear to derive specifically from perception of phonetic information.     

Characterizing the influence of native language experience on adult speech perception

Previous cross-language research has indicated that some speech contrasts present greater perceptual difficulty for adult non-native listeners than others do. It has been hypothesized that phonemic, phonetic, and acoustic factors contribute to this variability. Two experiments were conducted to evaluate systematically the role of phonemic status and phonetic familiarity in the perception of non-native speech contrasts and to test predictions derived from a model proposed by Best, McRoberts, and Sithole (1988). Experiment 1 showed that perception of an unfamiliar phonetic contrast was not less difficult for subjects who had experience with an analogous phonemic distinction in their native language than for subjects without such analogous experience. These results suggest that substantive phonetic experience influences the perception of non-native contrasts, and thus should contribute to a conceptualization of native language-processing skills. In Experiment 2, English listeners’ perception of two related nonphonemic place contrasts was not consistently different as had been expected on the basis of phonetic familiarity. A clear order effect in the perceptual data suggests that interactions between different perceptual assimilation patterns or acoustic properties of the two contrasts, or interactions involving both of these factors, underlie the perception of the two contrasts in this experiment. It was concluded that both phonetic familiarity and acoustic factors are potentially important to the explanation of variability in perception of nonphonemic contrasts. The explanation of how linguistic experience shapes speech perception will require characterizing the relative contribution of these factors, as well as other factors, including individual differences and variables that influence a listener’s orientation to speech stimuli.     

Effects of timbre and tempo change on memory for music

We investigated the effects of different encoding tasks and of manipulations of two supposedly surface parameters of music on implicit and explicit memory for tunes. In two experiments, participants were first asked to either categorize instrument or judge familiarity of 40 unfamiliar short tunes. Subsequently, participants were asked to give explicit and implicit memory ratings for a list of 80 tunes, which included 40 previously heard. Half of the 40 previously heard tunes differed in timbre (Experiment 1) or tempo (Experiment 2) in comparison with the first exposure. A third experiment compared similarity ratings of the tunes that varied in timbre or tempo. Analysis of variance (ANOVA) results suggest first that the encoding task made no difference for either memory mode. Secondly, timbre and tempo change both impaired explicit memory, whereas tempo change additionally made implicit tune recognition worse. Results are discussed in the context of implicit memory for nonsemantic materials and the possible differences in timbre and tempo in musical representations.     

„Die Botschaft hör ich wohl…“

From an evolutionary standpoint language developes from music and sounds. On grounds of that theory the role of intonation, timbre, volume and rhythm in speaking is considered as well as the messages that are transmitted beyond the lexical-semantic level primarily on the phonetic level. The vocal apparatus as a generator of tones, vibrations and rhythm is connected to the body as a whole and functions as a ?resonance body‘. Thereby, it gives the sound, when speaking or listening, a specific quality and meaning. That principle of giving and developing meaning in a sensory way exists already in the communication between mother and child before birth. Besides the secondary linguistical representations it remains lifelong of central importance in the generation and passing on of implicit knowledge. A psychoanalytic treatment is described as an example on how the phonetic level in speech influences the process and advances of therapy.     

A study of perceptual development for musical tuning

Musical tuning perception in infancy and adulthood was explored in three experiments. In Experiment 1, Western adults were tested in detection of randomly located mistunings in a melody based on musical interval patterns from native and nonnative musical scales. Subjects performed better in a Western major scale context than in either a Western augmented or--a Javanese pelog scale context. Because the major scale is used frequently in Western music and, therefore, is more perceptually familiar than either the augmented scale or the pelog scale are, the adults’ pattern of performance is suggestive of musical acculturation. Experiments 2 and3 were designed to explore the onset of culturally specific perceptual reorganization for music in the age period that has been found to be important in linguistically specific perceptual reorganization for speech. In Experiment 2, 1-year-olds had a pattern of performance similar to that of the adults, but 6-month-olds could not detect mistunings reliably better than chance. In Experiment 3, another group of 6-month-olds was tested, and a larger degree of mistuning was used so that floor effects might be avoided. These 6-month-olds performed better in the major and augmented scale contexts than in the pelog context, without a reliable performance difference between the major and augmented contexts. Comparison of the results obtained with 6-month-olds and 1-year-olds suggests that culturally specific perceptual reorganization for musical tuning begins to affect perception between these ages, but the 6-month-olds’ pattern of results considered alone is not as clear. The 6-month-olds’ better performance on the major and augmented interval patterns than on the pelog interval pattern is potentially attributable to either the 6-month.olds’ lesser perceptual acculturation than that of the 1-year-olds or perhaps to an innate predisposition for processing of music based on a single fundamental interval, in this case the semitone.     

Perceptual interactions between musical pitch and timbre.

These experiments examined perceptual interactions between musical pitch and timbre. Experiment 1, through the use of the Garner classification tasks, found that pitch and timbre of isolated tones interact. Classification times showed interference from uncorrelated variation in the irrelevant attribute and facilitation from correlated variation; the effects were symmetrical. Experiments 2 and 3 examined how musical pitch and timbre function in longer sequences. In recognition memory tasks, a target tone always appeared in a fixed position in the sequences, and listeners were instructed to attend to either its pitch or its timbre. For successive tones, no interactions between timbre and pitch were found. That is, changing the pitches of context tones did not affect timbre recognition, and vice versa. The tendency to perceive pitch in relation to other context pitches was strong and unaffected by whether timbre was constant or varying. In contrast, the relative perception of timbre was weak and was found only when pitch was constant. These results suggest that timbre is perceived more in absolute than in relative terms. Perceptual implications for creating patterns in music with timbre variations are discussed.     

Beat It! Music Overloads Novice Dancers

Together with melody, harmony, and timbre, rhythm and beat provide temporal structure for movement timing. Such musical features may act as cues to the phrasing and dynamics of a dance choreographed to the music. Novice dancers (N = 54) learned to criterion a novel 32‐s dance‐pop routine, either to full music or to the rhythm of that music. At test, participants recalled the dance to the same music, rhythm, new music, and in silence. If musical features aid memory, then full music during learning and test should result in superior dance recall, whereas if rhythm alone aids memory, then rhythm during learning and test should result in superior recall. The presence of a rhythm accompaniment during learning provided a significantly greater memory advantage for the recall of dance‐pop steps than full music. After learning to full music, silence at test enhanced recall. Findings are discussed in terms of entrainment and cognitive load. Copyright © 2014 John Wiley & Sons, Ltd.     

An investigation of young infants' perceptual representations of speech sounds

The present study examined the ability of newborns and 2-month-olds to detect phonetic differences between syllables. By relying on the modified high-amplitude sucking procedure, which did not permit the infants to use a simple same-different response, the present experiments tapped the perceptual representations of the speech sounds. Infants as young as a few days old displayed some capacity to represent differences in a set of syllables varying in their phonetic composition, although there was no convincing evidence that their representations were structured in terms of phonetic segments. Finally, evidence of developmental changes in speech processing were noted for the first time with infants in this age range. The change noted was a tendency from global toward more specific representations on the part of the older infants.     

Perception of syllable-final stop consonants by 2-month-old infants

Previous research has shown that infants are capable of perceiving many phonetic distinctions between initial segments of syllables. The present study demonstrates that 2-month-old infants have the ability to distinguish syllables differing only in their final segments. Infants were found to be sensitive to place-of-articulation differences for stop consonants in final segments of both consonant-vowel-consonant (CVC) and vowel-consonant (VC) syllable pairs. Contrary to previous reports for older infants (Shvachkin, 1973), there was no indication that 2-month-olds have any more difficulty with contrasts of final-stop consonant:3 than they do with initial ones.     

A study of perceptual development for musical tuning.

Musical tuning perception in infancy and adulthood was explored in three experiments. In Experiment 1, Western adults were tested in detection of randomly located mistunings in a melody based on musical interval patterns from native and nonnative musical scales. Subjects performed better in a Western major scale context than in either a Western augmented or a Javanese pelog scale context. Because the major scale is used frequently in Western music and, therefore, is more perceptually familiar than either the augmented scale or the pelog scale are, the adults' pattern of performance is suggestive of musical acculturation. Experiments 2 and 3 were designed to explore the onset of culturally specific perceptual reorganization for music in the age period that has been found to be important in linguistically specific perceptual reorganization for speech. In Experiment 2, 1-year-olds had a pattern of performance similar to that of the adults, but 6-month-olds could not detect mistunings reliably better than chance. In Experiment 3, another group of 6-month-olds was tested, and a larger degree of mistuning was used so that floor effects might be avoided. These 6-month-olds performed better in the major and augmented scale contexts than in the pelog context, without a reliable performance difference between the major and augmented contexts. Comparison of the results obtained with 6-month-olds and 1-year-olds suggests that culturally specific perceptual reorganization for musical tuning begins to affect perception between these ages, but the 6-month-olds' pattern of results considered alone is not as clear. The 6-month-olds' better performance on the major and augmented interval patterns than on the pelog interval pattern is potentially attributable to either the 6-month-olds' lesser perceptual acculturation than that of the 1-year-olds or perhaps to an innate predisposition for processing of music based on a single fundamental interval, in this case the semitone.     

Musical stem completion: humming that note

This study looked at how people store and retrieve tonal music explicitly and implicitly using a production task. Participants completed an implicit task (tune stem completion) followed by an explicit task (cued recall). The tasks were identical except for the instructions at test time. They listened to tunes and were then presented with tune stems from previously heard tunes and novel tunes. For the implicit task, they were asked to sing a note they thought would come next musically. For the explicit task, they were asked to sing the note they remembered as coming next. Experiment 1 found that people correctly completed significantly more old stems than new stems. Experiment 2 investigated the characteristics of music that fuel retrieval by varying a surface feature of the tune (same timbre or different timbre) from study to test and the encoding task (semantic or nonsemantic). Although we did not find that implicit and explicit memory for music were significantly dissociated for levels of processing, we did find that surface features of music affect semanticjudgments and subsequent explicit retrieval.     

Congenital amusia: A short-term memory deficit for non-verbal,but not verbal sounds

Congenital amusia refers to a lifelong disorder of music processing and is linked to pitch-processing deficits. The present study investigated congenital amusics’ short-term memory for tones, musical timbres and words. Sequences of five events (tones, timbres or words) were presented in pairs and participants had to indicate whether the sequences were the same or different. The performance of congenital amusics confirmed a memory deficit for tone sequences, but showed normal performance for word sequences. For timbre sequences, amusics’ memory performance was impaired in comparison to matched controls. Overall timbre performance was found to be correlated with melodic contour processing (as assessed by the Montreal Battery of Evaluation of Amusia). The present findings show that amusics’ deficits extend to non-verbal sound material other than pitch, in this case timbre, while not affecting memory for verbal material. This is in line with previous suggestions about the domain-specificity of congenital amusia.     

Discovering phonetic coherence in acoustic patterns

Despite spectral and temporal discontinuities in the speech signal, listeners normally report coherent phonetic patterns corresponding to the phonemes of a language that they know. What is the basis for the internal coherence of phonetic segments? According to one account, listeners achieve coherence by extracting and integrating discrete cues; according to another, coherence arises automatically from general principles of auditory form perception; according to a third, listeners perceive speech patterns as coherent because they are the acoustic consequences of coordinated articulatory gestures in a familiar language. We tested these accounts in three experiments by training listeners to hear a continuum of three-tone, modulated sine wave patterns, modeled after a minimal pair contrast between three-formant synthetic speech syllables, either as distorted speech signals carrying a phonetic contrast (speech listeners) or as distorted musical chords carrying a nonspeech auditory contrast (music listeners). The music listeners could neither integrate the sine wave patterns nor perceive their auditory coherence to arrive at consistent, categorical percepts, whereas the speech listeners judged the patterns as speech almost as reliably as the synthetic syllables on which they were modeled. The outcome is consistent with the hypothesis that listeners perceive the phonetic coherence of a speech signal by recognizing acoustic patterns that reflect the coordinated articulatory gestures from which they arose.     

Individual differences in the acquisition of second language phonology

Perceptual training was employed to characterize individual differences in non-native speech sound learning. Fifty-nine adult English speakers were trained to distinguish the Hindi dental–retroflex contrast, as well as a tonal pitch contrast. Training resulted in overall group improvement in the ability to identify and to discriminate the phonetic and the tonal contrasts, but there were considerable individual differences in performance. A category boundary effect during the post-training discrimination of the Hindi but not of the tonal contrast suggests different learning mechanisms for these two stimulus types. Specifically, our results suggest that successful learning of the speech sounds involves the formation of a long-term memory category representation for the new speech sound.     

Congenital amusia: A short-term memory deficit for non-verbal,but not verbal sounds

Congenital amusia refers to a lifelong disorder of music processing and is linked to pitch-processing deficits. The present study investigated congenital amusics’ short-term memory for tones, musical timbres and words. Sequences of five events (tones, timbres or words) were presented in pairs and participants had to indicate whether the sequences were the same or different. The performance of congenital amusics confirmed a memory deficit for tone sequences, but showed normal performance for word sequences. For timbre sequences, amusics’ memory performance was impaired in comparison to matched controls. Overall timbre performance was found to be correlated with melodic contour processing (as assessed by the Montreal Battery of Evaluation of Amusia). The present findings show that amusics’ deficits extend to non-verbal sound material other than pitch, in this case timbre, while not affecting memory for verbal material. This is in line with previous suggestions about the domain-specificity of congenital amusia.     

Speech perception and memory coding in relation to reading ability

Previous work has demonstrated that children who are poor readers have short-term memory deficits in tasks in which the stimuli lend themselves to phonetic coding. The aim of the present study was to explore whether the poor readers' memory dificit may have its origin in perception with the encoding of the stimuli. Three experiments were conducted with third grade good and poor readers. As in earlier experiments, the poor readers were found to perform less well on recall of random word strings and to be less affected by the phonetic characteristics (rhyming or not rhyming) of the items (Experiment 1). In addition, the poor readers produced more errors of transposition (in the nonrhyming strings) than did the good readers, a further indication of the poor readers' problems with memory for order. The subjects were tested on two auditory perception tasks, one employing words (Experiment 2) and the other nonspeech environmental sounds (Experiment 3). Each was presented under two conditions: with a favorable signal-to-noise ratio and with masking. The poor readers made significantly more errors than the good readers when listening to speech in noise, but did not differ in perception of speech without noise or in perception of nonspeech environmental sounds, whether noise-masked or not. Together, the results of the perception studies suggest that poor readers have a perceptual difficulty that is specific to speech. It is suggested that the short-term memory deficits characteristic of poor readers may stem from material-specific problems of perceptual processing.     

婴儿期母语音位范畴习得：来自言语知觉的证据

婴儿期言语知觉研究表明婴儿最初（1~4个月）可以分辨几乎所有的语音范畴对比;随母语经验增加,婴儿言语知觉逐渐表现出母语语音特征的影响,辅音知觉表现为对母语语音范畴界限敏感性的提高和对非母语范畴界限敏感性的下降,非母语范畴开始同化到母语音系中去,母语元音知觉表现出知觉磁体效应。这些证据表明婴儿逐渐习得母语音位范畴,音位范畴习得顺序可能依赖范畴例子本身的声学特征、发生频率等因素     

Learning words’ sounds before learning how words sound: 9-Month-olds use distinct objects as cues to categorize speech information

One of the central themes in the study of language acquisition is the gap between the linguistic knowledge that learners demonstrate, and the apparent inadequacy of linguistic input to support induction of this knowledge. One of the first linguistic abilities in the course of development to exemplify this problem is in speech perception: specifically, learning the sound system of one’s native language. Native-language sound systems are defined by meaningful contrasts among words in a language, yet infants learn these sound patterns before any significant numbers of words are acquired. Previous approaches to this learning problem have suggested that infants can learn phonetic categories from statistical analysis of auditory input, without regard to word referents. Experimental evidence presented here suggests instead that young infants can use visual cues present in word-labeling situations to categorize phonetic information. In Experiment 1, 9-month-old English-learning infants failed to discriminate two non-native phonetic categories, establishing baseline performance in a perceptual discrimination task. In Experiment 2, these infants succeeded at discrimination after watching contrasting visual cues (i.e., videos of two novel objects) paired consistently with the two non-native phonetic categories. In Experiment 3, these infants failed at discrimination after watching the same visual cues, but paired inconsistently with the two phonetic categories. At an age before which memory of word labels is demonstrated in the laboratory, 9-month-old infants use contrastive pairings between objects and sounds to influence their phonetic sensitivity. Phonetic learning may have a more functional basis than previous statistical learning mechanisms assume: infants may use cross-modal associations inherent in social contexts to learn native-language phonetic categories.