Tonal strength and melody recognition after long and short delays

In a continuous-running-memory task, subjects heard novel seven-note melodies that were tested after delays of 11 sec (empty) or 39 sec (filled). Test items were transposed to new pitch levels (to moderately distant keys in the musical sense)and included exact transpositions (targets), same-contour lures with altered pitch intervals, and new-contour lures. Melodies differed in tonal strength (degree of conformity to a musical key) and were tonally strong, tonally weak, or atonal. False alarms to same-contour lures decreased over the longer delay period, but only for tonal stimuli. In agreement with previous studies, discrimination of detailed changes in pitch intervals improved with increased delay, whereas discrimination of more global contour information declined, again only for tonal stimuli. These results suggest that poor short-delay performance in rejecting same-contour lures arises from confusion that is based on the similarity of tonality between standard stimuli and lures. If a test item has the same contour and a similar tonality to a just-presented item, subjects tend to accept it. After a delay filled with melodies in other tonalities, the salience of key information recedes, and subjects base their judgments on more detailed pattern information (namely, exact pitch intervals). The fact that tonality affects judgments of melodic contour indicates that contour is not an entirely separable feature of melodies but rather that a melody with its contour constitutes an integrated perceptual whole.     

Tonal strength and melody recognition after long and short delays.

In a continuous-running-memory task, subjects heard novel seven-note melodies that were tested after delays of 11 sec (empty) or 39 sec (filled). Test items were transposed to new pitch levels (to moderately distant keys in the musical sense) and included exact transpositions (targets), same-contour lures with altered pitch intervals, and new-contour lures. Melodies differed in tonal strength (degree of conformity to a musical key) and were tonally strong, tonally weak, or atonal. False alarms to same-contour lures decreased over the longer delay period, but only for tonal stimuli. In agreement with previous studies, discrimination of detailed changes in pitch intervals improved with increased delay, whereas discrimination of more global contour information declined, again only for tonal stimuli. These results suggest that poor short-delay performance in rejecting same-contour lures arises from confusion that is based on the similarity of tonality between standard stimuli and lures. If a test item has the same contour and a similar tonality to a just-presented item, subjects tend to accept it. After a delay filled with melodies in other tonalities, the salience of key information recedes, and subjects base their judgments on more detailed pattern information (namely, exact pitch intervals). The fact that tonality affects judgments of melodic contour indicates that contour is not an entirely separable feature of melodies but rather that a melody with its contour constitutes an integrated perceptual whole.     

Recognition of transposed melodies: Effects of pitch distance and harmonic distance

People easily recognize a familiar melody in a previously unheard key, but they also retain some key-specific information. Does the recognition of a transposed melody depend on either pitch distance or harmonic distance from the initially learned instances? Previous research has shown a stronger effect of pitch closeness than of harmonic similarity, but did not directly test for an additional effect of the latter variable. In the present experiment, we familiarized participants with a simple eight-note melody in two different keys (C and D) and then tested their ability to discriminate the target melody from foils in other keys. The transpositions included were to the keys of C# (close in pitch height, but harmonically distant), G (more distant in pitch, but harmonically close), and F# (more distant in pitch and harmonically distant). Across participants, the transpositions to F# and G were either higher or lower than the initially trained melodies, so that their average pitch distances from C and D were equated. A signal detection theory analysis confirmed that discriminability (d′) was better for targets and foils that were close in pitch distance to the studied exemplars. Harmonic similarity had no effect on discriminability, but it did affect response bias (c), in that harmonic similarity to the studied exemplars increased both hits and false alarms. Thus, both pitch distance and harmonic distance affect the recognition of transposed melodies, but with dissociable effects on discrimination and response bias.     

The perception of interleaved melodies

If the notes of two melodies whose pitch ranges do not overlap are interleaved in time so that successive tones come from the different melodies, the resulting sequence of tones is perceptually divided into groups that correspond to the two melodies. Such “melodic fission” demonstrates perceptual grouping based on pitch alone, and has been used extensively in music.Experiment I showed that the identification of interleaved pairs of familiar melodies is possible if their pitch ranges do not overlap, but difficult otherwise. A short-term recognition-memory paradigm (Expt II) showed that interleaving a “background” melody with an unfamiliar melody interferes with same-different judgments regardless of the separation of their pitch ranges, but that range separation attenuates the interference effect. When pitch ranges overlap, listeners can overcome the interference effect and recognize a familiar target melody if the target is prespecified, thereby permitting them to search actively for it (Expt III). But familiarity or prespecification of the interleaved background melody appears not to reduce its interfering effects on same-different judgments concerning unfamiliar target melodies (Expt IV).     

The generation of temporal and melodic expectancies during musical listening

When listening to a melody, we are often able to anticipate not onlywhat tonal intervals will occur next but alsowhen in time these will appear. The experiments reported here were carried out to investigate what types of structural relations support the generation of temporal expectancies in the context of a melody recognition task. The strategy was to present subjects with a set of folk tunes in which temporal accents (i.e., notes with a prolonged duration) always occurred in the first half of a melody, so that expectancies, if generated, could carry over to an isochronous sequence of notes in the latter half ofthe melody. The ability to detect deviant pitch changes in the final variation as a function of rhythmic context was then evaluated. Accuracy and reaction time data from Experiment 1 indicated that expectancy formation jointly depends on an invariant periodicity of temporal accentuation and the attentional highlighting ofcertain melodic relations (i.e., phrase ending points). In Experiment 2, once these joint expectancies were generated, the temporal dimension had a greater facilitating effectupon melody recognition than did the melodic one. These results are discussed in terms of their implications for the perceptual processing of musical events.     

Ideomotor effects of pitch on continuation tapping

The ideomotor principle predicts that perception will modulate action where overlap exists between perceptual and motor representations of action. This effect is demonstrated with auditory stimuli. Previous perceptual evidence suggests that pitch contour and pitch distance in tone sequences may elicit tonal motion effects consistent with listeners' implicit awareness of the lawful dynamics of locomotive bodies. To examine modulating effects of perception on action, participants in a continuation tapping task produced a steady tempo. Auditory tones were triggered by each tap. Pitch contour randomly and persistently varied within trials. Pitch distance between successive tones varied between trials. Although participants were instructed to ignore them, tones systematically affected finger dynamics and timing. Where pitch contour implied positive acceleration, the following tap and the intertap interval (ITI) that it completed were faster. Where pitch contour implied negative acceleration, the following tap and the ITI that it completed were slower. Tempo was faster with greater pitch distance. Musical training did not predict the magnitude of these effects. There were no generalized effects on timing variability. Pitch contour findings demonstrate how tonal motion may elicit the spontaneous production of accents found in expressive music performance.     

Detectability of duration and intensity increments in melody tones: A partial connection between music perception and performance

Two experiments demonstrate positional variation in the relative detectability of, respectively, local temporal and dynamic perturbations in an isochronous and isodynamic sequence of melody tones, played on a computer-controlled piano. This variation may reflect listeners’ expectations of expressive performance microstructure (thetop-down hypothesis), or it may be due to psychoacoustic (pitch-related) stimulus factors (thebottom-up hypothesis). Percent correct scores for increments in tone duration correlated significantly with the average timing profile of pianists’ expressive performances of the music, as predicted specifically by the top-down hypothesis. For intensity increments, the analogous perception-performance correlation was weak and the bottom-up factors of relative pitch height and/or direction of pitch change accounted for some of the perceptual variation. Subjects’ musical training increased overall detection accuracy but did not affect the positional variation in accuracy scores in either experiment. These results are consistent with the top-down hypothesis for timing, but they favor the bottom-up hypothesis for dynamics. The perception-performance correlation for timing may also be viewed as being due to complex stimulus properties such as tonal motion and tension/relaxation that influence performers and listeners in similar ways.     

Maximum key-profile correlation (MKC) as a measure of tonal structure in music

Tonal structure is musical organization on the basis of pitch, in which pitches vary in importance and rate of occurrence according to their relationship to a tonal center. Experiment 1 evaluated the maximum key-profile correlation (MKC), a product of Krumhansl and Schmuckler’s key-finding algorithm (Krumhansl, 1990), as a measure of tonal structure. The MKC is the maximum correlation coefficient between the pitch class distribution in a musical sample and key profiles,which indicate the stability of pitches with respect to particular tonal centers. The MKC values of melodies correlated strongly with listeners’ ratings of tonal structure. To measure the influence of the temporal order of pitches on perceived tonal structure, three measures (fifth span, semitone span, and pitch contour) taken from previous studies of melody perception were also correlated with tonal structure ratings. None of the temporal measures correlated as strongly or as consistently with tonal structure ratings as did the MKC, and nor did combining them with the MKC improve prediction of tonal structure ratings. In Experiment 2, the MKC did not correlate with recognition memory of melodies. However, melodies with very low MKC values were recognized less accurately than melodies with very high MKC values. Although it does not incorporate temporal, rhythmic, or harmonic factors that may influence perceived tonal structure, the MKC can be interpreted as a measure of tonal structure, at least for brief melodies.     

Dynamic processes in music perception

A schema-based theory of music perception that describes the dynamic interaction between the musical event and the listener’s knowledge of the underlying regularities in tonal music is proposed. Three properties of musical schema are evaluated in a recognition memory experiment: (1) The schema engages a subset of the abstract knowledge system that is determined by the predominant key of the musical sequence, (2) the schema evaluates both interval relations and the functions of the sounded elements within the established tonal framework, and (3) the schema interacts continuously with the musical event in time to process pitch information in its temporal context. Listeners are required to identify the serial position of a chord that is changed between two successive chord sequences that are otherwise identical. The experiment measures the magnitude and the temporal extent of the disruptive effect of including in the sequence an element outside the tonal framework. The results show temporally specific effects on memory for pitch relations consistent with the operation of a musical schema. Comparisons are made with schema-based theories applied in other perceptual and cognitive domains.     

The effect of task and pitch structure on pitch-time interactions in music

Musical pitch-time relations were explored by investigating the effect of temporal variation on pitch perception. In Experiment 1, trained musicians heard a standard tone followed by a tonal context and then a comparison tone. They then performed one of two tasks. In the cognitive task, they indicated whether the comparison tone was in the key of the context. In the perceptual task, they judged whether the comparison tone was higher or lower than the standard tone. For both tasks, the comparison tone occurred early, on time, or late with respect to temporal expectancies established by the context. Temporal variation did not affect accuracy in either task. Experiment 2 used the perceptual task and varied the pitch structure by employing either a tonal or an atonal context. Temporal variation did not affect accuracy for tonal contexts, but did for atonal contexts. Experiment 3 replicated these results and controlled potential confounds. We argue that tonal contexts bias attention toward pitch and eliminate effects of temporal variation, whereas atonal contexts do not, thus fostering pitch-time interactions. Psychonomic Society, Inc.     

Repetition priming in music

The authors explore priming effects of pitch repetition in music in 3 experiments. Musically untrained participants heard a short melody and sang the last pitch of the melody as quickly as possible. Each experiment manipulated (a) whether or not the tone to be sung (target) was heard earlier in the melody (primed) and (b) the prime-target distance (measured in events). Experiment 1 used variable-length melodies, whereas Experiments 2 and 3 used fixed-length melodies. Experiment 3 changed the timbre of the target tone. In all experiments, fast-responding participants produced repeated tones faster than nonrepeated tones, and this repetition benefit decreased as prime-target distances increased. All participants produced expected tonic endings faster than less expected nontonic endings. Repetition and tonal priming effects are compared with harmonic priming effects in music and with repetition priming effects in language.     

Effects of Phonetic Similarity in the Identification of Mandarin Tones

Tonal languages differ in how they use phonetic correlates e.g. average pitch height and pitch direction, for tonal contrasts. Thus, native speakers of a tonal language may need to adjust their attention to familiar or unfamiliar phonetic cues when perceiving non-native tones. On the other hand, speakers of a non-tonal language may need to develop sensitivity to tonal correlates absent from their native system. The current study examines and compares five language groups’ perception of two synthesized Mandarin tones: the high level tone and the high falling tone. It aims to examine how listeners from tonal and non-tonal backgrounds identify and categorize acoustically equidistant pitches varying along two phonetic dimensions: pitch onset and slope. Results reveal “universal” perceptual patterns across groups and also tendencies caused by native tonal systems. Our findings confirm that L1 tonal and prosodic systems affect speakers’ sensitivity to novel perceptual cues and their abilities to discern relevant phonetic differences.     

Cross-octave masking of single tones and musical sequences: The effects of structure on auditory recognition

A series of experiments explored the role of structural information in the auditory recognition process, within the context of a backward recognition masking paradigm. A masking tone presented after a test tone has been found to interfere with the perceptual processing of the test tone, the degree of interference decreasing with increased durations of the silent intertone interval between the test and masking tones. In the current studies, the task was modified to utilize three-tone sequences as the test stimuli. Six test sequences were employed (LMH, LHM, MLH, MHL, HLM, HML), where L, M, and H represent the lowest, middle, and highest frequencies in the melody. The observers identified these six possible sequences when the three tones of the test sequence were interleaved with three presentations of a single masking tone. All three tones of the test sequence were drawn from the same octave, while the masking tones could be drawn from any of three octaves, symmetrical around the octave containing the test tones. Under these conditions, interference occurred primarily from masking tones drawn from the same octave as the test tones. Masking tones drawn from other octaves were found to produce little, if any, interference with perception of the test tones. This effect was found to occur only for the identification of tonal sequences. Substantial masking of single-tone targets occurred with masking tones drawn from octaves other than that containing the targets. The results make apparent the use of structural information during auditory recognition. A theoretical interpretation was advanced which suggests that, while single tones are perceived on the basis of absolute pitch, the presence of auditory structure may allow relational information, such as exact pitch intervals or melodic contour, to facilitate perception of the tonal sequence.     

Online recognition of music is influenced by relative and absolute pitch information

Three experiments explored online recognition in a nonspeech domain, using a novel experimental paradigm. Adults learned to associate abstract shapes with particular melodies, and at test they identified a played melody's associated shape. To implicitly measure recognition, visual fixations to the associated shape versus a distractor shape were measured as the melody played. Degree of similarity between associated melodies was varied to assess what types of pitch information adults use in recognition. Fixation and error data suggest that adults naturally recognize music, like language, incrementally, computing matches to representations before melody offset, despite the fact that music, unlike language, provides no pressure to execute recognition rapidly. Further, adults use both absolute and relative pitch information in recognition. The implicit nature of the dependent measure should permit use with a range of populations to evaluate postulated developmental and evolutionary changes in pitch encoding.     

Harmonic,melodic, and frequency height influences in the perception of multivoiced music

Two experiments addressed the influences of harmonic relations, melody location, and relative frequency height on the perceptual organization of multivoiced music. In Experiment 1, listeners detected pitch changes in multivoiced piano music. Harmonically related pitch changes and those in the middle-frequency range were least noticeable. All pitch changes were noticeable in the high-frequency voice containing the melody (the most important voice), suggesting that melody can dominate harmonic relations. However, the presence of upper partials in the piano timbre used may have accounted for the harmonic effects. Experiment 2 employed pure sine tones, and replicated the effects of Experiment 1. In addition, the influence of the high-frequency melody on the noticeability of harmonically related pitches was lessened by the presence of a second melody. These findings suggest that harmonic, melodic, and relative frequency height relationships among voices interact in the perceptual organization of multivoiced music.     

Perceptual hysteresis in the judgment of auditory pitch shift

Perceptual hysteresis can be defined as the enduring influence of the recent past on current perception. Here, hysteresis was investigated in a basic auditory task: pitch comparisons between successive tones. On each trial, listeners were presented with pairs of tones and asked to report the direction of subjective pitch shift, as either “up” or “down.” All tones were complexes known as Shepard tones (Shepard, 1964), which comprise several frequency components at octave multiples of a base frequency. The results showed that perceptual judgments were determined both by stimulus-related factors (the interval ratio between the base frequencies within a pair) and by recent context (the intervals in the two previous trials). When tones were presented in ordered sequences, for which the frequency interval between tones was varied in a progressive manner, strong hysteresis was found. In particular, ambiguous stimuli that led to equal probabilities of “up” and “down” responses within a randomized context were almost fully determined within an ordered context. Moreover, hysteresis did not act on the direction of the reported pitch shift, but rather on the perceptual representation of each tone. Thus, hysteresis could be observed within sequences in which listeners varied between “up” and “down” responses, enabling us to largely rule out confounds related to response bias. The strength of the perceptual hysteresis observed suggests that the ongoing context may have a substantial influence on fundamental aspects of auditory perception, such as how we perceive the changes in pitch between successive sounds.     

Auditory feedback in music performance: the role of transition-based similarity

Past research has suggested that the disruptive effect of altered auditory feedback depends on how structurally similar the sequence of feedback events is to the planned sequence of actions. Three experiments pursued one basis for similarity in musical keyboard performance: matches between sequential transitions in spatial targets for movements and the melodic contour of auditory feedback. Trained pianists and musically untrained persons produced simple tonal melodies on a keyboard while hearing feedback sequences that either matched the planned melody or were contour-preserving variations of that melody. Sequence production was disrupted among pianists when feedback events were serially shifted by one event, similarly for shifts of planned melodies and tonal variations but less so for shifts of atonal variations. Nonpianists were less likely to be disrupted by serial shifts of variations but showed similar disruption to pianists for shifts of the planned melody. Thus, transitional properties and tonal schemata may jointly determine perception-action similarity during musical sequence production, and the tendency to generalize from a planned sequence to variations of it may develop with the acquisition of skill.     

Perceptual adaptation to structure-from-motion depends on the size of adaptor and probe objects,but not on the similarity of their shapes

Perceptual adaptation destabilizes the phenomenal appearance of multistable visual displays. Prolonged dominance of a perceptual state fatigues the associated neural population, lowering the likelihood of renewed perception of the same appearance (Nawrot & Blake in Perception & Psychophysics, 49, 230–44, 1991). Here, we used a selective adaptation paradigm to investigate perceptual adaptation for the illusory rotation of ambiguous structure-from-motion (SFM) displays. Specifically, we generated SFM objects with different three-dimensional shapes and presented them in random order, separating successive objects by brief blank periods, which included a mask. To assess the specificity of perceptual adaptation to the shape of SFM objects, we established the probability that a perceived direction of rotation persisted between successive objects of similar or dissimilar shape. We found that the strength of negative aftereffects depended on the volume, but not the shape, of adaptor and probe objects. More voluminous objects were both more effective as adaptor objects and more sensitive as probe objects. Surprisingly, we found these volume effects to be completely independent, since any relationship between two shapes (such as overlap between volumes, similarity of shape, or similarity of velocity profiles) failed to modulate the negative aftereffect. This pattern of results was the opposite of that observed for sensory memory of SFM objects, which reflects similarity between objects, but not volume of individual objects (Pastukhov et al. in Attention, Perception & Psychophysics, 75, 1215–1229, 2013). The disparate specificities of perceptual adaptation and sensory memory for identical SFM objects suggest that the two aftereffects engage distinct neural representations, consistent with recent brain imaging results (Schwiedrzik et al. in Cerebral Cortex, 2012).     

Encoding lexical tones in jTRACE: a simulation of monosyllabic spoken word recognition in Mandarin Chinese

Despite its prevalence as one of the most highly influential models of spoken word recognition, the TRACE model has yet to be extended to consider tonal languages such as Mandarin Chinese. A key reason for this is that the model in its current state does not encode lexical tone. In this report, we present a modified version of the jTRACE model in which we borrowed on its existing architecture to code for Mandarin phonemes and tones. Units are coded in a way that is meant to capture the similarity in timing of access to vowel and tone information that has been observed in previous studies of Mandarin spoken word recognition. We validated the model by first simulating a recent experiment that had used the visual world paradigm to investigate how native Mandarin speakers process monosyllabic Mandarin words (Malins & Joanisse, 2010). We then subsequently simulated two psycholinguistic phenomena: (1) differences in the timing of resolution of tonal contrast pairs, and (2) the interaction between syllable frequency and tonal probability. In all cases, the model gave rise to results comparable to those of published data with human subjects, suggesting that it is a viable working model of spoken word recognition in Mandarin. It is our hope that this tool will be of use to practitioners studying the psycholinguistics of Mandarin Chinese and will help inspire similar models for other tonal languages, such as Cantonese and Thai.     

Laterality effects in processing tonal and atonal melodies with affective and nonaffective task instructions

Right-handed university subjects were presented with monaural melodies that either conformed to the rules of the Western tonal system (tonal melodies) or that systematically deviated from it (atonal melodies) while containing similar contours and pitch skips. Subjects were tested under two different task instructions. One group was requested to judge whether each melody sounded correct or not (the nonaffective task); the other group had to judge whether each melody sounded pleasant or not (the affective task). The nonaffective task was found to elicit essentially no ear difference. In contrast, the affective instruction induced opposite and reliable laterality effects, depending on the valence of the response. The pleasant responses were indicative of a left hemisphere predominance and the unpleasant responses of a right hemisphere predominance. The results are consistent with the claim that the left hemisphere is biased toward positive emotions and the right to negative emotions. Moreover, the results suggest that affective appreciation of melodies is dissociable from their nonaffective judgment.