Perceptual interactions between musical pitch and timbre.

These experiments examined perceptual interactions between musical pitch and timbre. Experiment 1, through the use of the Garner classification tasks, found that pitch and timbre of isolated tones interact. Classification times showed interference from uncorrelated variation in the irrelevant attribute and facilitation from correlated variation; the effects were symmetrical. Experiments 2 and 3 examined how musical pitch and timbre function in longer sequences. In recognition memory tasks, a target tone always appeared in a fixed position in the sequences, and listeners were instructed to attend to either its pitch or its timbre. For successive tones, no interactions between timbre and pitch were found. That is, changing the pitches of context tones did not affect timbre recognition, and vice versa. The tendency to perceive pitch in relation to other context pitches was strong and unaffected by whether timbre was constant or varying. In contrast, the relative perception of timbre was weak and was found only when pitch was constant. These results suggest that timbre is perceived more in absolute than in relative terms. Perceptual implications for creating patterns in music with timbre variations are discussed.     

Illusory conjunctions of pitch and duration in unfamiliar tone sequences

In 3 experiments, the authors examined short-term memory for pitch and duration in unfamiliar tone sequences. Participants were presented a target sequence consisting of 2 tones (Experiment 1) or 7 tones (Experiments 2 and 3) and then a probe tone. Participants indicated whether the probe tone matched 1 of the target tones in both pitch and duration. Error rates were relatively low if the probe tone matched 1 of the target tones or if it differed from target tones in pitch, duration, or both. Error rates were remarkably high, however, if the probe tone combined the pitch of 1 target tone with the duration of a different target tone. The results suggest that illusory conjunctions of these dimensions frequently occur. A mathematical model is presented that accounts for the relative contribution of pitch errors, duration errors, and illusory conjunctions of pitch and duration.     

The influence of linguistic experience on the cognitive processing of pitch in speech and nonspeech sounds

In the present experiment, the authors tested Mandarin and English listeners on a range of auditory tasks to investigate whether long-term linguistic experience influences the cognitive processing of nonspeech sounds. As expected, Mandarin listeners identified Mandarin tones significantly more accurately than English listeners; however, performance did not differ across the listener groups on a pitch discrimination task requiring fine-grained discrimination of simple nonspeech sounds. The crucial finding was that cross-language differences emerged on a nonspeech pitch contour identification task: The Mandarin listeners more often misidentified flat and falling pitch contours than the English listeners in a manner that could be related to specific features of the sound structure of Mandarin, which suggests that the effect of linguistic experience extends to nonspeech processing under certain stimulus and task conditions.     

On the Reality of Illusory Conjunctions

The reality of illusory conjunctions in perception has been sometimes questioned, arguing that they can be explained by other mechanisms. Most relevant experiments are based on migrations along the space dimension. But the low rate of illusory conjunctions along space can easily hide them among other types of errors. As migrations over time are a more frequent phenomenon, illusory conjunctions can be disentangled from other errors. We report an experiment in which series of colored letters were presented in several spatial locations, allowing for migrations over both space and time. The distribution of frequencies were fit by several multinomial tree models based on alternative hypothesis about illusory conjunctions and the potential sources of free-floating features. The best-fit model acknowledges that most illusory conjunctions are migrations in the time domain. Migrations in space are probably present, but the rate is very low. Other conjunction errors, as those produced by guessing or miscategorizations of the to-be-reported feature, are also present in the experiment. The main conclusion is that illusory conjunctions do exist.     

Distant melodies: statistical learning of nonadjacent dependencies in tone sequences

Human listeners can keep track of statistical regularities among temporally adjacent elements in both speech and musical streams. However, for speech streams, when statistical regularities occur among nonadjacent elements, only certain types of patterns are acquired. Here, using musical tone sequences, the authors investigate nonadjacent learning. When the elements were all similar in pitch range and timbre, learners acquired moderate regularities among adjacent tones but did not acquire highly consistent regularities among nonadjacent tones. However, when elements differed in pitch range or timbre, learners acquired statistical regularities among the similar, but temporally nonadjacent, elements. Finally, with a moderate grouping cue, both adjacent and nonadjacent statistics were learned, indicating that statistical learning is governed not only by temporal adjacency but also by Gestalt principles of similarity.     

The role of attention in illusory conjunctions

In five experiments, we investigated the effects of attention on illusory conjunctions formed between features of unrelated objects. The first three experiments used a weak manipulation of attention and found that illusory conjunctions formed either among features receiving high attentional priority or among features receiving low attentional priority were not more frequent than were conjunctions formed between mixed features of different attentional priority. The last two experiments used a strong manipulation of attention and failed to reveal any evidence of true illusory conjunctions. The results are inconsistent with the feature-integration theory, which predicts that when attention is focused on a subset of items, illusory conjunctions ought to occur within and outside of the attended subset, but not between the attended and unattended items.     

Short-term forgetting without interference

In the 1st reported experiment, we demonstrate that auditory memory is robust over extended retention intervals (RIs) when listeners compare the timbre of complex tones, even when active or verbal rehearsal is difficult or impossible. Thus, our tones have an abstract timbre that resists verbal labeling, they differ across trials so that no "standard" comparison stimulus is built up, and the spectral change to be discriminated is very slight and therefore does not shift stimuli across verbal categories. Nonetheless, performance in this nonverbal immediate memory task was better at short (1-, 2-, or 4-s) than long (8-, 16-, or 32-s) RIs, an outcome predicted by temporal distinctiveness theory whereby at long RIs, tones are closer in time to tones on previous trials. We reject this account in the 2nd experiment, where we demonstrate that the ratio of RI to intertrial interval makes absolutely no difference to performance. We suggest that steady forgetting is consistent with a psychoacoustically derived conception of an auditory memory (the timbre memory model) that embodies time-based forgetting in the absence of feature-specific interference.     

Illusory conjunctions are alive and well: a reply to Donk (1999).

When presented with a red T and a green O, observers occasionally make conjunction responses and indicate that they saw a green T. These errors have been interpreted as reflecting separable processing stages of feature detection and integration with the illusory conjunctions arising from a failure at the integration stage. Recently, M. Donk (1999) asserted that the phenomenon of illusory conjunctions is an artifact. Conjunction reports are actually the result of confusing a nontarget item (O in the example above) for a target item (the letter T) and (correctly) reporting the color associated with the (incorrectly) selected target. The authors demonstrate that although target-nontarget confusion errors are a potential source of conjunction reports, there is a plethora of findings that cannot be accounted for by this confusion model. A review of the literature indicates that in many studies, illusory conjunctions do result from a failure to properly integrate features.     

Rate and structure in memory for auditory patterns

Musically knowledgeable listeners heard auditory patterns based on sets of six (Study 1) or eight tones (Study 2). In the first study, listeners ordered events from patterns generated by hierarchical rule trees and which possessed different pitch space and time structures: one type (nondistance nested) was more likely to produce auditory streaming than the other (distance nested). In the second study, different listeners reconstructed pitch intervals contained in one of eight patterns. Patterns differed according to (1) levels of pitch distance occur), (2) levels of pattern contour (two), and (3) rate (two). In both studies, fast patterns with many large pitch distances were more difficult to recollect. Listeners in the second study telided to “telescope” pitch distances. Most difficult were those rapid sequences with large patch intervals combined into a changing contour (nondistance nested); these patterns streamed. A third study replicated effects due to differences in pitch distances observed in Study 2. Results were interpreted in terms of a rhythmic theory of memory.     

Emergent features, attention, and object perception

The perceptual processing of arrows and triangles and of their component angles and lines was explored in a number of different tasks. The results suggest that some analysis of shapes into simpler parts occurs preattentively, because these parts can recombine to form illusory conjunctions when attention is divided. The presence of "emergent features," such as closure or arrow junctions, was inferred from predicted correlations in the pattern of performance across tasks and across individual subjects. Thus triangles (for most subjects) and arrows (for some subjects) behave as if they had a perceptual feature that is absent from their parts and that mediates parallel detection in search and easy texture segregation. For some subjects, circles could apparently supply the additional feature (presumably closure) required to form illusory triangles from their component lines, whereas for other subjects circles had no effect. The fact that triangle lines can form illusory conjunctions with another shape makes it unlikely that triangles are perceived holistically and strengthens the interpretation that relies on emergent features.     

Binding “what” and “where” in auditory working memory: An asymmetrical association between sound identity and sound location

Contrasting results in visual and auditory working memory studies suggest that the mechanisms of association between location and identity of stimuli depend on the sensory modality of the input. In this auditory study, we tested whether the association of two features both encoded in the “what” stream is different from the association between a “what” and a “where” feature. In an old–new recognition task, blindfolded participants were presented with sequences of sounds varying in timbre, pitch and location. They were required to judge if either the timbre, pitch or location of a single-probe stimulus was identical or different to the timbre, pitch or location of one of the sounds of the previous sequence. Only variations in one of the three features were relevant for the task, whereas the other two features could vary, with task-irrelevant changes. Results showed that task-irrelevant variations in the “what” features (either timbre or pitch) caused an impaired recognition of sound location and in the other task-relevant “what” feature, whereas changes in sound location did not affect the recognition of either one of the “what” features. We conclude that the identity of sounds is incidentally processed even when not required by the task, whereas sound location is not maintained when task irrelevant.     

Infant music perception: domain-general or domain-specific mechanisms?

We review the literature on infants' perception of pitch and temporal patterns, relating it to comparable research with human adult and non-human listeners. Although there are parallels in relative pitch processing across age and species, there are notable differences. Infants accomplish such tasks with ease, but non-human listeners require extensive training to achieve very modest levels of performance. In general, human listeners process auditory sequences in a holistic manner, and non-human listeners focus on absolute aspects of individual tones. Temporal grouping processes and categorization on the basis of rhythm are evident in non-human listeners and in human infants and adults. Although synchronization to sound patterns is thought to be uniquely human, tapping to music, synchronous firefly flashing, and other cyclic behaviors can be described by similar mathematical principles. We conclude that infants' music perception skills are a product of general perceptual mechanisms that are neither music- nor species-specific. Along with general-purpose mechanisms for the perceptual foundations of music, we suggest unique motivational mechanisms that can account for the perpetuation of musical behavior in all human societies.     

An interval size illusion: the influence of timbre on the perceived size of melodic intervals

In four experiments, we investigated the influence of timbre on perceived interval size. In Experiment 1, musically untrained participants heard two successive tones and rated the pitch distance between them. Tones were separated by six or seven semitones and varied in timbre. Pitch changes were accompanied by a congruent timbre change (e.g., ascending interval involving a shift from a dull to a bright timbre), an incongruent timbre change (e.g., ascending interval involving a shift from a bright to a dull timbre), or no timbre change. Ratings of interval size were strongly influenced by timbre. The six-semitone interval with a congruent timbre change was perceived to be larger than the seven-semitone interval with an incongruent timbre change (interval illusion). Experiment 2 revealed similar effects for musically trained participants. In Experiment 3, participants compared the size of two intervals presented one after the other. Effects of timbre were again observed, including evidence of an interval illusion. Experiment 4 confirmed that timbre manipulations did not distort the perceived pitch of tones. Changes in timbre can expand or contract the perceived size of intervals without distorting individual pitches. We discuss processes underlying interval size perception and their relation to pitch perception mechanisms.     

On the role of space and time in auditory processing

Unlike visual and tactile stimuli, auditory signals that allow perception of timbre, pitch and localization are temporal. To process these, the auditory nervous system must either possess specialized neural machinery for analyzing temporal input, or transform the initial responses into patterns that are spatially distributed across its sensory epithelium. The former hypothesis, which postulates the existence of structures that facilitate temporal processing, is most popular. However, I argue that the cochlea transforms sound into spatiotemporal response patterns on the auditory nerve and central auditory stages; and that a unified computational framework exists for central auditory, visual and other sensory processing. Specifically, I explain how four fundamental concepts in visual processing play analogous roles in auditory processing.     

The octave illusion revisited: suppression or fusion between ears?

The octave illusion occurs when each ear receives a sequence of tones alternating by 1 octave but with the high and low tones in different ears. Most listeners perceive these stimuli as a high pitch in one ear alternating with a low pitch in the other ear. D. Deutsch and P. L. Roll (1976) interpreted this phenomenon as evidence for a what-where division of auditory processing caused by sequential interactions between the tones. They argued that the pitch follows the frequency presented to the dominant ear but is lateralized toward the higher frequency component. This model was examined in 4 experiments. Results indicate that the perceived pitch approximates the fundamental frequency and that the illusion does not depend on sequential interactions. The octave illusion may arise from an interaction between dichotic fusion and binaural diplacusis rather than from suppression as proposed by Deutsch.     

The development of separability in visual perception

Under conditions that do not allow focused attention, reports of illusory conjunctions (i.e. errors that wrongly recombine the features of different objects) constitute evidence of the separate registering of features at an early processing level. The occurrence of illusory conjunctions was used to determine whether there is preattentive analysis of component dimensions (colour and form) and of parts of shapes (triangles and arrows) in young children aged 5 to 8 years. Evidence of preattentive analysis was found, even for the youngest children, for colour and form but not for parts of shapes. Although developmental effects hardly reached significance, inspection of abilities assumed to affect the illusory conjunctions phenomenon suggests that at the preattentive stage children can integrate spatially separate segments but lack the capacity to fully analyse connected segments.     

Perceptual hysteresis in the judgment of auditory pitch shift

Perceptual hysteresis can be defined as the enduring influence of the recent past on current perception. Here, hysteresis was investigated in a basic auditory task: pitch comparisons between successive tones. On each trial, listeners were presented with pairs of tones and asked to report the direction of subjective pitch shift, as either “up” or “down.” All tones were complexes known as Shepard tones (Shepard, 1964), which comprise several frequency components at octave multiples of a base frequency. The results showed that perceptual judgments were determined both by stimulus-related factors (the interval ratio between the base frequencies within a pair) and by recent context (the intervals in the two previous trials). When tones were presented in ordered sequences, for which the frequency interval between tones was varied in a progressive manner, strong hysteresis was found. In particular, ambiguous stimuli that led to equal probabilities of “up” and “down” responses within a randomized context were almost fully determined within an ordered context. Moreover, hysteresis did not act on the direction of the reported pitch shift, but rather on the perceptual representation of each tone. Thus, hysteresis could be observed within sequences in which listeners varied between “up” and “down” responses, enabling us to largely rule out confounds related to response bias. The strength of the perceptual hysteresis observed suggests that the ongoing context may have a substantial influence on fundamental aspects of auditory perception, such as how we perceive the changes in pitch between successive sounds.     

Is Posner's "beam" the same as Treisman's "glue"?: On the relation between visual orienting and feature integration theory

In the present study we investigated whether the visually allocated "beam" studied by Posner and others is the same visual attentional resource that performs the role of feature integration in Treisman's model. Subjects were cued to attend to a certain spatial location by a visual cue, and performance at expected and unexpected stimulus locations was compared. Subjects searched for a target letter (R) with distractor letters that either could give rise to illusory conjunctions (PQ) or could not (PB). Results from three separate experiments showed that orienting attention in response to central cues (endogenous orienting) showed similar effects for both conjunction and feature search. However, when attention was oriented with peripheral visual cues (exogenous orienting), conjunction search showed larger effects of attention than did feature search. It is suggested that the attentional systems that are oriented in response to central and peripheral cues may not be the same and that only the latter performs a role in feature integration. Possibilities for future research are discussed.     

Ear of input as a determinant of pitch-memory interference

Six experiments to evaluate the effect of presentation ear on pitch-memory interference were conducted using undergraduates as listeners. The task was to compare the pitch of two tones that were separated by an interval that included eight interpolated tones; the interpolated tones were presented either ipsilaterally or contralaterally to the presentation ear of the comparison tones. When the ear of interpolated-tone presentations was blocked, and therefore predictable, ipsilateral interference was greater than contralateral. In contrast, when the interpolated-tone presentation ear was varied randomly from trial to trial, ipsilateral and contralateral interferences were equivalent. These results are analogous to results found in previously reported auditory backward recognition masking (ABRM) experiments and suggest that the ABRM effect may be due, in part, to pitch-memory interference. Implications for theories of auditory processing and memory are discussed.     

Memory for surface features of unfamiliar melodies: independent effects of changes in pitch and tempo

A melody’s identity is determined by relations between consecutive tones in terms of pitch and duration, whereas surface features (i.e., pitch level or key, tempo, and timbre) are irrelevant. Although surface features of highly familiar recordings are encoded into memory, little is known about listeners’ mental representations of melodies heard once or twice. It is also unknown whether musical pitch is represented additively or interactively with temporal information. In two experiments, listeners heard unfamiliar melodies twice in an initial exposure phase. In a subsequent test phase, they heard the same (old) melodies interspersed with new melodies. Some of the old melodies were shifted in key, tempo, or key and tempo. Listeners’ task was to rate how well they recognized each melody from the exposure phase while ignoring changes in key and tempo. Recognition ratings were higher for old melodies that stayed the same compared to those that were shifted in key or tempo, and detrimental effects of key and tempo changes were additive in between-subjects (Experiment 1) and within-subjects (Experiment 2) designs. The results confirm that surface features are remembered for melodies heard only twice. They also imply that key and tempo are processed and stored independently.