Seeing what you hear: Visual feedback improves pitch recognition

The present study examined the effect of visual feedback on the ability to recognise and consolidate pitch information. We trained two groups of nonmusicians to play a piano piece by ear, having one group receiving uninterrupted audiovisual feedback, while allowing the other only to hear, but not see their hand on the keyboard. Results indicate that subjects for whom visual information was deprived showed significantly poorer ability to recognise pitches from the musical piece they had learned. These results are interesting since pitch recognition ability would not intuitively seem to rely on visual feedback. In addition, we show that subjects with previous experience in computer touch-typing made fewer errors during training when trained with no visual feedback, but did not show improved pitch recognition ability posttraining. Our results demonstrate how sensory redundancy increases robustness of learning, and further encourage the use of audiovisual training procedures for facilitating the learning of new skills.     

Binding “what” and “where” in auditory working memory: An asymmetrical association between sound identity and sound location

Contrasting results in visual and auditory working memory studies suggest that the mechanisms of association between location and identity of stimuli depend on the sensory modality of the input. In this auditory study, we tested whether the association of two features both encoded in the “what” stream is different from the association between a “what” and a “where” feature. In an old–new recognition task, blindfolded participants were presented with sequences of sounds varying in timbre, pitch and location. They were required to judge if either the timbre, pitch or location of a single-probe stimulus was identical or different to the timbre, pitch or location of one of the sounds of the previous sequence. Only variations in one of the three features were relevant for the task, whereas the other two features could vary, with task-irrelevant changes. Results showed that task-irrelevant variations in the “what” features (either timbre or pitch) caused an impaired recognition of sound location and in the other task-relevant “what” feature, whereas changes in sound location did not affect the recognition of either one of the “what” features. We conclude that the identity of sounds is incidentally processed even when not required by the task, whereas sound location is not maintained when task irrelevant.     

Functional ear (a)symmetry in brainstem neural activity relevant to encoding of voice pitch: a precursor for hemispheric specialization?

Pitch processing is lateralized to the right hemisphere; linguistic pitch is further mediated by left cortical areas. This experiment investigates whether ear asymmetries vary in brainstem representation of pitch depending on linguistic status. Brainstem frequency-following responses (FFRs) were elicited by monaural stimulation of the left and right ear of 15 native speakers of Mandarin Chinese using two synthetic speech stimuli that differ in linguistic status of tone. One represented a native lexical tone (Tone 2: T2); the other, T2′, a nonnative variant in which the pitch contour was a mirror image of T2 with the same starting and ending frequencies. Two 40-ms portions of f₀ contours were selected in order to compare two regions (R1, early; R2 late) differing in pitch acceleration rate and perceptual saliency. In R2, linguistic status effects revealed that T2 exhibited a larger degree of FFR rightward ear asymmetry as reflected in f₀ amplitude relative to T2′. Relative to midline (ear asymmetry = 0), the only ear asymmetry reaching significance was that favoring left ear stimulation elicited by T2′. By left- and right-ear stimulation separately, FFRs elicited by T2 were larger than T2′ in the right ear only. Within T2′, FFRs elicited by the earlier region were larger than the later in both ears. Within T2, no significant differences in FFRS were observed between regions in either ear. Collectively, these findings support the idea that origins of cortical processing preferences for perceptually-salient portions of pitch are rooted in early, preattentive stages of processing in the brainstem.     

Pitch Range Variations Improve Cognitive Processing of Audio Messages

This study explores the effect of different speaker intonation strategies in audio messages on attention, autonomic arousal, and memory. In this study, participants listened to 16 radio commercials, produced to vary in pitch range across sentences. Dependent variables were self‐reported effectiveness and adequacy, psychophysiological arousal and attention, immediate word recall and recognition of information. Results showed that messages conveyed with pitch variations achieved better scores compared to commercials with a homogenous pitch range across the sentences. This was especially the case when high pitch intonation was followed by low pitch within a sentence. The results increase our understanding of the influence of pitch range on processing by establishing a concrete strategy as a best practice for improving attention and memory.     

Contrast Is in the Eye of the Beholder: Infelicitous Beat Gesture Increases Cognitive Load During Online Spoken Discourse Comprehension

We investigated how two cues to contrast—beat gesture and contrastive pitch accenting—affect comprehenders' cognitive load during processing of spoken referring expressions. In two visual-world experiments, we orthogonally manipulated the presence of these cues and their felicity, or fit, with the local (sentence-level) referential context in critical referring expressions while comprehenders' task-evoked pupillary responses (TEPRs) were examined. In Experiment 1, beat gesture and contrastive accenting always matched the referential context of filler referring expressions and were therefore relatively felicitous on the global (experiment) level, whereas in Experiment 2, beat gesture and contrastive accenting never fit the referential context of filler referring expressions and were therefore infelicitous on the global level. The results revealed that both beat gesture and contrastive accenting increased comprehenders' cognitive load. For beat gesture, this increase in cognitive load was driven by both local and global infelicity. For contrastive accenting, this increase in cognitive load was unaffected when cues were globally felicitous but exacerbated when cues were globally infelicitous. Together, these results suggest that comprehenders' cognitive resources are taxed by processing infelicitous use of beat gesture and contrastive accenting to convey contrast on both the local and global levels.