Effects of intelligibility on working memory demand for speech perception

Understanding low-intelligibility speech is effortful. In three experiments, we examined the effects of intelligibility on working memory (WM) demands imposed by perception of synthetic speech. In all three experiments, a primary speeded word recognition task was paired with a secondary WM-load task designed to vary the availability of WM capacity during speech perception. Speech intelligibility was varied either by training listeners to use available acoustic cues in a more diagnostic manner (as in Experiment 1) or by providing listeners with more informative acoustic cues (i.e., better speech quality, as in Experiments 2 and 3). In the first experiment, training significantly improved intelligibility and recognition speed; increasing WM load significantly slowed recognition. A significant interaction between training and load indicated that the benefit of training on recognition speed was observed only under low memory load. In subsequent experiments, listeners received no training; intelligibility was manipulated by changing synthesizers. Improving intelligibility without training improved recognition accuracy, and increasing memory load still decreased it, but more intelligible speech did not produce more efficient use of available WM capacity. This suggests that perceptual learning modifies the way available capacity is used, perhaps by increasing the use of more phonetically informative features and/or by decreasing use of less informative ones.     

情绪的动机强度与工作记忆任务类型的协调性对自我控制任务的影响

考察趋近动机强度不同的积极情绪和回避动机强度不同的消极情绪与工作记忆的协调性对自我控制任务的影响。结果发现,与高趋近动机的积极情绪、高回避动机的消极情绪以及中性情绪相比,在低趋近动机的积极情绪下完成言语工作记忆以及在低回避动机的消极情绪下完成空间工作记忆,工作记忆成绩提高,而Stroop分数降低,这表明情绪与工作记忆的协调性受到情绪动机强度的影响。     

Word recognition reflects dimension-based statistical learning

Speech processing requires sensitivity to long-term regularities of the native language yet demands listeners to flexibly adapt to perturbations that arise from talker idiosyncrasies such as nonnative accent. The present experiments investigate whether listeners exhibit dimension-based statistical learning of correlations between acoustic dimensions defining perceptual space for a given speech segment. While engaged in a word recognition task guided by a perceptually unambiguous voice-onset time (VOT) acoustics to signal beer, pier, deer, or tear, listeners were exposed incidentally to an artificial "accent" deviating from English norms in its correlation of the pitch onset of the following vowel (F0) to VOT. Results across four experiments are indicative of rapid, dimension-based statistical learning; reliance on the F0 dimension in word recognition was rapidly down-weighted in response to the perturbation of the correlation between F0 and VOT dimensions. However, listeners did not simply mirror the short-term input statistics. Instead, response patterns were consistent with a lingering influence of sensitivity to the long-term regularities of English. This suggests that the very acoustic dimensions defining perceptual space are not fixed and, rather, are dynamically and rapidly adjusted to the idiosyncrasies of local experience, such as might arise from nonnative-accent, dialect, or dysarthria. The current findings extend demonstrations of "object-based" statistical learning across speech segments to include incidental, online statistical learning of regularities residing within a speech segment.     

Individual Differences in Categorization Gradience As Predicted by Online Processing of Phonetic Cues During Spoken Word Recognition: Evidence From Eye Movements

Recent studies have documented substantial variability among typical listeners in how gradiently they categorize speech sounds, and this variability in categorization gradience may link to how listeners weight different cues in the incoming signal. The present study tested the relationship between categorization gradience and cue weighting across two sets of English contrasts, each varying orthogonally in two acoustic dimensions. Participants performed a four-alternative forced-choice identification task in a visual world paradigm while their eye movements were monitored. We found that (a) greater categorization gradience derived from behavioral identification responses corresponds to larger secondary cue weights derived from eye movements; (b) the relationship between categorization gradience and secondary cue weighting is observed across cues and contrasts, suggesting that categorization gradience may be a consistent within-individual property in speech perception; and (c) listeners who showed greater categorization gradience tend to adopt a buffered processing strategy, especially when cues arrive asynchronously in time.     

Working memory load and distraction: dissociable effects of visual maintenance and cognitive control

We establish a new dissociation between the roles of working memory (WM) cognitive control and visual maintenance in selective attention as measured by the efficiency of distractor rejection. The extent to which focused selective attention can prevent distraction has been shown to critically depend on the level and type of load involved in the task. High perceptual load that consumes perceptual capacity leads to reduced distractor processing, whereas high WM load that reduces WM ability to exert priority-based executive cognitive control over the task results in increased distractor processing (e.g., Lavie, Trends in Cognitive Sciences, 9(2), 75–82, 2005). WM also serves to maintain task-relevant visual representations, and such visual maintenance is known to recruit the same sensory cortices as those involved in perception (e.g., Pasternak & Greenlee, Nature Reviews Neuroscience, 6(2), 97–107, 2005). These findings led us to hypothesize that loading WM with visual maintenance would reduce visual capacity involved in perception, thus resulting in reduced distractor processing—similar to perceptual load and opposite to WM cognitive control load. Distractor processing was assessed in a response competition task, presented during the memory interval (or during encoding; Experiment 1a) of a WM task. Loading visual maintenance or encoding by increased set size for a memory sample of shapes, colors, and locations led to reduced distractor response competition effects. In contrast, loading WM cognitive control with verbal rehearsal of a random letter set led to increased distractor effects. These findings confirm load theory predictions and provide a novel functional distinction between the roles of WM maintenance and cognitive control in selective attention.     

Enhanced music sensitivity in 9-month-old bilingual infants

This study explores the influence of bilingualism on the cognitive processing of language and music. Specifically, we investigate how infants learning a non-tone language perceive linguistic and musical pitch and how bilingualism affects cross-domain pitch perception. Dutch monolingual and bilingual infants of 8–9 months participated in the study. All infants had Dutch as one of the first languages. The other first languages, varying among bilingual families, were not tone or pitch accent languages. In two experiments, infants were tested on the discrimination of a lexical (N = 42) or a violin (N = 48) pitch contrast via a visual habituation paradigm. The two contrasts shared identical pitch contours but differed in timbre. Non-tone language learning infants did not discriminate the lexical contrast regardless of their ambient language environment. When perceiving the violin contrast, bilingual but not monolingual infants demonstrated robust discrimination. We attribute bilingual infants’ heightened sensitivity in the musical domain to the enhanced acoustic sensitivity stemming from a bilingual environment. The distinct perceptual patterns between language and music and the influence of acoustic salience on perception suggest processing diversion and association in the first year of life. Results indicate that the perception of music may entail both shared neural network with language processing, and unique neural network that is distinct from other cognitive functions.     

Temporally nonadjacent nonlinguistic sounds affect speech categorization

Speech perception is an ecologically important example of the highly context-dependent nature of perception; adjacent speech, and even nonspeech, sounds influence how listeners categorize speech. Some theories emphasize linguistic or articulation-based processes in speech-elicited context effects and peripheral (cochlear) auditory perceptual interactions in non-speech-elicited context effects. The present studies challenge this division. Results of three experiments indicate that acoustic histories composed of sine-wave tones drawn from spectral distributions with different mean frequencies robustly affect speech categorization. These context effects were observed even when the acoustic context temporally adjacent to the speech stimulus was held constant and when more than a second of silence or multiple intervening sounds separated the nonlinguistic acoustic context and speech targets. These experiments indicate that speech categorization is sensitive to statistical distributions of spectral information, even if the distributions are composed of nonlinguistic elements. Acoustic context need be neither linguistic nor local to influence speech perception.     

Selective Association Between Tetris Game Play and Visuospatial Working Memory: A Preliminary Investigation

Recent experimental and clinical research has suggested that Tetris game play can disrupt maladaptive forms of mental imagery because Tetris competes for limited cognitive resources within visuospatial working memory (WM) that contribute to imagery. Whether or not Tetris performance is selectively associated with visuospatial WM remains to be tested. In this study, young adults (N = 46) completed six standardized measures indexing verbal and non‐verbal reasoning, verbal and visuospatial short‐term memory, and verbal and visuospatial WM. They also played Tetris. Consistent with the hypothesis that visuospatial WM resources support Tetris game play, there was a significant moderate positive relationship between Tetris scores and visuospatial WM performance but no association with other cognitive ability measures. Findings suggest that Tetris game play involves both storage and processing resources within visuospatial WM. These preliminary results can inform interventions involving computer games to disrupt the development of maladaptive visual imagery, for example, intrusive memories of trauma. © 2017 The Authors. Applied Cognitive Psychology Published by John Wiley & Sons Ltd.     

工作记忆负荷对无意注意的影响

通过探讨工作记忆负荷对无意注意的影响,考察工作记忆的认知控制功能对选择性注意的作用机制。为了避免工作记忆和选择注意的目标或者干扰子之间的知觉注意资源的竞争,采用言语工作记忆任务和视觉空间位置辨别任务,同时视觉干扰刺激采用被试无法识别的韩语。结果发现,在高负荷条件下,干扰效应显著;而在低和中负荷条件下,没有观察到干扰效应。说明在高负荷条件下,被试没有额外的资源来抑制干扰加工,从而出现了显著的干扰效应。由此,本研究支持负荷理论的观点：工作记忆在选择性注意中执行认知控制功能。     

Evidence for hierarchical categorization of coarticulated phonemes

The reported research investigates how listeners recognize coarticulated phonemes. First, 2 data sets from experiments on the recognition of coarticulated phonemes published by D. H. Whalen (1989) are reanalyzed. The analyses indicate that listeners used categorization strategies involving a hierarchical dependency. Two new experiments are reported investigating the production and perception of fricative-vowel syllables. On the basis of measurements of acoustic cues on a large set of natural utterances, it was predicted that listeners would use categorization strategies involving a dependency of the fricative categorization on the perceived vowel. The predictions were tested in a perception experiment using a 2-dimensional synthetic fricative-vowel continuum. Model analyses of the results pooled across listeners confirmed the predictions. Individual analyses revealed some variability in the categorization dependencies used by different participants.     

Cognitive basis of individual differences in speech perception,production and representations: The role of domain general attentional switching

This study investigated whether individual differences in cognitive functions, attentional abilities in particular, were associated with individual differences in the quality of phonological representations, resulting in variability in speech perception and production. To do so, we took advantage of a tone merging phenomenon in Cantonese, and identified three groups of typically developed speakers who could differentiate the two rising tones (high and low rising) in both perception and production [+Per+Pro], only in perception [+Per–Pro], or in neither modalities [–Per–Pro]. Perception and production were reflected, respectively, by discrimination sensitivity d′ and acoustic measures of pitch offset and rise time differences. Components of event-related potential (ERP)—the mismatch negativity (MMN) and the ERPs to amplitude rise time—were taken to reflect the representations of the acoustic cues of tones. Components of attention and working memory in the auditory and visual modalities were assessed with published test batteries. The results show that individual differences in both perception and production are linked to how listeners encode and represent the acoustic cues (pitch contour and rise time) as reflected by ERPs. The present study has advanced our knowledge from previous work by integrating measures of perception, production, attention, and those reflecting quality of representation, to offer a comprehensive account for the underlying cognitive factors of individual differences in speech processing. Particularly, it is proposed that domain-general attentional switching affects the quality of perceptual representations of the acoustic cues, giving rise to individual differences in perception and production.     

Acoustic differences, listener expectations, and the perceptual accommodation of talker variability

Two talkers' productions of the same phoneme may be quite different acoustically, whereas their productions of different speech sounds may be virtually identical. Despite this lack of invariance in the relationship between the speech signal and linguistic categories, listeners experience phonetic constancy across a wide range of talkers, speaking styles, linguistic contexts, and acoustic environments. The authors present evidence that perceptual sensitivity to talker variability involves an active cognitive mechanism: Listeners expecting to hear 2 different talkers differing only slightly in average pitch showed performance costs typical of adjusting to talker variability, whereas listeners hearing the same materials but expecting a single talker or given no special instructions did not show these performance costs. The authors discuss the implications for understanding phonetic constancy despite variability between talkers (and other sources of variability) and for theories of speech perception. The results provide further evidence for active, controlled processing in real-time speech perception and are consistent with a model of talker normalization that involves contextual tuning.     

Why does working memory span predict complex cognition? Testing the strategy affordance hypothesis

We introduce and empirically evaluate the strategy affordance hypothesis, which holds that individual differences in strategy use will mediate the relationship between performances on a working memory (WM) span task and another cognitive task only when the same strategies are afforded by both tasks. One hundred forty-eight participants completed basic memory tasks and verbal span tasks that afford the same strategies, such as imagery and sentence generation, and completed reading comprehension tasks that afford different ones, such as self-questioning and summarization. Effective strategy use on WM span tasks accounted for variance in the span-memory relationship, but not for the span-comprehension relationship, supporting the strategy affordance hypothesis. Strategy use mediated the span-cognition relationship only when both tasks afforded the same strategies.     

Expanding the mind’s workspace: Training and transfer effects with a complex working memory span task

In the present study, a novel working memory (WM) training paradigm was used to test the malleability of WM capacity and to determine the extent to which the benefits of this training could be transferred to other cognitive skills. Training involved verbal and spatial versions of a complex WM span task designed to emphasize simultaneous storage and processing requirements. Participants who completed 4 weeks of WM training demonstrated significant improvements on measures of temporary memory. These WM training benefits generalized to performance on the Stroop task and, in a novel finding, promoted significant increases in reading comprehension. The results are discussed in relation to the hypothesis that WM training affects domain-general attention control mechanisms and can thereby elicit far-reaching cognitive benefits. Implications include the use of WM training as a general tool for enhancing important cognitive skills.     

Prior experience with negative spectral correlations promotes information integration during auditory category learning

Complex sounds vary along a number of acoustic dimensions. These dimensions may exhibit correlations that are familiar to listeners due to their frequent occurrence in natural sounds—namely, speech. However, the precise mechanisms that enable the integration of these dimensions are not well understood. In this study, we examined the categorization of novel auditory stimuli that differed in the correlations of their acoustic dimensions, using decision bound theory. Decision bound theory assumes that stimuli are categorized on the basis of either a single dimension (rule based) or the combination of more than one dimension (information integration) and provides tools for assessing successful integration across multiple acoustic dimensions. In two experiments, we manipulated the stimulus distributions such that in Experiment 1, optimal categorization could be accomplished by either a rule-based or an information integration strategy, while in Experiment 2, optimal categorization was possible only by using an information integration strategy. In both experiments, the pattern of results demonstrated that unidimensional strategies were strongly preferred. Listeners focused on the acoustic dimension most closely related to pitch, suggesting that pitch-based categorization was given preference over timbre-based categorization. Importantly, in Experiment 2, listeners also relied on a two-dimensional information integration strategy, if there was immediate feedback. Furthermore, this strategy was used more often for distributions defined by a negative spectral correlation between stimulus dimensions, as compared with distributions with a positive correlation. These results suggest that prior experience with such correlations might shape short-term auditory category learning.     

The influence of linguistic experience on the cognitive processing of pitch in speech and nonspeech sounds

In the present experiment, the authors tested Mandarin and English listeners on a range of auditory tasks to investigate whether long-term linguistic experience influences the cognitive processing of nonspeech sounds. As expected, Mandarin listeners identified Mandarin tones significantly more accurately than English listeners; however, performance did not differ across the listener groups on a pitch discrimination task requiring fine-grained discrimination of simple nonspeech sounds. The crucial finding was that cross-language differences emerged on a nonspeech pitch contour identification task: The Mandarin listeners more often misidentified flat and falling pitch contours than the English listeners in a manner that could be related to specific features of the sound structure of Mandarin, which suggests that the effect of linguistic experience extends to nonspeech processing under certain stimulus and task conditions.     

Dialectal differences in hemispheric specialization for Japanese lexical pitch accent

Language experience can alter perceptual abilities and the neural specialization for phonological contrasts. Here we investigated whether dialectal differences in the lexical use of pitch information lead to differences in functional lateralization for pitch processing. We measured cortical hemodynamic responses to pitch pattern changes in native speakers of Standard (Tokyo) Japanese, which has a lexical pitch accent system, and native speakers of ‘accentless’ dialects, which do not have any lexical tonal phenomena. While the Standard Japanese speakers showed left-dominant responses in temporal regions to pitch pattern changes within words, the accentless dialects speakers did not show such left-dominance. Pitch pattern changes within harmonic-complex tones also elicited different brain activation patterns between the two groups. These results indicate that the neural processing of pitch information differs depending on the listener’s native dialect, and that listeners’ linguistic experiences may further affect the processing of pitch changes even for non-linguistic sounds.     

The effect of familiarity on within-person age judgements from voices

Listeners can perceive a person’s age from their voice with above chance accuracy. Studies have usually established this by asking listeners to directly estimate the age of unfamiliar voices. The recordings used mostly include cross-sectional samples of voices, including people of different ages to cover the age range of interest. Such cross-sectional samples likely include not only cues to age in the sound of the voice but also socio-phonetic cues, encoded in how a person speaks. How age perpcetion accuracy is affected when minimizing socio-phonetic cues by sampling the same voice at different time points remains largely unknown. Similarly, with the voices in age perception studies being usually unfamiliar to listeners, it is unclear how familiarity with a voice affects age perception. We asked listeners who were either familiar or unfamiliar with a set of four voices to complete an age discrimination task: listeners heard two recordings of the same person’s voice, recorded 15 years apart, and were asked to indicate in which recording the person was younger. Accuracy for both familiar and unfamiliar listeners was above chance. While familiarity advantages were apparent, accuracy was not particularly high: familiar and unfamiliar listeners were correct for 68.2% and 62.7% of trials, respectively (chance = 50%). Familiarity furthermore interacted with the voices included. Overall, our findings indicate that age perception from voices is not a trivial task at all times – even when listeners are familiar with a voice. We discuss our findings in the light of how reliable voice may be as a signal for age.     

Planning sentences while doing other things at the same time: effects of concurrent verbal and visuospatial working memory load

This study investigated to what extent advance planning during sentence production is affected by a concurrent cognitive load. In two picture–word interference experiments in which participants produced subject–verb–object sentences while ignoring auditory distractor words, we assessed advance planning at a phonological (lexeme) and at an abstract–lexical (lemma) level under visuospatial or verbal working memory (WM) load. At the phonological level, subject and object nouns were found to be activated before speech onset with concurrent visuospatial WM load, but only subject nouns were found to be activated with concurrent verbal WM load, indicating a reduced planning scope as a function of type of WM load (Experiment 1). By contrast, at the abstract–lexical level, subject and object nouns were found to be activated regardless of type of concurrent load (Experiment 2). In both experiments, sentence planning had a more detrimental effect on concurrent verbal WM task performance than on concurrent visuospatial WM task performance. Overall, our results suggest that advance planning at the phonological level is more affected by a concurrently performed verbal WM task than advance planning at the abstract–lexical level. Also, they indicate an overlap of resources allocated to phonological planning in speech production and verbal WM.