Privileged Access by Irrelevant Speech to Short-term Memory: The Role of Changing State

Memory for visually presented items is impaired by speech that is played as an irrelevant background. The paper presents the view that changing state of the auditory material is an important prerequisite for this disruption. Four experiments studied the effects of sounds varying in complexity in an attempt to establish which features of changing state in the auditory signal lead to diminished recall. Simple unvarying or repetitive speech sounds were not sufficient to induce the irrelevant speech effect (Experiment 1): in addition, simple analogues of speech, possessing regular or irregular envelopes and using a range of carriers, failed to imitate the action of speech (Experiment 2). Variability of between-utterance phonology in the irrelevant stream (Experiment 3) emerged as a crucial factor. Moreover, predictability of the syllable sequence did not reduce the degree of disruption (Experiment 4) suggesting that supra-syllabic characteristics of the speech are of little importance. The results broadly support the idea that disruption of short-term memory only occurs when the speech stream changes in state. It is argued that disruption occurs in memory when cues to serial order based on phonological representations of heard material interfere with the phonological codes of visual origin. It is suggested that cues to changing state of the speech input contaminate those associated with items of visual origin, which are already in a phonological store.     

Modeling the effects of irrelevant speech on memory

The feature model (Nairne, 1990) is extended to account for the effects of irrelevant speech and concomitant interactions in immediate serial recall. In the feature model, both articulatory suppression and irrelevant speech are seen as adding noise to the memory representation, the difference being that articulatory suppression diverts more resources than does irrelevant speech. The addition of noise impairs recall because it reduces the probability of successful redintegration. When a competitor is incorrectly recalled, rather than the correct item, this competitor is recalled out of order, producing an increase in order errors. Six simulations are reported that show that the model accounts for (1) the impairment by both irrelevant speech and articulatory suppression, (2) the irrelevance of the phonological and semantic composition of the irrelevant speech, (3) greater disruption when the irrelevant speech tokens vary, (4) the abolition of the phonological similarity effect for visual, but not for auditory, items, (5) the abolition of the word length effect for both visual and auditory items, and (6) the abolition of the irrelevant speech effect under articulatory suppression for both visual and auditory items. The feature model is compared with the two other major views of irrelevant speech, the phonological store hypothesis and the changing state hypothesis.     

Disruption of order information by irrelevant items: a serial recognition paradigm

Irrelevant speech effect (ISE) is defined as a decrement in visually presented digit-list short-term memory performance due to exposure to irrelevant auditory material. Perhaps the most successful theoretical explanation of the effect is the changing state hypothesis. This hypothesis explains the effect in terms of confusion between amodal serial order cues, and represents a view based on the interference caused by the processing of similar order information of the visual and auditory materials. An alternative view suggests that the interference occurs as a consequence of the similarity between the visual and auditory contents of the stimuli. An important argument for the former view is the observation that ISE is almost exclusively observed in tasks that require memory for serial order. However, most short-term memory tasks require that both item and order information be retained in memory. An ideal task to investigate the sensitivity of maintenance of serial order to irrelevant speech would be one that calls upon order information but not item information. One task that is particularly suited to address this issue is serial recognition. In a typical serial recognition task, a list of items is presented and then probed by the same list in which the order of two adjacent items has been transposed. Due to the re-presentation of the encoding string, serial recognition requires primarily the serial order to be maintained while the content of the presented items is deemphasized. In demonstrating a highly significant ISE of changing versus steady-state auditory items in a serial recognition task, the present finding lends support for and extends previous empirical findings suggesting that irrelevant speech has the potential to interfere with the coding of the order of the items to be memorized.     

Irrelevant Speech,Phonological Similarity,and Presentation Modality

The phonological similarity effect refers to the finding that a list of similarsounding items is recalled less well than a list of dissimilar-sounding items. A strong prediction of the phonological store hypothesis, based on Baddeley's (1986, 1992) working memory framework, is that irrelevant speech should interact with the phonological similarity effect. We report three experiments that demonstrate that irrelevant speech eliminates the phonological similarity effect for visual but not for auditory items. In this respect, irrelevant speech functions much like articulatory suppression. Implications for the phonological store and changing state hypotheses are discussed.     

Disruption of short-term memory by changing and deviant sounds: support for a duplex-mechanism account of auditory distraction

The disruption of short-term memory by to-be-ignored auditory sequences (the changing-state effect) has often been characterized as attentional capture by deviant events (deviation effect). However, the present study demonstrates that changing-state and deviation effects are functionally distinct forms of auditory distraction: The disruption of visual-verbal serial recall by changing-state speech was independent of the effect of a single deviant voice embedded within the speech (Experiment 1); a voice-deviation effect, but not a changing-state effect, was found on a missing-item task (Experiment 2); and a deviant voice repetition within the context of an alternating-voice irrelevant speech sequence disrupted serial recall (Experiment 3). The authors conclude that the changing-state effect is the result of a conflict between 2 seriation processes being applied concurrently to relevant and irrelevant material, whereas the deviation effect reflects a more general attention-capture process.     

Word Dose in the Disruption of Serial Recall by Irrelevant Speech: Phonological Confusions or Changing State?

Irrelevant background speech disrupts serial recall of visually presented lists of verbal material. Three experiments tested the hypothesis that the degree of disruption is dependent on the number of words heard (i.e. word dose) whilst the task was undertaken. Experiments 1 and 2 showed that more disruption is produced if the word dose is increased, thereby providing evidence to support the experimental hypothesis. It was concluded from the first two experiments that the word-dose effect might be the result of increasing the amount of changing-state information in the speech. The results of Experiment 3 supported this conclusion by showing an interaction between word dose and changing-state information. It was noted however that the results might be explained within the working memory account of the disruptive action of irrelevant speech. A further two experiments cast doubt on this possibility by failing to replicate the finding that the phonological similarity between heard and seen material affects the degree of interference (Salame & Baddeley, 1982). The findings are discussed in relation to the changing state hypothesis of the irrelevant speech effect (e.g. Jones, Madden, & Miles, 1992).     

Irrelevant speech effects and statistical learning

Immediate serial recall of visually presented verbal stimuli is impaired by the presence of irrelevant auditory background speech, the so-called irrelevant speech effect. Two of the three main accounts of this effect place restrictions on when it will be observed, limiting its occurrence either to items processed by the phonological loop (the phonological loop hypothesis) or to items that are not too dissimilar from the irrelevant speech (the feature model). A third, the object-oriented episodic record (O-OER) model, requires only that the memory task involves seriation. The present studies test these three accounts by examining whether irrelevant auditory speech will interfere with a task that does not involve the phonological loop, does not use stimuli that are compatible with those to be remembered, but does require seriation. Two experiments found that irrelevant speech led to lower levels of performance in a visual statistical learning task, offering more support for the O-OER model and posing a challenge for the other two accounts.     

Can preference for background music mediate the irrelevant sound effect?

Research suggests that listening to background music prior to task performance increases cognitive processes, such as attention and memory, through the mechanism of increasing arousal and positive mood. However, music preference has not been explored with regard to a more common and realistic scenario of concurrent music and cognition, namely the ‘irrelevant sound effect’ (ISE). To examine this, serial recall was tested under quiet, liked and disliked music sound conditions as well as steady‐state (repetition of ‘3’) and changing‐state speech (random digits 1–9). Results revealed performance to be poorer for both music conditions and the changing‐state speech compared to quiet and steady‐state speech conditions. The lack of difference between both music conditions suggests that preference does not affect serial recall performance. These findings are discussed within the music and cognition and auditory distraction literatures. Copyright © 2010 JohnWiley & Sons, Ltd.     

Disruption of visual short-term memory by changing-state auditory stimuli: The role of segmentation

Typically, serial recall performance can be disrupted by the presence of an irrelevant stream of background auditory stimulation, but only if the background stream changes over time (the auditory changing-state effect). It was hypothesized that segmentation of the auditory stream is necessary for changing state to be signified. In Experiment 1, continuous random pitch glides failed to disrupt serial recall, but glides interrupted regularly by silence brought about the usual auditory changing-state effect. In Experiment 2, a physically continuous stream of synthesized vowel sounds was found to have disruptive effects. In Experiment 3, the technique of auditory induction showed that preattentive organization rather than critical features of the sound could account for the disruption by glides. With pitch glides, silence plays a preeminent role in the temporal segmentation of the sound stream, but speech contains corr-elated-time-varying changes in frequency and amplitude that make silent intervals superfluous.     

Disruption of memory for lip-read lists by irrelevant speech: Further support for the changing state hypothesis

Two experiments critically re-examine the finding of Campbell and Dodd (1984, Experiment 2), which suggests that irrelevant speech disrupts the encoding of visual material for serial recall. Support is sought for the competing view that the effect of irrelevant speech is on storage by comparing the effect of a range of acoustic conditions on memory for graphic and lip-read lists. Initially, serial short-term recall of visually presented lists was examined with irrelevant speech that was both asynchronous with the visually presented items and of varied speech content (Experiment 1a). In this experiment substantial impairments in recall of both graphic and lip-read lists were found. However, with unvarying asynchronous speech (Experiment 1b) the effect of speech was small and non-significant. Experiment 2 examined the effect of changing state and of synchrony of speech with lip movements. When conditions of synchronous and asynchronous unvarying speech were contrasted, no significant effect of synchrony or irrelevant speech was found (Experiment 2a and 2c). In contrast, when the speech was varying in content, a strong effect of irrelevant speech was found; moreover, the effect was roughly the same for synchronous and asynchronous materials (Experiment 2b). The contrast in outcome with varying and unvarying speech provides strong support for the “changing state” model of the irrelevant speech effect. Coupled with the absence of an effect of synchrony in Experiment 2, these experiments reinforce the view that disruption by irrelevant speech occurs in memory, not at encoding.     

Interruption of phonological coding in conduction aphasia

A case study of conduction aphasia, investigating single word repetition, phonological coding, and short-term memory, is reported. Evidence from intact adults suggests that repetition can occur through either a lexical route or a direct auditory-articulatory link. For this conduction aphasic, E.A., the direct link was impaired, although the lexical route could be used to produce accurate single word repetition. Several experiments demonstrated a significant impairment in the generation and maintenance of an abstract phonological code. The consequences of a disruption of phonological coding on speech perception and on verbal short-term memory are discussed.     

Disorder in sequential speech perception: a case study on pure word deafness

We described disorders of a patient which were uniquely restricted to speech perception of syllable sequences after brain damage. The results of series of experiments using syllable sequences showed "negative recency effect," in which the subject's repetition performance at the latter syllable position was remarkably poor. Experimental analyses suggested that the "negative recency effect" could be due to dual factors: the lower rate of processing of speech sounds and the memory load of holding processes of preceding syllables imposed on the succeeding phonological processing. The results also suggested that the holding processes which imposed the memory load on the succeeding auditory phonological coding processing were modality nonspecific.     

The phonological loop model of working memory: An ERP study of irrelevant speech and phonological similarity effects

The phonological loop model for retention of auditory verbal material in working memory, developed by Baddeley, assumes that irrelevant speech and phonological similarity influence only one and the same element of the system—that is, the phonological short-term store. We tested this idea by recording eventrelated potentials (ERPs) to auditorily presented letters that were phonologically similar or dissimilar and were to be memorized in the presence of more or less disturbing irrelevant speech. Irrelevant speech and phonological similarity caused ERP effects with clearly different scalp topographies, indicating that these factors influence different brain systems and hence probably different cognitive elements. Moreover, ERPs indicated that the phonological similarity effect might involve processes at the level of phonological analysis. Our data also support recent suggestions that the irrelevant speech effect is not based on the phonological similarity between relevant and irrelevant material, but on the phonological variability within the irrelevant stream.     

Hearing by eye

Recent work on integration of auditory and visual information during speech perception has indicated that adults are surprisingly good at, and rely extensively on, lip reading. The conceptual status of lip read information is of interest: such information is at the same time both visual and phonological. Three experiments investigated the nature of short term coding of lip read information in hearing subjects. The first experiment used asynchronous visual and auditory information and showed that a subject's ability to repeat words, when heard speech lagged lip movements, was unaffected by the lag duration, both quantitatively and qualitatively. This suggests that lip read information is immediately recoded into a durable code. An experiment on serial recall of lip read items showed a serial position curve containing a recency effect (characteristic of auditory but not visual input). It was then shown that an auditory suffix diminishes the recency effect obtained with lip read stimuli. These results are consistent with the hypothesis that seen speech, that is not heard, is encoded into a durable code which has some shared properties with heard speech. The results of the serial recall experiments are inconsistent with interpretations of the recency and suffix effects in terms of precategorical acoustic storage, for they demonstrate that recency and suffix effects can be supra-modal.     

Irrelevant sound disrupts speech production: Exploring the relationship between short-term memory and experimentally induced slips of the tongue

To explore the relationship between short-term memory and speech production, we developed a speech error induction technique. The technique, which was adapted from a Japanese word game, exposed participants to an auditory distractor word immediately before the utterance of a target word. In Experiment 1, the distractor words that were phonologically similar to the target word led to a greater number of errors in speaking the target than did the dissimilar distractor words. Furthermore, the speech error scores were significantly correlated with memory span scores. In Experiment 2, memory span scores were again correlated with the rate of the speech errors that were induced from the task-irrelevant speech sounds. Experiment 3 showed a strong irrelevant-sound effect in the serial recall of nonwords. The magnitude of the irrelevant-sound effects was not affected by phonological similarity between the to-be-remembered nonwords and the irrelevant-sound materials. Analysis of recall errors in Experiment 3 also suggested that there were no essential differences in recall error patterns between the dissimilar and similar irrelevant-sound conditions. Weproposed two different underlying mechanisms in immediate memory, one operating via the phonological short-term memory store and the other via the processes underpinning speech production.     

Phonological short‐term memory and vocabulary development: further evidence on the nature of the relationship

The nature and generality of the developmental association between phonological short‐term memory and vocabulary knowledge was explored in two studies. Study 1 investigated whether the link between vocabulary and verbal memory arises from the requirement to articulate memory items at recall or from earlier processes involved in the encoding and storage of the verbal material. Four‐year‐old children were tested on immediate memory measures which required either spoken recall (nonword repetition and digit span) or recognition of a sequence of nonwords. The phonological memory–vocabulary association was found to be as strong for the serial recognition as recall‐based measures, favouring the view that it is phonological short‐term memory capacity rather than speech output skills which constrain word learning. In Study 2, the association between phonological memory skills and vocabulary knowledge was found to be strong in teenaged as well as younger children, indicating that phonological memory constraints on word learning remain significant throughout childhood. Copyright © 1999 John Wiley & Sons, Ltd.     

A visuospatial “phonological loop” in working memory: Evidence from American Sign Language

In two experiments, the question of whether working memory could support an articulatory rehearsal loop in the visuospatial domain was investigated. Deaf subjects fluent in American Sign Language (ASL) were tested on immediate serial recall. In Experiment 1, using ASL stimuli, evidence for manual motoric coding (worse recall under articulatory suppression) was found, replicating findings of ASL-based phonological coding (worse recall for phonologically similar lists). The two effects did not interact, suggesting separate components which both contribute to performance. Stimuli in Experiment 2 were namable pictures, which had to be recoded for ASL-based rehearsal to occur. Under these conditions, articulatory suppression eliminated the phonological similarity effect. Thus, an articulatory process seems to be used in translating pictures into a phonological code for memory maintenance. These results indicate a configuration of components similar to the phonological loop for speech, suggesting that working memory can develop a language-based rehearsal loop in the visuospatial modality.     

Habituation to irrelevant speech: effects on a visual short-term memory task

The Baddeley and Hitch (1974) formulation of short-term or working memory embodied a phonological store within the articulatory loop component of the model. Later formulations specifically postulated an acoustic filter that endowed only speech-like stimuli with obligatory access to this phonological store. This paper presents evidence that this phonological store may have two filters, one of which is subject to habituation and can therefore attenuate the entry of irrelevant speech, thus undermining the obligatory access assumption of the model. An experiment is reported in which subjects were presented with a habituation period comprising 20 min of irrelevant speech--speech to be ignored by the subject--before a test phase in which a visually presented serial recall task with concurrent irrelevant speech was performed. The effect of irrelevant speech, which impairs performance on the primary task when there is no habituation phase, is reduced markedly in those conditions where the speech used in the habituation phase is the same as that used in the test phase, if the irrelevant speech is in a language different from that presented during the subsequent trials or if the habituator is a hummed version of the irrelevant speech passage. When a nonspeech sound (pink noise) is used in the habituation phase, a large irrelevant speech effect is found in the test phase. Morris, Quayle, and Jones (1989) found that humming did not produce an irrelevant speech effect, which suggests that the first filter is permeable to humming but that the second filter is not.(ABSTRACT TRUNCATED AT 250 WORDS)     

Artificially induced valence of distractor words increases the effects of irrelevant speech on serial recall

In a game context, nonwords either were artificially associated with negative valence or were in some sense neutral or irrelevant. Subsequently, participants memorized target words in silence or while attempting to ignore the negatively valent, irrelevant, or neutral auditory distractor nonwords. The presence of distractor nonwords impaired recall performance, but negative distractor nonwords caused more disruption than neutral and irrelevant distractors, which did not differ in how much disruption they caused. These findings conceptually replicate earlier results showing disruption due to valence with natural language words and extend them by demonstrating that auditory features that may possibly be confounded with valence in natural language words cannot be the cause of the observed disruption. Explanations of the irrelevant speech effect within working memory models that specify an explicit role of attention in the maintenance of information for immediate serial recall can explain this pattern of results, whereas structural models of working memory cannot.     

The impact of broadband noise on serial memory: changes in band-pass frequency increase disruption

Irrelevant sound consisting of bursts of broadband noise, in which centre frequency changes with each burst, markedly impaired short-term memory for order. In contrast, a sequence of irrelevant sound in which the same band-pass noise burst was repeated did not produce significant disruption. Serial recall for both visual-verbal (Experiment 1) and visual-spatial items (Experiment 2) was sensitive to the increased disruption produced by changing irrelevant noise. The results provide evidence that sounds that are largely aperiodic can produce marked disruption of serial recall in a similar manner to periodic sounds (e.g., speech, musical streams, and tones), and thus show a changing-state effect.