Short-term implicit memory: Visual, auditory, and cross-modality priming

Short-term implicit memory was examined for mixed auditory (A) and visual (V) stimuli. In lexical decision, words and nonwords were repeated at lags of 0, 1, 3, and 6 intervening trials, in four prime-target combinations (VV, VA, AV, AA). Same-modality repetition priming showed a lag x lexicality interaction for visual stimuli (nonwords decayed faster), but not for auditory stimuli (longer lasting smooth decay for both words and nonwords). These modality differences suggest that short-term priming has a perceptual locus, with the phonological lexicon maintaining stimuli active longer than the orthographic lexicon and treating pseudowords as potential words. We interpret these differences in terms of the different memory needs of speech recognition and text reading. Weak cross-modality short-term priming was present for words and nonwords, indicating recoding between perceptual forms.     

The interaction between response effects during the acquisition of response priming

In three experiments we investigated the role of associative learning in the acquisition of response priming by effect stimuli, by examining their interaction during response-effect learning. Having replicated the ability of visual effect stimuli to prime their associated responses, we paired a response with a compound consisting of visual and auditory effects before assessing the ability of the auditory effect stimulus to prime the response. This priming was reduced if the visual stimulus had been pre-trained as an effect of the response. By contrast, priming by the visual effect stimulus was potentiated when the auditory effect had been pre-trained. We interpret these interactions in terms of contemporary associative learning theory derived from studies of conditioning.     

Effects of laterality and pitch height of an auditory accessory stimulus on horizontal response selection: The Simon effect and the SMARC effect

In the present article, we investigated the effects of pitch height and the presented ear (laterality) of an auditory stimulus, irrelevant to the ongoing visual task, on horizontal response selection. Performance was better when the response and the stimulated ear spatially corresponded (Simon effect), and when the spatial—musical association of response codes (SMARC) correspondence was maintained—that is, right (left) response with a high-pitched (low-pitched) tone. These findings reveal an automatic activation of spatially and musically associated responses by task-irrelevant auditory accessory stimuli. Pitch height is strong enough to influence the horizontal responses despite modality differences with task target.     

Modality and domain specific components in auditory and visual working memory tasks

In the tripartite model of working memory (WM) it is postulated that a unique part system—the visuo-spatial sketchpad (VSSP)—processes non-verbal content. Due to behavioral and neurophysiological findings, the VSSP was later subdivided into visual object and visual spatial processing, the former representing objects’ appearance and the latter spatial information. This distinction is well supported. However, a challenge to this model is the question how spatial information from non-visual sensory modalities, for example the auditory one, is processed. Only a few studies so far have directly compared visual and auditory spatial WM. They suggest that the distinction of two processing domains—one for object and one for spatial information—also holds true for auditory WM, but that only a part of the processes is modality specific. We propose that processing in the object domain (the item’s appearance) is modality specific, while spatial WM as well as object-location binding relies on modality general processes.     

Episodic retrieval processes take place automatically in auditory negative priming

Episodic retrieval processes involved in negative priming have been argued to be susceptible to the proportion of attended repetition trials. The more trials with the same prime and probe response, the more beneficial should it be to retrieve the prime episode, particularly its response. Retrieval of the prime episode, however, is task-inappropriate for ignored repetition trials, leading to negative priming. Correspondingly, visual negative priming increases with the proportion of attended repetition trials. We tested whether the same is true for the auditory modality. Three attended repetition proportion groups (0–25–50%) showed the same amount of negative priming. All groups committed more prime response errors in ignored repetition than in control trials, implying that prime response retrieval took place. Thus, retrieval processes in auditory negative priming appear to be automatic and cannot be influenced as easily as in the visual domain. In Experiment 2, the proportion of ignored repetition trials was manipulated (25–50–75%) to test whether auditory negative priming can be strategically manipulated at all. Similar to the visual modality negative priming was reduced with increasing proportion of ignored repetition trials. Differences between visual and auditory short-term memory are discussed to account for the results.     

Effects of spatial response coding on distractor processing: Evidence from auditory spatial negative priming tasks with keypress,joystick, and head movement responses

Attention, Perception, & Psychophysics - Prior studies of spatial negative priming indicate that distractor-assigned keypress responses are inhibited as part of visual, but not auditory,...     

Normal auditory negative priming in schizophrenic patients

Performance of 28 schizophrenic patients and 28 matched controls was compared in an auditory priming task. A large auditory negative priming effect was obtained for the patients as well as for the control group, and the size of the negative priming effect was approximately the same for both groups. Under the same conditions, positive or repetition priming for the patients was enhanced compared to that of the control group. The present findings from an auditory priming task are consistent with a growing body of evidence from the visual domain showing normal rather than reduced or eliminated negative priming in schizophrenic patients.     

Normal auditory negative priming in schizophrenic patients

Performance of 28 schizophrenic patients and 28 matched controls was compared in an auditory priming task. A large auditory negative priming effect was obtained for the patients as well as for the control group, and the size of the negative priming effect was approximately the same for both groups. Under the same conditions, positive or repetition priming for the patients was enhanced compared to that of the control group. The present findings from an auditory priming task are consistent with a growing body of evidence from the visual domain showing normal rather than reduced or eliminated negative priming in schizophrenic patients.     

Cross-modal priming and explicit memory in patients with verbal production deficits

Implicit memory is often thought to reflect an influence of past experience on perceptual processes, yet priming effects are found when the perceptual format of stimuli changes between study and test episodes. Such cross-modal priming effects have been hypothesized to depend upon stimulus recoding processes whereby a stimulus presented in one modality is converted to other perceptual formats. The present research examined recoding accounts of cross-modal priming by testing patients with verbal production deficits that presumably impair the conversion of visual words into auditory/phonological forms. The patients showed normal priming in a visual stem completion task following visual study (Experiment 1), but showed impairments following auditory study in both implicit (Experiment 2) and explicit (Experiment 3) stem completion. The results are consistent with the hypothesis that verbal production processes contribute to the recoding of visual stimuli and support cross-modal priming. The results also indicate that shared processes contribute to both explicit memory and cross-modal implicit memory.     

Switching in the cocktail party: exploring intentional control of auditory selective attention

Using a novel variant of dichotic selective listening, we examined the control of auditory selective attention. In our task, subjects had to respond selectively to one of two simultaneously presented auditory stimuli (number words), always spoken by a female and a male speaker, by performing a numerical size categorization. The gender of the task-relevant speaker could change, as indicated by a visual cue prior to auditory stimulus onset. Three experiments show clear performance costs with instructed attention switches. Experiment 2 varied the cuing interval to examine advance preparation for an attention switch. Experiment 3 additionally isolated auditory switch costs from visual cue priming by using two cues for each gender, so that gender repetition could be indicated by a changed cue. Experiment 2 showed that switch costs decreased with prolonged cuing intervals, but Experiment 3 revealed that preparation did not affect auditory switch costs but only visual cue priming. Moreover, incongruent numerical categories in competing auditory stimuli produced interference and substantially increased error rates, suggesting continued processing of task-relevant information that often leads to responding to the incorrect auditory source. Together, the data show clear limitations in advance preparation of auditory attention switches and suggest a considerable degree of inertia in intentional control of auditory selection criteria.     

Linking sight and sound: fMRI evidence of primary auditory cortex activation during visual word recognition

We describe two studies that used repetition priming paradigms to investigate brain activity during the reading of single words. Functional magnetic resonance images were collected during a visual lexical decision task in which nonword stimuli were manipulated with regard to phonological properties and compared to genuine English words. We observed a region in left-hemisphere primary auditory cortex linked to a repetition priming effect. The priming effect activity was observed only for stimuli that sound like known words; moreover, this region was sensitive to strategic task differences. Thus, a brain region involved in the most basic aspects of auditory processing appears to be engaged in reading even when there is no environmental oral or auditory component.     

Multisensory identification of natural objects in a two-way crossmodal priming paradigm

The question of how vision and audition interact in natural object identification is currently a matter of debate. We developed a large set of auditory and visual stimuli representing natural objects in order to facilitate research in the field of multisensory processing. Normative data was obtained for 270 brief environmental sounds and 320 visual object stimuli. Each stimulus was named, categorized, and rated with regard to familiarity and emotional valence by N=56 participants (Study 1). This multimodal stimulus set was employed in two subsequent crossmodal priming experiments that used semantically congruent and incongruent stimulus pairs in a S1-S2 paradigm. Task-relevant targets were either auditory (Study 2) or visual stimuli (Study 3). The behavioral data of both experiments expressed a crossmodal priming effect with shorter reaction times for congruent as compared to incongruent stimulus pairs. The observed facilitation effect suggests that object identification in one modality is influenced by input from another modality. This result implicates that congruent visual and auditory stimulus pairs were perceived as the same object and demonstrates a first validation of the multimodal stimulus set.     

Cross-modal priming evidence for phonology-to-orthography activation in visual word recognition

Subjects were asked to indicate which item of a word/nonword pair was a word. On critical trials the nonword was a pseudohomophone of the word. RTs of dyslexics were shorter in blocks of trials in which a congruent auditory prime was simultaneously presented with the visual stimuli. RTs of normal readers were longer for high frequency words when there was auditory priming. This provides evidence that phonology can activate orthographic representations; the size and direction of the effect of auditory priming on visual lexical decision appear to be a function of the relative speeds with which sight and hearing activate orthography.     

Evidence of vocal and manual event files in auditory negative priming

Negative priming with auditory as well as with visual stimuli has been shown to involve the retrieval of prime response information as evidenced by an increase of prime response errors to the probes of ignored repetition trials compared to control trials. We investigated whether prime response retrieval processes were also present for response modalities other than manual responding. In an auditory four alternative forced choice task participants either vocally or manually identified a target sound while ignoring a distractor sound. Negative priming was of equal size in both response modalities. What is more, for both response modalities, there was evidence of increased prime response errors in ignored repetition trials compared to control trials. The findings suggest that retrieval of event files of the prime episode including prime response information is a general mechanism underlying the negative priming phenomenon irrespective of stimulus or response modality.     

Location-specific versus hemisphere-specific adaptation of processing selectivity

Current theories of cognitive control assume that processing selectivity is adjusted according to the utility of processing task-irrelevant stimulus features. Consistently, interference evoked by flanker stimuli is reduced when the proportion of incompatible trials—in which flankers are associated with an incorrect response—is increased. Consistent with the idea that the cerebral hemispheres select processing strategies independently of each other, Corballis and Gratton (2003) demonstrated that flanker interference for stimuli presented in either the left or right visual hemifield is affected by the ratio of compatible and incompatible target-flanker pairings presented in the same—but not in the other—hemifield. Presenting stimuli at four different locations, we demonstrated independent effects of the ratio of compatible and incompatible trials for stimulus locations in different hemifields as well as for stimulus locations within the same hemifield. Independent selectivity adjustment regarding the left and right visual hemifields thus appears to be a special case of a larger class of location-specific adaptation effects and might not be informative regarding hemisphere-specific processing.     

Capuchin monkeys (Cebus apella) use positive, but not negative, auditory cues to infer food location

Nonhuman primates appear to capitalize more effectively on visual cues than corresponding auditory versions. For example, studies of inferential reasoning have shown that monkeys and apes readily respond to seeing that food is present (“positive” cuing) or absent (“negative” cuing). Performance is markedly less effective with auditory cues, with many subjects failing to use this input. Extending recent work, we tested eight captive tufted capuchins (Cebus apella) in locating food using positive and negative cues in visual and auditory domains. The monkeys chose between two opaque cups to receive food contained in one of them. Cup contents were either shown or shaken, providing location cues from both cups, positive cues only from the baited cup, or negative cues from the empty cup. As in previous work, subjects readily used both positive and negative visual cues to secure reward. However, auditory outcomes were both similar to and different from those of earlier studies. Specifically, all subjects came to exploit positive auditory cues, but none responded to negative versions. The animals were also clearly different in visual versus auditory performance. Results indicate that a significant proportion of capuchins may be able to use positive auditory cues, with experience and learning likely playing a critical role. These findings raise the possibility that experience may be significant in visually based performance in this task as well, and highlight that coming to grips with evident differences between visual versus auditory processing may be important for understanding primate cognition more generally.     

Inhibition and decay of motor and nonmotor priming

Motor responses can be facilitated by congruent visual stimuli and prolonged by incongruent visual stimuli that are made invisible by masking (direct motor priming). Recent studies on direct motor priming showed a reversal of these priming effects when a three-stimulus paradigm was used in which a prime was followed by a mask and a target stimulus was presented after a delay. A similar three-stimulus paradigm on nonmotor priming, however, showed no reversal of priming effects when the mask was used as a cue for processing of the following target stimulus (cue priming). Experiment 1 showed that the time interval between mask and target is crucial for the reversal of priming. Therefore, the time interval between mask and target was varied in three experiments to see whether cue priming is also subject to inhibition at a certain time interval. Cues indicated (1) the stimulus modality of the target stimulus, (2) the task to be performed on a multidimensional auditory stimulus, or (3) part of the motor response. Whereas direct motor priming showed the reversal of priming about 100 msec after mask presentation, cue priming effects simply decayed during the 300 msec after mask presentation. These findings provide boundary conditions for accounts of inverse priming effects.     

Visual-auditory events: cross-modal perceptual priming and recognition memory.

Modality specificity in priming is taken as evidence for independent perceptual systems. However, Easton, Greene, and Srinivas (1997) showed that visual and haptic cross-modal priming is comparable in magnitude to within-modal priming. Where appropriate, perceptual systems might share like information. To test this, we assessed priming and recognition for visual and auditory events, within- and across- modalities. On the visual test, auditory study resulted in no priming. On the auditory priming test, visual study resulted in priming that was only marginally less than within-modal priming. The priming results show that visual study facilitates identification on both visual and auditory tests, but auditory study only facilitates performance on the auditory test. For both recognition tests, within-modal recognition exceeded cross-modal recognition. The results have two novel implications for the understanding of perceptual priming: First, we introduce visual and auditory priming for spatio-temporal events as a new priming paradigm chosen for its ecological validity and potential for information exchange. Second, we propose that the asymmetry of the cross-modal priming observed here may reflect the capacity of these perceptual modalities to provide cross-modal constraints on ambiguity. We argue that visual perception might inform and constrain auditory processing, while auditory perception corresponds to too many potential visual events to usefully inform and constrain visual perception.     

Crossmodal transfer of arousal, but not pleasantness, from the musical to the visual domain

Arousal and valence (pleasantness) are considered primary dimensions of emotion. However, the degree to which these dimensions interact in emotional processing across sensory modalities is poorly understood. We addressed this issue by applying a crossmodal priming paradigm in which auditory primes (Romantic piano solo music) varying in arousal and/or pleasantness were sequentially paired with visual targets (IAPS pictures). In Experiment 1, the emotion spaces of 120 primes and 120 targets were explored separately in addition to the effects of musical training and gender. Thirty-two participants rated their felt pleasantness and arousal in response to primes and targets on equivalent rating scales as well as their familiarity with the stimuli. Musical training was associated with elevated familiarity ratings for high-arousing music and a trend for elevated arousal ratings, especially in response to unpleasant musical stimuli. Males reported higher arousal than females for pleasant visual stimuli. In Experiment 2, 40 nonmusicians rated their felt arousal and pleasantness in response to 20 visual targets after listening to 80 musical primes. Arousal associated with the musical primes modulated felt arousal in response to visual targets, yet no such transfer of pleasantness was observed between the two modalities. Experiment 3 sought to rule out the possibility of any order effect of the subjective ratings, and responses of 14 nonmusicians replicated results of Experiment 2. This study demonstrates the effectiveness of the crossmodal priming paradigm in basic research on musical emotions.     

Auditory spatial negative priming: What is remembered of irrelevant sounds and their locations?

The categorization and identification of previously ignored visual or auditory stimuli is typically slowed down—a phenomenon that has been called the negative priming effect and can be explained by the episodic retrieval of response-inadequate prime information and/or an inhibitory model. A similar after-effect has been found in visuospatial tasks: participants are slowed down in localizing a visual stimulus that appears at a previously ignored location. In the auditory modality, however, such an after-effect of ignoring a sound at a specific location has never been reported. Instead, participants are impaired in their localization performance when the sound at the previously ignored location changes identity, a finding which is compatible with the so-called feature-mismatch hypothesis. Here, we describe the properties of auditory spatial in contrast to visuospatial negative priming and report two experiments that specify the nature of this auditory after-effect. Experiment 1 shows that the detection of identity-location mismatches is a genuinely auditory phenomenon that can be replicated even when the sound sources are invisible. Experiment 2 reveals that the detection of sound-identity mismatches in the probe depends on the processing demands in the prime. This finding implies that the localization of irrelevant sound sources is not the inevitable consequence of processing the auditory prime scenario but depends on the difficulty of the target search process among distractor sounds.