The nature of cross-modal color-word interference effects

Cowan and Barron (1987) and Cowan (1989b) reported that color-naming performance was slowed by spoken color names drawn from the same set but presented in an order unrelated to the printed colors. Although Miles, Madden, and Jones (1989) and Miles and Jones (1989) were unable to replicate this cross-modal effect, it is replicated here in two experiments with much better experimental control than before. However, the effect is shown to depend upon the relative timing of the color and word in a way that conflicts with the theoretical account that Cowan and Barron offered. While Cowan and Barron suggested that an irrelevant color word would contaminate the response set if this word occupied short-term memory when the color was about to be named, it appears that interference actually occurs only if the memory representation was formed very recently and had not been inhibited. Further implications for processing are discussed.     

Lip-read me now, hear me better later: cross-modal transfer of talker-familiarity effects

There is evidence that for both auditory and visual speech perception, familiarity with the talker facilitates speech recognition. Explanations of these effects have concentrated on the retention of talker information specific to each of these modalities. It could be, however, that some amodal, talker-specific articulatory-style information facilitates speech perception in both modalities. If this is true, then experience with a talker in one modality should facilitate perception of speech from that talker in the other modality. In a test of this prediction, subjects were given about 1 hr of experience lipreading a talker and were then asked to recover speech in noise from either this same talker or a different talker. Results revealed that subjects who lip-read and heard speech from the same talker performed better on the speech-in-noise task than did subjects who lip-read from one talker and then heard speech from a different talker.     

Asymmetric cross-modal effects in time perception

It is common to judge the duration of an audiovisual event, and yet it remains controversial how the judgment of duration is affected by signals from other modalities. We used an oddball paradigm to examine the effect of sound on the judgment of visual duration and that of a visual object on the judgment of an auditory duration. In a series of standards and oddballs, the participants compared the duration of the oddballs to that of the standards. Results showed asymmetric cross-modal effects, supporting the auditory dominance hypothesis: a sound extends the perceived visual duration, whereas a visual object has no effect on perceived auditory duration. The possible mechanisms (pacemaker or mode switch) proposed in the Scalar Expectancy Theory [Gibbon, J., Church, R. M., & Meck, W. H. (1984). Scalar timing in memory. In J. Gibbon & L. Allan (Eds.), Annals of the New York Academy of Sciences: Vol. 423. Timing and time perception (pp. 52–77). New York: New York Academy of Sciences] were examined using different standard durations. We conclude that sound increases the perceived visual duration by accelerating the pulse rate in the visual pacemaker.     

The limits of newborn's grasping to detect texture in a cross-modal transfer task

Three experiments were conducted to investigate human newborns’ ability to perceive texture property tactually, either in a cross-modal transfer task or in an intra-modal tactual discrimination task. In Experiment 1, newborns failed to tactually recognize the texture (smooth vs. granular) of flat objects that they had previously seen, when they held flat objects. This failure was mainly due to a lack of intra-modal tactual discrimination between the two objects (Experiment 2). In contrast, Experiment 3 showed that newborns were able to tactually recognize the texture of previously seen surfaces when they held volumetric objects. Taken together, the results suggest that cross-modal transfer of texture from vision to touch stem from a peripheral mechanism, not a central mechanism. Grasping only allows newborns to perceive the texture of volumetric but not flat objects. As a consequence, this study reveals the limits of newborns’ grasping to detect and process information about texture. The results also suggest that more mature exploratory procedures, such as the “lateral motion” procedure exhibited by adults [Lederman, S. J., & Klatzky, R. (1987). Hand movements: A window into haptic object recognition. Cognitive Psychology, 19, 342–368], might be necessary for detecting the texture of flat objects in newborn infants.     

Visual and tactile cross-modal mere exposure effects

The affective system enables people to perceive and judge the emotional content of stimuli from various sensory modalities. Cross-modal interactions in affective processes, however, are less well understood. Using novel three-dimensional objects, we investigated cross-modal mere exposure effects between vision and touch. Previewing objects increased the preference judged by hand, while pre-touching did not modulate the preference judged by vision. Moreover, these effects were found to be independent of recognition performance, suggesting a dissociation between affective and cognitive processing. Our demonstration of a cross-modal mere exposure effect suggests that the affective system integrates inputs from visual and tactile modalities asymmetrically.     

The roles of imagery,language, and metamemory in cross-modal transfer in children

Summary The relative strengths of association of imagery, language, and metamemory to cross-modal transfer were evaluated in 5-year-olds and 7-year-olds. Children were administered either tactual-visual or visual-tactual cross-modal transfer tasks, as well as two visual imagery tasks (spatial recall and mental rotation), two language competence tasks (vocabulary and word usage), metamemory questions, and a general intelligence task. With intelligence and handedness controlled, only visual imagery related significantly to cross-modal transfer. These findings suggest that mental imagery, as opposed to language competence or metamemory, is a principal component of cross-modal transfer, and they are consistent with the observation that preverbal human infants and infrahuman primates exhibit cross-modal transfer.     

Spatial constraints on visual-tactile cross-modal distractor congruency effects

Across three experiments, participants made speeded elevation discrimination responses to vibrotactile targets presented to the thumb (held in a lower position) or the index finger (upper position) of either hand, while simultaneously trying to ignore visual distractors presented independently from either the same or a different elevation. Performance on the vibrotactile elevation discrimination task was slower and less accurate when the visual distractor was incongruent with the elevation of the vibrotactile target (e.g., a lower light during the presentation of an upper vibrotactile target to the index finger) than when they were congruent, showing that people cannot completely ignore vision when selectively attending to vibrotactile information. We investigated the attentional, temporal, and spatial modulation of these cross-modal congruency effects by manipulating the direction of endogenous tactile spatial attention, the stimulus onset asynchrony between target and distractor, and the spatial separation between the vibrotactile target, any visual distractors, and the participant’s two hands within and across hemifields. Our results provide new insights into the spatiotemporal modulation of crossmodal congruency effects and highlight the utility of this paradigm for investigating the contributions of visual, tactile, and proprioceptive inputs to the multisensory representation of peripersonal space.     

Temporal specificity in cross-modal transfer of the rabbit nictitating membrane response

Two experiments examine cross-modal transfer of response features specific to the interstimulus interval (ISI) between a conditioned stimulus (CS) and an unconditioned stimulus. Rabbits were given initial training with a stimulus (CSA) in one modality (e.g., tone) at a designated ISI (e.g., 600 ms). Training was then shifted to a new stimulus (CSB) in another modality (e.g., light) at a new ISI (e.g., 400 ms). The timing of early conditioned responses (CRs) to CSB reflected the ISI of CSA. Ultimately, CRs to CSB shifted to a temporal location conforming to the ISI of CSB. When the ISI of CSB was shorter than that of CSA, CRs to CSA also shifted to a locus conforming to the ISI of CSB. The present results confirmed previous findings that training in one CS modality accelerates CR acquisition to a CS in another modality. The findings are compared with the transfer of response patterns in instrumental learning sets and are discussed regarding their implications for theories of cross-modal transfer.     

Tactual learning and cross-modal transfer of an oddity problem in young children

The ability to solve tactual oddity problems, and transfer of oddity learning across the visual and the tactual modalities, was studied in 3- to 8-year-old children (N = 294). Oddity tasks consisting of one odd and two equal objects were made from stimuli that were easily discriminated visually and tactually. The results showed that tactual oddity learning increased gradually with age. The growth in tactual performance begins later than visual, suggesting that children are more adept at encoding visual stimulus invariances or relational properties than tactual ones. Bidirectional cross-modal transfer of oddity learning was found, supporting the suggestion that such transfer occurs when training and transfer oddity tasks share a common vehicle dimension. The cross-modal effect also shows that oddity learning is independent of a specific modality-labeled perceptual context. Our results are consistent with the view that development of oddity learning depends on a single rather than a dual process, and that the oddity relation may be treated as an amodal stimulus feature.     

Lingual vibrotactile and auditory cross-modal matching: frequency effects

5 subjects matched pairs of auditory and vibrotactile stimuli on intensity, making judgments of suprathreshold magnitudes. Slope values for the 100-Hz cross-modal lingual vibrotactile stimulus-standard frequency condition were steeper than those for 250- or 400-Hz conditions. Slope values became steeper at about 25- to 30-dB SL, so frequency seems an important parameter to control in such research.     

Learning across senses: cross-modal effects in multisensory statistical learning

It is currently unknown whether statistical learning is supported by modality-general or modality-specific mechanisms. One issue within this debate concerns the independence of learning in one modality from learning in other modalities. In the present study, the authors examined the extent to which statistical learning across modalities is independent by simultaneously presenting learners with auditory and visual streams. After establishing baseline rates of learning for each stream independently, they systematically varied the amount of audiovisual correspondence across 3 experiments. They found that learners were able to segment both streams successfully only when the boundaries of the audio and visual triplets were in alignment. This pattern of results suggests that learners are able to extract multiple statistical regularities across modalities provided that there is some degree of cross-modal coherence. They discuss the implications of their results in light of recent claims that multisensory statistical learning is guided by modality-independent mechanisms.     

Developmental invariance in distinctiveness effects in memory

Improvements in 5- and 7-year-olds' acquisition and retention of related concept pairings were examined when additional similarities and differences between pair members were provided. Using a standard paired-associate learning paradigm, children learned 18 related picture pairs; some of the children either were given or produced additional similarities or differences between pair members at the time of learning. Three weeks after learning was complete, children attempted to recall the pairs. Using a model to determine the storage and retrieval loci of these effects, the results showed that (a) all children benefited from self-generated elaborations, regardless of whether these were similarities or differences, and these benefits were storage related, and (b) difference elaborations improved children's retention regardless of whether they were self- or experimenter-generated, and these effects were primarily retrieval based. These results are consistent with theories that (a) view retrieval as the locus of distinctiveness effects and (b) view storage as the locus of self-generated memory improvements.     

Auditory psychophysical scaling exposure effects: magnitude estimation and cross-modal matching

The purpose of the present study was to investigate possible effects of exposure upon suprathreshold psychological responses when auditory magnitude estimation and cross-modal matching with audition as the standard are conducted within the same experiment. Four groups of 10 subjects each whose over-all age range was 18 to 23 yr. were employed. During the cross-modal matching task the Groups 1 and 2 subjects adjusted a vibrotactile stimulus presented to the dorsal surface of the tongue and the Groups 3 and 4 subjects adjusted a vibrotactile stimulus presented to the thenar eminence of the right hand to match binaurally presented auditory stimuli. The magnitude-estimation task was conducted before the cross-modal matching task for Groups 1 and 3 and the cross-modal matching task was conducted before the magnitude-estimation task for Groups 2 and 4. The psychophysical methods of magnitude estimation and cross-modal matching showed no effect of one upon the other when used in the same experiment.     

Sound enhances visual perception: cross-modal effects of auditory organization on vision

Six experiments demonstrated cross-modal influences from the auditory modality on the visual modality at an early level of perceptual organization. Participants had to detect a visual target in a rapidly changing sequence of visual distractors. A high tone embedded in a sequence of low tones improved detection of a synchronously presented visual target (Experiment 1), but the effect disappeared when the high tone was presented before the target (Experiment 2). Rhythmically based or order-based anticipation was unlikely to account for the effect because the improvement was unaffected by whether there was jitter (Experiment 3) or a random number of distractors between successive targets (Experiment 4). The facilitatory effect was greatly reduced when the tone was less abrupt and part of a melody (Experiments 5 and 6). These results show that perceptual organization in the auditory modality can have an effect on perceptibility in the visual modality.