Semantic facilitation across sensory modalities in the processing of individual words and sentences

The effect of semantic priming upon lexical decisions made for words in isolation (Experiment 1) and during sentence comprehension (Experiment 2) was investigated using a cross-modal lexical decision task. In Experiment 1, subjects made lexical decisions to both auditory and visual stimuli. Processing auditorily presented words facilitated subsequent lexical decisions on semantically related visual words. In Experiment 2, subjects comprehended auditorily presented sentences while simultaneously making lexical decisions for visually presented stimuli. Lexical decisions were facilitated when a visual word appeared immediately following a related word in the sentential material. Lexical decisions were also facilitated when the visual word appeared three syllables following closure of the clause containing the related material. Arguments are made for autonomy of semantic priming during sentence comprehension.     

Task coordination between and within sensory modalities: effects on distraction

Load theory predictions for the effects of task coordination between and within sensory modalities (vision and hearing or vision only) on the level of distraction were tested. Response competition effects in a visual flanker task when it was coordinated with an auditory discrimination task (between-modality conditions) or a visual discrimination task (within-modality conditions) were compared with single-task conditions. In the between-modality conditions, response competition effects were greater in the two- (vs. single-) task conditions irrespective of the level of discrimination task difficulty. In the within-modality conditions, response competition effects were greater in the two-task (vs. single-task) conditions only when these involved a more difficult visual discrimination task. The results provided support for the load theory prediction that executive control load leads to greater distractor interference while highlighting the effects of task modality.     

Stimulus context and absolute magnitude estimation: a study of individual differences

The effect of stimulus context on absolute-magnitude-estimation (AME) judgments was examined by determining whether the loudness judgment of a tone is influenced by the intensities of other tones presented within the session. A group of 18 subjects was tested in separate sessions in which they judged stimuli within either a low (10-60 dB SL) or a high (40-90 dB SL) range of intensities. Examination of the results of individual subjects revealed that judgments of stimuli common to the two ranges were, in most subjects, unaffected or only slightly affected by the position of the range. The judgments of 2 subjects who failed to follow the instructions, however, showed very large context effects due to changing the stimulus range. The results of a second experiment, in which 22 subjects judged the loudness of tones within either a narrow (35-65 dB SL) or a wide (20-80 dB SL) range, revealed that, in all but 1 subject, the width of the range had no systematic effect on the loudness judgments of stimuli common to both ranges. This was also true 1 month later when 16 of the subjects returned to the laboratory to judge the loudness of tones within an even wider range of 10-90 dB SL. It was concluded that AME judgments are relatively insensitive to the potential biasing influences of stimulus context.     

Cross-modal dynamic capture: congruency effects in the perception of motion across sensory modalities

This study investigated multisensory interactions in the perception of auditory and visual motion. When auditory and visual apparent motion streams are presented concurrently in opposite directions, participants often fail to discriminate the direction of motion of the auditory stream, whereas perception of the visual stream is unaffected by the direction of auditory motion (Experiment 1). This asymmetry persists even when the perceived quality of apparent motion is equated for the 2 modalities (Experiment 2). Subsequently, it was found that this visual modulation of auditory motion is caused by an illusory reversal in the perceived direction of sounds (Experiment 3). This "dynamic capture" effect occurs over and above ventriloquism among static events (Experiments 4 and 5), and it generalizes to continuous motion displays (Experiment 6). These data are discussed in light of related multisensory phenomena and their support for a "modality appropriateness" interpretation of multisensory integration in motion perception.     

Compounding of discriminative stimuli from the same and different sensory modalities

Rats' responding was maintained by fixed-interval schedules of reinforcement in the presence of a tone or two separate lights. The lights were either of low, moderate, or high intensity. Compounds of these single discriminative stimuli each maintained a greater frequency of response than did the single stimuli, and the compound composed of stimuli from different sensory modalities (light+tone) maintained a greater level of responding than did the compound composed of stimuli from the same sensory modality (light+light). Combining lights of different intensity had no differential effect on responding. However, in the second experiment, a compound composed of a light and a tone, each of greater intensity than the light and tone of another compound, initially maintained a higher frequency of response, demonstrating intensity effects during stimulus compounding when the increase in intensity occurs through the component stimuli. This intensity effect, however, was only transitory.     

An fMRI investigation on image generation in different sensory modalities: The influence of vividness

In the present fMRI study the issue of the specific cortices activation during imagery generation in different sensory modalities is addressed. In particular, we tested whether the vividness variability of imagery was reflected in the BOLD signal within specific sensory cortices. Subjects were asked to generate a mental image for each auditory presented sentence. Each imagery modality was contrasted with an abstract sentence condition. In addition, subjects were asked to fill the Italian version of the Questionnaire Upon Mental Imagery (QMI) prior to each neuroimaging session. In general, greater involvement of sensory specific cortices in high-vivid versus low-vivid subjects was found for visual (occipital), gustatory (anterior insula), kinaesthetic (pre-motor), and tactile and for somatic (post-central parietal) imagery modalities. These results support the hypothesis that vividness is related to image format: high-vivid subjects would create more analogical representations relying on the same specific neural substrates active during perception with respect to low-vivid subjects. Results are also discussed according to the simulation perspective.     

Identification and integration of sensory modalities: Neural basis and relation to consciousness

A key question in studying consciousness is how neural operations in the brain can identify streams of sensory input as belonging to distinct modalities, which contributes to the representation of qualitatively different experiences. The basis for identification of modalities is proposed to be constituted by self-organized comparative operations across a network of unimodal and multimodal sensory areas. However, such network interactions alone cannot answer the question how sensory feature detectors collectively account for an integrated, yet phenomenally differentiated experiential content. This problem turns out to be different from, although related to, the binding problem. It is proposed that the neural correlate of an enriched, multimodal experience is constituted by the attractor state of a dynamic associative network. Within this network, unimodal and multimodal sensory maps continuously interact to influence each other’s attractor state, so that a feature change in one modality results in a fast re-coding of feature information in another modality. In this scheme, feature detection is coded by firing-rate, whereas firing phase codes relational aspects.     

Stimulus significance effects in habituation of the phasic and tonic orienting reflex

Phasic and tonic measures of electrodermal activity were examined in a simple habituation paradigm, using innocuous visual stimuli. Separate groups were used to investigate the effects of stimulus significance, as manipulated by instructions. One group had no stimulus-related task (indifferent group), while the other was asked to silently count the stimuli to report to the experimenter later (significant group). Prestimulus skin conductance levels were considered as measures of the arousal level at each stimulus presentation, and the subsequent electrodermal responses were taken as the phasic orienting reflex (OR) elicited by each stimulus. Changes in prestimulus arousal were taken as measures of the tonic OR to the experimental series. Marked group differences were found in both phasic and tonic components of the OR. Some, but not all, of the significance effects in the phasic OR were attributable to differences in arousal. The results are discussed in the context of theoretical accounts of the OR.     

The perceived importance of sensory modalities in product usage: a study of self-reports

Although popular belief holds that vision dominates human experience, this does not necessarily imply that people regard vision as the most important sensory modality during the interaction with every product. Instead, the relative importance of the different modalities is likely to depend on the type of product and on the task performed. In Study 1, respondents reported how important they found vision, audition, touch, taste, and smell during the use of 45 different products. In Study 2, the respondents answered a similar question for the evaluation of the safety, ease of use, and enjoyment experienced for 15 products. Importance ratings for the sensory modalities differed considerably between the products. Differences due to the types of evaluations were smaller. Averaged over products and evaluation types, vision was the most important sensory modality for product evaluations, followed by touch, smell, audition, and taste. However, for about half of the individual products, the importance ratings for vision were lower than for one of the other modalities. These findings are in line with the view that vision is regarded the dominant modality, because it plays an important part in many and an irrelevant part in virtually no product experiences.     

Specialization and semantic organization: Evidence for multiple semantics linked to sensory modalities

The present article reviews the case for multiple systems in semantic memory and empirically evaluates a multiple semantics proposal based on sensory modalities. In the experiments, a conceptual switching cost paradigm was used (Pecher, Zeelenberg, & Barsalou, 2003), in which participants verified properties for concepts and verification time was compared for target trials (e.g., a dog can bark) that were preceded by context trials of either the same or a different modality (e.g., a bee can buzz or a horse can have spots). Experiment 1 involved a modality switch while controlling for the concept's category and demonstrated that when modalities were switched, a processing cost was incurred that could not be attributed to the latter dimension. Experiment 2 further supported this conclusion by showing that, in a reverse situation, the cost incurred when category was switched was at least smaller. The results are discussed by considering possible alternative amodal explanations and other data that have shown the influence of sensory information in conceptual processing.     

Phonological coding in reading comprehension: The importance of individual differences

Abstract

In two experiments, texts were presented word by word at a fixed pace to subjects high or low in test anxiety, under various concurrent interference conditions: no interference, articulatory suppression, finger tapping, unattended meaningful speech, reversed (meaningless) speech and nonword (meaningless) speech. High-anxiety subjects produced overt articulation more frequently than low-anxiety subjects, especially in the speech conditions. There was an interaction between anxiety and interference on comprehension performance: Conditions thought to interfere with phonological coding (i.e. articulatory suppression and nonword speech) were detrimental to comprehension for high but not for low-anxiety subjects; in contrast, there were no differences in comprehension performance as a function of anxiety under conditions causing no (proper) phonological interference (i.e. no interference, finger tapping and reversed speech). There were mixed results in the meaningful speech condition, partly attributable to its semantic interference component. These findings reveal that the use and importance of phonological coding in reading comprehension vary with differences in test anxiety-high-anxiety subjects are vulnerable to disruptive effects on phonological coding. This suggests that high-anxiety individuals are especially likely to resort to speech-based coding during reading, and that phonological coding has a compensatory role for them.     

Age effects in coding tasks: componential analysis and test of the sensory deficit hypothesis

Multiple forms of a symbol-digit substitution task were used to provide a componential analysis of age differences in coding task performance. The results demonstrated age differences in feature encoding, memory, and visual search. A 2nd experiment was conducted with young adults to investigate a sensory deficit as a locus of age differences. The spatial contrast sensitivity deficit of older adults was simulated on forms by applying a digital filter. Persons in the age-simulated contrast condition performed worse than those in the normal contrast condition. The stimulus degradation effect was linked to visual search speed. The study illustrates the utility of componential analysis and offers direct support for the hypothesis that sensory deficits affect performance on tasks used to assess intelligence.     

Ensemble coding of faces occurs in children and develops dissociably from coding of individual faces

Ensemble coding allows adults to access useful information about average properties of groups, sometimes even in the absence of detailed representations of individual group members. This form of coding may emerge early in development with initial reports of ensemble coding for simple properties (size, numerosity) in young children and even infants. Here we demonstrate that ensemble coding of faces, which provides information about average properties of social groups, is already present in 6–8‐year‐old children. This access to average information increases with age from 6 to 18 years and its development is dissociable from age‐related improvements in the coding of individual face identities. This dissociation provides the first direct evidence that distinct processes underlie ensemble and individual coding of face identity, evidence that has been lacking from adult studies. More generally, our results add to the emerging evidence for impressively mature sensitivity to statistical properties of the visual environment in children. They indicate that children have access to gist information about social groups that may facilitate adaptive social behaviour.     

Haptic modality takes its time: Dynamic of activations of sensory modalities in perceptual and memory processes

Based on claims resulting from grounded cognition theory that perceptual and memory processes are using the same distributed systems, the present study investigated the temporal aspect of access to memory traces through haptic and auditory modalities. Unlike in the case of visual or auditory components, the perception of a vibrotactile component is more sequential in nature and therefore cannot be fully processed before the end of the signal. The present study explores the dynamic of components activation in a situation of audio‐vibrotactile asynchrony. We used a short‐term priming paradigm consisting of an association phase (between a vibration and sound) and a test phase testing priming effect of a vibrotactile stimulation on the processing of a target sound. Results showed an interference with a simultaneous processing and a facilitation with a sequential processing. The temporality process of perceptual components is also important at a memory level.