Processes underlying dimensional interactions: Correspondences between linguistic and nonlinguistic dimensions

In six experiments, we examined speeded classification when one dimension was linguistic and the other was nonlinguistic. In five of these, attributes on the dimensions corresponded meaningfully, having in common the concepts "high" and "low." For example, in Experiment 1, the visually presented words HI and LO were paired with high- or low-pitched tones; in Experiment 2, the dimensions were visual words and vertical position, in Experiment 3, they were spoken words and position, and in Experiments 4 and 5, spoken words and pitch. For each dimension in each pair, subjects suffered Garner interference when dimensions were varied orthogonally. Garner interference remained constant across 15 blocks of trials (Experiment 5). Subjects also showed significant congruity effects in all experiments, with attributes from congruent stimuli (e.g., HI/high pitch) classified faster than attributes from incongruent stimuli (e.g., HI/low pitch). These results differ from those obtained previously with noncorresponding pairs of linguistic-nonlinguistic dimensions. The results also differ from those obtained with traditional Stroop dimensions (colors and color words; Experiment 6), which showed minimal Garner interference and diminishing congruity effects across blocks of trials. We conclude that the interactions found here represent cross-talk between channels within a semantic level of processing. We contrast our view with current models of dimensional interaction.     

Multisensory synesthetic interactions in the speeded classification of visual size

In the present study, we attempted to demonstrate a synesthetic relationship between auditory frequency and visual size. In Experiment 1, participants performed a speeded visual size discrimination task in which they had to judge whether a variable-sized disk was bigger or smaller than a standard reference disk. A task-irrelevant sound that was either synesthetically congruent with the relative size of the disk (e.g., a low-frequency sound presented with a bigger disk) or synesthetically incongruent with it (e.g., a low-frequency sound presented with a smaller disk) was sometimes presented together with the variable disk. Reaction times were shorter in the synesthetically congruent condition than in the incongruent condition. Verbal labeling and semantic mediation interpretations of this interaction were explored in Experiment 2, in which high- and low-frequency sounds were presented in separate blocks of trials, and in Experiment 3, in which the tones were replaced by the spoken words "high" and "low." Response priming/bias explanations were ruled out in Experiment 4, in which a synesthetic congruency effect was still reported even when participants made same-versus-different discrimination responses regarding the relative sizes of the two disks. Taken together, these results provide the first empirical demonstration that the relative frequency of an irrelevant sound can influence the speed with which participants judge the size of visual stimuli when the sound varies on a trial-by-trial basis along a synesthetically compatible dimension. The possible cognitive bases for this synesthetic association are also discussed.     

Dimensional interactions in vibrotactile information processing

The effect of irrelevant dimensional variation on the processing of vibrotactile stimuli was measured. Six observers performed a speeded classification task with stimuli varying along the dimensions of pitch and loudness. Choice reaction times were obtained for stimuli differing on one dimension alone, on two correlated dimensions, or on two orthogonally varied dimensions. Compared to one-dimension performance, reaction times were faster in the correlated condition and slower in the orthogonal condition. In general, these findings agreed with similar experiments in other modalities, with the exception that the effects in this study tended to be stronger for cases in which loudness, rather than pitch, was the relevant dimension. The results are explained in terms of the integrality of pitch and loudness and of the relative discriminability of dimensions.     

Some perceptual dependencies in speeded classification of vowel color and pitch

Wood (1975) suggested that information specifying consonant identity is dependent on the earlier processing of pitch. Is vowel identification also dependent on the prior processing of pitch? In contrast with the results obtained with consonants, Kuhl (1975, 1976) reported that infants responded selectively to differences in vowel color when pitch varied but not to differences in pitch when vowel color varied. Miller (1978) has also reported that adults show mutual, symmetric interference effects in speeded classification of vowel color and pitch, In the present study, judgments of vowel color and pitch were examined in a reaction time task in order to assess the effects of the relative discriminability of vowel and pitch quality on speeded classification. In addition, we also compared the classification of isolated vowels to vowels in consonantal context. The overall results were consistent with both Kuhl’s and Miller’s earlier findings but refine our understanding of the interaction between various dimensions by showing that vowel identification is also dependent on the processing of pitch information. Such an interaction, however, becomes evident only when the processing dependencies are examined across a wide range of stimulus values for each dimension. The present findings provide additional information about the nature of processing dependencies among dimensions in speech and the methods by which such dependencies may be studied.     

Are crossmodal correspondences relative or absolute? Sequential effects on speeded classification

Crossmodal correspondences have often been demonstrated using congruency effects between pairs of stimuli in different sensory modalities that vary along separate dimensions. To date, however, it is still unclear the extent to which these correspondences are relative versus absolute in nature: that is, whether they result from pre-defined values that rigidly link the two dimensions or rather result from flexible values related to the previous occurrence of the crossmodal stimuli. Here, we investigated this issue in a speeded classification task featuring the correspondence between auditory pitch and visual size (e.g., congruent correspondence between high pitch/small disc and low pitch/large disc). Participants classified the size of the visual stimuli (large vs. small) while hearing concurrent high- or low-pitched task-irrelevant sounds. On some trials, visual stimuli were paired instead with “intermediate” pitch, that could be interpreted differently according to the auditory stimulus on the preceding trial (i.e., as “lower” following the presentation of a high pitch tone, but as “higher” following the presentation of a low pitch tone). Performance on sequence-congruent trials (e.g., when a small disc paired with the intermediate-pitched tone was preceded by a low pitch tone) was compared to sequence-incongruent trials (e.g., when a small disc paired with the intermediate-pitch tone was by a high-pitched tone). The results revealed faster classification responses on sequence-congruent than on sequence-incongruent trials. This demonstrates that the effect of the pitch/size correspondence is relative in nature, and subjected to trial-by-trial interpretation of the stimulus pair.     

Perceptual primacy of dimensions: support for a model of dimensional interaction

Do Ss always process multidimensional stimuli according to psychologically primary dimensions? Our hypothesis is that they do: Primary dimensions provide one component of a new model of dimensional interaction, a model that distinguishes information processed at the level of attributes from information processed at the level of the stimulus. By using sound stimuli created from the dimensions pitch-loudness (Experiments 1 and 2), pitch-timbre (Experiment 3), and loudness-timbre (Experiment 4), we tested performance in selective- and divided-attention tasks at each of three orientations of axes: 0 degrees, 22.5 degrees, and 45 degrees. Each experiment revealed strong evidence of primacy: As axes rotated from 0 degrees to 45 degrees, selective attention deteriorated, but divided attention improved, producing a distinct pattern of convergence. Each experiment also revealed effects of congruity: Attributes from corresponding poles of a dimension (e.g., high pitch and loud) were classified faster than those from noncorresponding poles. The results fit well with our new conception but are inconsistent with other current models of dimensional interaction.     

Similarity relations among synesthetic stimuli and their attributes

Four experiments examined the similarity relations that exist among bimodal attributes that correspond synesthetically (e.g., white color and high pitch) and among stimuli formed by combining these attributes either congruently (e.g., white/high, black/low) or incongruently (e.g., white/low, black/high). Previous research suggests two hypotheses: (a) Synesthetic stimuli are compared as wholes on the basis of their overall similarity, and (b) nonidentical congruent stimuli are more dissimilar than nonidentical incongruent stimuli. In these four experiments, similarity among either individual attributes (Experiments 1 and 4) or bimodal stimuli (Experiments 2 and 3) was measured by either ratings or response latencies; similarity judgments were scaled with an individual differences scaling procedure (SINDSCAL). Stimulus comparisons were fit well by a Euclidean but not a city-block metric, supporting the overall similarity hypothesis. However, there was little evidence that subjects perceived congruity/incongruity among stimulus wholes, even though subjects were sensitive to correspondence/noncorrespondence among attributes. These results were replicated in four additional experiments using larger stimulus sets. A two-process account is proposed in which stimulus formation (intersensory processing) occurs independently of the abstraction of cross-sensory meaning (figurative processing).     

The relationship between allocentric and egocentric frames of reference and categorical and coordinate spatial information processing

We report two experiments on the relationship between allocentric/egocentric frames of reference and categorical/coordinate spatial relations. Jager and Postma (2003) suggest two theoretical possibilities about their relationship: categorical judgements are better when combined with an allocentric reference frame and coordinate judgements with an egocentric reference frame (interaction hypothesis); allocentric/egocentric and categorical/coordinate form independent dimensions (independence hypothesis). Participants saw stimuli comprising two vertical bars (targets), one above and the other below a horizontal bar. They had to judge whether the targets appeared on the same side (categorical) or at the same distance (coordinate) with respect either to their body-midline (egocentric) or to the centre of the horizontal bar (allocentric). The results from Experiment 1 showed a facilitation in the allocentric and categorical conditions. In line with the independence hypothesis, no interaction effect emerged. To see whether the results were affected by the visual salience of the stimuli, in Experiment 2 the luminance of the horizontal bar was reduced. As a consequence, a significant interaction effect emerged indicating that categorical judgements were more accurate than coordinate ones, and especially so in the allocentric condition. Furthermore, egocentric judgements were as accurate as allocentric ones with a specific improvement when combined with coordinate spatial relations. The data from Experiment 2 showed that the visual salience of stimuli affected the relationship between allocentric/egocentric and categorical/coordinate dimensions. This suggests that the emergence of a selective interaction between the two dimensions may be modulated by the characteristics of the task.     

Interaction among auditory dimensions: timbre, pitch, and loudness

In two experiments, we examined whether or not pairs of auditory dimensions--timbre-loudness (Experiment 1) and timbre-pitch (Experiment 2)--interact in speeded classification. Subjects classified values from one dimension while the other dimension was (1) held constant (baseline), (2) varied orthogonally (filtering), or (3) correlated linearly. The subjects showed substantial Garner interference when classifying all dimensions--that is, poor performance at filtering relative to baseline. Timbre and loudness displayed redundancy gain (i.e., performance faster than baseline) when correlated positively, but redundancy loss (i.e., interference) when correlated negatively. Timbre and pitch displayed redundancy gain however dimensions were correlated. Both pairs of dimensions showed substantial effects of congruity: Attributes from one dimension were classified faster when paired with "congruent" attributes from the other dimension. The results are interpreted in terms of an interactive multichannel model of auditory processing.     

Effects of pitch distance and likelihood on the perceived duration of deviant auditory events

When a deviant (oddball) stimulus is presented within a series of otherwise identical (standard) stimuli, the duration of the oddball tends to be overestimated. Two experiments investigated factors affecting systematic distortions in the perceived duration of oddball stimuli. Both experiments used an auditory oddball paradigm where oddball tones varied in both their pitch distance from the pitch of a standard tone and their likelihood of occurrence. Experiment 1 revealed that (1) far-pitch oddballs were perceived to be longer than near-pitch oddballs, (2) effects of pitch distance were greater in low-likelihood conditions, and (3) oddballs in later serial positions were perceived to be longer than oddballs in earlier serial positions. The above effects held regardless of whether oddballs were higher or lower in pitch than the standard. Experiment 2 revealed a pattern of response times in an oddball detection task that generally paralleled the pattern of data observed in Experiment 1; across conditions, there was a negative correlation between detection times and perceived duration. Taken together, the results suggest that the observed effects of oddball pitch, likelihood, and position on perceived duration are at least partly driven by how quickly individuals are able to initiate timing the oddball following its onset. Implications for different theoretical accounts of the oddball effect are discussed.     

Influence of auditory and audiovisual stimuli on the right–left prevalence effect

When auditory stimuli are used in two-dimensional spatial compatibility tasks, where the stimulus and response configurations vary along the horizontal and vertical dimensions simultaneously, a right–left prevalence effect occurs in which horizontal compatibility dominates over vertical compatibility. The right–left prevalence effects obtained with auditory stimuli are typically larger than that obtained with visual stimuli even though less attention should be demanded from the horizontal dimension in auditory processing. In the present study, we examined whether auditory or visual dominance occurs when the two-dimensional stimuli are audiovisual, as well as whether there will be cross-modal facilitation of response selection for the horizontal and vertical dimensions. We also examined whether there is an additional benefit of adding a pitch dimension to the auditory stimulus to facilitate vertical coding through use of the spatial-musical association of response codes (SMARC) effect, where pitch is coded in terms of height in space. In Experiment 1, we found a larger right–left prevalence effect for unimodal auditory than visual stimuli. Neutral, non-pitch coded, audiovisual stimuli did not result in cross-modal facilitation, but did show evidence of visual dominance. The right–left prevalence effect was eliminated in the presence of SMARC audiovisual stimuli, but the effect influenced horizontal rather than vertical coding. Experiment 2 showed that the influence of the pitch dimension was not in terms of influencing response selection on a trial-to-trial basis, but in terms of altering the salience of the task environment. Taken together, these findings indicate that in the absence of salient vertical cues, auditory and audiovisual stimuli tend to be coded along the horizontal dimension and vision tends to dominate audition in this two-dimensional spatial stimulus–response task.     

The relationship between allocentric and egocentric frames of reference and categorical and coordinate spatial information processing

We report two experiments on the relationship between allocentric/egocentric frames of reference and categorical/coordinate spatial relations. Jager and Postma (2003) suggest two theoretical possibilities about their relationship: categorical judgements are better when combined with an allocentric reference frame and coordinate judgements with an egocentric reference frame (interaction hypothesis); allocentric/egocentric and categorical/coordinate form independent dimensions (independence hypothesis). Participants saw stimuli comprising two vertical bars (targets), one above and the other below a horizontal bar. They had to judge whether the targets appeared on the same side (categorical) or at the same distance (coordinate) with respect either to their body-midline (egocentric) or to the centre of the horizontal bar (allocentric). The results from Experiment 1 showed a facilitation in the allocentric and categorical conditions. In line with the independence hypothesis, no interaction effect emerged. To see whether the results were affected by the visual salience of the stimuli, in Experiment 2 the luminance of the horizontal bar was reduced. As a consequence, a significant interaction effect emerged indicating that categorical judgements were more accurate than coordinate ones, and especially so in the allocentric condition. Furthermore, egocentric judgements were as accurate as allocentric ones with a specific improvement when combined with coordinate spatial relations. The data from Experiment 2 showed that the visual salience of stimuli affected the relationship between allocentric/egocentric and categorical/coordinate dimensions. This suggests that the emergence of a selective interaction between the two dimensions may be modulated by the characteristics of the task.     

Stroop interference is affected in inhibition of return

In previous research, we have shown that the processing of targets that are presented to locations subject to inhibition of return (IOR) is affected by an inhibitory tagging mechanism. This mechanism acts by disconnecting activated representations of stimuli at inhibited locations from their associated responses. In two experiments, we assessed whether this inhibitory tagging mechanism of visual attention is also applied to task-irrelevant but prepotent dimensions of target stimuli, such as words in the Stroop task. To test this hypothesis, we examined the Stroop effect in an IOR procedure. The results showed that (1) IOR can be found in a color discrimination task, (2) the Stroop interference was reduced (Experiment 1) or eliminated (Experiment 2) when stimuli appeared at cued locations, as compared with cases in which they were presented at uncued locations, and (3) the effect of inhibitory tagging was limited to the shortest stimulus onset asynchrony value, replicating previous findings. These results agree with the idea that inhibitory tagging, occurring in IOR, affects the efficiency with which color words compete for responses in Stroop-like situations.     

Does crossmodal correspondence modulate the facilitatory effect of auditory cues on visual search?

The ??pip-and-pop effect?? refers to the facilitation of search for a visual target (a horizontal or vertical bar whose color changes frequently) among multiple visual distractors (tilted bars also changing color unpredictably) by the presentation of a spatially uninformative auditory cue synchronized with the color change of the visual target. In the present study, the visual stimuli in the search display changed brightness instead of color, and the crossmodal congruency between the pitch of the auditory cue and the brightness of the visual target was manipulated. When cue presence and cue congruency were randomly varied between trials (Experiment 1), both congruent cues (low-frequency tones synchronized with dark target states or high-frequency tones synchronized with bright target states) and incongruent cues (the reversed mapping) facilitated visual search performance equally, relative to a no-cue baseline condition. However, when cue congruency was blocked and the participants were informed about the pitch?Cbrightness mapping in the cue-present blocks (Experiment 2), performance was significantly enhanced when the cue and target were crossmodally congruent as compared to when they were incongruent. These results therefore suggest that the crossmodal congruency between auditory pitch and visual brightness can influence performance in the pip-and-pop task by means of top-down facilitation.     

Individual differences in stroop and local-global processing: a possible role of interhemispheric interaction

Two experiments are reported examining individual differences in the processing of centrally presented stimuli containing two dimensions of information lateralized to opposite cerebral hemispheres. Left-handers, arising from (a) their lesser degree of functional lateralization and (b) their greater degree of callosal connectivity, were hypothesized to exhibit greater interdimensional (and presumably interhemispheric) interaction. Experiment 1 utilized local-global stimuli, and left-handers were found to be impaired at keeping the two dimensions independent and superior at integrating the two dimensions. Experiment 2 used Stroop stimuli, and left-handers again were impaired at keeping the two dimensions independent (i.e., showed greater Stroop interference). Correlational analyses indicated that the mechanisms of interdimensional integration versus independence are at least partially independent from one another. Results suggest that aspects of interhemispheric interaction can be addressed via the use of nonlateralized input.     

Redundancy effects in short-term memory

Five experiments are reported whose purpose was to demonstrate that short-term memory is improved by redundancy within the material. In Experiment I “tune” containing two, three, four and five tones of differing frequencies had to be coded into digits 1-5, to indicate the order of the pitches in a tune. Performance on stimuli containing correlated amplitude and duration were compared with the uni-dimensional condition. Experiment II repeated I, but required intensity to be coded. Experiment III required pitch coding under three conditions including that when amplitude and frequency were uncorrelated, and compared the performance of musically trained sunjects with nonmusicians. Experiment IV repeated III, but subjects were informed of the relation between dimensions. Experiment V involved “shadowing” the tunes by whistling simultaneously with the stimulus.

It was concluded (a) that intercorrelation improves, but zero correlation impairs short-term memory; (b) that knowledge of the relation between dimensions improves performance in the correlated condition, but does not prevent impairment under zero correlation; and (c) the performance of musically trained subjects exceeds that of controls and is unaffected by the presence of a correlated or uncorrelated dimension.     

Automatic shifts of attention in the Dimensional Change Card Sort task: Subtle changes in task materials lead to flexible switching

Two experiments tested a hypothesis that reducing demands on executive control in a Dimensional Change Card Sort task will lead to improved performance in 3-year-olds. In Experiment 1, the shape dimension was represented by two dissimilar values (stars and flowers), and the color dimension was represented by two similar values (red and pink). This configuration of stimuli rendered shape more salient than color. In Experiment 2, attentional weights of each dimension value were manipulated by using two versus four values to represent the dimensions of shape and color. The results indicated that increasing saliency of the postswitch dimension (Experiment 1) and reducing attentional weights of individual dimension values (Experiment 2) lead to a marked improvement in the postswitch sorting accuracy in 3-year-olds.     

Primacy of dimensions in vibrotactile perception: an evaluation of early holistic models.

Early holistic models of perception presume that stimuli composed of interacting dimensions can be experienced initially as undifferentiated. This view, formalized through recourse to a Euclidean geometry of perceptual space, predicts that the orientation of axes used to create stimulus sets is unimportant to performance in speeded classification. We tested this idea by using the interacting vibrotactile dimensions of pitch and loudness. Despite perceivers' relatively poor experience with these dimensions, we showed that the orientation corresponding to pitch and loudness was unique in vibrotactile perceptual space; subjects classified stimuli more efficiently at this orientation than at other orientations. Certain holistic models also claim that when stimulus differences are small, perceivers can recognize change without distinguishing the kind of change. We tested this idea by using a signal detection analysis of unspeeded same-different decisions. We found that subjects' ability to notice the kind of change equaled their ability to notice the change alone. In view of these results, which indicate that pitch and loudness are primary in vibrotactile perception, we detail a new conception of dimensional interaction.     

Exemplar effects in the context of a categorization rule: Featural and holistic influences

Brooks and colleagues (S. W. Allen & L. R. Brooks, 1991; G. Regehr & L. R. Brooks, 1993) have shown that the classification of transfer stimuli is influenced by their similarity to training stimuli, even when a perfect classification rule is available. It is argued that the original effect obtained by Brooks and colleagues might have resulted from two potential confounding variables. Once these confounds were controlled, the current authors did not replicate Brooks and colleagues' results in Experiment 1. Exemplar effects appeared in Experiment 2 when transfer stimuli were perceptually more similar to training stimuli than in Experiment 1. In Experiment 3, the authors obtained exemplar effects with separated stimuli, a finding that was not predicted by Brooks and colleagues' model. The authors suggest that a close perceptual match between training and transfer stimuli is necessary for the effect to occur, for both integrated and separated stimuli. The nature of this perceptual match, holistic or featural, is discussed.