Musical space synesthesia: Automatic,explicit and conceptual connections between musical stimuli and space

In musical–space synesthesia, musical pitches are perceived as having a spatially defined array. Previous studies showed that symbolic inducers (e.g., numbers, months) can modulate response according to the inducer’s relative position on the synesthetic spatial form. In the current study we tested two musical–space synesthetes and a group of matched controls on three different tasks: musical–space mapping, spatial cue detection and a spatial Stroop-like task. In the free mapping task, both synesthetes exhibited a diagonal organization of musical pitch tones rising from bottom left to the top right. This organization was found to be consistent over time. In the subsequent tasks, synesthetes were asked to ignore an auditory or visually presented musical pitch (irrelevant information) and respond to a visual target (i.e., an asterisk) on the screen (relevant information). Compatibility between musical pitch and the target’s spatial location was manipulated to be compatible or incompatible with the synesthetes’ spatial representations. In the spatial cue detection task participants had to press the space key immediately upon detecting the target. In the Stroop-like task, they had to reach the target by using a mouse cursor. In both tasks, synesthetes’ performance was modulated by the compatibility between irrelevant and relevant spatial information. Specifically, the target’s spatial location conflicted with the spatial information triggered by the irrelevant musical stimulus. These results reveal that for musical–space synesthetes, musical information automatically orients attention according to their specific spatial musical-forms. The present study demonstrates the genuineness of musical–space synesthesia by revealing its two hallmarks—automaticity and consistency. In addition, our results challenge previous findings regarding an implicit vertical representation for pitch tones in non-synesthete musicians.     

Superior encoding enhances recall in color-graphemic synesthesia

Synesthesia is a phenomenon in which particular stimuli, such as letters or sound, generate a secondary sensory experience in particular individuals. Reports of enhanced memory in synesthetes raise the question of its cognitive and neurological substrates. Enhanced memory in synesthetes could arise from the explicit or implicit use of a synesthetic cue to aid memory, from changes unique to the synesthete brain, or from both, depending on the task. To assess this question, we tested nine color-graphemic synesthetes using standardized neuropsychological measures that should not trigger color-graphemic synesthesia (visuo-spatial tests) and measures that should trigger color-graphemic synesthesia (verbal tasks). We found a synesthetic advantage on both types of tests, primarily in the initial encoding of information. The pattern of results adds to existing evidence of advantages in synesthetic memory, as well as provides novel evidence that synesthetes may have enhanced encoding rather than superior recall. Synesthetes learn more initially, rather than forgetting less over time.     

Do the colors of your letters depend on your language? Language-dependent and universal influences on grapheme-color synesthesia in seven languages

Grapheme-color synesthetes experience graphemes as having a consistent color (e.g., “N is turquoise”). Synesthetes’ specific associations (which letter is which color) are often influenced by linguistic properties such as phonetic similarity, color terms (“Y is yellow”), and semantic associations (“D is for dog and dogs are brown”). However, most studies of synesthesia use only English-speaking synesthetes. Here, we measure the effect of color terms, semantic associations, and non-linguistic shape-color associations on synesthetic associations in Dutch, English, Greek, Japanese, Korean, Russian, and Spanish. The effect size of linguistic influences (color terms, semantic associations) differed significantly between languages. In contrast, the effect size of non-linguistic influences (shape-color associations), which we predicted to be universal, indeed did not differ between languages. We conclude that language matters (outcomes are influenced by the synesthete’s language) and that synesthesia offers an exceptional opportunity to study influences on letter representations in different languages.     

Consistency and strength of grapheme-color associations are separable aspects of synesthetic experience

Consistency of synesthetic associations over time is a widely used test of synesthesia. Since many studies suggest that consistency is not a completely reliable feature, we compared the consistency and strength of synesthetes’ grapheme-color associations. Consistency was measured by scores on the Synesthesia Battery and by the Euclidean distance in color space for the specific graphemes tested for each participant. Strength was measured by congruency magnitudes on the Implicit Association Test. The strength of associations was substantially greater for synesthetes than non-synesthetes, suggesting that this is a novel, objective marker of synesthesia. Although, intuitively, strong associations should also be consistent, consistency and strength were uncorrelated, indicating that they are likely independent, at least for grapheme-color synesthesia. These findings have implications for our understanding of synesthesia and for estimates of its prevalence since synesthetes who experience strong, but inconsistent, associations may not be identified by tests that focus solely on consistency.     

When 9 is not on the right: Implications from number-form synesthesia

Number-form synesthetes consciously experience numbers in spatially-defined locations. For non-synesthete individuals, a similar association of numbers and space appears in the form of an implicit mental number line as signified by the distance effect–reaction time decreases as the numerical distance between compared numbers increases. In the current experiment, three number-form synesthetes and two different non-synesthete control groups (Hebrew speaking and English speaking) performed a number comparison task. Synesthete participants exhibited a sizeable distance effect only when presented numbers were congruent with their number-form. In contrast, the controls exhibited a distance effect regardless of congruency or presentation type. The findings suggest that: (a) number-form synesthesia impairs the ability to represent numbers in a flexible manner according to task demands; (b) number-form synesthesia is a genuine tangible experience, triggered involuntarily; and (c) the classic mental number line can be more pliable than previously thought and appears to be independent of cultural-lingo direction.     

Is it for real? Evaluating authenticity of musical pitch-space synesthesia

In spatial-sequence synesthesia, ordinal sequences are visualized in explicit spatial locations. We examined a recently documented subtype in which musical notes are represented in spatial configurations, to verify consistency and automaticity of musical pitch-space (M-S) synesthesia. An M-S synesthete performed a mapping pre-task (Exp. 1) used to indicate the locations of 7 auditory or visually presented notes, in 2 sessions a month apart. Results revealed strong correlations between sessions, suggesting the synesthete's musical forms were consistent over time. Experiment 2 assessed automaticity of M-S synesthesia. The same synesthete and matched controls preformed a spatial Stroop-like task. Participants were presented with an auditory or visual musical note and then had to reach for an asterisk (target) with a mouse cursor. The target appeared in a compatible or incompatible location (according to the synesthete's spatial representation). A compatibility effect was found for the synesthete but not for controls. The synesthete was significantly faster when the target appeared in compatible locations than in incompatible ones. Our findings show that for synesthetes, auditory and visually presented notes automatically trigger attention to specific spatial locations according to their specific M-S associations.     

数形联觉——数字空间表征研究“新”取向

数字空间表征是人类对数字进行表征的重要方式。数形联觉（number-form synesthesia）是一种数字可以有意识地引起空间知觉的独特现象,与此类似的是非联觉者中广泛存在的无意识的心理数字线（mental number line）现象。两者在行为和脑机制上存在着很多重叠,也存在着值得思考的差异。数形联觉的研究能够提供实质性的行为和脑机制数据,用以解决数字空间表征研究中出现的问题,加强对于数字空间表征的理解;也为更加全面深入地开展进一步研究提供了新的启示,成为数字空间表征研究中值得推崇的新取向。     

Implicit response-irrelevant number information triggers the SNARC effect: Evidence using a neural overlap paradigm

There is evidence from the SNARC (spatial–numerical association of response codes) effect and NDE (numerical distance effect) that number activates spatial representations. Most of this evidence comes from tasks with explicit reference to number, whether through presentation of Arabic digits (SNARC) or through magnitude decisions to nonsymbolic representations (NDE). Here, we report four studies that use the neural overlap paradigm developed by Fias, Lauwereyns, and Lammertyn (2001) to examine whether the presentation of implicit and task-irrelevant numerosity information (nonsymbolic arrays and auditory numbers) is enough to activate a spatial representation of number. Participants were presented with either numerosity arrays (1–9 circles or triangles) to which they made colour (Experiment 1) or orientation (Experiment 2) judgements, or auditory numbers coupled with an on-screen stimulus to which they made a colour (Experiment 3) or orientation (Experiment 4) judgement. SNARC effects were observed only for the orientation tasks. Following the logic of Fias et al., we argue that this SNARC effect occurs as a result of overlap in parietal processing for number and orientation judgements irrespective of modality. Furthermore, we found stronger SNARC effects in the small number range (1–4) than in the larger number range (6–9) for both nonsymbolic displays and auditory numbers. These results suggest that quantity is extracted (and interferes with responses in the orientation task) but this is not exact for the entire number range. We discuss a number of alternative models and mechanisms of numerical processing that may account for such effects.     

Implicit response-irrelevant number information triggers the SNARC effect: Evidence using a neural overlap paradigm

There is evidence from the SNARC (spatial-numerical association of response codes) effect and NDE (numerical distance effect) that number activates spatial representations. Most of this evidence comes from tasks with explicit reference to number, whether through presentation of Arabic digits (SNARC) or through magnitude decisions to nonsymbolic representations (NDE). Here, we report four studies that use the neural overlap paradigm developed by Fias, Lauwereyns, and Lammertyn (2001) to examine whether the presentation of implicit and task-irrelevant numerosity information (nonsymbolic arrays and auditory numbers) is enough to activate a spatial representation of number. Participants were presented with either numerosity arrays (1-9 circles or triangles) to which they made colour (Experiment 1) or orientation (Experiment 2) judgements, or auditory numbers coupled with an on-screen stimulus to which they made a colour (Experiment 3) or orientation (Experiment 4) judgement. SNARC effects were observed only for the orientation tasks. Following the logic of Fias et al., we argue that this SNARC effect occurs as a result of overlap in parietal processing for number and orientation judgements irrespective of modality. Furthermore, we found stronger SNARC effects in the small number range (1-4) than in the larger number range (6-9) for both nonsymbolic displays and auditory numbers. These results suggest that quantity is extracted (and interferes with responses in the orientation task) but this is not exact for the entire number range. We discuss a number of alternative models and mechanisms of numerical processing that may account for such effects.     

Synesthesia and memory: color congruency, von Restorff, and false memory effects

In the current study, we explored the influence of synesthesia on memory for word lists. We tested 10 grapheme-color synesthetes who reported an experience of color when reading letters or words. We replicated a previous finding that memory is compromised when synesthetic color is incongruent with perceptual color. Beyond this, we found that, although their memory for word lists was superior overall, synesthetes did not exhibit typical color- or semantic-defined von Restorff isolation effects (von Restorff, 1933) compared with control participants. Moreover, our synesthetes exhibited a reduced Deese-Roediger-McDermott false memory effect (Deese, 1959; Roediger & McDermott, 1995). Taken as a whole, these findings are consistent with the idea that color-grapheme synesthesia can lead people to place a greater emphasis on item-specific processing and surface form characteristics of words in a list (e.g., the letters that make them up) relative to relational processing and more meaning-based processes. (PsycINFO Database Record (c) 2010 APA, all rights reserved).     

Synesthesia and creativity in fine arts students: An Empirical Look

Abstract:

A survey of 358 fine arts students at three large universities indicated that 23% (84) experienced synesthesia in a spontaneous and consistent manner, and 49% (175) reported no such experience. From this subject pool, 61 synesthetes and 61 controls, equated on gender, major, year in school, and verbal intelligence, received a battery of four creativity measures. The synesthetes scored significantly higher than controls on all four creativity measures; and their self‐rated frequency and vividness of synesthesia though significantly intercorrelated, correlated marginally with the measures of creativity. The results are consistent with anecdotal reports that synesthetes are often involved in musical and artistic fields, and possessive of a high degree of creativity.     

Comparing implicit and synaesthetic number–space associations: Visuospatial and verbal spatial–numerical associations of response codes

In the theory of the mental number line, number, and space are implicitly associated, giving rise to the spatial–numerical association of response codes (SNARC) effect, in which smaller numbers are more readily associated with the left side of space and larger numbers with the right, during a parity judgement task. Others, however, have argued that the SNARC effect is flexible and better explained by verbal rather than spatial associations. A few single-case studies on the SNARC effect have tested number–space synaesthetes, who make explicit associations between number and space. Here, we present data from experiments conducted on groups of synaesthetes and nonsynaesthetes on the classic SNARC parity judgement task with lateralized response keys and a modified version in which they responded to labels appearing on screen. Synaesthetes' behaviour was expected to differ from nonsynaesthetes' behaviour due to the explicit, fixed nature of their number–space associations, but both experiments show the two groups behaving in the same way, indicating that parity judgement tasks may not be tapping the same representation of number that gives rise to synaesthetic number–space experiences.     

Two plus blue equals green: Grapheme-color synesthesia allows cognitive access to numerical information via color

In grapheme-color synesthesia, graphemes (e.g., numbers or letters) evoke color experiences. It is generally reported that the opposite is not true: colors will not generate experiences of graphemes or their associated information. However, recent research has provided evidence that colors can implicitly elicit symbolic representations of associated graphemes. Here, we examine if these representations can be cognitively accessed. Using a mathematical verification task replacing graphemes with color patches, we find that synesthetes can verify such problems with colors as accurately as with graphemes. Doing so, however, takes time: ～250 ms per color. Moreover, we find minimal reaction time switch-costs for switching between computing with graphemes and colors. This demonstrates that given specific task demands, synesthetes can cognitively access numerical information elicited by physical colors, and they do so as accurately as with graphemes. We discuss these results in the context of possible cognitive strategies used to access the information.     

Synesthetic colors are elicited by sound quality in Japanese synesthetes

Determinants of synesthetic color choice for Japanese phonetic characters were studied in six Japanese synesthetes. The study used Hiragana and Katakana characters, which represent the same set of syllables although their visual forms are dissimilar. From a palette of 138 colors, synesthetes selected a color corresponding to each character. Results revealed that synesthetic color choices for Hiragana characters and those for their Katakana counterparts were remarkably consistent, indicating that color selection depended on character-related sounds and not visual form. This Hiragana–Katakana invariance cannot be regarded as the same phenomenon as letter case invariance, usually reported for English grapheme-color synesthesia, because Hiragana and Katakana characters have different identities whereas upper and lower case letters have the same identity. This involvement of phonology suggests that cross-activation between an inducer (i.e., letter/character) brain region and that of the concurrent (i.e., color) area in grapheme-color synesthesia is mediated by higher order cortical processing areas.     

Reduced audiovisual integration in synesthesia – evidence from bimodal speech perception

Recent research suggests synesthesia as a result of a hypersensitive multimodal binding mechanism. To address the question whether multimodal integration is altered in synesthetes in general, grapheme‐colour and auditory‐visual synesthetes were investigated using speech‐related stimulation in two behavioural experiments. First, we used the McGurk illusion to test the strength and number of illusory perceptions in synesthesia. In a second step, we analysed the gain in speech perception coming from seen articulatory movements under acoustically noisy conditions. We used disyllabic nouns as stimulation and varied signal‐to‐noise ratio of the auditory stream presented concurrently to a matching video of the speaker. We hypothesized that if synesthesia is due to a general hyperbinding mechanism this group of subjects should be more susceptible to McGurk illusions and profit more from the visual information during audiovisual speech perception. The results indicate that there are differences between synesthetes and controls concerning multisensory integration – but in the opposite direction as hypothesized. Synesthetes showed a reduced number of illusions and had a reduced gain in comprehension by viewing matching articulatory movements in comparison to control subjects. Our results indicate that rather than having a hypersensitive binding mechanism, synesthetes show weaker integration of vision and audition.     

Synesthesia: II. Psychophysical relations in the synesthesia of geometrically shaped taste and colored hearing

The psychophysical function in two synesthetic subjects, one with geometrically shaped taste and the other with colored hearing, is investigated. Each synesthete is compared with three control subjects. Pilot studies are used to construct appropriate response domains. Compared to controls, synesthetes give a restricted set of responses that are also asymmetric with respect to the range of choices available in various subsets of the response domain. A combination of absolute and relative effects is also distinctive of synesthetic percepts. It is concluded that synesthesia is a distinct phenomenon unlike ordinary mediated associations, and occupies an intermediate position in the range of simple to complex brain mechanisms.     

Is there an internal association of numbers to hands? The task set influences the nature of the SNARC effect

The SNARC effect refers to the association of smaller numbers with the left and of larger numbers with the right side of extracorporal space (Dehaene, Bossini, & Giraux, 1993). We tested the assumption that, in addition to these associations, numbers are also related to participants' hands. We report two experiments with vertically arranged buttons in which the nature of the SNARC effect depended on whether the task set was button or hand related: In the first case, a vertical location-related SNARC effect occurred, whereas in the second a hand-related SNARC effect was found. Our third experiment confirmed that space-related number representations dominate the SNARC effect when the buttons are arranged horizontally. We concluded that both effector- and space-related number representations can influence and modify the SNARC effect.     

Effects of non-symbolic numerical information suggest the existence of magnitude-space synesthesia

In number-space synesthesia, numbers are visualized in spatially defined arrays. In a recent study (Gertner et al. in Cortex, doi: 10.1016/j.cortex.2012.03.019 , 2012), we found that the size congruency effect (SiCE) for physical judgments (i.e., comparing numbers' physical sizes while ignoring their numerical values, for example, 8) was modulated by the spatial position of the presented numbers. Surprisingly, we found that the neutral condition, which is comprise solely of physical sizes (e.g., 3), was affected as well. This pattern gave rise to the idea that number-space synesthesia might entail not only discrete, ordered, meaningful symbols (i.e., Arabic numbers) but also continuous non-symbolic magnitudes (i.e., sizes, length, luminance, etc.). We tested this idea by assessing the performance of two number-space synesthetes and 12 matched controls in 3 comparative judgment tasks involving symbolic and non-symbolic stimuli: (1) Arabic numbers, (2) dot clusters, and (3) sizes of squares. The spatial position of the presented stimuli was manipulated to be compatible or incompatible with respect to the synesthetic number-space perceptions. Results revealed that for synesthetes, but not for controls, non-symbolic magnitudes (dot clusters) as well as symbolic magnitudes (i.e., Arabic numbers) interacted with space. Our study suggests that number-space synesthetes might have a general magnitude-space association that is not restricted to concrete symbolic stimuli. These findings support recent theories on the perception and evaluation of sizes in numerical cognition.     

The impact of inhibition capacities and age on number–space associations

Numerical and spatial representations are tightly linked, i.e., when doing a binary classification judgment on Arabic digits, participants are faster to respond with their left/right hand to small/large numbers, respectively (Spatial-Numerical Association of Response Codes, SNARC effect, Dehaene et al. in J Exp Psychol Gen 122:371–396, 1993). To understand the underlying mechanisms of the well-established SNARC effect, it seems essential to explore the considerable inter-individual variability characterizing it. The present study assesses the respective roles of inhibition, age, working memory (WM) and response speed. Whereas these non-numerical factors have been proposed as potentially important factors to explain individual differences in SNARC effects, none (except response speed) has so far been explored directly. Confirming our hypotheses, the results show that the SNARC effect was stronger in participants that had weaker inhibition abilities (as assessed by the Stroop task), were relatively older and had longer response times. Interestingly, whereas a significant part of the age influence was mediated by cognitive inhibition, age also directly impacted the SNARC effect. Similarly, cognitive inhibition abilities explained inter-individual variability in number–space associations over and above the factors age, WM capacity and response speed. Taken together our results provide new insights into the nature of number–space associations by describing how these are influenced by the non-numerical factors age and inhibition.     

Precision of synesthetic color matching resembles that for recollected colors rather than physical colors

Grapheme-color synesthesia is an atypical condition in which individuals experience sensations of color when reading printed graphemes such as letters and digits. For some grapheme-color synesthetes, seeing a printed grapheme triggers a sensation of color, but hearing the name of a grapheme does not. This dissociation allowed us to compare the precision with which synesthetes are able to match their color experiences triggered by visible graphemes, with the precision of their matches for recalled colors based on the same graphemes spoken aloud. In six synesthetes, color matching for printed graphemes was equally variable relative to recalled experiences. In a control experiment, synesthetes and age-matched controls either matched the color of a circular patch while it was visible on a screen, or they judged its color from memory after it had disappeared. Both synesthetes and controls were more variable when matching from memory, and the variance of synesthetes' recalled color judgments matched that associated with their synesthetic judgments for visible graphemes in the first experiment. Results suggest that synesthetic experiences of color triggered by achromatic graphemes are analogous to recollections of color. (PsycINFO Database Record (c) 2012 APA, all rights reserved).