Colored-Speech Synaesthesia Is Triggered by Multisensory, Not Unisensory, Perception

ABSTRACT— Although it is estimated that as many as 4% of people experience some form of enhanced cross talk between (or within) the senses, known as synaesthesia, very little is understood about the level of information processing required to induce a synaesthetic experience. In work presented here, we used a well-known multisensory illusion called the McGurk effect to show that synaesthesia is driven by late, perceptual processing, rather than early, unisensory processing. Specifically, we tested 9 linguistic-color synaesthetes and found that the colors induced by spoken words are related to what is perceived (i.e., the illusory combination of audio and visual inputs) and not to the auditory component alone. Our findings indicate that color-speech synaesthesia is triggered only when a significant amount of information processing has occurred and that early sensory activation is not directly linked to the synaesthetic experience.     

What is the relationship between Aphantasia,Synaesthesia and Autism?

For people with aphantasia, visual imagery is absent or markedly impaired. Here, we investigated the relationship between aphantasia and two other neurodevelopmental conditions also linked to imagery differences: synaesthesia, and autism. In Experiment 1a and 1b, we asked whether aphantasia and synaesthesia can co-occur, an important question given that synaesthesia is linked to strong imagery. Taking grapheme-colour synaesthesia as a test case, we found that synaesthesia can be objectively diagnosed in aphantasics, suggesting visual imagery is not necessary for synaesthesia to occur. However, aphantasia influenced the type of synaesthesia experienced (favouring ‘associator’ over ‘projector’ synaesthesia - a distinction tied to the phenomenology of the synaesthetic experience). In Experiment 2, we asked whether aphantasics have traits associated with autism, an important question given that autism – like aphantasia – is linked to weak imagery. We found that aphantasics reported more autistic traits than controls, with weaknesses in imagination and social skills.     

An exploratory study of linguistic–colour associations across languages in multilingual synaesthetes

In one of the most common forms of synaesthesia, linguistic–colour synaesthesia, colour is induced by stimuli such as numbers, letters, days of the week, and months of the year. It is not clear, however, whether linguistic–colour synaesthesia is determined more by higher level semantic information—that is, word meaning—or by lower level grapheme or phoneme structure. To explore this issue, we tested whether colour is consistently induced by grapheme or phoneme form or word meaning in bilingual and trilingual linguistic–colour synaesthetes. We reasoned that if the induced colour was related to word meaning, rather than to the acoustic or visual properties of the words, then the induced colours would remain consistent across languages. We found that colours were not consistently related to word meaning across languages. Instead, induced colours were more related to form properties of the word across languages, particularly visual structure. However, the type of inducing stimulus influenced specific colour associations. For example, colours to months of the year were more consistent across languages than were colours to numbers or days of the week. Furthermore, the effect of inducing stimuli was also associated with the age of acquisition of additional languages. Our findings are discussed with reference to a critical period in language acquisition on synaesthesia.     

Synaesthesia is associated with enhanced, self-rated visual imagery

Although the condition known as synaesthesia is currently undergoing a scientific resurgence, to date the literature has largely focused on the heterogeneous nature of synaesthesia across individuals. In order to provide a better understanding of synaesthesia, however, general characteristics need to be investigated. Synaesthetic experiences are often described as occurring 'internally' or in the 'mind's eye', which is remarkably similar to how we would describe our experience of visual mental imagery. We assessed the role of visual imagery in synaesthesia by sampling a large group of synaesthetes and found that they report experiencing more vivid mental images than controls. These findings have important implications for our general understanding of synaesthesia and, in particular, emphasize the need to control for visual imagery in behavioural and neuroimaging paradigms.     

Do sequence-space synaesthetes have better spatial imagery skills? Maybe not

Sequence-space synaesthesia is a type of visuo-spatial imagery in which numbers or calendar units are experienced to occupy locations in space. Previous studies have claimed that these synaesthetes (1) have stronger self-reported visual (but not spatial) imagery and (2) perform unusually well on mental rotation tasks that are usually taken to reflect spatial (but not visual) imagery. To further investigate whether this form of synaesthesia is related to spatial imagery, we compared synaesthetes to controls on the Object Spatial Imagery Questionnaire, a paper folding test and a mental rotation task. The synaesthetes did not differ from controls in self-reported spatial imagery, but showed a strong trend to report better visual imagery, replicating previously reported data patterns. Consistent with this, their paper folding and mental rotation performance was no better than controls. We also confirmed that, in our pooled sample, performance on both these tasks was positively correlated with self-reported spatial imagery. We suggest our data are more consistent with the view that sequence-space synaesthesia is related to visual than to spatial imagery, and we suggest reasons why previous studies may have found superior mental rotation performance.     

Synaesthetic perception of colour and visual space in a blind subject: an fMRI case study

In spatial sequence synaesthesia (SSS) ordinal stimuli are perceived as arranged in peripersonal space. Using fMRI, we examined the neural bases of SSS and colour synaesthesia for spoken words in a late-blind synaesthete, JF. He reported days of the week and months of the year as both coloured and spatially ordered in peripersonal space; parts of the days and festivities of the year were spatially ordered but uncoloured. Words that denote time-units and triggered no concurrents were used in a control condition. Both conditions inducing SSS activated the occipito-parietal, infero-frontal and insular cortex. The colour area hOC4v was engaged when the synaesthetic experience included colour. These results confirm the continued recruitment of visual colour cortex in this late-blind synaesthetes. Synaesthesia also involved activation in inferior frontal cortex, which may be related to spatial memory and detection, and in the insula, which might contribute to audiovisual integration related to the processing of inducers and concurrents.     

Synaesthesia: learned or lost?

The question why synaesthesia, an atypical binding within or between modalities, occurs is both enduring and important. Two explanations have been provided: (1) a congenital explanation: we are all born as synaesthetes but most of us subsequently lose the experience due to brain development; (2) a learning explanation: synaesthesia is related to some learning process during childhood. Three recent studies provide conflicting support for these explanations. Two studies supported the idea that synaesthesia is learned by showing that the frequency of everyday language implicitly modulates the synaesthetic experience. Another study argued that synaesthesia reflects basic, innate magnitude representations. In this paper we reassess these points of view, and show that it is possible for both to be valid. These findings are integrated into an interactive specialization account of development in order to explain the neuronal mechanism underlying synaesthesia.     

Is there a burden attached to synaesthesia? Health screening of synaesthetes in the general population

Synaesthesia has long been considered a benign alternative form of perception most often associated with positive rather than negative outcomes. The condition has been associated with a variety of cognitive and perceptual advantages, including benefits in memory, processing speed, and creativity. It is not currently recognized in the DSM‐IV. Recently, however, several studies have raised the question of a possible link between synaesthesia and clinical conditions. Here, we present the first large‐scale screening of the general population in which we (1) objectively identified grapheme‐colour synaesthetes and (2) elicited information from our participants about a range of clinical conditions. We compared the prevalence rates of these conditions in synaesthetes versus non‐synaesthetes to establish whether any conditions were found at a higher rate among synaesthetes. In our initial study, screening 3,742 people (95 synaesthetes and 3,647 controls), we found initially that grapheme‐colour was significantly comorbid with two conditions (anxiety disorder and obsessive compulsive disorder). In our second study, screening a new population of 120 synaesthetes and 166 non‐synaesthetes, we replicated our finding that grapheme‐colour synaesthesia is comorbid with anxiety disorder. At the same time, we also addressed a methodological concern that likely elevated rates of OCD in Study 1. We consider the aetiology of synaesthesia to determine whether there may be a shared genetic or neurological basis with anxiety disorder, and we question the status of synaesthesia within a mental health framework.     

Brain areas involved in synaesthesia: a review

Despite a recent upsurge of research, much remains unknown about the neurobiological mechanisms underlying synaesthesia. By integrating results obtained so far in Magnetic Resonance Imaging (MRI) studies, this contribution sheds light on the role of particular brain regions in synaesthetic experiences. First, in accordance with its sensory nature, it seems that the sensory brain areas corresponding to the type of synaesthetic experience are activated. Synaesthetic colour experiences can activate colour regions in occipito-temporal cortex, but this is not necessarily restricted to V4. Furthermore, sensory and motor brain regions have been obtained that extend beyond the particular type of synaesthesia studied. Second, differences in experimental setup, number and type of synaesthetes tested, and method to delineate regions of interest may help explain inconsistent results obtained in the BOLD-MRI (Blood Oxygen Level Dependent functional MRI) studies. Third, an overview of obtained results shows that a network of brain areas rather than a single brain region underlies synaesthesia. Six brain regions of overlapping results emerge, these regions are in sensory and motor regions as well as 'higher level' regions in parietal and frontal lobe. We propose that these regions are related to three different cognitive processes inherently part of synaesthesia; the sensory processes, the (attentional) 'binding' processes, and cognitive control processes. Finally, we discuss how these functional and structural brain properties might relate to the development of synaesthesia. In particular, we believe this relationship is better understood by separating the question what underlies the presence of synaesthesia ('trait') from what determines particular synaesthetic associations ('type').     

Redefining synaesthesia?

In a thought-provoking paper, Simner (2012) highlights and criticizes a number of assumptions concerning synaesthesia. She specifically takes issue with the following assumptions: (1) synaesthesia is strictly a sensory-perceptual phenomenon; (2) consistency of inducer-concurrent pairs is the gold standard for establishing the authenticity of an individual's synaesthesia; and (3) synaesthesia is not heterogeneous. In the wake of this critique, Simner advances a working definition of synaesthesia as a neurological hyper-association that aims to be more inclusive of its variants. We are very sympathetic to Simner's approach and believe that it raises important points that will advance our understanding of synaesthesia. Here we supplement, and sometimes challenge, some of these ideas.     

Is synaesthesia an X-linked dominant trait with lethality in males?

In previous research the inheritance patterns of synaesthesia (eg experiencing colours from graphemes) has been studied and it was concluded that synaesthesia is most likely to be the outcome of a single gene passed on the X chromosome in a dominant fashion. In addition, it has been reported that the female-male ratio of synaesthetes is as high as 6:1 and the families of synaesthetes contain more female than male members. This raises the possibility that the gene may be associated with lethality in males. In this study we replicate and extend previous research by investigating the female-male ratio and inheritance patterns in a large sample of synaesthetic families (N = 85). We were able to verify the authenticity of grapheme-colour associates in at least one proband from each family using internal consistency. As before, our results show a female-male bias and are broadly consistent with an X-linked dominant mode of inheritance. However, there was no evidence of male lethality (eg synaesthetes are just as likely to give birth to sons as to daughters). Moreover, our female-male ratio of synaesthetes within families was 2:1--considerably lower than previous estimates. We speculate that men may be more reluctant to disclose synaesthesia than women (indeed, our female-male ratio based on self-referral was 3.7: 1). Finally, we discuss how the genotype may give rise to the phenotype in terms of changes in synaptogenesis or plasticity extending into childhood, to be subsequently shaped by the environment.     

Synaesthesia,creativity and art: What is the link?

It has been suggested that individuals with synaesthesia may show heightened creativity as a result of being able to form meaningful associations between disparate stimuli (e.g. colour, sound). In this study, a large sample (N=82) of people with various kinds of synaesthesia were given two psychometric tests of creativity (Remote Associates Test, Alternate Uses Test) and were also asked about the amount of time engaged in creative arts (visual art, music). There was a significant tendency for synaesthetes to spend more time engaged in creative arts and this was, at least in part, dependent upon the type of synaesthesia experienced. For example, synaesthetes experiencing vision from music were far more likely to play an instrument than their other synaesthetic counterparts. There was no relationship between this tendency and the psychometric measures of creativity, but synaesthetes did outperform controls on one of the two psychometric measures (Remote Associates). We conclude that the tendency for synaesthetes to be more engaged in art is likely to have a different mechanism to psychometric measures of creativity, and that there is no direct link between them. Although synaesthetes may well perform better on some measures of creativity, we suggest that synaesthetes have better bottom‐up access to certain associations, but are not necessarily better able to use them flexibly (in divergent thinking).     

Mental imagery and synaesthesia: Is synaesthesia from internally-generated stimuli possible?

Previous studies provide empirical support for the reported colour experience in grapheme-colour synaesthesia by measuring the synaesthetic experience from an externally presented grapheme. The current study explored the synaesthetic experience resulting from a visual mental image of a grapheme. Grapheme-colour synaesthetes (N=6) and matched controls (N=10 per synaesthete) completed a visual mental imagery task that involved visualising a letter and making a size-based decision about it. The background colour that the grapheme was visualised against was manipulated so that it was congruent or incongruent with the synaesthetic colour for the grapheme being visualised. Compared to matched controls, an effect of colour condition was found for four of the six synaesthetes, although importantly the direction of the effect varied between synaesthetes. In addition, a significant effect of group was found, as the synaesthetes were all faster than the matched controls at the imagery task, regardless of background colour. We conclude that there is some support for subjective reports of imagery-induced synaesthesia, but there are important individual differences. These findings are discussed in relation to both the visual imagery and synaesthesia literature.     

What is the relationship between synaesthesia and visuo-spatial number forms?

This study compares the tendency for numerals to elicit spontaneous perceptions of colour or taste (synaesthesia) with the tendency to visualise numbers as occupying particular visuo-spatial configurations (number forms). The prevalence of number forms was found to be significantly higher in synaesthetes experiencing colour compared both to synaesthetes experiencing taste and to control participants lacking any synaesthetic experience. This suggests that the presence of synaesthetic colour sensations enhances the tendency to explicitly represent numbers in a visuo-spatial format although the two symptoms may nevertheless be logically independent (i.e. it is possible to have number forms without colour, and coloured numbers without forms). Number forms are equally common in men and women, unlike previous reports of synaesthesia that have suggested a strong female bias. Individuals who possess a number form are also likely to possess visuo-spatial forms for other ordinal sequences (e.g. days, months, letters) which suggests that it is the ordinal nature of numbers rather than numerical quantity that gives rise to this particular mode of representation. Finally, we also describe some consequences of number forms for performance in a number comparison task.     

A systematic, large-scale study of synaesthesia: implications for the role of early experience in lexical-colour associations

For individuals with synaesthesia, stimuli in one sensory modality elicit anomalous experiences in another modality. For example, the sound of a particular piano note may be 'seen' as a unique colour, or the taste of a familiar food may be 'felt' as a distinct bodily sensation. We report a study of 192 adult synaesthetes, in which we administered a structured questionnaire to determine the relative frequency and characteristics of different types of synaesthetic experience. Our data suggest the prevalence of synaesthesia in the adult population is approximately 1 in 1150 females and 1 in 7150 males. The incidence of left-handedness in our sample was within the normal range, contrary to previous claims. We did, however, find that synaesthetes are more likely to be involved in artistic pursuits, consistent with anecdotal reports. We also examined responses from a subset of 150 synaesthetes for whom letters, digits and words induce colour experiences ('lexical-colour' synaesthesia). There was a striking consistency in the colours induced by certain letters and digits in these individuals. For example, 'R' elicited red for 36% of the sample, 'Y' elicited yellow for 45%, and 'D' elicited brown for 47%. Similar trends were apparent for a group of non-synaesthetic controls who were asked to associate colours with letters and digits. Based on these findings, we suggest that the development of lexical-colour synaesthesia in many cases incorporates early learning experiences common to all individuals. Moreover, many of our synaesthetes experienced colours only for days of the week, letters or digits, suggesting that inducers that are part of a conventional sequence (e.g. Monday, Tuesday, Wednesday...; A, B, C...; 1, 2, 3...) may be particularly important in the development of synaesthetic inducer-colour pairs. We speculate that the learning of such sequences during an early critical period determines the particular pattern of lexical-colour links, and that this pattern then generalises to other words.     

Cross-modal personality attributions in synaesthetes and non-synaesthetes

Those variants of synaesthesia that trigger colour are well studied, although comparatively less is known about variants that involve cognitive constructs such as personality types. Here we investigate sequence-personality synaesthesia (also known as ordinal linguistic personification, OLP) in which sequenced units (e.g., letters) become associated to personalities or genders. We present the first group study of this variant, showing similarities and differences between synaesthetes and non-synaesthetes. In Experiment 1, we show that synaesthetes differ from the general population in the phenomenology of their reports, the depth of their personality associations, and the consistency of those associations over time. In Experiment 2, we show that synaesthetes are similar to the general population in the underlying rules that link their personalities to letters. Specifically, we show that these mappings are not random, but are based on a shared rule system linking linguistic qualities of letters with quantitative dimensions of personality (based on Goldberg's Big Five personality traits; Goldberg, 1990, 1992). Synaesthetes tend to associate high-frequency letters with high agreeable and low neurotic personalities, and non-synaesthetes share these tendencies at an implicit level. Together, these data show that synaesthetes differ from the general population in phenomenological ways, but that their underlying mechanisms may be common to all people.     

The prevalence of synaesthesia depends on early language learning

According to one theory, synaesthesia develops, or is preserved, because it helps children learn. If so, it should be more common among adults who faced greater childhood learning challenges. In the largest survey of synaesthesia to date, the incidence of synaesthesia was compared among native speakers of languages with transparent (easier) and opaque (more difficult) orthographies. Contrary to our prediction, native speakers of Czech (transparent) were more likely to be synaesthetes than native speakers of English (opaque). However, exploratory analyses suggested that this was because more Czechs learned non-native second languages, which was strongly associated with synaesthesia, consistent with the learning hypothesis. Furthermore, the incidence of synaesthesia among speakers of opaque languages was double that among speakers of transparent languages other than Czech, also consistent with the learning hypothesis. These findings contribute to an emerging understanding of synaesthetic development as a complex and lengthy process with multiple causal influences.     

Lexical-gustatory synaesthesia: linguistic and conceptual factors

This study documents an unusual case of developmental synaesthesia, in which speech sounds induce an involuntary sensation of taste that is subjectively located in the mouth. JIW shows a highly structured, non-random relationship between particular combinations of phonemes (rather than graphemes) and the resultant taste, and this is influenced by a number of fine-grained phonemic properties (e.g. allophony, phoneme ordering). The synaesthesia is not found for environmental sounds. The synaesthesia, in its current form, is likely to have originated during vocabulary acquisition, since it is guided by learned linguistic and conceptual knowledge. The phonemes that trigger a given taste tend to also appear in the name of the corresponding foodstuff (e.g. /I/, /n/ and /s/ can trigger a taste of mince /mIns/) and there is often a semantic association between the triggering word and taste (e.g. the word blue tastes "inky"). The results suggest that synaesthesia does not simply reflect innate connections from one perceptual system to another, but that it can be mediated and/or influenced by a symbolic/conceptual level of representation.     

Psychophysiological evidence for the genuineness of swimming-style colour synaesthesia

Recently, swimming-style colour synaesthesia was introduced as a new form of synaesthesia. A synaesthetic Stroop test was used to establish its genuineness. Since Stroop interference can occur for any type of overlearned association, in the present study we used a modified Stroop test and psychophysiological synaesthetic conditioning to further establish the genuineness of this form of synaesthesia. We compared the performance of a swimming-style colour synaesthete and a control who was trained on swimming-style colour associations. Our results showed that behavioural aspects of swimming-style colour synaesthesia can be mimicked in a trained control. Importantly, however, our results showed a psychophysiological conditioning effect for the synaesthete only. We discuss the theoretical relevance of swimming-style colour synaesthesia according to different models of synaesthesia. We conclude that swimming-style colour synaesthesia is a genuine form of synaesthesia, can be mimicked behaviourally in non-synaesthetes, and is best explained by a re-entrant feedback model.