Training synaesthesia

Synaesthetic inducers such as graphemes are typically cultural artifacts. Thus, a learning component seems evident in synaesthesia (Simner et al, 2009 Brain 132 57-64). Normally, synaesthetes report to have their experiences since they can remember. Nevertheless, a recent training study suggests that synaesthesia can be mimicked in non-synaesthetes. To date, the role of learning during the development of synaesthesia is still debated. It is not clear whether synaesthesia can be learned or trained at all. To address this question, we compared a non-adaptive and an adaptive training for their effectiveness. We assessed their impact on two types of priming tasks, before and after the training. We found stronger priming in the adaptive training group suggesting that adaptive training is more efficient to mimic synaesthesia.     

Human spatial navigation via a visuo-tactile sensory substitution system

Spatial navigation within a real 3-D maze was investigated to study space perception on the sole basis of tactile information transmitted by means of a 'tactile vision substitution system' (TVSS) allowing the conversion of optical images-collected by a micro camera-into 'tactile images' via a matrix in contact with the skin. The development of such a device is based on concepts of cerebral and functional plasticity, enabling subjective reproduction of visual images from tactile data processing. Blindfolded sighted subjects had to remotely control the movements of a robot on which the TVSS camera was mounted. Once familiarised with the cues in the maze, the subjects were given two exploration sessions. Performance was analysed according to an objective point of view (exploration time, discrimination capacity), as well as a subjective one (speech). The task was successfully carried out from the very first session. As the subjects took a different path during each navigation, a gradual improvement in performance (discrimination and exploration time) was noted, generating a phenomenon of learning. Moreover, subjective analysis revealed an evolution of the spatialisation process towards distal attribution. Finally, some emotional expressions seemed to reflect the genesis of 'qualia' (emotional qualities of stimulation).     

Handedness and calendar orientations in time-space synaesthesia

In one common variant of time-space synaesthesia, individuals report the consistent experience of months bound to a spatial arrangement, commonly described as a circle extending outside of the body. Whereas the layout of these calendars has previously been thought to be relatively random and to differ greatly between synaesthetes, Study 1 provides the first evidence suggesting one critical aspect of these calendars is mediated by handedness: clockwise versus counter-clockwise orientation. A study of 34 time-space synaesthetes revealed a strong association between handedness and the orientation of circular calendars. That is, left-handed time-space synaesthetes tended to report counter-clockwise arrangements and right-handed synaesthetes clockwise. Study 2 tested whether a similar bias was present in non-synaesthetes whose task was to memorize and recall the spatial configuration of a clockwise and counter-clockwise calendar. Non-synaesthetes' relative performance on these two sorts of calendars was significantly correlated with their handedness scores in a pattern similar to synaesthetes. Specifically, left-handed controls performed better on counter-clockwise calendars compared to clockwise, and right-handed controls on clockwise over counter-clockwise. We suggest that the implicit biases seen in controls are mediated by similar mechanisms as in synaesthesia, highlighting the graded nature of synaesthetic associations.     

Lexical-gustatory synaesthesia: linguistic and conceptual factors

This study documents an unusual case of developmental synaesthesia, in which speech sounds induce an involuntary sensation of taste that is subjectively located in the mouth. JIW shows a highly structured, non-random relationship between particular combinations of phonemes (rather than graphemes) and the resultant taste, and this is influenced by a number of fine-grained phonemic properties (e.g. allophony, phoneme ordering). The synaesthesia is not found for environmental sounds. The synaesthesia, in its current form, is likely to have originated during vocabulary acquisition, since it is guided by learned linguistic and conceptual knowledge. The phonemes that trigger a given taste tend to also appear in the name of the corresponding foodstuff (e.g. /I/, /n/ and /s/ can trigger a taste of mince /mIns/) and there is often a semantic association between the triggering word and taste (e.g. the word blue tastes "inky"). The results suggest that synaesthesia does not simply reflect innate connections from one perceptual system to another, but that it can be mediated and/or influenced by a symbolic/conceptual level of representation.     

Frequency correlates in grapheme-color synaesthesia

Individuals with synaesthesia experience certain stimuli in more than one sensory modality. Most common is the linkage of letters and digits (graphemes) to colors. Whereas synaesthesia might be partly genetically determined, the linkages to specific colors are assumed to be learned. We present a systematic statistical analysis of synaesthetic color perception based on subjects' reproduction of individual colors for each grapheme, instead of simple verbal categorizations. The statistical analysis revealed that the color perceptions, measured with the HSL (hue, saturation, and luminance) scale, varied systematically among the different digits and letters. The frequencies of the digits and letters (in the German language) partly explained these systematic variations. However, digit frequency was more strongly related to color perception in the synaesthetes than was letter frequency. The results for digit and letter frequency indicate that experience with graphemes may shape synaesthetic color perception.     

Perception with compensatory devices: from sensory substitution to sensorimotor extension

Sensory substitution devices provide through an unusual sensory modality (the substituting modality, e.g., audition) access to features of the world that are normally accessed through another sensory modality (the substituted modality, e.g., vision). In this article, we address the question of which sensory modality the acquired perception belongs to. We have recourse to the four traditional criteria that have been used to define sensory modalities: sensory organ, stimuli, properties, and qualitative experience ( Grice, 1962 ), to which we have added the criteria of behavioral equivalence ( Morgan, 1977 ), dedication ( Keeley, 2002 ), and sensorimotor equivalence ( O'Regan & Noë, 2001 ). We discuss which of them are fulfilled by perception through sensory substitution devices and whether this favors the view that perception belongs to the substituting or to the substituted modality. Though the application of a number of criteria might be taken to point to the conclusion that perception with a sensory substitution device belongs to the substituted modality, we argue that the evidence leads to an alternative view on sensory substitution. According to this view, the experience after sensory substitution is a transformation, extension, or augmentation of our perceptual capacities, rather than being something equivalent or reducible to an already existing sensory modality. We develop this view by comparing sensory substitution devices to other "mind-enhancing tools" such as pen and paper, sketchpads, or calculators. An analysis of sensory substitution in terms of mind-enhancing tools unveils it as a thoroughly transforming perceptual experience and as giving rise to a novel form of perceptual interaction with the environment.     

Visual experiences in the blind induced by an auditory sensory substitution device

In this report, the phenomenology of two blind users of a sensory substitution device – “The vOICe” – that converts visual images to auditory signals is described. The users both report detailed visual phenomenology that developed within months of immersive use and has continued to evolve over a period of years. This visual phenomenology, although triggered through use of The vOICe, is likely to depend not only on online visualization of the auditory signal but also on the users’ previous (albeit distant) experience of veridical vision (e.g. knowledge of shapes and visual perspective). Once established, the sensory substitution mapping between the auditory and visual domains is not confined to when the device is worn and, thus, may constitute an example of acquired synaesthesia.     

Brain areas involved in synaesthesia: a review

Despite a recent upsurge of research, much remains unknown about the neurobiological mechanisms underlying synaesthesia. By integrating results obtained so far in Magnetic Resonance Imaging (MRI) studies, this contribution sheds light on the role of particular brain regions in synaesthetic experiences. First, in accordance with its sensory nature, it seems that the sensory brain areas corresponding to the type of synaesthetic experience are activated. Synaesthetic colour experiences can activate colour regions in occipito-temporal cortex, but this is not necessarily restricted to V4. Furthermore, sensory and motor brain regions have been obtained that extend beyond the particular type of synaesthesia studied. Second, differences in experimental setup, number and type of synaesthetes tested, and method to delineate regions of interest may help explain inconsistent results obtained in the BOLD-MRI (Blood Oxygen Level Dependent functional MRI) studies. Third, an overview of obtained results shows that a network of brain areas rather than a single brain region underlies synaesthesia. Six brain regions of overlapping results emerge, these regions are in sensory and motor regions as well as 'higher level' regions in parietal and frontal lobe. We propose that these regions are related to three different cognitive processes inherently part of synaesthesia; the sensory processes, the (attentional) 'binding' processes, and cognitive control processes. Finally, we discuss how these functional and structural brain properties might relate to the development of synaesthesia. In particular, we believe this relationship is better understood by separating the question what underlies the presence of synaesthesia ('trait') from what determines particular synaesthetic associations ('type').     

Ovals of time: time-space associations in synaesthesia

We examine a condition in which units of time, such as months of the year, are associated with specific locations in space. For individuals with this time-space synaesthesia, contiguous time units such as months are spatially linked forming idiosyncratically shaped patterns such as ovals, oblongs or circles. For some individuals, each time unit appears in a highly specific colour. For instance, one of the synaesthetes we studied experienced December as a red area located at arms length to the left of their body. For the same individual May was a blue area located roughly at arms length to the right of their body. We studied four synaesthetes who report spatial associations for the months of the year. We found that the time-space associations experienced by these individuals were consistent across test-retest. In addition, month names directed visual attention to particular locations in space. For some synaesthetes, this directing of spatial attention was quite rapid-in accord with their reports that month names involuntarily bring to mind spatial locations.     

A self-organizing learning account of number-form synaesthesia

Some people automatically and involuntarily “see” mental images of numbers in spatial arrays when they think of numbers. This phenomenon, called number forms, shares three key characteristics with the other types of synaesthesia, within-individual consistency, between-individual variety, and mixture of regularity and randomness. A theoretical framework called SOLA (self-organizing learning account of number forms) is proposed, which explains the generation process of number forms and the origin of those three characteristics. The simulations replicated the qualitative properties of the shapes of number forms, the property that numbers are aligned in order of size, that discontinuity usually occurs at the point of carry, and that continuous lines tend to have many bends.     

Colours sometimes count: awareness and bidirectionality in grapheme-colour synaesthesia

Three experiments were conducted with 10 grapheme-colour synaesthetes and 10 matched controls to investigate (a) whether awareness of the inducer grapheme is necessary for synaesthetic colour induction and (b) whether grapheme-colour synaesthesia may be bidirectional in the sense that not only do graphemes induce colours, but that colours influence the processing of graphemes. Using attentional blink and Stroop paradigms with digit targets, we found that some synaesthetes did report “seeing” synaesthetic colours even when they were not able to report the inducing digit. Moreover, congruency effects (effects of matching the colour of digit presentation with the synaesthetic colour associated with that digit) suggested that grapheme-colour synaesthesia can be bidirectional, at least for some synaesthetes.     

Familial patterns and the origins of individual differences in synaesthesia

The term synaesthesia has been applied to a range of different sensory-perceptual and cognitive experiences, yet how these experiences are related to each other is not well understood. Not only are there disparate types of synaesthesia, but even within types there are vast individual differences in the way that stimuli induce synaesthesia and in the subjective synaesthetic experience. An investigation of the inheritance patterns of different types of synaesthesia is likely to elucidate whether a single underlying mechanism can explain all types. This study is the first to systematically survey all types of synaesthesia within a familial framework. We recruited 53 synaesthetes and 42% of these probands reported a first-degree relative with synaesthesia. We then directly contacted as many first-degree relatives as possible and collected complete data on synaesthetic status for all family members for 17 families. We found that different types of synaesthesia can occur within the same family and that the qualitative nature of the experience can differ between family members. Our findings strongly indicate that various types of synaesthesia are fundamentally related at the genetic level, but that the explicit associations and the individual differences between synaesthetes are influenced by other factors. Synaesthesia thus provides a good model to explore the interplay of all these factors in the development of cognitive traits in general.     

Tasty non-words and neighbours: The cognitive roots of lexical-gustatory synaesthesia

For lexical-gustatory synaesthetes, words trigger automatic, associated food sensations (e.g., for JB, the word slope tastes of over-ripe melon). Our study tests two claims about this unusual condition: that synaesthetic tastes are associated with abstract levels of word representation (concepts/lemmas), and that the first tastes to crystallise in early development are those triggered by food-names (e.g., apple tastes of apple; Simner & Ward, 2006). This concept/lemma-based proposal is difficult to immediately reconcile with the finding that non-words may also generate tastes (Ward, Simner, & Auyeung, 2005), since non-words have no concept/lemma representations. We manipulated the characteristics of non-words to provide three types of evidence that non-word tastes in fact stem from real word neighbours: Non-words with neighbours (e.g., keach) are more likely to generate tastes than those with no neighbours (e.g., vilps); pseudo-homophone non-words that are orthographically close to real words (e.g., peeple) are more likely to generate tastes than those that are more distant (baybee); and finally, the tastes of non-words are less consistent, and less intense, than those of real words. Additionally, we test the hypothesis that synaesthetic tastes develop initially from food-names by showing that non-words are more intensely flavoured if they are homophonic with food-names (e.g., toffie) versus non-foods (e.g., peeple). From this we conclude that synaesthetic tastes develop from food-names, and that tasty non-words do not challenge a concept/lemma-based account of lexical-gustatory synaesthesia.     

Cost of mental set reconfiguration between digits and their photisms in synaesthesia

In this study we present an experiment investigating the reconfiguration process elicited by the task switching paradigm in synaesthesia. We study the time course of the operations involved in the activation of photisms. In the experimental Group, four digit-color synaesthetes alternated between an odd-even task and a color task (to indicate the photism elicited by each digit). In both tasks, the target stimuli were numbers between 1 and 9 written in white. One of the control groups ran the same tasks but this time with colored numbers (Naive Control Group). The results of these studies showed the expected pattern for the control group in the case of regular shift: a significant task switch cost with an abrupt offset and a cost reduction in long RSI. However for the experimental group, we found switch cost asymmetry in the short RSI and non-significant cost in the long RSI. A second control group performed exactly the same tasks as the experimental group (with white numbers as targets and a second imaginary color task) -Trained Control Group-. We found no cost for this second control group. This means that the cost of mental set reconfiguration between numbers (inducers) and their photisms (concurrent sensations) occurs, that there is a specific cost asymmetry (from photisms to inducers) and that this cost cannot be explained by associative learning. The results are discussed in terms of exogenous and endogenous components of mental set reconfiguration.     

Seeing double: the role of meaning in alphanumeric-colour synaesthesia

When PD, an alphanumeric-colour synaesthete, is shown black digits or letters, each grapheme elicits a highly specific colour, called a "photism" (e.g., a 2 induces green, a Z induces brown). Previous experiments showed that photisms interfere with video-colour naming when the photism colour and the video colour are incongruent. Here we used coloured ambiguous graphemes that could be interpreted as either digits or letters depending on context (e.g., an ambiguous grapheme was interpreted as a 2 if presented within a block of digit trials, but as a Z if presented within a block of letter trials). Ambiguous graphemes were presented in video colours that were either congruent or incongruent with PD's photisms. Crucially, what was a congruent trial in one context was an incongruent trial in the other context. PD's pattern of response times indicated that identical graphemes could induce differently coloured photisms depending on their interpretation. This suggests that the meaning of graphemes ultimately determines their synaesthetic colour.     

An independent component analysis of fMRI data of grapheme-colour synaesthesia

The independent component analysis (ICA) method was applied to fMRI data from two synaesthetes and their matched controls while they performed the coloured-word Stroop task and the single-letter (synaesthetic) Stroop task. ICA identified an 'attention' network, a 'perceptual' network as well as a 'conflict monitoring' network. Increased activity was observed in right V4 during the single-letter Stroop task for synaesthetes only. The finding confirms that the same neural substrate that is known to support the experience of physical colours also supports the experience of synaesthetic colours.     

Psychophysiological evidence for the genuineness of swimming-style colour synaesthesia

Recently, swimming-style colour synaesthesia was introduced as a new form of synaesthesia. A synaesthetic Stroop test was used to establish its genuineness. Since Stroop interference can occur for any type of overlearned association, in the present study we used a modified Stroop test and psychophysiological synaesthetic conditioning to further establish the genuineness of this form of synaesthesia. We compared the performance of a swimming-style colour synaesthete and a control who was trained on swimming-style colour associations. Our results showed that behavioural aspects of swimming-style colour synaesthesia can be mimicked in a trained control. Importantly, however, our results showed a psychophysiological conditioning effect for the synaesthete only. We discuss the theoretical relevance of swimming-style colour synaesthesia according to different models of synaesthesia. We conclude that swimming-style colour synaesthesia is a genuine form of synaesthesia, can be mimicked behaviourally in non-synaesthetes, and is best explained by a re-entrant feedback model.