Stimulus Parameters Underlying Sound-Symbolic Mapping of Auditory Pseudowords to Visual Shapes

Sound symbolism refers to non-arbitrary mappings between the sounds of words and their meanings and is often studied by pairing auditory pseudowords such as “maluma” and “takete” with rounded and pointed visual shapes, respectively. However, it is unclear what auditory properties of pseudowords contribute to their perception as rounded or pointed. Here, we compared perceptual ratings of the roundedness/pointedness of large sets of pseudowords and shapes to their acoustic and visual properties using a novel application of representational similarity analysis (RSA). Representational dissimilarity matrices (RDMs) of the auditory and visual ratings of roundedness/pointedness were significantly correlated crossmodally. The auditory perceptual RDM correlated significantly with RDMs of spectral tilt, the temporal fast Fourier transform (FFT), and the speech envelope. Conventional correlational analyses showed that ratings of pseudowords transitioned from rounded to pointed as vocal roughness (as measured by the harmonics-to-noise ratio, pulse number, fraction of unvoiced frames, mean autocorrelation, shimmer, and jitter) increased. The visual perceptual RDM correlated significantly with RDMs of global indices of visual shape (the simple matching coefficient, image silhouette, image outlines, and Jaccard distance). Crossmodally, the RDMs of the auditory spectral parameters correlated weakly but significantly with those of the global indices of visual shape. Our work establishes the utility of RSA for analysis of large stimulus sets and offers novel insights into the stimulus parameters underlying sound symbolism, showing that sound-to-shape mapping is driven by acoustic properties of pseudowords and suggesting audiovisual cross-modal correspondence as a basis for language users' sensitivity to this type of sound symbolism.     

Implicit phonetic symbolism in voicing of consonants and visual lightness using Garner's speeded classification task

The present study examines implicit phonetic symbolism which posits that arbitrary linguistic sound is related to certain aspects of characteristics of other modalities, such as color, touch, or emotion. In consonant discrimination and lightness discrimination using Garner's speeded classification paradigm, spoken sounds (voiced/voiceless consonants) and lightness of visual stimuli (black/white squares) were systematically varied to assess cross-modal interactions. Congruent audio-visual pairs (voiced consonants and black, and between voiceless consonants and white) facilitated consonant discrimination. In lightness discrimination, no congruent facilitation or congruence effect was observed. These results indicated that cross-modal interactions in implicit phonetic symbolism can be found in correlations between linguistic spoken sounds and visual lightness.     

语音象征的产生机制：基于敏感期的先天后天作用模型

Bouba-Kiki效应(简称BK效应)指语音和形状特征之间的映射关系。针对BK效应的产生机制, 先天论和后天论之间争论激烈。先天论的观点认为人们对语音象征的敏感性是出生时便存在的一种语言机制。而后天论的观点则强调语音象征是语言经验的产物。上述理论均获得大量研究证据的支持, 且均无法完全否定对方。这表明, 上述理论可能均未完整揭示语音象征的产生机制。鉴于此, 针对BK效应的产生机制, 梳理先天论与后天论的支持证据, 并率先提出语言相关的BK效应敏感期假设。同时, 梳理了支持BK效应敏感期的初步研究证据及可能的影响因素。进而, 以语言相关的BK效应敏感期假设为基础, 提出语音象征产生的先天后天相互作用模型, 以整合以往研究中的矛盾。最后, 展望了语音象征的未来研究进展和方向。     

Toward an exploration of feeling of strangeness in schizophrenia: perspectives on acousmatic and everyday listening

The aim of this study was to investigate abnormal perceptual experiences in schizophrenia, in particular the feeling of strangeness, which is commonly found in patients' self-reports. The experimental design included auditory complex stimuli within 2 theoretical frameworks based on "sensory gating deficit" and "aberrant salience," inspired from conventional perceptual scales. A specific sound corpus was designed with environmental (meaningful) and abstract (meaningless) sounds. The authors compared sound evaluations on 3 perceptual dimensions (bizarre, familiar, and invasive) and 2 emotional dimensions (frightening and reassuring) between 20 patients with schizophrenia (SCZ) and 20 control participants (CTL). The perceptual judgment was rated on independent linear scales for each sound. In addition, the conditioning-testing P50 paradigm was conducted on 10 SCZ and 10 CTL. Both behavioral and electrophysiological data confirmed the authors' expectations according to the 2 previous theoretical frameworks and showed that abnormal perceptual experiences in SCZ consisted of perceiving meaningful sounds in a distorted manner and as flooding/inundating but also in perceiving meaningless sounds as things that become meaningful by assigning them some significance. In addition, the use of independent scales to each perceptual dimension highlighted an unexpected ambivalence on familiarity and bizarreness in SCZ compatible with the explanation of semantic process impairment. The authors further suggested that this ambivalence might be due to a conflicting coactivation of 2 types of listening, that is, every day and musical (or acousmatic) listening.     

Sound symbolism facilitates early verb learning

Some words are sound-symbolic in that they involve a non-arbitrary relationship between sound and meaning. Here, we report that 25-month-old children are sensitive to cross-linguistically valid sound-symbolic matches in the domain of action and that this sound symbolism facilitates verb learning in young children. We constructed a set of novel sound-symbolic verbs whose sounds were judged to match certain actions better than others, as confirmed by adult Japanese- as well as English speakers, and by 2- and 3-year-old Japanese-speaking children. These sound-symbolic verbs, together with other novel non-sound-symbolic verbs, were used in a verb learning task with 3-year-old Japanese children. In line with the previous literature, 3-year-olds could not generalize the meaning of novel non-sound-symbolic verbs on the basis of the sameness of action. However, 3-year-olds could correctly generalize the meaning of novel sound-symbolic verbs. These results suggest that iconic scaffolding by means of sound symbolism plays an important role in early verb learning.     

Consistency of magnitude estimations with conceptual data dimensions used for sonification

In two experiments listeners assessed how sounds represented data, as might be used in sonification. In Experiment 1, 209 undergraduates used magnitude estimation to define the relationship between three sound attributes (frequency, tempo, modulation index) and 10 data dimensions (size, temperature, pressure, velocity, number of dollars, urgency, proximity, attractiveness, danger, mass). Polarities and slopes (i.e. power function exponents) are reported and compared to predictions from the literature. In Experiment 2, 226 new participants demonstrated polarities and slopes are stable across a direct replication. Results show that listener expectations depend on both sound and data dimensions in use. While there are some unanimous expectations across listeners, there are also differences due to different mental models formed by participants, which may relate to listening experience. Copyright © 2006 John Wiley & Sons, Ltd.     

Understanding consumers' brand‐induced food taste perception: A comparison of ‘brand familiarity’ – and ‘consumer value – brand symbolism (in)congruity’ – accounts

Past research has shown that familiar brands can boost consumers' food taste experiences. On the other hand, more recent evidence suggests that the (in)congruity between consumer values and brand symbolism can affect the food taste perception. This study is the first one to integrate these two accounts into one single conceptual framework and to empirically evaluate their relative roles in explaining consumers' brand‐induced taste perception. Two experiments involving taste trials (blind vs brand‐cued sensory evaluation) were conducted. The first experiment analysed the brand familiarity effect, whereas the second experiment addressed also the taste perception of yogurts with differing brand symbolism amongst food consumers with distinct value orientations to find support for the (in)congruity effects. This research implies that congruity is not responsible for enhancing consumers' taste perception beyond the level that is produced by the brand familiarity. In contrast, the incongruity effect appears capable of neutralising the brand familiarity effect. Therefore, these two explanations may operate independently. More generally, this study speaks for the importance of incorporating consumer value – brand symbolism incongruity mechanism into food consumption studies; even owners' of strong food brands cannot trust the ability of their brands to boost a consumer's taste experience if there is no correspondence between his or her central values and brand symbolism. Thus, an objectively better taste is not necessarily decisive; satisfactory sensory quality can suffice if it is coupled with imaginative and daring brand marketing that delivers unique emotional and functional benefits for well‐defined food consumer target segments. Copyright © 2011 John Wiley & Sons, Ltd.     

An Investigation of Sound-Symbolism in the Context of Tactile Feeling

Sound symbolism suggests a non-arbitrary relationship between speech sounds and the concepts to which those sounds refer (Hinton, Nichols, &; Ohala, 2006 Hinton, L., Nichols, J., &; Ohala, J. J. (2006). Sound symbolism. Cambridge, UK: Cambridge University Press. [Google Scholar]). Supporting evidence comes primarily from studies investigating how speech sounds relate to semantically compatible visual concepts. The present study therefore attempted to examine sound symbolism in the context of tactile perception. Contrary to the propositions of sound symbolism, participants in Experiment 1 did not consistently assign names with plosive consonant to objects with curved frames. Experiment 2, however, found that names with fricative consonants were more likely to be applied to materials with rough surfaces. The results suggested the existence of a direct relationship between speech sounds and their referent concepts that could be crucial in revealing the phenomenon of sound symbolism. A future study was also proposed to study the contributions of mouth shape and airflow to associations between speech sounds and tactile feelings. (161 words)     

Sound symbolic naming of novel objects is a graded function

Although linguistic traditions of the last century assumed that there is no link between sound and meaning (i.e., arbitrariness), recent research has established a nonarbitrary relation between sound and meaning (i.e., sound symbolism). For example, some sounds (e.g., /u/ as in took) suggest bigness whereas others (e.g., /i/ as in tiny) suggest smallness. We tested whether sound symbolism only marks contrasts (e.g., small versus big things) or whether it marks object properties in a graded manner (e.g., small, medium, and large things). In two experiments, participants viewed novel objects (i.e., greebles) of varying size and chose the most appropriate name for each object from a list of visually or auditorily presented nonwords that varied incrementally in the number of “large” and “small” phonemes. For instance, “wodolo” contains all large-sounding phonemes, whereas “kitete” contains all small-sounding phonemes. Participants' choices revealed a graded relationship between sound and size: The size of the object linearly predicted the number of large-sounding phonemes in its preferred name. That is, small, medium, and large objects elicited names with increasing numbers of large-sounding phonemes. The results are discussed in relation to cross-modal processing, gesture, and vocal pitch.     

The symbolic implications of vowels and of their orthographic representations in two natural languages

Ninety-two Hebrew-speaking subjects judged the magnitude, brightness, and hardness symbolism of orthographic characters designating five vowel phonemes in Hindi and in Japanese. For both languages and all three symbolic dimensions, the figural symbolism of the orthographic characters was found to replicate very closely the sound symbolism of their phonemic referents. The ranking of the five vowel characters in order of increasing magnitude and decreasing brightness and hardness was as follows:i, e, a, u, o. The results were interpreted to suggest that sound patterns and visual patterns tend to carry cross-culturally consistent connotations, and that the symbolic implications of sounds have been embodied in the pattern of orthographic characters in natural languages.This study was supported by a grant from the Human Development Center, The Hebrew University.     

The additivity of loudness across critical bands: A conjoint measurement approach

Five subjects were required in each trial to compare directly two sounds and to indicate which sound was louder. Each of the 64 sounds employed consisted of a combination of one of eight intensity levels of a 2-kHz tone and one of eight intensities of a 5-kHz tone. If, as Fletcher and Munson (1933) argued, loudness is additive for tone combinations in which the frequencies are widely separated, then subjects’ judgments should reflect the summed loudnesses of the 2- and 5-kHz tones in a two-tone combination. Judgments of individual subjects were shown to satisfy the conditions for an additive structure, and individual loudness scales were constructed. These loudness scales varied from subject to subject. Since this paired comparison procedure minimized response biases, the results suggest substantial individual differences in the sensory representation of sound intensity. The relations among sensory scales derived from other structured sensory judgments, such as binaural loudness, are discussed.     

Phonation Types Matter in Sound Symbolism

Sound symbolism is a non-arbitrary correspondence between sound and meaning. The majority of studies on sound symbolism have focused on consonants and vowels, and the sound-symbolic properties of suprasegmentals, particularly phonation types, have been largely neglected. This study examines the size and shape symbolism of four phonation types: modal and creaky voices, falsetto, and whisper. Japanese speakers heard 12 novel words (e.g., /íbi/, /ápa/) pronounced with the four types of phonation and rated the size and roundedness/pointedness each of the 48 stimuli seemed to represent on seven-point scales. The results showed that phonation types as well as consonantal and vocalic features influenced the ratings. Creaky voice was associated with larger and more pointed images than modal voice, which was in turn associated with larger and more pointed images than whisper. Falsetto was also associated with roundedness but not with smallness. These results shed new light on the acoustic approaches to sound symbolism and suggest the significance of phonation types and other suprasegmental features in the phenomenon.     

Poor hand-pointing to sounds in right brain-damaged patients: Not just a problem of spatial-hearing

We asked 22 right brain-damaged (RBD) patients and 11 elderly healthy controls to perform hand-pointing movements to free-field unseen sounds, while modulating two non-auditory variables: the initial position of the responding hand (left, centre or right) and the presence or absence of task-irrelevant ambient vision. RBD patients suffering from visual neglect, unlike RBD patients without neglect and healthy controls, showed a systematic rightward error in sound localisation, which was modulated by the non-auditory variables. Localisation errors were exacerbated by initial hand-position to the right of the body-midline, and reduced by the leftwards initial hand-position. Moreover, for the visual neglect patients, mere presence of ambient vision worsened localisation errors. These results demonstrate that although hand-pointing to sounds has often been considered a straightforward approach to investigate sound-localisation abilities in brain-damaged patients, in some patients it may actually reveal localisation deficits that reflect a combination of impaired spatial-hearing and spatial biases from other sensory modalities (i.e., vision and proprioception).     

The role of long-term familiarity and attentional maintenance in short-term memory for timbre

We study short-term recognition of timbre using familiar recorded tones from acoustic instruments and unfamiliar transformed tones that do not readily evoke sound-source categories. Participants indicated whether the timbre of a probe sound matched with one of three previously presented sounds (item recognition). In Exp. 1, musicians better recognised familiar acoustic compared to unfamiliar synthetic sounds, and this advantage was particularly large in the medial serial position. There was a strong correlation between correct rejection rate and the mean perceptual dissimilarity of the probe to the tones from the sequence. Exp. 2 compared musicians' and non-musicians' performance with concurrent articulatory suppression, visual interference, and with a silent control condition. Both suppression tasks disrupted performance by a similar margin, regardless of musical training of participants or type of sounds. Our results suggest that familiarity with sound source categories and attention play important roles in short-term memory for timbre, which rules out accounts solely based on sensory persistence.     

Poor hand-pointing to sounds in right brain-damaged patients: not just a problem of spatial-hearing

We asked 22 right brain-damaged (RBD) patients and 11 elderly healthy controls to perform hand-pointing movements to free-field unseen sounds, while modulating two non-auditory variables: the initial position of the responding hand (left, centre or right) and the presence or absence of task-irrelevant ambient vision. RBD patients suffering from visual neglect, unlike RBD patients without neglect and healthy controls, showed a systematic rightward error in sound localisation, which was modulated by the non-auditory variables. Localisation errors were exacerbated by initial hand-position to the right of the body-midline, and reduced by the leftwards initial hand-position. Moreover, for the visual neglect patients, mere presence of ambient vision worsened localisation errors. These results demonstrate that although hand-pointing to sounds has often been considered a straightforward approach to investigate sound-localisation abilities in brain-damaged patients, in some patients it may actually reveal localisation deficits that reflect a combination of impaired spatial-hearing and spatial biases from other sensory modalities (i.e., vision and proprioception).     

The role of sound symbolism in language learning

Certain correspondences between the sound and meaning of words can be observed in subsets of the vocabulary. These sound-symbolic relationships have been suggested to result in easier language acquisition, but previous studies have explicitly tested effects of sound symbolism on learning category distinctions but not on word learning. In 2 word learning experiments, we varied the extent to which phonological properties related to a rounded-angular shape distinction and we distinguished learning of categories from learning of individual words. We found that sound symbolism resulted in an advantage for learning categories of sound-shape mappings but did not assist in learning individual word meanings. These results are consistent with the limited presence of sound symbolism in natural language. The results also provide a reinterpretation of the role of sound symbolism in language learning and language origins and a greater specification of the conditions under which sound symbolism proves advantageous for learning. (PsycINFO Database Record (c) 2012 APA, all rights reserved).     

Symbouki: a meta‐analysis on the emergence of sound symbolism in early language acquisition

Adults and toddlers systematically associate pseudowords such as “bouba” and “kiki” with round and spiky shapes, respectively, a sound symbolic phenomenon known as the “bouba‐kiki effect”. To date, whether this sound symbolic effect is a property of the infant brain present at birth or is a learned aspect of language perception remains unknown. Yet, solving this question is fundamental for our understanding of early language acquisition. Indeed, an early sensitivity to such sound symbolic associations could provide a powerful mechanism for language learning, playing a bootstrapping role in the establishment of novel sound–meaning associations. The aim of the present meta‐analysis (SymBouKi) is to provide a quantitative overview of the emergence of the bouba‐kiki effect in infancy and early childhood. It allows a high‐powered assessment of the true sound symbolic effect size by pooling over the entire set of 11 extant studies (six published, five unpublished), entailing data from 425 participants between 4 and 38 months of age. The quantitative data provide statistical support for a moderate, but significant, sound symbolic effect. Further analysis found a greater sensitivity to sound symbolism for bouba‐type pseudowords (i.e., round sound‐shape correspondences) than for kiki‐type pseudowords (i.e., spiky sound‐shape correspondences). For the kiki‐type pseudowords, the effect emerged with age. Such discrepancy challenges the view that sensitivity to sound symbolism is an innate language mechanism rooted in an exuberant interconnected brain. We propose alternative hypotheses where both innate and learned mechanisms are at play in the emergence of sensitivity to sound symbolic relationships.     

An investigation of the relationship between teachers’ expectations and teachers’ perceptions of student attributes

Little is known about factors other than students’ abilities and background variables that shape teachers’ achievement expectations. This study was aimed at investigating the role of teachers’ perceptions of students attributes (working habits, popularity, self-confidence, student–teacher relationships, and classroom behavior) in shaping teachers’ expectations. The sample analyzed consisted of 5316 students and 469 classes in grade 6 in Dutch primary education. Teachers had higher expectations for students who they perceived as self-confident and having positive work habits. Differences in expectations between boys and girls could partly be explained by the teachers’ perceptions of students’ work habits. Teachers differed in the extent to which they let their perceptions of student attributes shape their expectations.     

Sound-symbolism: A Piece in the Puzzle of Word Learning

Sound-symbolism is the idea that the relationship between word sounds and word meaning is not arbitrary for all words, but rather that there are subsets of words in the world’s languages for which sounds and their symbols have some degree of correspondence. The present research investigates sound-symbolism as a possible route to the learning of an unknown word’s meaning. Three studies compared the guesses that adult participants made regarding the potential meanings of sound-symbolic and non-sound symbolic obsolete words. In each study, participants were able to generate better definitions for sound-symbolic words when compared to non-sound symbolic words. Participants were also more likely to recognize the meanings of sound symbolic words. The superior performance on sound-symbolic words held even when definitions generated on the basis of sound association were eliminated. It is concluded that sound symbolism is a word property that influences word learning.     

Sound symbolic naming of novel objects is a graded function

Although linguistic traditions of the last century assumed that there is no link between sound and meaning (i.e., arbitrariness), recent research has established a nonarbitrary relation between sound and meaning (i.e., sound symbolism). For example, some sounds (e.g., /u/ as in took) suggest bigness whereas others (e.g., /i/ as in tiny) suggest smallness. We tested whether sound symbolism only marks contrasts (e.g., small versus big things) or whether it marks object properties in a graded manner (e.g., small, medium, and large things). In two experiments, participants viewed novel objects (i.e., greebles) of varying size and chose the most appropriate name for each object from a list of visually or auditorily presented nonwords that varied incrementally in the number of "large" and "small" phonemes. For instance, "wodolo" contains all large-sounding phonemes, whereas "kitete" contains all small-sounding phonemes. Participants' choices revealed a graded relationship between sound and size: The size of the object linearly predicted the number of large-sounding phonemes in its preferred name. That is, small, medium, and large objects elicited names with increasing numbers of large-sounding phonemes. The results are discussed in relation to cross-modal processing, gesture, and vocal pitch.