Implicit Association Test (IAT) Studies Investigating Pitch-Shape Audiovisual Cross-modal Associations Across Language Groups

Previous studies have shown that Chinese speakers and non-Chinese speakers exhibit different patterns of cross-modal congruence for the lexical tones of Mandarin Chinese, depending on which features of the pitch they attend to. But is this pattern of language-specific listening a conscious cultural strategy or an automatic processing effect? If automatic, does it also apply when the same pitch contours no longer sound like speech? Implicit Association Tests (IATs) provide an indirect measure of cross-modal association. In a series of IAT studies, conducted with participants with three kinds of language backgrounds (Chinese-dominant bilinguals, Chinese balanced bilinguals, and English speakers with no Chinese experience) we find language-specific congruence effects for Mandarin lexical tones but not for matched sine-wave stimuli. That is, for linguistic stimuli, non-Chinese speakers show advantages for pitch-height congruence (high-pointy, low-curvy); no congruence effects were found for Chinese speakers. For non-linguistic stimuli, all participant groups showed advantages for pitch-height congruence. The present findings suggest that non-lexical tone congruence (high-pointy, low-curvy) is a basic congruence pattern, and the acquisition of a language with lexical tone can alter this perception.     

象似性和手语学习经验对中国手语-汉语跨模态语义启动效应的影响

采用启动条件下的词汇判断任务,考察熟练手语使用者和无手语经验成年听人的跨模态语义启动效应。结果发现：1)在象似词条件下,两组被试判断汉语语义相关词的反应时均快于语义无关词,说明手语象似词和汉语词之间存在跨模态语义启动效应。2)在非象似词条件下,仅手语熟练被试判断汉语语义相关词的反应时快于语义无关词,无手语经验被试判断汉语语义相关词和无关词的速度没有差异。这是由于前者心理词库中的手语词和口语词共享语义表征,而后者主要依赖手语象似词的视觉模拟性。整个研究表明,中国手语和汉语间存在跨模态语义启动效应,但该效应受到手语词象似性和手语学习经历的调节。     

Sound-symbolism: A Piece in the Puzzle of Word Learning

Sound-symbolism is the idea that the relationship between word sounds and word meaning is not arbitrary for all words, but rather that there are subsets of words in the world’s languages for which sounds and their symbols have some degree of correspondence. The present research investigates sound-symbolism as a possible route to the learning of an unknown word’s meaning. Three studies compared the guesses that adult participants made regarding the potential meanings of sound-symbolic and non-sound symbolic obsolete words. In each study, participants were able to generate better definitions for sound-symbolic words when compared to non-sound symbolic words. Participants were also more likely to recognize the meanings of sound symbolic words. The superior performance on sound-symbolic words held even when definitions generated on the basis of sound association were eliminated. It is concluded that sound symbolism is a word property that influences word learning.     

Sound symbolic naming of novel objects is a graded function

Although linguistic traditions of the last century assumed that there is no link between sound and meaning (i.e., arbitrariness), recent research has established a nonarbitrary relation between sound and meaning (i.e., sound symbolism). For example, some sounds (e.g., /u/ as in took) suggest bigness whereas others (e.g., /i/ as in tiny) suggest smallness. We tested whether sound symbolism only marks contrasts (e.g., small versus big things) or whether it marks object properties in a graded manner (e.g., small, medium, and large things). In two experiments, participants viewed novel objects (i.e., greebles) of varying size and chose the most appropriate name for each object from a list of visually or auditorily presented nonwords that varied incrementally in the number of “large” and “small” phonemes. For instance, “wodolo” contains all large-sounding phonemes, whereas “kitete” contains all small-sounding phonemes. Participants' choices revealed a graded relationship between sound and size: The size of the object linearly predicted the number of large-sounding phonemes in its preferred name. That is, small, medium, and large objects elicited names with increasing numbers of large-sounding phonemes. The results are discussed in relation to cross-modal processing, gesture, and vocal pitch.     

Japanese Sound‐Symbolism Facilitates Word Learning in English‐Speaking Children

Sound‐symbolism is the nonarbitrary link between the sound and meaning of a word. Japanese‐speaking children performed better in a verb generalization task when they were taught novel sound‐symbolic verbs, created based on existing Japanese sound‐symbolic words, than novel nonsound‐symbolic verbs ( Imai, Kita, Nagumo, & Okada, 2008 ). A question remained as to whether the Japanese children had picked up regularities in the Japanese sound‐symbolic lexicon or were sensitive to universal sound‐symbolism. The present study aimed to provide support for the latter. In a verb generalization task, English‐speaking 3‐year‐olds were taught novel sound‐symbolic verbs, created based on Japanese sound‐symbolism, or novel nonsound‐symbolic verbs. English‐speaking children performed better with the sound‐symbolic verbs, just like Japanese‐speaking children. We concluded that children are sensitive to universal sound‐symbolism and can utilize it in word learning and generalization, regardless of their native language.     

The Sound of Grasp Affordances: Influence of Grasp‐Related Size of Categorized Objects on Vocalization

Previous research shows that simultaneously executed grasp and vocalization responses are faster when the precision grip is performed with the vowel [i] and the power grip is performed with the vowel [ɑ]. Research also shows that observing an object that is graspable with a precision or power grip can activate the grip congruent with the object. Given the connection between vowel articulation and grasping, this study explores whether grasp‐related size of observed objects can influence not only grasp responses but also vowel pronunciation. The participants had to categorize small and large objects into natural and manufactured categories by pronouncing the vowel [i] or [ɑ]. As predicted, [i] was produced faster when the object's grasp‐related size was congruent with the precision grip while [ɑ] was produced faster when the size was congruent with the power grip (Experiment 1). The effect was not, however, observed when the participants were presented with large objects that are not typically grasped by the power grip (Experiment 2). This study demonstrates that vowel production is systematically influenced by grasp‐related size of a viewed object, supporting the account that sensory‐motor processes related to grasp planning and representing grasp‐related properties of viewed objects interact with articulation processes. The paper discusses these findings in the context of size–sound symbolism, suggesting that mechanisms that transform size‐grasp affordances into corresponding grasp‐ and articulation‐related motor programs might provide a neural basis for size‐sound phenomena that links small objects with closed‐front vowels and large objects with open‐back vowels.     

声音概念垂直空间隐喻表征的初步探讨

本文通过2个实验,初步探讨汉语声音概念词的垂直空间隐喻表征。实验1先视觉呈现声音概念词,要求被试对随后在屏幕上方或者下方呈现的图形进行形状判断;实验2要求被试先对声音词进行音高判断,再判断图形。结果只在实验2中发现隐喻表征,说明声音词的垂直空间隐喻表征需要一定的条件。后期问卷调查后发现,声音词普遍存在多种表象,且声音并不是首先激活的。意味着声音词的垂直空间隐喻表征需要较高的声音音高维度的语义凸显度,需对声音概念进行较深层次的语义加工,才可激活垂直空间隐喻表征。     

“Big” Sounds Bigger in More Widely Spoken Languages

Larger communities face more communication barriers. We propose that languages spoken by larger communities adapt and overcome these greater barriers by increasing their reliance on sound symbolism, as sound symbolism can facilitate communication. To test whether widely spoken languages are more sound symbolic, participants listened to recordings of the words big and small in widely spoken and less common languages and guessed their meanings. Accuracy was higher for words from widely spoken languages providing evidence that widely spoken languages harbor more sound symbolism. Preliminary results also suggest that widely spoken languages rely on different sound symbolic patterns than less common languages. Community size can thus shape linguistic forms and influence the tools that languages use to facilitate communication.     

Auditory space perception in left- and right-handers

Several studies have shown that handedness has an impact on visual spatial abilities. Here we investigated the effect of laterality on auditory space perception. Participants (33 right-handers, 20 left-handers) completed two tasks of sound localization. In a dark, anechoic, and sound-proof room, sound stimuli (broadband noise) were presented via 21 loudspeakers mounted horizontally (from 80° on the left to 80° on the right). Participants had to localize the target either by using a swivel hand-pointer or by head-pointing. Individual lateral preferences of eye, ear, hand, and foot were obtained using a questionnaire. With both pointing methods, participants showed a bias in sound localization that was to the side contralateral to the preferred hand, an effect that was unrelated to their overall precision. This partially parallels findings in the visual modality as left-handers typically have a more rightward bias in visual line bisection compared with right-handers. Despite the differences in neural processing of auditory and visual spatial information these findings show similar effects of lateral preference on auditory and visual spatial perception. This suggests that supramodal neural processes are involved in the mechanisms generating laterality in space perception.