Degree of Language Experience Modulates Visual Attention to Visible Speech and Iconic Gestures During Clear and Degraded Speech Comprehension

Visual information conveyed by iconic hand gestures and visible speech can enhance speech comprehension under adverse listening conditions for both native and non‐native listeners. However, how a listener allocates visual attention to these articulators during speech comprehension is unknown. We used eye‐tracking to investigate whether and how native and highly proficient non‐native listeners of Dutch allocated overt eye gaze to visible speech and gestures during clear and degraded speech comprehension. Participants watched video clips of an actress uttering a clear or degraded (6‐band noise‐vocoded) action verb while performing a gesture or not, and were asked to indicate the word they heard in a cued‐recall task. Gestural enhancement was the largest (i.e., a relative reduction in reaction time cost) when speech was degraded for all listeners, but it was stronger for native listeners. Both native and non‐native listeners mostly gazed at the face during comprehension, but non‐native listeners gazed more often at gestures than native listeners. However, only native but not non‐native listeners' gaze allocation to gestures predicted gestural benefit during degraded speech comprehension. We conclude that non‐native listeners might gaze at gesture more as it might be more challenging for non‐native listeners to resolve the degraded auditory cues and couple those cues to phonological information that is conveyed by visible speech. This diminished phonological knowledge might hinder the use of semantic information that is conveyed by gestures for non‐native compared to native listeners. Our results demonstrate that the degree of language experience impacts overt visual attention to visual articulators, resulting in different visual benefits for native versus non‐native listeners.     

Why Colour Primitivism?

Primitivism is the view that colors are sui generis properties of physical objects. The basic insight underlying primitivism is that colours are as we see them, i.e. they are categorical properties of physical objects—simple, monadic, constant, etc.—just like shapes. As such, they determine the content of colour experience. Accepting the premise that colours are sui generis properties of physical objects, this paper seeks to show that ascribing primitive properties to objects is, ipso facto, ascribing to objects irreducible dispositions to look coloured, and that anything that primitive redness can do, the non-reductive disposition to look red can do just as well. What makes primitivism suspect is not the commitment to sui generis properties, but instead the claim that colours are more than dispositions. Since, as I show, whatever primitivism appeals to for the purpose of arguing that colours are more than dispositions—objectivity, explanation, causation, phenomenology, constancy, etc.—can also be invoked by non-reductive dispositionalism, the feature that purportedly renders colours more than dispositions remains mysterious.     

Feature Statistics Modulate the Activation of Meaning During Spoken Word Processing

Understanding spoken words involves a rapid mapping from speech to conceptual representations. One distributed feature‐based conceptual account assumes that the statistical characteristics of concepts’ features—the number of concepts they occur in (distinctiveness/sharedness) and likelihood of co‐occurrence (correlational strength)—determine conceptual activation. To test these claims, we investigated the role of distinctiveness/sharedness and correlational strength in speech‐to‐meaning mapping, using a lexical decision task and computational simulations. Responses were faster for concepts with higher sharedness, suggesting that shared features are facilitatory in tasks like lexical decision that require access to them. Correlational strength facilitated responses for slower participants, suggesting a time‐sensitive co‐occurrence‐driven settling mechanism. The computational simulation showed similar effects, with early effects of shared features and later effects of correlational strength. These results support a general‐to‐specific account of conceptual processing, whereby early activation of shared features is followed by the gradual emergence of a specific target representation.     

How might degrees of belief shift? On action conflicting with professed beliefs

People often act in ways that appear incompatible with their sincere assertions (such as trembling in fear when their death becomes an imminent possibility, despite earlier professing that “Death is not bad!”). But how might we explain such cases? On the shifting view, subjects’ degrees of belief (or degrees of confidence) may be highly sensitive to changes in context. This paper articulates and refines this view, after defending it against recent criticisms. It details two mechanisms by which degrees of beliefs may shift.     

Latent semantic analysis cosines as a cognitive similarity measure: Evidence from priming studies

In distributional semantics models (DSMs) such as latent semantic analysis (LSA), words are represented as vectors in a high-dimensional vector space. This allows for computing word similarities as the cosine of the angle between two such vectors. In two experiments, we investigated whether LSA cosine similarities predict priming effects, in that higher cosine similarities are associated with shorter reaction times (RTs). Critically, we applied a pseudo-random procedure in generating the item material to ensure that we directly manipulated LSA cosines as an independent variable. We employed two lexical priming experiments with lexical decision tasks (LDTs). In Experiment 1 we presented participants with 200 different prime words, each paired with one unique target. We found a significant effect of cosine similarities on RTs. The same was true for Experiment 2, where we reversed the prime-target order (primes of Experiment 1 were targets in Experiment 2, and vice versa). The results of these experiments confirm that LSA cosine similarities can predict priming effects, supporting the view that they are psychologically relevant. The present study thereby provides evidence for qualifying LSA cosine similarities not only as a linguistic measure, but also as a cognitive similarity measure. However, it is also shown that other DSMs can outperform LSA as a predictor of priming effects.     

Learning of novel semantic relationships via sudden comprehension is associated with a hippocampus-independent network

Sudden comprehension—or insight—during problem-solving can enhance learning, but the underlying neural processes are largely unknown. We investigated neural correlates of learning from sudden comprehension using functional magnetic resonance imaging and a verbal problem-solving task. Solutions and “solutions” to solvable and unsolvable verbal problems, respectively, were presented to induce sudden comprehension or continued incomprehension. We found activations of the hippocampus, medial prefrontal cortex (mPFC), amygdala, and striatum during sudden comprehension. Notably, however, mPFC and temporo-parietal neocortical structures rather than the hippocampus were associated with later learning of suddenly comprehended solutions. Moreover, difficult compared to easy sudden comprehension elicited midbrain activations and was associated with successful learning, pointing to learning via intrinsic reward. Sudden comprehension of novel semantic associations may constitute a special case of long-term memory formation primarily mediated by the mPFC, expanding our knowledge of its role in prior-knowledge-dependent memory.     

Interactions of reading and semantics along the ventral visual processing stream

Converging evidence supports a distributed-plus-hub view of semantic processing, in which there are distributed modular semantic sub-systems (e.g., for shape, colour, and action) connected to an amodal semantic hub. Furthermore, object semantic processing of colour and shape, and lexical reading and identification, are processed mainly along the ventral stream, while action semantic processing occurs mainly along the dorsal stream. In Experiment 1, participants read a prime word that required imagining either the object or action referent, and then named a lexical word target. In Experiments 2 and 3, participants performed a lexical decision task (LDT) with the same targets as in Experiment 1, in the presence of foils that were legal nonwords (NW; Experiment 2) or pseudohomophones (PH; Experiment 3). Semantic priming was similar in effect size regardless of prime type for naming, but was greater for object primes than action primes for the LDT with PH foils, suggesting a shared-stream advantage when the task demands focus on orthographic lexical processing. These experiments extend the distributed-plus-hub model, and provide a novel paradigm for further research.     

The influence of speech rate and accent on access and use of semantic information

Circumstances in which the speech input is presented in sub-optimal conditions generally lead to processing costs affecting spoken word recognition. The current study indicates that some processing demands imposed by listening to difficult speech can be mitigated by feedback from semantic knowledge. A set of lexical decision experiments examined how foreign accented speech and word duration impact access to semantic knowledge in spoken word recognition. Results indicate that when listeners process accented speech, the reliance on semantic information increases. Speech rate was not observed to influence semantic access, except in the setting in which unusually slow accented speech was presented. These findings support interactive activation models of spoken word recognition in which attention is modulated based on speech demands.     

Activation of semantic information at the sublexical level during handwriting production: Evidence from inhibition effects of Chinese semantic radicals in the picture‐word interference paradigm

Using a non‐alphabetic language (e.g., Chinese), the present study tested a novel view that semantic information at the sublexical level should be activated during handwriting production. Over 80% of Chinese characters are phonograms, in which semantic radicals represent category information (e.g., 椅 ‘chair,’ 桃 ‘peach,’ 橙 ‘orange’ are related to plants) while phonetic radicals represent phonetic information (e.g., 狼 ‘wolf,’ 朗 ‘brightness,’ 郎 ‘male,’ are all pronounced /lang/). Under different semantic category conditions at the lexical level (semantically related in Experiment 1; semantically unrelated in Experiment 2), the orthographic relatedness and semantic relatedness of semantic radicals in the picture name and its distractor were manipulated under different SOAs (i.e., stimulus onset asynchrony, the interval between the onset of the picture and the onset of the interference word). Two questions were addressed: (1) Is it possible that semantic information could be activated in the sublexical level conditions? (2) How are semantic and orthographic information dynamically accessed in word production? Results showed that both orthographic and semantic information were activated under the present picture‐word interference paradigm, dynamically under different SOAs, which supported our view that discussions on semantic processes in the writing modality should be extended to the sublexical level. The current findings provide possibility for building new orthography‐phonology‐semantics models in writing.     

Non‐Arbitrariness in Mapping Word Form to Meaning: Cross‐Linguistic Formal Markers of Word Concreteness

Arbitrary symbolism is a linguistic doctrine that predicts an orthogonal relationship between word forms and their corresponding meanings. Recent corpora analyses have demonstrated violations of arbitrary symbolism with respect to concreteness, a variable characterizing the sensorimotor salience of a word. In addition to qualitative semantic differences, abstract and concrete words are also marked by distinct morphophonological structures such as length and morphological complexity. Native English speakers show sensitivity to these markers in tasks such as auditory word recognition and naming. One unanswered question is whether this violation of arbitrariness reflects an idiosyncratic property of the English lexicon or whether word concreteness is a marked phenomenon across other natural languages. We isolated concrete and abstract English nouns (N  = 400), and translated each into Russian, Arabic, Dutch, Mandarin, Hindi, Korean, Hebrew, and American Sign Language. We conducted offline acoustic analyses of abstract and concrete word length discrepancies across languages. In a separate experiment, native English speakers (N  = 56) with no prior knowledge of these foreign languages judged concreteness of these nouns (e.g., Can you see, hear, feel, or touch this? Yes/No). Each naïve participant heard pre‐recorded words presented in randomized blocks of three foreign languages following a brief listening exposure to a narrative sample from each respective language. Concrete and abstract words differed by length across five of eight languages, and prediction accuracy exceeded chance for four of eight languages. These results suggest that word concreteness is a marked phenomenon across several of the world's most widely spoken languages. We interpret these findings as supportive of an adaptive cognitive heuristic that allows listeners to exploit non‐arbitrary mappings of word form to word meaning.