Manifold embedding for zero-shot recognition

Zero-Shot Recognition (ZSR) has gained its popularity recently owing to its promising characteristic that extends the classifiers to the unseen classes. It is typically addressed by resorting to a class semantic space to transfer the knowledge from the seen classes to unseen ones. Therefore, constructing the effective interactions between the visual space and the class semantic space is the key for ZSR. In this paper, under the assumption that the distribution of the semantic categories in the semantic space has an intrinsic manifold structure, we propose two manifold embedding-based ZSR approaches to capture the intrinsic structures of both the visual space and the class semantic space, i.e., ME-ZSR and MCCA-ZSR. Specifically, ME-ZSR builds embedding from visual space to semantic space, while MCCA-ZSR explores to embed both visual and semantic features into a common space. The linear, closed-form solutions make both methods efficient to optimize. Extensive experiments on three popular datasets AwA, CUB and NAB validate the effectiveness of both methods.     

Modelling the effects of semantic ambiguity in word recognition

Most words in English are ambiguous between different interpretations; words can mean different things in different contexts. We investigate the implications of different types of semantic ambiguity for connectionist models of word recognition. We present a model in which there is competition to activate distributed semantic representations. The model performs well on the task of retrieving the different meanings of ambiguous words, and is able to simulate data reported by Rodd, Gaskell, and Marslen-Wilson [J. Mem. Lang. 46 (2002) 245] on how semantic ambiguity affects lexical decision performance. In particular, the network shows a disadvantage for words with multiple unrelated meanings (e.g., bark) that coexists with a benefit for words with multiple related word senses (e.g., twist). The ambiguity disadvantage arises because of interference between the different meanings, while the sense benefit arises because of differences in the structure of the attractor basins formed during learning. Words with few senses develop deep, narrow attractor basins, while words with many senses develop shallow, broad basins. We conclude that the mental representations of word meanings can be modelled as stable states within a high-dimensional semantic space, and that variations in the meanings of words shape the landscape of this space.     

Evolution of word meanings through metaphorical mapping: Systematicity over the past millennium

One way that languages are able to communicate a potentially infinite set of ideas through a finite lexicon is by compressing emerging meanings into words, such that over time, individual words come to express multiple, related senses of meaning. We propose that overarching communicative and cognitive pressures have created systematic directionality in how new metaphorical senses have developed from existing word senses over the history of English. Given a large set of pairs of semantic domains, we used computational models to test which domains have been more commonly the starting points (source domains) and which the ending points (target domains) of metaphorical mappings over the past millennium. We found that a compact set of variables, including externality, embodiment, and valence, explain directionality in the majority of about 5000 metaphorical mappings recorded over the past 1100 years. These results provide the first large-scale historical evidence that metaphorical mapping is systematic, and driven by measurable communicative and cognitive principles.     

Language knowledge and event knowledge in language use

This paper examines how semantic knowledge is used in language comprehension and in making judgments about events in the world. We contrast knowledge gleaned from prior language experience (“language knowledge”) and knowledge coming from prior experience with the world (“world knowledge”). In two corpus analyses, we show that previous research linking verb aspect and event representations have confounded language and world knowledge. Then, using carefully chosen stimuli that remove this confound, we performed four experiments that manipulated the degree to which language knowledge or world knowledge should be salient and relevant to performing a task, finding in each case that participants use the type of knowledge most appropriate to the task. These results provide evidence for a highly context-sensitive and interactionist perspective on how semantic knowledge is represented and used during language processing.     

The discovery and comparison of symbolic magnitudes

Humans and other primates are able to make relative magnitude comparisons, both with perceptual stimuli and with symbolic inputs that convey magnitude information. Although numerous models of magnitude comparison have been proposed, the basic question of how symbolic magnitudes (e.g., size or intelligence of animals) are derived and represented in memory has received little attention. We argue that symbolic magnitudes often will not correspond directly to elementary features of individual concepts. Rather, magnitudes may be formed in working memory based on computations over more basic features stored in long-term memory. We present a model of how magnitudes can be acquired and compared based on BARTlet, a representationally simpler version of Bayesian Analogy with Relational Transformations (BART; Lu, Chen, & Holyoak, 2012). BARTlet operates on distributions of magnitude variables created by applying dimension-specific weights (learned with the aid of empirical priors derived from pre-categorical comparisons) to more primitive features of objects. The resulting magnitude distributions, formed and maintained in working memory, are sensitive to contextual influences such as the range of stimuli and polarity of the question. By incorporating psychological reference points that control the precision of magnitudes in working memory and applying the tools of signal detection theory, BARTlet is able to account for a wide range of empirical phenomena involving magnitude comparisons, including the symbolic distance effect and the semantic congruity effect. We discuss the role of reference points in cognitive and social decision-making, and implications for the evolution of relational representations.     

A review of literature on the link between action observation and action language: advancing a shared semantic theory

Understanding each other is a core concept of social cohesion and, consequently, has immense value in human society. Importantly, shared information leading to cohesion can come from two main sources: observed action and/or language (word) processing. In this paper, we propose a theoretical framework for the link between action observation and action verb processing. Based on the activation of common semantic representations of actions through semantic resonance, this model can account for the neurophysiological, behavioral and neuropsychological domains in the link between action observation and language. Semantic resonance is hypothesized to play a role beyond that of the mere observation of others and can benefit future studies trying to connect action production and language.     

Russian-language neurosemantics: Clustering of word meaning and sense from oral narratives

This article is a part of a large-scale brain mapping project aimed at finding the relations among semantic categories in oral Russian-language texts and brain activity as measured using functional magnetic resonance imaging (fMRI). The goal of present study in particular is to examine the nature of lexical semantic relations and find an appropriate lexical space, homeomorphic to the activation patterns in the brain. Participants were presented with oral narratives, which described significant social issues from the first-person perspective. Stimuli were annotated using a dictionary and a vector approach. Results show that fMRI signal and clusters of related words have similar patterns of brain activation across participants. Results also show that annotation by a list of features more strongly contributes to prediction of the observed activation patterns. Findings confirm the hypothesis of situational semantic representation in the brain.     

Characterizing the Dynamics of Learning in Repeated Reference Games

The language we use over the course of conversation changes as we establish common ground and learn what our partner finds meaningful. Here we draw upon recent advances in natural language processing to provide a finer-grained characterization of the dynamics of this learning process. We release an open corpus (>15,000 utterances) of extended dyadic interactions in a classic repeated reference game task where pairs of participants had to coordinate on how to refer to initially difficult-to-describe tangram stimuli. We find that different pairs discover a wide variety of idiosyncratic but efficient and stable solutions to the problem of reference. Furthermore, these conventions are shaped by the communicative context: words that are more discriminative in the initial context (i.e., that are used for one target more than others) are more likely to persist through the final repetition. Finally, we find systematic structure in how a speaker's referring expressions become more efficient over time: Syntactic units drop out in clusters following positive feedback from the listener, eventually leaving short labels containing open-class parts of speech. These findings provide a higher resolution look at the quantitative dynamics of ad hoc convention formation and support further development of computational models of learning in communication.     

Acquisition of the Meaning of the Word Orange Requires Understanding of the Meanings of Red,Pink, and Purple: Constructing a Lexicon as a Connected System

This research investigated how children build up the language-specific system of the color lexicon, examining factors that play important roles for the construction of an adult-like color lexicon. We had 3-, 4-, and 5-year-old Japanese-speaking children and adults (n = 20, 18, 19, and 19, respectively) produce names for 93 color swatches. The results showed that children of all ages were able to apply most of the chromatic words to the colors close to the center of each category, but even 5-year-olds struggle to delineate the boundaries between the words. Furthermore, the model analyses revealed that broad-covering and high-frequency words are mapped to the center of the lexical category earlier. However, cross-individual consistency in adults' use contributed most strongly for the adult-like boundary delineation. The results suggest that the process of system construction consists of at least two steps (i.e., mapping words to their category center and finding appropriate boundaries between neighboring words), with the quantity and the quality of the input contributing differently to the steps.