Word Senses as Clusters of Meaning Modulations: A Computational Model of Polysemy

Most words in natural languages are polysemous; that is, they have related but different meanings in different contexts. This one-to-many mapping of form to meaning presents a challenge to understanding how word meanings are learned, represented, and processed. Previous work has focused on solutions in which multiple static semantic representations are linked to a single word form, which fails to capture important generalizations about how polysemous words are used; in particular, the graded nature of polysemous senses, and the flexibility and regularity of polysemy use. We provide a novel view of how polysemous words are represented and processed, focusing on how meaning is modulated by context. Our theory is implemented within a recurrent neural network that learns distributional information through exposure to a large and representative corpus of English. Clusters of meaning emerge from how the model processes individual word forms. In keeping with distributional theories of semantics, we suggest word meanings are generalized from contexts of different word tokens, with polysemy emerging as multiple clusters of contextually modulated meanings. We validate our results against a human-annotated corpus of polysemy focusing on the gradedness, flexibility, and regularity of polysemous sense individuation, as well as behavioral findings of offline sense relatedness ratings and online sentence processing. The results provide novel insights into how polysemy emerges from contextual processing of word meaning from both a theoretical and computational point of view.     

Phonological priming and cohort effects in toddlers

Adult word recognition is influenced by prior exposure to phonologically or semantically related words (cup primes cat or plate) compared to unrelated words (door), suggesting that words are organised in the adult lexicon based on their phonological and semantic properties and that word recognition implicates not just the heard word, but also related words. We investigate the phonological organisation of the toddler lexicon with two experiments using a picture priming technique. Twenty-four month olds showed inhibition of target recognition in related primed trials compared to unrelated primed trials (Experiment 1) and also in related primed trials compared to unprimed trials (Experiment 2). Further analysis of children’s responding found that this inhibition effect was modulated by the cohort and neighbourhood size of the words tested. Overall, the results indicate a lexical basis for the reported effects and suggest that the phonological properties provide an organisational basis for words in the toddler lexicon.     

Language, thought, and real nouns

We test the claim that acquiring a mass-count language, like English, causes speakers to think differently about entities in the world, relative to speakers of classifier languages like Japanese. We use three tasks to assess this claim: object-substance rating, quantity judgment, and word extension. Using the first two tasks, we present evidence that learning mass-count syntax has little effect on the interpretation of familiar nouns between Japanese and English, and that speakers of these languages do not divide up referents differently along an individuation continuum, as claimed in some previous reports [Gentner, D., & Boroditsky, L. (2001). Individuation, relativity, and early word learning. In M. Bowerman, & S. Levinson (Eds.), Language acquisition and conceptual development (pp. 215-256). Cambridge University Press]. Instead, we argue that previous cross-linguistic differences [Imai, M., & Gentner, D. (1997). A cross-linguistic study of early word meaning: Universal ontology and linguistic influence. Cognition, 62, 169-200] are attributable to “lexical statistics” [Gleitman, L., & Papafragou, A. (2005). Language and thought. In K. Holyoak, & R. Morrison (Eds.), Cambridge handbook of thinking and reasoning (pp. 633-661). Cambridge University Press]. Speakers of English are more likely to think that a novel ambiguous expression like “the blicket” refers to a kind of object (relative to speakers of Japanese) because speakers of English are likely to assume that “blicket” is a count noun rather than a mass noun, based on the relative frequency of each kind of word in English. This is confirmed by testing Mandarin-English bilinguals with a word extension task. We find that bilinguals tested in English with mass-count ambiguous syntax extend novel words like English monolinguals (and assume that a word like “blicket” refers to a kind of object). In contrast, bilinguals tested in Mandarin are significantly more likely to extend novel words by material. Thus, online lexical statistics, rather than non-linguistic thought, mediate cross-linguistic differences in word extension. We suggest that speakers of Mandarin, English, and Japanese draw on a universal set of lexical meanings, and that mass-count syntax allows speakers of English to select among these meanings.     

Object labeling influences infant phonetic learning and generalization

Different kinds of speech sounds are used to signify possible word forms in every language. For example, lexical stress is used in Spanish (/‘be.be/, ‘he/she drinks’ versus /be.’be/, ‘baby’), but not in French (/‘be.be/ and /be.’be/ both mean ‘baby’). Infants learn many such native language phonetic contrasts in their first year of life, likely using a number of cues from parental speech input. One such cue could be parents’ object labeling, which can explicitly highlight relevant contrasts. Here we ask whether phonetic learning from object labeling is abstract—that is, if learning can generalize to new phonetic contexts. We investigate this issue in the prosodic domain, as the abstraction of prosodic cues (like lexical stress) has been shown to be particularly difficult. One group of 10-month-old French-learners was given consistent word labels that contrasted on lexical stress (e.g., Object A was labeled /‘ma.bu/, and Object B was labeled /ma.’bu/). Another group of 10-month-olds was given inconsistent word labels (i.e., mixed pairings), and stress discrimination in both groups was measured in a test phase with words made up of new syllables. Infants trained with consistently contrastive labels showed an earlier effect of discrimination compared to infants trained with inconsistent labels. Results indicate that phonetic learning from object labeling can indeed generalize, and suggest one way infants may learn the sound properties of their native language(s).     

Processing polysemous words in context: Evidence for interrelated meanings

Two experiments are reported which examined whether the various meanings of polysemous adjectives (e.g.,firm as insolid orfirm as instrict) are functionally independent during language comprehension. In Experiment 1 a priming technique was used, similar to that which has been used to investigate the processing of homonyms, to test whether alternative meanings of polysemous words become active even if they are irrelevant in the context. The results showed that this was only reliably the case for central meanings of the words (eg.,firm as insolid), for which effects were found as much as 1100 msec after prime onset. No significant priming of targets related to noncentral meanings (eg.,firm as instrict) was obtained in irrelevant contexts. This happened despite the fact that both types of target were equally primed when the prime occurred in isolation. Experiment 2 obtained similar results using a relatedness judgment task. The asymmetrical priming of central and noncentral targets in irrelevant contexts is discussed in terms of a hierarchical meaning structure relating the diverse uses of polysemous words.     

Modelling the effects of semantic ambiguity in word recognition

Most words in English are ambiguous between different interpretations; words can mean different things in different contexts. We investigate the implications of different types of semantic ambiguity for connectionist models of word recognition. We present a model in which there is competition to activate distributed semantic representations. The model performs well on the task of retrieving the different meanings of ambiguous words, and is able to simulate data reported by Rodd, Gaskell, and Marslen-Wilson [J. Mem. Lang. 46 (2002) 245] on how semantic ambiguity affects lexical decision performance. In particular, the network shows a disadvantage for words with multiple unrelated meanings (e.g., bark) that coexists with a benefit for words with multiple related word senses (e.g., twist). The ambiguity disadvantage arises because of interference between the different meanings, while the sense benefit arises because of differences in the structure of the attractor basins formed during learning. Words with few senses develop deep, narrow attractor basins, while words with many senses develop shallow, broad basins. We conclude that the mental representations of word meanings can be modelled as stable states within a high-dimensional semantic space, and that variations in the meanings of words shape the landscape of this space.     

Feature Statistics Modulate the Activation of Meaning During Spoken Word Processing

Understanding spoken words involves a rapid mapping from speech to conceptual representations. One distributed feature‐based conceptual account assumes that the statistical characteristics of concepts’ features—the number of concepts they occur in (distinctiveness/sharedness) and likelihood of co‐occurrence (correlational strength)—determine conceptual activation. To test these claims, we investigated the role of distinctiveness/sharedness and correlational strength in speech‐to‐meaning mapping, using a lexical decision task and computational simulations. Responses were faster for concepts with higher sharedness, suggesting that shared features are facilitatory in tasks like lexical decision that require access to them. Correlational strength facilitated responses for slower participants, suggesting a time‐sensitive co‐occurrence‐driven settling mechanism. The computational simulation showed similar effects, with early effects of shared features and later effects of correlational strength. These results support a general‐to‐specific account of conceptual processing, whereby early activation of shared features is followed by the gradual emergence of a specific target representation.     

Probing Lexical Ambiguity: Word Vectors Encode Number and Relatedness of Senses

Lexical ambiguity—the phenomenon of a single word having multiple, distinguishable senses—is pervasive in language. Both the degree of ambiguity of a word (roughly, its number of senses) and the relatedness of those senses have been found to have widespread effects on language acquisition and processing. Recently, distributional approaches to semantics, in which a word's meaning is determined by its contexts, have led to successful research quantifying the degree of ambiguity, but these measures have not distinguished between the ambiguity of words with multiple related senses versus multiple unrelated meanings. In this work, we present the first assessment of whether distributional meaning representations can capture the ambiguity structure of a word, including both the number and relatedness of senses. On a very large sample of English words, we find that some, but not all, distributional semantic representations that we test exhibit detectable differences between sets of monosemes (unambiguous words; N = 964), polysemes (with multiple related senses; N = 4,096), and homonyms (with multiple unrelated senses; N = 355). Our findings begin to answer open questions from earlier work regarding whether distributional semantic representations of words, which successfully capture various semantic relationships, also reflect fine-grained aspects of meaning structure that influence human behavior. Our findings emphasize the importance of measuring whether proposed lexical representations capture such distinctions: In addition to standard benchmarks that test the similarity structure of distributional semantic models, we need to also consider whether they have cognitively plausible ambiguity structure.     

Personality, event-related potential (ERP) and heart rate (HR): An investigation of Gray's theory

Following Gray's theory of personality, the aim of the study was to examine the role of the personality dimensions, behavioural inhibition and activation, in mediating event-related potentials (ERPs) and the level of anticipatory heart rate (HR) deceleration response during two visual-stimulus recognition tasks. In the first task ERPs and anticipatory HR changes were elicited by feedback words informing the subjects about the quality of their response (‘correct’, ‘incorrect’). In the second task ERPs and HR responses were elicited from 62 women by feedback words (‘losing’, ‘winning’) indicating losing or winning, of amounts of money. The Gray-Wilson Personality Questionnaire (GWPQ), the Eysenck Personality Questionnaire (EPQ) and the I₇ questionnaire were used to measure variables which refer to the function of the activation and inhibition system. The N200, P300, N400, P650 and N800 peak amplitudes of the ERPs and the level of anticipatory HR deceleration response to the feedback signals were measured. Personality and physiological responses were first analysed by using a split-plot ANOVA design and second by examining multiple relationships with factor analysis. Separate ANOVAs were performed across Ss selected from the total group (n = 62) on the basis of extreme scores (high or low) on Approach (APPR), Passive Avoidance (PASS.AV), Extinction (EXTI), Extraversion (E), Neuroticism (N) and Impulsiveness (I). In line with Gray's prediction, high APPR Ss exhibited larger P600 peak amplitudes to signals indicating winning and low APPR Ss exhibited larger P600 amplitudes to signals indicating losing. Neurotic Ss produced larger N800 peak amplitudes to stimuli indicating losing as compared with stimuli indicating winning. Stable Ss, in contrast, did not display differences between feedback stimuli. Heart-rate deceleration response to feedback signals was successful in distinguishing high and low N Ss as well as Introverts and Extraverts. Neurotic Ss, compared with Stable ones, showed a more pronounced anticipatory HR slowing for both of the feedback signals. Introverts displayed more pronounced HR decelerations for punishment compared with reward signals. Extraverts, in contrast, showed greater HR decelerations for reward compared with punishment signals. These results were all in line with predictions that can be derived from Gray's theory. However, in the opposite direction was the relationship between PASS.AV and N200 peak: High PASS.AV Ss displayed greater N200 peak amplitudes to winning signals compared with signals indicating a correct response. Results from factor analysis gave rise to a three-factor solution wherein personality dimensions sensitive to signals of reward and that sensitive to signals of punishment were loading together with different physiological factors.     

Neurocognitive dimensions of lexical complexity in Polish

Neuroimaging studies of English suggest that speech comprehension engages two interdependent systems: a bilateral fronto-temporal network responsible for general perceptual and cognitive processing, and a specialised left-lateralised network supporting specifically linguistic processing. Using fMRI we test this hypothesis in Polish, a Slavic language with rich and diverse morphology. We manipulated general perceptual complexity (presence or absence of an onset-embedded stem, e.g. kotlet ‘cutlet’ vs. kot ‘cat’) and specifically linguistic complexity (presence of an inflectional affix, e.g. dom ‘house, Nom’ vs. dom-u ‘house, Gen’). Non-linguistic complexity activated a bilateral network, as in English, but we found no differences between inflected and uninflected nouns. Instead, all types of words activated left inferior frontal areas, suggesting that all Polish words can be considered linguistically ‘complex’ in processing terms. The results support a dual network hypothesis, but highlight differences between languages like English and Polish, and underline the importance of cross-linguistic comparisons.     

Some hypotheses concerning the evolution of polysemous words

The evolution of polysemous words is examined in three studies. Study 1 indicates that older words are more polysemous than recent words, and that frequently used words are more polysemous than infrequently used words. These effects were replicated, controlling for word concreteness (Study 2) and evaluation (Study 3), using two other samples of words. If words evolve additional meanings through metaphor (MacCormac, 1985), then variables (such as concreteness and evaluation) that play a role in metaphor may mediate the evolution of polysemy. The correlation between concreteness and polysemy was negligible (Study 2). As hypothesized, evaluation is a significant predictor of polysemy: Negative words are more polysemous than positive words, and relatively neutral words are more polysemous than evaluatively extreme words. These findings are consistent with the notion that polysemous words evolve, in part, through metaphor.     

Levels of number knowledge during early childhood

Researchers have long disagreed about whether number concepts are essentially continuous (unchanging) or discontinuous over development. Among those who take the discontinuity position, there is disagreement about how development proceeds. The current study addressed these questions with new quantitative analyses of children’s incorrect responses on the Give-N task. Using data from 280 children, ages 2 to 4 years, this study showed that most wrong answers were simply guesses, not counting or estimation errors. Their mean was unrelated to the target number, and they were lower-bounded by the numbers children actually knew. In addition, children learned the number-word meanings one at a time and in order; they treated the number words as mutually exclusive; and once they figured out the cardinal principle of counting, they generalized this principle to the rest of their count list. Findings support the ‘discontinuity’ account of number development in general and the ‘knower-levels’ account in particular.     

Polysemy and the acquisition of the cognitive internal state lexicon

Previous research (Hall, Scholnick, & Hughes, 1987) suggested that cognitive word meanings denote six increasingly abstract and complex cognitive processes. In this paper, we further investigated two interrelated aspects of this framework. A set of 36 conversations with children aged 4:5 to 5 provided the data. In Study 1, issues related to the optimal ordering of levels were examined by comparing the original sequence with a restructured model. In Study 2, the hypothesis that, in addition toknow, other polysemous cognitive internal state words would be hierarchically organized was tested. The findings lent support to the restructured model, but rejected the hypothesis that polysemous words within the domain were characterized by equivalent,hhierarchically organized levels. We conclude by proposing that cognitive internal state words were related by complementarity of levels rather than equivalence.The research on which this paper is based was supported by a grant from the Carnegie Corporation of New York to William S. Hall. Data analysis was supported by the Computer Science Center at the University of Maryland, College Park.     

Nosology and Causal Necessity; The Relation BetweenDefining a Disease and Discovering its Necessary Cause

The problem of disease definition is related to theproblem of proving that a certain agent is thenecessary cause of a certain disease. Natural kindterms like rheumatoid arthritis and AIDS refer toessences which are discoverable rather thanpredeterminate. No statement about such diseases isa priori necessarily true. Because theories onnecessary causes involve natural kind semantics,Koch's postulates cannot be used to falsify or verifysuch theories. Instead of proving that agent A is thenecessary cause of disease D, we include A in atheoretical definition of D, take this to representthe real meaning of D, and discard thepretheoretical definition. This is illustrated byKoch's own attempt to prove he had discovered thenecessary cause of tuberculosis. Methodologicalarguments about disease causation require a clear viewof our use of diagnostic terms. Medical lexicographersshould do more to provide such a view.     

Word learning does not end at fast-mapping: evolution of verb meanings through reorganization of an entire semantic domain

This paper explores the process through which children sort out the relations among verbs belonging to the same semantic domain. Using a set of Chinese verbs denoting a range of action events that are labeled by carrying or holding in English as a test case, we looked at how Chinese-speaking 3-, 5-, and 7-year-olds and adults apply 13 different verbs to a range of carrying/holding events. We asked how children learning Chinese originally divide and label the semantic space in this domain, how they discover the boundaries between different words, and how the meanings of verbs in the domain as a whole evolve toward the representations of adults. We also addressed the question of what factors make verb meaning acquisition easy or hard. Results showed that the pattern of children’s verb use is largely different from that of adults and that it takes a long time for children to be able to use all verbs in this domain in the way adults do. We also found that children start to use broad-covering and frequent verbs the earliest, but use of these verbs tends to converge on adult use more slowly because children could not use these verbs as adults did until they had identified boundaries between these verbs and other near-synonyms with more specific meanings. This research highlights the importance of systematic investigation of words that belong to the same domain as a whole, examining how word meanings in a domain develop as parts of a connected system, instead of examining each word on its own: learning the meaning of a verb invites restructuring of the meanings of related, neighboring verbs.     

The nature, prevalence and correlates of generativity among men in middle career

Multiple methods were used to explore the character, contexts, and correlates of generativity among 41 men aged 45–55. Generativity in the role of worker was unrelated to generativity in men’s roles as father, citizen and ‘leisurite’. Individuals who were generative in their work reported greater job satisfaction and subjective career success. These associations were stronger for generativity targeting societal well-being and fellow-workers’ growth than for creative/productive generativity. As some theorists have argued, therefore, nurturant forms of generativity may be more adaptive in mid/late career than other varieties, and more indicative of psychosocial maturity. The status of creativity/productivity within the generativity construct seems problematic. Our findings offer qualified support for McAdams and de St Aubin’s (McAdams, D.P., & de St Aubin, E. (1992). A theory of generativity and its assessment through self-report, behavioral acts, and narrative themes in autobiography. Journal of Personality & Social Psychology, 62 (6), 1003–1015) contention that generativity is expressed differentially in the domains of concern, commitment and action, and that the different relationships of these generativity domains to third variables can be interpreted in meaningful ways.     

Spatial distance effects on incremental semantic interpretation of abstract sentences: Evidence from eye tracking

A large body of evidence has shown that visual context information can rapidly modulate language comprehension for concrete sentences and when it is mediated by a referential or a lexical-semantic link. What has not yet been examined is whether visual context can also modulate comprehension of abstract sentences incrementally when it is neither referenced by, nor lexically associated with, the sentence. Three eye-tracking reading experiments examined the effects of spatial distance between words (Experiment 1) and objects (Experiment 2 and 3) on participants’ reading times for sentences that convey similarity or difference between two abstract nouns (e.g., ‘Peace and war are certainly different...’). Before reading the sentence, participants inspected a visual context with two playing cards that moved either far apart or close together. In Experiment 1, the cards turned and showed the first two nouns of the sentence (e.g., ‘peace’, ‘war’). In Experiments 2 and 3, they turned but remained blank. Participants’ reading times at the adjective (Experiment 1: first-pass reading time; Experiment 2: total times) and at the second noun phrase (Experiment 3: first-pass times) were faster for sentences that expressed similarity when the preceding words/objects were close together (vs. far apart) and for sentences that expressed dissimilarity when the preceding words/objects were far apart (vs. close together). Thus, spatial distance between words or entirely unrelated objects can rapidly and incrementally modulate the semantic interpretation of abstract sentences.     

The processing of lexical ambiguity: homonymy and polysemy in the mental lexicon

Under the theoretical assumption that lexical ambiguity is not a homogeneous phenomenon, but rather that it is subdivided into two distinct types, namely homonymy and polysemy, the present study investigated whether these different types of lexical ambiguity are psychologically real. Four types of ambiguous words, homonymous words (e.g., "pen"), polysemous words with metaphorical extensions (e.g., "eye"), polysemous words with a count/mass metonymic extension (e.g., "turkey"), and polysemous words with a producer/product metonymic extension (e.g., "Dali"), were used in a cross-modal sentence-priming lexical decision task. Overall, the theoretical distinction between homonymy and polysemy was reflected in the results of the present study, which revealed differential processing depending on the type of ambiguity.     

Feature overlap slows lexical selection: Evidence from the picture–word interference paradigm

How does the presence of a categorically related word influence picture naming latencies? In order to test competitive and noncompetitive accounts of lexical selection in spoken word production, we employed the picture–word interference (PWI) paradigm to investigate how conceptual feature overlap influences naming latencies when distractors are category coordinates of the target picture. Mahon et al. (2007. Lexical selection is not by competition: A reinterpretation of semantic interference and facilitation effects in the picture-word interference paradigm. Journal of Experimental Psychology. Learning, Memory, and Cognition, 33(3), 503–535. doi:10.1037/0278-7393.33.3.503) reported that semantically close distractors (e.g., zebra) facilitated target picture naming latencies (e.g., HORSE) compared to far distractors (e.g., whale). We failed to replicate a facilitation effect for within-category close versus far target–distractor pairings using near-identical materials based on feature production norms, instead obtaining reliably larger interference effects (Experiments 1 and 2). The interference effect did not show a monotonic increase across multiple levels of within-category semantic distance, although there was evidence of a linear trend when unrelated distractors were included in analyses (Experiment 2). Our results show that semantic interference in PWI is greater for semantically close than for far category coordinate relations, reflecting the extent of conceptual feature overlap between target and distractor. These findings are consistent with the assumptions of prominent competitive lexical selection models of speech production.