Exploring What Is Encoded in Distributional Word Vectors: A Neurobiologically Motivated Analysis

The pervasive use of distributional semantic models or word embeddings for both cognitive modeling and practical application is because of their remarkable ability to represent the meanings of words. However, relatively little effort has been made to explore what types of information are encoded in distributional word vectors. Knowing the internal knowledge embedded in word vectors is important for cognitive modeling using distributional semantic models. Therefore, in this paper, we attempt to identify the knowledge encoded in word vectors by conducting a computational experiment using Binder et al.'s (2016) featural conceptual representations based on neurobiologically motivated attributes. In an experiment, these conceptual vectors are predicted from text-based word vectors using a neural network and linear transformation, and prediction performance is compared among various types of information. The analysis demonstrates that abstract information is generally predicted more accurately by word vectors than perceptual and spatiotemporal information, and specifically, the prediction accuracy of cognitive and social information is higher. Emotional information is also found to be successfully predicted for abstract words. These results indicate that language can be a major source of knowledge about abstract attributes, and they support the recent view that emphasizes the importance of language for abstract concepts. Furthermore, we show that word vectors can capture some types of perceptual and spatiotemporal information about concrete concepts and some relevant word categories. This suggests that language statistics can encode more perceptual knowledge than often expected.     

Speech parts as Poisson processes

This paper presents evidence that six of the seven parts of speech occur in written text as Poisson processes, simple or recurring. The six major parts are nouns, verbs, adjectives, adverbs, prepositions, and conjunctions, with the interjection occurring too infrequently to support a model. The data consist of more than the first 5000 words of works by four major authors coded to label the parts of speech, as well as periods (sentence terminators). Sentence length is measured via the period and found to be normally distributed with no stochastic model identified for its occurrence. The models for all six speech parts but the noun significantly distinguish some pairs of authors and likewise for the joint use of all words types. Any one author is significantly distinguished from any other by at least one word type and sentence length very significantly distinguishes each from all others. The variety of word type use, measured by Shannon entropy, builds to about 90% of its maximum possible value. The rate constants for nouns are close to the fractions of maximum entropy achieved. This finding together with the stochastic models and the relations among them suggest that the noun may be a primitive organizer of written text.     

The effect of oral vocabulary on reading visually novel words: a comparison of the dual-route-cascaded and triangle frameworks

Dual-route-cascaded (DRC) (e.g. Coltheart & Rastle, Journal of Experimental Psychology: Human Perception and Performance 20 (1994) 1197) and triangle framework (e.g. Seidenberg & McClelland, Psychological Review 96 (1989) 523) predictions were tested regarding the effect of having a word in oral vocabulary prior to reading that same word. Over two sessions, at intervals of 2--3 days, 44 Grade 1 (6--7-year-old) children were aurally familiarized with the sound and meaning of ten novel words (semantic oral instantiation), and with just the sound of another ten novel words (non-semantic oral instantiation). Two to three days later non-word naming performance was significantly more accurate for aurally trained novel words compared to pseudohomophones, which were in turn advantaged over untrained non-words. The semantic manipulation had no effect. Experiment 2 manipulated articulation during (non-semantic) training. Forty Grade 1 children participated. Again, aurally trained items were named more accurately and quickly than equivalently trained pseudohomophones, which were in turn advantaged over untrained non-words. The articulation manipulation had no effect. The results suggest that word-specific phonological information is represented in the reading system independently of semantic or articulatory influences. The results are interpreted as being problematic for both the DRC and triangle frameworks, but more so for the latter.     

无语境条件下汉语词类歧义词的意义激活

以单字词为实验材料,采用启动命名任务,在孤立词条件下考察了汉语词类歧义词不同意义激活的时间进程特点。实验发现,歧义词的两种意义都能被激活,但主要意义激活得较早,次要意义激活速度较慢;当两种意义都激活之后,次要意义的激活水平降低,而主要意义仍维持在一定激活水平。结果表明,汉语词类歧义词的多重激活过程会受到意义相对频率的影响。     

"修辞立其诚"的语义学诠释

“修辞立其减”本为孔子读《易》解经之语,但它却对后世的文学评论及修辞学产生了深刻影响,高频率的使用还使它凝结为词——“修辞立诚”。如此,“修辞”与“立诚”在结构上成为平行并列的两个词。但从修辞角度看,二者则有密切的从属关系:“诚”内“辞”外,“诚”主“辞”从,“辞”以“诚”存。“诚”在儒家思想中的主导地位也说明,修辞应以“诚”为本,“辞”要服从并服务于“诚”。所谓“修辞立其诚”,是指以“诚”为前提并以“诚”为主导的“修辞”。     

Promoting Generalization of Oral Reading Fluency: Providing Drill versus Practice Opportunities

Extensive evidence exists demonstrating the benefits of repeated readings (RR) interventions at increasing students’ fluency on intervention passages. Few studies however have examined the extent to which repeatedly reading one passage improves students’ reading fluency on similar passages. Using an alternating treatment design, we examined the extent to which two interventions resulted in improvements in students’ fluency on generalization passages. While both interventions incorporated RR, one intervention involved students reading one passage four times and the other involved students reading two similar passages each twice. Intervention effects were evaluated by having students read a generalization passage prior to and following intervention implementation. Results indicate that both interventions were effective in increasing students reading fluency on generalization passages. For 3 participants the RR intervention produced greater gains in fluency on the generalization passages, while data for the remaining 3 participants are inconclusive. Implications of these findings for practice and for better understanding application of the instructional hierarchy to the development of reading interventions are discussed.

Lexical configuration and lexical engagement: when adults learn new words

People know thousands of words in their native language, and each of these words must be learned at some time in the person's lifetime. A large number of these words will be learned when the person is an adult, reflecting the fact that the mental lexicon is continuously changing. We explore how new words get added to the mental lexicon, and provide empirical support for a theoretical distinction between what we call lexical configuration and lexical engagement. Lexical configuration is the set of factual knowledge associated with a word (e.g., the word's sound, spelling, meaning, or syntactic role). Almost all previous research on word learning has focused on this aspect. However, it is also critical to understand the process by which a word becomes capable of lexical engagement--the ways in which a lexical entry dynamically interacts with other lexical entries, and with sublexical representations. For example, lexical entries compete with each other during word recognition (inhibition within the lexical level), and they also support the activation of their constituents (top-down lexical-phonemic facilitation, and lexically-based perceptual learning). We systematically vary the learning conditions for new words, and use separate measures of lexical configuration and engagement. Several surprising dissociations in behavior demonstrate the importance of the theoretical distinction between configuration and engagement.     

言语吸引力影响因素探索：物理属性和社会特征

语音吸引力是个体吸引力的重要组成部分。为探索言语吸引力影响因素,以汉语词汇为材料,设计实验分析语音基频、共振峰等物理属性以及词义、声音受喜欢程度和语音性别等社会属性对语音吸引力的影响。结果表明：分析物理属性,基频、共振峰等并未能预测语音吸引力;分析社会属性,较之词语意义,声音本身具有的吸引力水平对言语吸引力评价影响更大;而男性和女性对两种性别言语吸引力评价差异不显著。言语吸引力评价在男性和女性间具有一致性：＂好声音＂比＂会说话＂更加容易被认为具有吸引力。     

A phonetic approach to consonant repetition in early words

The goal of this study was to evaluate movement-based principles for understanding early speech output patterns. Consonant repetition patterns within children's actual productions of word forms were analyzed using spontaneous speech data from 10 typically developing American-English learning children between 12 and 36 months of age. Place of articulation, word level patterns, and developmental trends in CVC and CVCV repeated word forms were evaluated. Labial and coronal place repetitions dominated. Regressive repetition (e.g., [gag] for “dog”) occurred frequently in CVC but not in CVCV word forms. Consonant repetition decreased over time. However, the children produced sound types available reported as being within young children's production system capabilities in consonant repetitions in all time periods. Findings suggest that a movement-based approach can provide a framework for comprehensively characterizing consonant place repetition patterns in early speech development.     

BALDEY: A database of auditory lexical decisions

In an auditory lexical decision experiment, 5541 spoken content words and pseudowords were presented to 20 native speakers of Dutch. The words vary in phonological make-up and in number of syllables and stress pattern, and are further representative of the native Dutch vocabulary in that most are morphologically complex, comprising two stems or one stem plus derivational and inflectional suffixes, with inflections representing both regular and irregular paradigms; the pseudowords were matched in these respects to the real words. The BALDEY (“biggest auditory lexical decision experiment yet”) data file includes response times and accuracy rates, with for each item morphological information plus phonological and acoustic information derived from automatic phonemic segmentation of the stimuli. Two initial analyses illustrate how this data set can be used. First, we discuss several measures of the point at which a word has no further neighbours and compare the degree to which each measure predicts our lexical decision response outcomes. Second, we investigate how well four different measures of frequency of occurrence (from written corpora, spoken corpora, subtitles, and frequency ratings by 75 participants) predict the same outcomes. These analyses motivate general conclusions about the auditory lexical decision task. The (publicly available) BALDEY database lends itself to many further analyses.