Entropy,semantic relatedness and proximity

Although word co-occurrences within a document have been demonstrated to be semantically useful, word interactions over a local range have been largely neglected by psychologists due to practical challenges. Shannon’s (Bell Systems Technical Journal, 27, 379–423, 623–665, 1948) conceptualization of information theory suggests that these interactions should be useful for understanding communication. Computational advances make an examination of local word–word interactions possible for a large text corpus. We used Brants and Franz’s (2006) dataset to generate conditional probabilities for 62,474 word pairs and entropy calculations for 9,917 words in Nelson, McEvoy, and Schreiber’s (Behavior Research Methods, Instruments, & Computers, 36, 402–407, 2004) free association norms. Semantic associativity correlated moderately with the probabilities and was stronger when the two words were not adjacent. The number of semantic associates for a word and the entropy of a word were also correlated. Finally, language entropy decreases from 11 bits for single words to 6 bits per word for four-word sequences. The probabilities and entropies discussed here are included in the supplemental materials for the article.     

Speech parts as Poisson processes

This paper presents evidence that six of the seven parts of speech occur in written text as Poisson processes, simple or recurring. The six major parts are nouns, verbs, adjectives, adverbs, prepositions, and conjunctions, with the interjection occurring too infrequently to support a model. The data consist of more than the first 5000 words of works by four major authors coded to label the parts of speech, as well as periods (sentence terminators). Sentence length is measured via the period and found to be normally distributed with no stochastic model identified for its occurrence. The models for all six speech parts but the noun significantly distinguish some pairs of authors and likewise for the joint use of all words types. Any one author is significantly distinguished from any other by at least one word type and sentence length very significantly distinguishes each from all others. The variety of word type use, measured by Shannon entropy, builds to about 90% of its maximum possible value. The rate constants for nouns are close to the fractions of maximum entropy achieved. This finding together with the stochastic models and the relations among them suggest that the noun may be a primitive organizer of written text.     

Modulation of working memory updating: Does long-term memory lexical association matter?

The aim of the present study was to investigate how working memory updating for verbal material is modulated by enduring properties of long-term memory. Two coexisting perspectives that account for the relation between long-term representation and short-term performance were addressed. First, evidence suggests that performance is more closely linked to lexical properties, that is, co-occurrences within the language. Conversely, other evidence suggests that performance is linked more to long-term representations which do not entail lexical/linguistic representations. Our aim was to investigate how these two kinds of long-term memory associations (i.e., lexical or nonlexical) modulate ongoing working memory activity. Therefore, we manipulated (between participants) the strength of the association in letters based on either frequency of co-occurrences (lexical) or contiguity along the sequence of the alphabet (nonlexical). Results showed a cost in working memory updating for strongly lexically associated stimuli only. Our findings advance knowledge of how lexical long-term memory associations between consonants affect working memory updating and, in turn, contribute to the study of factors which impact the updating process across memory systems.     

Adult Age Differences in Memory for Verbs and Nouns

Younger and older adults were asked to remember noun pairs (e.g., head – cap), verb pairs (e.g., bounce – throw), and verb-noun pairs (e.g., break – stick). For half of the pairs, participants used imagined objects and performed an action or series of related actions for each pair. For the other half of the pairs, participants read but did not perform the pairs. Free recall and cued recall tests revealed that age differences in memory for both performed and nonperformed items were larger for verbs than for nouns. The recall advantage of nouns over verbs was larger for older than for younger adults. Verbs are hypothesized to be more difficult for older adults to remember because they are more and less specific than nouns and because it is more difficult to integrate verbs with other words than to integrate nouns with other words.     

The Dependency Structure of Coordinate Phrases: A Corpus Approach

Hudson (1990) proposes that each conjunct in a coordinate phrase forms dependency relations with heads or dependents outside the coordinate phrase (the “multi-head” view). This proposal is tested through corpus analysis of Wall Street Journal text. For right-branching constituents (such as direct-object NPs), a short-long preference for conjunct ordering is observed; this is predicted by the multi-head view, under the assumption that structures resulting in shorter dependencies are preferred. A short-long preference is also observed for left-branching constituents (such as subject NPs), which is less obviously accommodated by the multi-head view but not incompatible with it. The repetition of determiners was also examined (the dog and cat versus the dog and the cat), and a stronger preference was found for repetition with singular count nouns as opposed to mass or plural nouns; this accords well with the multi-head view, under the reasoning that single-determiner constructions require crossing dependencies with count nouns but not with plural or mass nouns.     

Dutch modality exclusivity norms: Simulating perceptual modality in space

Perceptual information is important for the meaning of nouns. We present modality exclusivity norms for 485 Dutch nouns rated on visual, auditory, haptic, gustatory, and olfactory associations. We found these nouns are highly multimodal. They were rated most dominant in vision, and least in olfaction. A factor analysis identified two main dimensions: one loaded strongly on olfaction and gustation (reflecting joint involvement in flavor), and a second loaded strongly on vision and touch (reflecting joint involvement in manipulable objects). In a second study, we validated the ratings with similarity judgments. As expected, words from the same dominant modality were rated more similar than words from different dominant modalities; but – more importantly – this effect was enhanced when word pairs had high modality strength ratings. We further demonstrated the utility of our ratings by investigating whether perceptual modalities are differentially experienced in space, in a third study. Nouns were categorized into their dominant modality and used in a lexical decision experiment where the spatial position of words was either in proximal or distal space. We found words dominant in olfaction were processed faster in proximal than distal space compared to the other modalities, suggesting olfactory information is mentally simulated as “close” to the body. Finally, we collected ratings of emotion (valence, dominance, and arousal) to assess its role in perceptual space simulation, but the valence did not explain the data. So, words are processed differently depending on their perceptual associations, and strength of association is captured by modality exclusivity ratings.     

Imageability and distributional typicality measures of nouns and verbs in contemporary English

Dissociations between noun and verb processing are not uncommon after brain injury; yet, precise psycholinguistic comparisons of nouns and verbs are hampered by the underrepresentation of verbs in published semantic word norms and by the absence of contemporary estimates for part-of-speech usage. We report herein imageability ratings and rating response times (RTs) for 1,197 words previously categorized as pure nouns, pure verbs, or words of balanced noun-verb usage on the basis of the Francis and Ku?era (1982) norms. Nouns and verbs differed in rated imageability, and there was a stronger correspondence between imageability rating and RT for nouns than for verbs. For all word types, the image-rating-RT function implied that subjects employed an image generation process to assign ratings. We also report a new measure of noun-verbtypicality that used the Hyperspace Analog to Language (HAL; Lund & Burgess, 1996) context vectors (derived from a large sample of Usenet text) to compute the mean context distance between each word and all of thepure nouns andpure verbs. For a subset of the items, the resulting HAL noun-verb difference score was compared with part-of-speech usage in a representative sample of the Usenet corpus. It is concluded that this score can be used to estimate the extent to which a given word occurs in typical noun or verb sentence contexts in informal contemporary English discourse. The item statistics given in Appendix B will enable experimenters to select representative examples of nouns and verbs or to compare typical with atypical nouns (or verbs), while holding constant or covarying rated imageability.     

The relation between visual imagery mediators and recall

Three hypotheses are discussed as explanations for the result that pairs of concrete nouns are more easily remembered than are pairs of abstract nouns: the imagery hypothesis, the familiarity hypothesis, and the concreteness hypothesis. Two experiments are reported in which the degree of visual imagery associated with the components of paired associate items was not indicative of the degree of visual imagery experienced during their learning or with the accuracy with which they were recalled. It was found that pairs of related abstract nouns were rated higher in imagery and familiarity than were pairs of unrelated concrete nouns, but recall of the higher imagery pairs was poorer. The concreteness hypothesis is discussed as the best explanation for the results. The concreteness hypothesis proposes that people learn to associate the labels of concrete objects by using their real-world knowledge of the potential relations between categories of objects. Dual coding theory and schema theory are also discussed as explanations for mediation learning, and the issue of visual imagery as an epiphenomenon is addressed.     

Pair-relational encoding of performed nouns and verbs

Summary Subjects had to learn lists of noun pairs and verb pairs. They were informed in advance about the test types and were tested for free recall (FR) and cued recall (CR). Three classes of encoding instructions were used: standard learning instructions, item-specific enactment instructions (to perform the denoted action of the verb or a typical action for the noun, and to do the same plus finding separate goals for the two elements of each pair), and enactment instructions that were completed by explicit instructions to integrate the word pairs (find a common goal, and find a common goal plus rating your success). There was no effect of encoding instructions on FR of nouns. There was a better FR under all enactment instructions than under standard instructions for verbs. CR decreased after item-specific enactment instructions, in contrast with standard learning instructions, but more for nouns than for verbs. CR increased after the instructions to integrate the pairs, in contrast with item-specific enactment instructions, but more for nouns than for verbs. It was concluded that enactment provides excellent item-specific information that can hardly be enhanced further, and that the item-specific information provided by concrete nouns is fundamentally good and is difficult to enhance by enactment. It is further assumed that enactment not only provides excellent item-specific information, but also hinders pair integration. Therefore, CR decreases after enactment. This decrease can only be overcome when subjects actively try to integrate the word pairs.     

The accessibility of characters in single sentences: proper names,common nouns,and first mention

Accessibility of characters in two-character sentences (e.g., The butler helped Calvin at the wedding reception) was investigated with a probe recognition task. Probes were either the first character (e.g., butler) or the second character (e.g., Calvin) in a sentence and were designated by proper names or common nouns crossed with name or noun nonprobes. Results show that (1) probes in first position are more accessible than those in second position, but not when noun probes are paired with name nonprobes, (2) characters designated by names are generally more accessible than those designated by nouns, and (3) the first name in a sentence is more available than other characters, regardless of position. Thus, accessibility of characters in a sentence seems dependent on discourse function, with named characters seen as main characters, rather than on nondiscourse-related factors, such as temporal distinctiveness.     

Modifiers in metaphors: Effects of constituent phrase similarity on the interpretation of figurative sentences

Previous research has shown that the ease of metaphor interpretation and judgments of metaphor goodness are correlated with the degree of similarity between the two nouns linked in a metaphor. This study was designed to investigate the effects of adding adjective modifiers to the nouns constituting metaphorical sentences. Four types of associative relationships, between adjectives and nouns were defined. It was found that different patterns of adjective modification influenced constituent phrase similarity (e.g., the ADJECTIVE-NOUN_A is an ADJECTIVE-NOUN_B), and such differences were consistent with changes in metaphor goodness and interpretability. However, the intercorrelations among these variables were a function of the level of similarity between unmodified constituent nouns. With initially similar constituent nouns, the three variables were about equally intercorrelated. With initially dissimilar constituent nouns, constituent phrase similarity and metaphor goodness were highly correlated, but interpretability was not predictable from a linear model. Results are discussed in terms of a cognitive-feature model of association and metaphor processing.     

Contextual self-organizing map: software for constructing semantic representations

In this article, we introduce a software package that applies a corpus-based algorithm to derive semantic representations of words. The algorithm relies on analyses of contextual information extracted from a text corpus—specifically, analyses of word co-occurrences in a large-scale electronic database of text. Here, a target word is represented as the combination of the average of all words preceding the target and all words following it in a text corpus. The semantic representation of the target words can be further processed by a self-organizing map (SOM; Kohonen, Self-organizing maps, 2001), an unsupervised neural network model that provides efficient data extraction and representation. Due to its topography-preserving features, the SOM projects the statistical structure of the context onto a 2-D space, such that words with similar meanings cluster together, forming groups that correspond to lexically meaningful categories. Such a representation system has its applications in a variety of contexts, including computational modeling of language acquisition and processing. In this report, we present specific examples from two languages (English and Chinese) to demonstrate how the method is applied to extract the semantic representations of words.     

Words in a sentence become less accessible when an anaphor is resolved

A noun that identifies an entity in a discourse becomes less accessible following an anaphoric reference to another entity. The phenomenon cannot be attributed to ad hoc strategies, memory decay, or context checking. It occurs for both common and proper nouns and for nouns that identify both characters and inanimate objects. It is stronger for nouns that identify important entities, as opposed to more peripheral ones.     

Data from eye-tracking corpora as evidence for theories of syntactic processing complexity

We evaluate the predictions of two theories of syntactic processing complexity, dependency locality theory (DLT) and surprisal, against the Dundee Corpus, which contains the eye-tracking record of 10 participants reading 51,000 words of newspaper text. Our results show that DLT integration cost is not a significant predictor of reading times for arbitrary words in the corpus. However, DLT successfully predicts reading times for nouns. We also find evidence for integration cost effects at auxiliaries, not predicted by DLT. For surprisal, we demonstrate that an unlexicalized formulation of surprisal can predict reading times for arbitrary words in the corpus. Comparing DLT integration cost and surprisal, we find that the two measures are uncorrelated, which suggests that a complete theory will need to incorporate both aspects of processing complexity. We conclude that eye-tracking corpora, which provide reading time data for naturally occurring, contextualized sentences, can complement experimental evidence as a basis for theories of processing complexity.     

Adjacent and Non-Adjacent Word Contexts Both Predict Age of Acquisition of English Words: A Distributional Corpus Analysis of Child-Directed Speech

Children show a remarkable degree of consistency in learning some words earlier than others. What patterns of word usage predict variations among words in age of acquisition? We use distributional analysis of a naturalistic corpus of child-directed speech to create quantitative features representing natural variability in word contexts. We evaluate two sets of features: One set is generated from the distribution of words into frames defined by the two adjacent words. These features primarily encode syntactic aspects of word usage. The other set is generated from non-adjacent co-occurrences between words. These features encode complementary thematic aspects of word usage. Regression models using these distributional features to predict age of acquisition of 656 early-acquired English words indicate that both types of features improve predictions over simpler models based on frequency and appearance in salient or simple utterance contexts. Syntactic features were stronger predictors of children's production than comprehension, whereas thematic features were stronger predictors of comprehension. Overall, earlier acquisition was predicted by features representing frames that select for nouns and verbs, and by thematic content related to food and face-to-face play topics; later acquisition was predicted by features representing frames that select for pronouns and question words, and by content related to narratives and object play.     

Hemispheric dissociation in judging semantic relations: complementarity for close and distant associates.

In two lateralized tachistoscopic experiments, we presented (i) pairs of nouns with close or distant semantic associations or (ii) pairs of nouns which were randomly matched and later rated by the subjects as to their semantic distance. In both experiments, words presented to the right visual field were more frequently judged as semantically close in meaning than words presented to the left visual field (LVF), whereas words presented to the LVF were more frequently judged as semantically distant. The results are discussed in relation to hemispheric language functions and current models of cerebral laterality.     

Temporal associative processes revealed by intrusions in paired-associate recall

Although much is known about the factors that influence the acquisition and retention of individual paired associates, the existence of temporally defined associations spanning multiple pairs has not been demonstrated. We report two experiments in which subjects studied randomly paired nouns for a subsequent cued recall test. When subjects recalled nontarget items, their intrusions tended to come from nearby pairs. This across-pair contiguity effect was graded, spanning noncontiguously studied word pairs. The existence of such long-range temporally defined associations lends further support to contextual-retrieval models of episodic association.     

Interword spacing in Chinese text layout

Three experiments using Chinese text were conducted to investigate word spacing and its effect on reading performance. In Exp. 1, a sonogram detector was used to analyze interword and intercharacter (within a word) time intervals from text read aloud by professional TV broadcasters versus college graduates. The results showed interword intervals were significantly longer than intercharacter intervals, indicating that interword spacing has psychological reality in speech. Exp. 2 examined the effect on reading performance due to separating the characters that compose a word. Separating the characters of a word did not decrease reading accuracy but did result in significantly longer reading times. Exp. 3 explored the effect of word spacing in Chinese sentences on reading performance. Analysis showed that word spacing did not affect reading accuracy, but half character and whole-character spacing significantly reduced reading time. The results of the present study suggest that word spacing in Chinese text layout enhances reading performance. Word spacing may help the reader to segment more quickly a string of characters into words and reduce the likelihood of misinterpretation. Also, ambiguity of sentence structure severely degraded reading accuracy. The implications of the results for word spacing design in Chinese text are discussed.     

Developmental differences in sensitivity to semantic relations among good and poor comprehenders: evidence from semantic priming

Semantic priming for category coordinates (e.g. CAT-DOG; AEROPLANE-TRAIN) and for pairs of words related through function (e.g. BROOM-FLOOR; SHAMPOO-HAIR) was assessed in children with good and poor reading comprehension, matched for decoding skill. Lexical association strength was also manipulated by comparing pairs of words that were highly associated with pairs that shared low association strength. Both groups of children showed priming for function-related words, but for the category co-ordinates, poor comprehenders only showed priming if the category pairs also shared high association strength. Good comprehenders showed priming for category-related targets, irrespective of the degree of prime-target association. These findings are related to models of language development in which category knowledge is gradually abstracted and refined from children's event-based knowledge and it is concluded that in the absence of explicit co-occurrence, poor comprehenders are less sensitive to abstract semantic relations than normal readers.