Age effects in a computational model of memory

Previous research demonstrates that semantic priming is relatively unaffected by age (Chiarello, Church, & Hoyer, 1985; Howard, 1988). To determine how age might affect representations in the HAL model of memory, the authors gathered text from older and younger adults and generated a global co-occurrence matrix for each. An analysis demonstrated that, as in humans, there was little difference in measures of semantic priming between memory matrices constructed from the two corpora. However, the authors discovered that semantic word neighborhoods generated by the older adult corpus were denser than corresponding neighborhoods derived from the younger adult corpus. This result has implications for changes in the semantic representation of words due to the aging process.     

WINDSOR: Windsor improved norms of distance and similarity of representations of semantics

Lexical co-occurrence models of semantic memory form representations of the meaning of a word on the basis of the number of times that pairs of words occur near one another in a large body of text. These models offer a distinct advantage over models that require the collection of a large number of judgments from human subjects, since the construction of the representations can be completely automated. Unfortunately, word frequency, a well-known predictor of reaction time in several cognitive tasks, has a strong effect on the co-occurrence counts in a corpus. Two words with high frequency are more likely to occur together purely by chance than are two words that occur very infrequently. In this article, we examine a modification of a successful method for constructing semantic representations from lexical co-occurrence. We show that our new method eliminates the influence of frequency, while still capturing the semantic characteristics of words.     

Exploring lexical co-occurrence space using HiDEx

Hyperspace analog to language (HAL) is a high-dimensional model of semantic space that uses the global co-occurrence frequency of words in a large corpus of text as the basis for a representation of semantic memory. In the original HAL model, many parameters were set without any a priori rationale. We have created and publicly released a computer application, the High Dimensional Explorer (HiDEx), that makes it possible to systematically alter the values of these parameters to examine their effect on the co-occurrence matrix that instantiates the model. We took an empirical approach to understanding the influence of the parameters on the measures produced by the models, looking at how well matrices derived with different parameters could predict human reaction times in lexical decision and semantic decision tasks. New parameter sets give us measures of semantic density that improve the model’s ability to predict behavioral measures. Implications for such models are discussed.     

Higher-order semantic structures in an African Grey parrot’s vocalizations: evidence from the hyperspace analog to language (HAL) model

Previous research has described the significant role that social interaction plays in both the acquisition and use of speech by parrots. The current study analyzed the speech of one home-raised African Grey parrot (Psittacus erithacus erithacus) across three different social contexts: owner interacting with parrot in the same room, owner and parrot interacting out of view in adjacent rooms, and parrot home alone. The purpose was to determine the extent to which the subject’s speech reflected an understanding of the contextual substitutability (e.g., the word street can be substituted in context for the word road) of the vocalizations that comprised the units in her repertoire (i.e., global co-occurrence of repertoire units; Burgess in Behav Res Methods Instrum Comput 30:188–198, 1998; Lund and Burgess in Behav Res Methods Instrum Comput 28:203–208, 1996). This was accomplished via the human language model hyperspace analog to language (HAL). HAL is contextually driven and bootstraps language “rules” from input without human intervention. Because HAL does not require human tutelage, it provided an objective measure to empirically examine the parrot’s vocalizations. Results indicated that the subject’s vocalization patterns did contain global co-occurrence. The presence of this quality in this nonhuman’s speech may be strongly indicative of higher-order cognitive skills.     

Imageability and distributional typicality measures of nouns and verbs in contemporary English

Dissociations between noun and verb processing are not uncommon after brain injury; yet, precise psycholinguistic comparisons of nouns and verbs are hampered by the underrepresentation of verbs in published semantic word norms and by the absence of contemporary estimates for part-of-speech usage. We report herein imageability ratings and rating response times (RTs) for 1,197 words previously categorized as pure nouns, pure verbs, or words of balanced noun-verb usage on the basis of the Francis and Ku?era (1982) norms. Nouns and verbs differed in rated imageability, and there was a stronger correspondence between imageability rating and RT for nouns than for verbs. For all word types, the image-rating-RT function implied that subjects employed an image generation process to assign ratings. We also report a new measure of noun-verbtypicality that used the Hyperspace Analog to Language (HAL; Lund & Burgess, 1996) context vectors (derived from a large sample of Usenet text) to compute the mean context distance between each word and all of thepure nouns andpure verbs. For a subset of the items, the resulting HAL noun-verb difference score was compared with part-of-speech usage in a representative sample of the Usenet corpus. It is concluded that this score can be used to estimate the extent to which a given word occurs in typical noun or verb sentence contexts in informal contemporary English discourse. The item statistics given in Appendix B will enable experimenters to select representative examples of nouns and verbs or to compare typical with atypical nouns (or verbs), while holding constant or covarying rated imageability.     

Effects of chronic non-clinical depression on the use of positive and negative words in language contexts

The purpose of this study was to investigate the effects of chronic non-clinical depression (ChD) on the use of ambiguous and unambiguous emotional words in language contexts. Participants with ChD and normal non-depressed (ND) participants' responses to positive and negative stimulus words on a free association word test were compared using the Hyperspace Analogue to Language (HAL) model of memory's context density metric, a measure of the range of contexts in which a word is used in language. ChD participants generated negative words that were used in more restricted contexts than ND participants. However, ND and ChD participants used positive words in a similar range of contexts. The results suggest that people with ChD tend to use negative words that are probably more negative in connotation, yet they are not impaired with respect to using words in contexts where positive words are required.     

From simple associations to the building blocks of language: Modeling meaning in memory with the HAL model

This paper presents a theoretical approach of how simple, episodic associations are transduced into semantic and grammatical categorical knowledge. The approach is implemented in the hyperspace analogue to language (HAL) model of memory, which uses a simple global co-occurrence learning algorithm to encode the context in which words occur. This encoding is the basis for the formation of meaning representations in a high-dimensional context space. Results are presented, and the argument is made that this simple process can ultimately provide the language-comprehension system with semantic and grammatical information required in the comprehension process.     

The effect of corpus size in predicting reaction time in a basic word recognition task: Moving on from Kučera and Francis

Word frequency is one of the strongest determiners of reaction time (RT) in word recognition tasks; it is an important theoretical and methodological variable. The Ku?era and Francis (1967) word frequency count (derived from the 1-million-word Brown corpus) is used by most investigators concerned with the issue of word frequency. Word frequency estimates from the Brown corpus were compared with those from a 131-million-word corpus (the HAL corpus; conversational text gathered from Usenet) in a standard word naming task with 32 subjects. RT was predicted equally well by both corpora for high-frequency words, but the larger corpus provided better predictors for low- and medium-frequency words. Furthermore, the larger corpus provides estimates for 97,261 lexical items; the smaller corpus, for 50,406 items.     

A computational approach to modeling population differences

Four experiments were conducted to determine whether the Hyperspace Analogue to Language (HAL) model of semantic memory could differentiate between two different populations. An analysis of the differences in densities (or average distances between word neighbors in semantic space) in HAL matrices—generated from text corpora derived from younger and older adults—confirmed that HAL was able to distinguish between the two age groups. This difference was again detected when structured interview data were used to build the corpora. A third experiment, designed to test the specificity of HAL in detecting differences between groups, did not detect any difference in the densities of the memory representations when older adults generated both the test corpora. The final experiment, conducted on the language of adults with Alzheimer’s and normal adults, again demonstrated that HAL could discriminate between the two populations. These results suggest that HAL is capable of modeling, on the basis of changes in mean density, some of the differences between populations without modifying the model itself but, rather, by changing the text corpus from which the model creates its representations in semantic space.     

Semantic distance norms computed from an electronic dictionary (WordNet).

WordNet, an electronic dictionary (or lexical database), is a valuable resource for computational and cognitive scientists. Recent work on the computing of semantic distances among nodes (synsets) in WordNet has made it possible to build a large database of semantic distances for use in selecting word pairs for psychological research. The database now contains nearly 50,000 pairs of words that have values for semantic distance, associative strength, and similarity based on co-occurrence. Semantic distance was found to correlate weakly with these other measures but to correlate more strongly with another measure of semantic relatedness, featural similarity. Hierarchical clustering analysis suggested that the knowledge structure underlying semantic distance is similar in gross form to that underlying featural similarity. In experiments in which semantic similarity ratings were used, human participants were able to discriminate semantic distance. Thus, semantic distance as derived from WordNet appears distinct from other measures of word pair relatedness and is psychologically functional. This database may be downloaded from www.psychonomic.org/archive/.     

Semantic distance norms computed from an electronic dictionary (WordNet)

WordNet, an electronic dictionary (or lexical database), is a valuable resource for computational and cognitive scientists. Recent work on the computing of semantic distances among nodes (synsets) in WordNet has made it possible to build a large database of semantic distances for use in selecting word pairs for psychological research. The database now contains nearly 50,000 pairs of words that have values for semantic distance, associative strength, and similarity based on co-occurrence. Semantic distance was found to correlate weakly with these other measures but to correlate more strongly with another measure of semantic relatedness, featural similarity. Hierarchical clustering analysis suggested that the knowledge structure underlying semantic distance is similar in gross form to that underlying featural similarity. In experiments in which semantic similarity ratings were used, human participants were able to discriminate semantic distance. Thus, semantic distance as derived from WordNet appears distinct from other measures of word pair relatedness and is psychologically functional. This database may be downloaded fromwww.psychonomic.org/archive/.     

Integrating constraints for learning word-referent mappings

Learning word-referent mappings is complex because the word and its referent tend to co-occur with multiple other words and potential referents. Such complexity has led to proposals for a host of constraints on learning, though how these constraints may interact has not yet been investigated in detail. In this paper, we investigated interactions between word co-occurrence constraints and cross-situational statistics in word learning. Analyses of child-directed speech revealed that when both object-referring and non-referring words occurred in the utterance, referring words were more likely to be preceded by a determiner than when the utterance contained only referring words. In a word learning study containing both referring and non-referring words, learning was facilitated when non-referring words contributed grammatical constraints analogous to determiners. The complexity of multi-word utterances provides an opportunity for co-occurrence constraints to contribute to word-referent mapping, and the learning mechanism is able to integrate these multiple sources of information.     

Saccade launch site as a predictor of fixation durations in reading: comments on Hand, Miellet, O'Donnell, and Sereno (2010)

An important question in research on eye movements in reading is whether word frequency and word predictability have additive or interactive effects on fixation durations. A fair number of studies have reported only additive effects of the frequency and predictability of a target word on reading times on that word, failing to show significant interactions. Recently, however, Hand, Miellet, O'Donnell, and Sereno (see record 2010-19099-001) reported interactive effects in a study that included the distance of the prior fixation from the target word (launch site). They reported that when the saccade into the target word was launched from very near to the word (within 3 characters), the predictability effect was larger for low frequency words, but when the saccade was launched from a medium distance (4-6 characters from the word) the predictability effect was larger for high frequency words. Hand et al. argued for the importance of including launch site in analyses of target word fixation durations. Here we describe several problems with Hand et al.'s use of analyses of variance in which launch site is divided into distinct ordinal levels. We describe a more appropriate way to analyze such data-linear mixed-effect models-and we use this method to show that launch site does not modulate the interaction between frequency and predictability in two other data sets.     

Semantic boost on episodic associations: an empirically-based computational model

Words become associated following repeated co-occurrence episodes. This process might be further determined by the semantic characteristics of the words. The present study focused on how semantic and episodic factors interact in incidental formation of word associations. First, we found that human participants associate semantically related words more easily than unrelated words; this advantage increased linearly with repeated co-occurrence. Second, we developed a computational model, SEMANT, suggesting a possible mechanism for this semantic-episodic interaction. In SEMANT, episodic associations are implemented through lateral connections between nodes in a pre-existent self-organized map of word semantics. These connections are strengthened at each instance of concomitant activation, proportionally with the amount of the overlapping activity waves of activated nodes. In computer simulations SEMANT replicated the dynamics of associative learning in humans and led to testable predictions concerning normal associative learning as well as impaired learning in a diffuse semantic system like that characteristic of schizophrenia.     

Distributional typicality: a new approach to estimating noun and verb usage from large scale text corpora

This paper reports a new approach to estimating the extent to which words have predominant noun and verb usages which do not require human judgments about parts of speech. The Hyperspace Analog to Language model (HAL, Lund & Burgess, 1996) was used to computationally estimate noun vs verb usage based on the statistical regularities present in a large-scale electronic text corpus. This measure can be used to estimate the extent to which a given word occurs in typical noun or verb sentence contexts (i.e., its distributional typicality) in informal contemporary discourse.     

The University of South Florida free association, rhyme, and word fragment norms.

Preexisting word knowledge is accessed in many cognitive tasks, and this article offers a means for indexing this knowledge so that it can be manipulated or controlled. We offer free association data for 72,000 word pairs, along with over a million entries of related data, such as forward and backward strength, number of competing associates, and printed frequency. A separate file contains the 5,019 normed words, their statistics, and thousands of independently normed rhyme, stem, and fragment cues. Other files provide n x n associative networks for more than 4,000 words and a list of idiosyncratic responses for each normed word. The database will be useful for investigators interested in cuing, priming, recognition, network theory, linguistics, and implicit testing applications. They also will be useful for evaluating the predictive value of free association probabilities as compared with other measures, such as similarity ratings and co-occurrence norms. Of several procedures for measuring preexisting strength between two words, the best remains to be determined. The norms may be downloaded from www.psychonomic.org/archive/.     

Cross-lingual numerical distance priming with second-language number words in native- to third-language number word translation

An important question in research about bilingualism is whether translation from a second language (L2) to the native language (L1) is semantically mediated or whether it occurs through word associations at the lexical level. Recent research has shown that both L1 and L2 number word translation imply semantic access, suggesting strong L2 lexicosemantic mappings for number words (Duyck & Brysbaert, 2004). In the present study, this assumption was further explored by means of a cross-lingual number distance priming paradigm with Dutch–English–French trilinguals. We found that number word translation from L1 to the third language (L3), and vice versa, was facilitated by L2 number masked word primes that represented the same magnitude as the target (translation equivalents), relative to primes that are numerically less close. This confirms the existence of strong L2 lexicosemantic mappings for number words and generalizes previous semantic effects in L1–L2 translation to translation processes between L1 and L3.     

Detection and identification of words and letters in simulated visual search of word lists

Ss were shown a rapid sequence of words and had to (a) make a speeded response to the presentation of a predefined target and (b) report a "response word" which they thought immediately followed the target in the sequence. In Experiment I presentation rate was varied orthogonally with the number of alternative targets. Detection errors and latency increased with target set size, as did the distance of the response word from the target in the list sequence. Increases in presentation rate produced greater target-response distance without affecting detection time. In Experiment II Ss were in some conditions given only the initial letter of the targets; the response could be either the whole word following the target or only its initial letter. The results indicated that Ss could, within limits, concurrently detect initial letters and identify words. Alternatives to hierarchical-type models of stimulus processing in visual search were discussed.     

Syllabic length effects in visual word recognition and naming

Two experiments investigated the role of the number of syllables in visual word recognition and naming. Experiment 1 (word and nonword naming) showed that effects of number of syllables on naming latencies were observed for nonwords and very low-frequency words but not for high-frequency words. In Experiment 2 (lexical decision), syllabic length effects were also obtained for very low-frequency words but not for high-frequency words and nonwords. These results suggest that visual word recognition and naming do require syllabic decomposition, at least for very low-frequency words in French. These data are compatible with the multiple-trace memory model for polysyllabic word reading [Psychol. Rev. 105 (1998) 678]. In this model, reading depends on the activity of two procedures: (1) a global procedure that operates in parallel across a letter string (and does not generate a strong syllabic length effect) and that is the predominant process in generating responses to high-frequency words, and (2) an analytic procedure that operates serially across a letter string (and generates a strong syllabic length effect) and that is the predominant process in generating responses to very low-frequency words. A modified version of the dual route cascaded model [Psychol. Rev. 108 (1) (2001) 204] can also explain the present results, provided that syllabic units are included in this model. However, the Parallel Distributed Processing model [Psychol. Rev. 96 (1989) 523; J. Exp. Psychol.: Human Perception Perform. 16 (1990) 92] has difficulties to account for these results.     

Frequent frames as a cue for grammatical categories in child directed speech

This paper introduces the notion of frequent frames, distributional patterns based on co-occurrence patterns of words in sentences, then investigates the usefulness of this information in grammatical categorization. A frame is defined as two jointly occurring words with one word intervening. Qualitative and quantitative results from distributional analyses of six different corpora of child directed speech are presented in two experiments. In the analyses, words that were surrounded by the same frequent frame were categorized together. The results show that frequent frames yield very accurate categories. Furthermore, evidence from behavioral studies suggests that infants and adults are sensitive to frame-like units, and that adults use them to categorize words. This evidence, along with the success of frames in categorizing words, provides support for frames as a basis for the acquisition of grammatical categories.