The differential role of phonological and distributional cues in grammatical categorisation

Recognising the grammatical categories of words is a necessary skill for the acquisition of syntax and for on-line sentence processing. The syntactic and semantic context of the word contribute as cues for grammatical category assignment, but phonological cues, too, have been implicated as important sources of information. The value of phonological and distributional cues has not, with very few exceptions, been empirically assessed. This paper presents a series of analyses of phonological cues and distributional cues and their potential for distinguishing grammatical categories of words in corpus analyses. The corpus analyses indicated that phonological cues were more reliable for less frequent words, whereas distributional information was most valuable for high frequency words. We tested this prediction in an artificial language learning experiment, where the distributional and phonological cues of categories of nonsense words were varied. The results corroborated the corpus analyses. For high-frequency nonwords, distributional information was more useful, whereas for low-frequency words there was more reliance on phonological cues. The results indicate that phonological and distributional cues contribute differentially towards grammatical categorisation.     

The secret is in the sound: from unsegmented speech to lexical categories

When learning language, young children are faced with many seemingly formidable challenges, including discovering words embedded in a continuous stream of sounds and determining what role these words play in syntactic constructions. We suggest that knowledge of phoneme distributions may play a crucial part in helping children segment words and determine their lexical category, and we propose an integrated model of how children might go from unsegmented speech to lexical categories. We corroborated this theoretical model using a two‐stage computational analysis of a large corpus of English child‐directed speech. First, we used transition probabilities between phonemes to find words in unsegmented speech. Second, we used distributional information about word edges – the beginning and ending phonemes of words – to predict whether the segmented words from the first stage were nouns, verbs, or something else. The results indicate that discovering lexical units and their associated syntactic category in child‐directed speech is possible by attending to the statistics of single phoneme transitions and word‐initial and final phonemes. Thus, we suggest that a core computational principle in language acquisition is that the same source of information is used to learn about different aspects of linguistic structure.     

Word category conversion causes processing costs: Evidence from adjectival passives

Constructing syntactic representations in language comprehension begins with the identification of word categories. Whether the category information is stored in the mental lexicon is a matter of debate in current linguistic theorizing. The standard view assumes that the syntactic category of a word is lexically specified (lexicalist approach). More recently, it has been proposed within the paradigm of distributed morphology that lexical elements (roots) are stored without any syntactic category information: The syntactic category of a lexical element is determined only by the syntactic context in which it appears (syntactic approach). For processing categoryambiguous words, different hypotheses can be derived from these two accounts. The lexicalist approach predicts that there are productive grammatical processes, such as nominalization and adjectivization, that convert a word of Category A into one of Category B. Such a conversion might be assumed to create additional processing costs. Within the syntactic approach, on the other hand, no additional processing step is expected, because there is no need for any category shift. In a self-paced reading study on so-called adjectival passives, we found evidence of costs predicted under the lexicalist approach (i.e., for a grammatical process that changes the category of a word). More specifically, the present study provides evidence for category conversion from a verbal participle into an adjectival one. We also discuss an alternative explanation for this finding in terms of frequency.     

Before the N400: effects of lexical-semantic violations in visual cortex

There exists an increasing body of research demonstrating that language processing is aided by context-based predictions. Recent findings suggest that the brain generates estimates about the likely physical appearance of upcoming words based on syntactic predictions: words that do not physically look like the expected syntactic category show increased amplitudes in the visual M100 component, the first salient MEG response to visual stimulation. This research asks whether violations of predictions based on lexical-semantic information might similarly generate early visual effects. In a picture-noun matching task, we found early visual effects for words that did not accurately describe the preceding pictures. These results demonstrate that, just like syntactic predictions, lexical-semantic predictions can affect early visual processing around ～100ms, suggesting that the M100 response is not exclusively tuned to recognizing visual features relevant to syntactic category analysis. Rather, the brain might generate predictions about upcoming visual input whenever it can. However, visual effects of lexical-semantic violations only occurred when a single lexical item could be predicted. We argue that this may be due to the fact that in natural language processing, there is typically no straightforward mapping between lexical-semantic fields (e.g., flowers) and visual or auditory forms (e.g., tulip, rose, magnolia). For syntactic categories, in contrast, certain form features do reliably correlate with category membership. This difference may, in part, explain why certain syntactic effects typically occur much earlier than lexical-semantic effects.     

Contributions of category and fine-grained information to location memory: When categories don't weigh in

Several studies have shown that people's memory for location can be influenced by categorical information. According to a model proposed by Huttenlocher, Hedges, and Duncan (1991), people estimate location by combining fine-grained item-level information in memory with category-level information. When the fine-grained information is inexact, category-level information is given greater weight, which leads to biased responses. We manipulated the distribution of locations presented in order to alter the usefulness of category information, and we manipulated background texture in order to alter accuracy of fine-grained memory. The distributional information reduced bias without altering overall accuracy of responding, whereas the background texture manipulation affected accuracy without changing bias. Our results suggest that category information may weigh in only when it is actively processed.     

Learnability of embedded syntactic structures depends on prosodic cues

The ability to process center-embedded structures has been claimed to represent a core function of the language faculty. Recently, several studies have investigated the learning of center-embedded dependencies in artificial grammar settings. Yet some of the results seem to question the learnability of these structures in artificial grammar tasks. Here, we tested under which exposure conditions learning of center-embedded structures in an artificial grammar is possible. We used naturally spoken syllable sequences and varied the presence of prosodic cues. The results suggest that mere distributional information does not suffice for successful learning. Prosodic cues marking the boundaries of the major relevant units, however, can lead to learning success. Thus, our data are consistent with the hypothesis that center-embedded syntactic structures can be learned in artificial grammar tasks if language-like acoustic cues are provided.     

Distributed cortical networks for syntax processing: Broca's area as the common denominator

Different types of syntactic information (word category, grammatical gender) are processed at different times during word recognition. However, it is an open issue which brain systems support these processes. In the present event-related fMRI study, subjects performed either a syntactic gender decision task on German nouns (GEN), a word category decision task (WC, nouns vs. prepositions), or a physical baseline task (BASE). Reaction times in WC were faster than in GEN, supporting earlier electrophysiological results. Relative to BASE, both syntactic tasks activated the inferior tip of BA 44. In addition, BA 45 showed activation in GEN, whereas BA 47 was activated in WC. The imaging data indicate that the inferior portion of BA 44 together with type-specific prefrontal areas supports both initial word category related and later syntactic processes.     

Gavagai Is as Gavagai Does: Learning Nouns and Verbs From Cross‐Situational Statistics

Learning to map words onto their referents is difficult, because there are multiple possibilities for forming these mappings. Cross‐situational learning studies have shown that word‐object mappings can be learned across multiple situations, as can verbs when presented in a syntactic context. However, these previous studies have presented either nouns or verbs in ambiguous contexts and thus bypass much of the complexity of multiple grammatical categories in speech. We show that noun word learning in adults is robust when objects are moving, and that verbs can also be learned from similar scenes without additional syntactic information. Furthermore, we show that both nouns and verbs can be acquired simultaneously, thus resolving category‐level as well as individual word‐level ambiguity. However, nouns were learned more quickly than verbs, and we discuss this in light of previous studies investigating the noun advantage in word learning.     

The phonological-distributional coherence hypothesis: cross-linguistic evidence in language acquisition

Several phonological and prosodic properties of words have been shown to relate to differences between grammatical categories. Distributional information about grammatical categories is also a rich source in the child's language environment. In this paper we hypothesise that such cues operate in tandem for developing the child's knowledge about grammatical categories. We term this the Phonological-Distributional Coherence Hypothesis (PDCH). We tested the PDCH by analysing phonological and distributional information in distinguishing open from closed class words and nouns from verbs in four languages: English, Dutch, French, and Japanese. We found an interaction between phonological and distributional cues for all four languages indicating that when distributional cues were less reliable, phonological cues were stronger. This provides converging evidence that language is structured such that language learning benefits from the integration of information about category from contextual and sound-based sources, and that the child's language environment is less impoverished than we might suspect.     

Rapid acquisition of phonological alternations by infants

We explore whether infants can learn novel phonological alternations on the basis of distributional information. In Experiment 1, two groups of 12-month-old infants were familiarized with artificial languages whose distributional properties exhibited either stop or fricative voicing alternations. At test, infants in the two exposure groups had different preferences for novel sequences involving voiced and voiceless stops and fricatives, suggesting that each group had internalized a different familiarization alternation. In Experiment 2, 8.5-month-olds exhibited the same patterns of preference. In Experiments 3 and 4, we investigated whether infants' preferences were driven solely by preferences for sequences of high transitional probability. Although 8.5-month-olds in Experiment 3 were sensitive to the relative probabilities of sequences in the familiarization stimuli, only 12-month-olds in Experiment 4 showed evidence of having grouped alternating segments into a single functional category. Taken together, these results suggest a developmental trajectory for the acquisition of phonological alternations using distributional cues in the input.     

Differences in Noun and Verb Processing in Lexical Decision Cannot be Attributed to Word Form and Morphological Complexity Alone

It has been suggested that the effect of word category in noun and verb processing reflects typical word class properties, which can be characterized in terms of semantic as well as syntactic and morphological features. The present study is aimed at differentiating and discussing the relative contribution of these aspects with a main focus on syntactic and morphological processing. Experiment 1 established a processing advantage for nouns in German visual lexical decision, using nouns denoting biological and man-made objects as compared to transitive and intransitive verbs. Experiment 2 showed that the noun advantage persisted even when the morphological differences between word categories were reduced by using identical suffixes in nouns and verbs. Overall results suggest that the processing differences cannot be reduced to variables such as frequency, word form, or morphological complexity. Reaction time differences between transitive and intransitive verbs strengthen the role of syntactic information. In line with previous accounts the observed effects are discussed in terms of a category-specific combination of linguistic parameters.     

The distributional structure of grammatical categories in speech to young children

We present a series of three analyses of young children's linguistic input to determine the distributional information it could plausibly offer to the process of grammatical category learning. Each analysis was conducted on four separate corpora from the CHILDES database (MacWhinney, 2000) of speech directed to children under 2;5. We showthat, in accord with other findings, a distributional analysis which categorizeswords based on their co‐occurrence patterns with surroundingwords successfully categorizes the majority of nouns and verbs. In Analyses 2 and 3, we attempt to make our analyses more closely relevant to natural language acquisition by adopting more realistic assumptions about howyoung children represent their input. In Analysis 2, we limit the distributional context by imposing phrase structure boundaries, and find that categorization improves even beyond that obtained from less limited contexts. In Analysis 3, we reduce the representation of input elements which young children might not fully process and we find that categorization is not adversely affected: Although noun categorization is worse than in Analyses 1 and 2, it is still good; and verb categorization actually improves. Overall, successful categorization of nouns and verbs is maintained across all analyses. These results provide promising support for theories of grammatical category formation involving distributional analysis, as long as these analyses are combined with appropriate assumptions about the child learner's computational biases and capabilities.     

Category induction from distributional cues in an artificial language

The ability to identify the grammatical category of a word (e.g., noun, verb, adjective) is a fundamental aspect of competence in a natural language. Children show evidence of categorization by as early as 18 months, and in some cases younger. However, the mechanisms that underlie this ability are not well understood. The lexical co-occurrence patterns of words in sentences could provide information about word categories--for example, words that follow the in English often belong to the same category. As a step in understanding the role distributional mechanisms might play in language learning, the present study investigated the ability of adults to categorize words on the basis of distributional information. Forty participants listened for approximately 6 min to sentences in an artificial language and were told that they would later be tested on their memory for what they had heard. Participants were next tested on an additional set of sentences and asked to report which sentences they recognized from the first 6 min. The results suggested that learners performed a distributional analysis on the initial set of sentences and recognized sentences on the basis of their memory of sequences of categories of words. Thus, mechanisms that would be useful in natural language learning were shown to be active in adults in an artificial language learning task.     

Linguistic Distributional Knowledge and Sensorimotor Grounding both Contribute to Semantic Category Production

The human conceptual system comprises simulated information of sensorimotor experience and linguistic distributional information of how words are used in language. Moreover, the linguistic shortcut hypothesis predicts that people will use computationally cheaper linguistic distributional information where it is sufficient to inform a task response. In a pre-registered category production study, we asked participants to verbally name members of concrete and abstract categories and tested whether performance could be predicted by a novel measure of sensorimotor similarity (based on an 11-dimensional representation of sensorimotor strength) and linguistic proximity (based on word co-occurrence derived from a large corpus). As predicted, both measures predicted the order and frequency of category production but, critically, linguistic proximity had an effect above and beyond sensorimotor similarity. A follow-up study using typicality ratings as an additional predictor found that typicality was often the strongest predictor of category production variables, but it did not subsume sensorimotor and linguistic effects. Finally, we created a novel, fully grounded computational model of conceptual activation during category production, which best approximated typical human performance when conceptual activation was allowed to spread indirectly between concepts, and when candidate category members came from both sensorimotor and linguistic distributional representations. Critically, model performance was indistinguishable from typical human performance. Results support the linguistic shortcut hypothesis in semantic processing and provide strong evidence that both linguistic and grounded representations are inherent to the functioning of the conceptual system. All materials, data, and code are available at https://osf.io/vaq56/ .     

The phonological-distributional coherence hypothesis: Cross-linguistic evidence in language acquisition

Several phonological and prosodic properties of words have been shown to relate to differences between grammatical categories. Distributional information about grammatical categories is also a rich source in the child’s language environment. In this paper we hypothesise that such cues operate in tandem for developing the child’s knowledge about grammatical categories. We term this the Phonological-Distributional Coherence Hypothesis (PDCH). We tested the PDCH by analysing phonological and distributional information in distinguishing open from closed class words and nouns from verbs in four languages: English, Dutch, French, and Japanese. We found an interaction between phonological and distributional cues for all four languages indicating that when distributional cues were less reliable, phonological cues were stronger. This provides converging evidence that language is structured such that language learning benefits from the integration of information about category from contextual and sound-based sources, and that the child’s language environment is less impoverished than we might suspect.     

Lexical categories at the edge of the word

Language acquisition may be one of the most difficult tasks that children face during development. They have to segment words from fluent speech, figure out the meanings of these words, and discover the syntactic constraints for joining them together into meaningful sentences. Over the past couple of decades, computational modeling has emerged as a new paradigm for gaining insights into the mechanisms by which children may accomplish these feats. Unfortunately, many of these models assume a computational complexity and linguistic knowledge likely to be beyond the abilities of developing young children. This article shows that, using simple statistical procedures, significant correlations exist between the beginnings and endings of a word and its lexical category in English, Dutch, French, and Japanese. Therefore, phonetic information can contribute to individuating higher level structural properties of these languages. This article also presents a simple 2-layer connectionist model that, once trained with an initial small sample of words labeled for lexical category, can infer the lexical category of a large proportion of novel words using only word-edge phonological information, namely the first and last phoneme of a word. The results suggest that simple procedures combined with phonetic information perceptually available to children provide solid scaffolding for emerging lexical categories in language development.     

Familiar words can serve as a semantic seed for syntactic bootstrapping

Young children can exploit the syntactic context of a novel word to narrow down its probable meaning. But how do they learn which contexts are linked to which semantic features in the first place? We investigate if 3‐ to 4‐year‐old children (n = 60) can learn about a syntactic context from tracking its use with only a few familiar words. After watching a 5‐min training video in which a novel function word (i.e., ‘ko’) replaced either personal pronouns or articles, children were able to infer semantic properties for novel words co‐occurring with the newly learned function word (i.e., objects vs. actions). These findings implicate a mechanism by which a distributional analysis, associated with a small vocabulary of known words, could be sufficient to identify some properties associated with specific syntactic contexts.     

Integration of pragmatic and phonetic cues in spoken word recognition

Although previous research has established that multiple top-down factors guide the identification of words during speech processing, the ultimate range of information sources that listeners integrate from different levels of linguistic structure is still unknown. In a set of experiments, we investigate whether comprehenders can integrate information from the 2 most disparate domains: pragmatic inference and phonetic perception. Using contexts that trigger pragmatic expectations regarding upcoming coreference (expectations for either he or she), we test listeners' identification of phonetic category boundaries (using acoustically ambiguous words on the /hi/～/∫i/ continuum). The results indicate that, in addition to phonetic cues, word recognition also reflects pragmatic inference. These findings are consistent with evidence for top-down contextual effects from lexical, syntactic, and semantic cues, but they extend this previous work by testing cues at the pragmatic level and by eliminating a statistical-frequency confound that might otherwise explain the previously reported results. We conclude by exploring the time course of this interaction and discussing how different models of cue integration could be adapted to account for our results.     

Distributional Models of Category Concepts Based on Names of Category Members

Cognitive scientists have long used distributional semantic representations of categories. The predominant approach uses distributional representations of category-denoting nouns, such as “city” for the category city. We propose a novel scheme that represents categories as prototypes over representations of names of its members, such as “Barcelona,” “Mumbai,” and “Wuhan” for the category city. This name-based representation empirically outperforms the noun-based representation on two experiments (modeling human judgments of category relatedness and predicting category membership) with particular improvements for ambiguous nouns. We discuss the model complexity of both classes of models and argue that the name-based model has superior explanatory potential with regard to concept acquisition.     

The Effect of Prosody on Distributional Learning in 12‐ to 13‐Month‐Old Infants

Distributional information is a potential cue for learning syntactic categories. Recent studies demonstrate a developmental trajectory in the level of abstraction of distributional learning in young infants. Here we investigate the effect of prosody on infants' learning of adjacent relations between words. Twelve‐ to thirteen‐month‐old infants were exposed to an artificial language comprised of 3‐word‐sentences of the form aXb and cYd, where X and Y words differed in the number of syllables. Training sentences contained a prosodic boundary between either the first and the second word or the second and the third word. Subsequently, infants were tested on novel test sentences that contained new X and Y words and also contained a flat prosody with no grouping cues. Infants successfully discriminated between novel grammatical and ungrammatical sentences, suggesting that the learned adjacent relations can be abstracted across words and prosodic conditions. Under the conditions tested, prosody may be only a weak constraint on syntactic categorization. Copyright © 2011 John Wiley & Sons, Ltd.