Learning the unlearnable: the role of missing evidence

Syntactic knowledge is widely held to be partially innate, rather than learned. In a classic example, it is sometimes argued that children know the proper use of anaphoric one, although that knowledge could not have been learned from experience. Lidz et al. [Lidz, J., Waxman, S., & Freedman, J. (2003). What infants know about syntax but couldn't have learned: Experimental evidence for syntactic structure at 18 months. Cognition, 89, B65-B73.] pursue this argument, and present corpus and experimental evidence that appears to support it; they conclude that specific aspects of this knowledge must be innate. We demonstrate, contra Lidz et al., that this knowledge may in fact be acquired from the input, through a simple Bayesian learning procedure. The learning procedure succeeds because it is sensitive to the absence of particular input patterns--an aspect of learning that is apparently overlooked by Lidz et al. More generally, we suggest that a prominent form of the "argument from poverty of the stimulus" suffers from the same oversight, and is as a result logically unsound.     

Learning antecedents for anaphoric one

Lidz et al. [Lidz, J., Waxman, S., & Freedman, J. (2003). What infants know about syntax but couldn't have learned: Experimental evidence for syntactic structure at 18 months. Cognition, 89, B65-B73.] claim experimental substantiation of an argument from the poverty of the stimulus, in the sense of Pullum and Scholz [Linguist. Rev. 19 (2002) 9]. They cite a specific feature of English--the assignment of appropriate antecedents for anaphoric one--that cannot possibly be learned from experience because the evidence needed is found only in utterances of a type too rare to be encountered. Their argument involves three empirical claims. In this note we dispute all three.     

Reaffirming the poverty of the stimulus argument: a reply to the replies

Lidz, Waxman, and Freedman [Lidz, J., Waxman, S., & Freedman, J. (2003). What infants know about syntax but couldn’t have learned: Evidence for syntactic structure at 18-months. Cognition, 89, B65-B73.] argue that acquisition of the syntactic and semantic properties of anaphoric one in English relies on innate knowledge within the learner. Several commentaries have now been published questioning this finding. We defend the original finding by identifying both empirical and logical flaws in the critiques.     

Uncovering the richness of the stimulus: structure dependence and indirect statistical evidence

The poverty of stimulus argument is one of the most controversial arguments in the study of language acquisition. Here we follow previous approaches challenging the assumption of impoverished primary linguistic data, focusing on the specific problem of auxiliary (AUX) fronting in complex polar interrogatives. We develop a series of corpus analyses of child-directed speech showing that there is indirect statistical information useful for correct auxiliary fronting in polar interrogatives and that such information is sufficient for distinguishing between grammatical and ungrammatical generalizations, even in the absence of direct evidence. We further show that there are simple learning devices, such as neural networks, capable of exploiting such statistical cues, producing a bias toward correct AUX questions when compared to their ungrammatical counterparts. The results suggest that the basic assumptions of the poverty of stimulus argument may need to be reappraised.     

What infants know about syntax but couldn't have learned: experimental evidence for syntactic structure at 18 months

Generative linguistic theory stands on the hypothesis that grammar cannot be acquired solely on the basis of an analysis of the input, but depends, in addition, on innate structure within the learner to guide the process of acquisition. This hypothesis derives from a logical argument, however, and its consequences have never been examined experimentally with infant learners. Challenges to this hypothesis, claiming that an analysis of the input is indeed sufficient to explain grammatical acquisition, have recently gained attention. We demonstrate with novel experimentation the insufficiency of this countervailing view. Focusing on the syntactic structures required to determine the antecedent for the pronoun one, we demonstrate that the input to children does not contain sufficient information to support unaided learning. Nonetheless, we show that 18-month-old infants do have command of the syntax of one. Because this syntactic knowledge could not have been gleaned exclusively from the input, infants' mastery of this aspect of syntax constitutes evidence for the contribution of innate structure within the learner in acquiring a grammar.     

Bigrams and the richness of the stimulus

Recent challenges to Chomsky's poverty of the stimulus thesis for language acquisition suggest that children's primary data may carry "indirect evidence" about linguistic constructions despite containing no instances of them. Indirect evidence is claimed to suffice for grammar acquisition, without need for innate knowledge. This article reports experiments based on those of Reali and Christiansen (2005) , who demonstrated that a simple bigram language model can induce the correct form of auxiliary inversion in certain complex questions. This article investigates the nature of the indirect evidence that supports this learning, and assesses how reliably it is available. Results confirm the original finding for one specific sentence type but show that the model's success is highly circumscribed. It performs poorly on inversion in related constructions in English and Dutch. Because other, more powerful statistical models have so far been shown to succeed only on the same limited subset of cases as the bigram model, it remains to be seen whether stimulus richness can be substantiated more generally.     

The phonological-distributional coherence hypothesis: cross-linguistic evidence in language acquisition

Several phonological and prosodic properties of words have been shown to relate to differences between grammatical categories. Distributional information about grammatical categories is also a rich source in the child's language environment. In this paper we hypothesise that such cues operate in tandem for developing the child's knowledge about grammatical categories. We term this the Phonological-Distributional Coherence Hypothesis (PDCH). We tested the PDCH by analysing phonological and distributional information in distinguishing open from closed class words and nouns from verbs in four languages: English, Dutch, French, and Japanese. We found an interaction between phonological and distributional cues for all four languages indicating that when distributional cues were less reliable, phonological cues were stronger. This provides converging evidence that language is structured such that language learning benefits from the integration of information about category from contextual and sound-based sources, and that the child's language environment is less impoverished than we might suspect.     

Poverty of the stimulus revisited

A central goal of modern generative grammar has been to discover invariant properties of human languages that reflect "the innate schematism of mind that is applied to the data of experience" and that "might reasonably be attributed to the organism itself as its contribution to the task of the acquisition of knowledge" (Chomsky, 1971). Candidates for such invariances include the structure dependence of grammatical rules, and in particular, certain constraints on question formation. Various "poverty of stimulus" (POS) arguments suggest that these invariances reflect an innate human endowment, as opposed to common experience: Such experience warrants selection of the grammars acquired only if humans assume, a priori, that selectable grammars respect substantive constraints. Recently, several researchers have tried to rebut these POS arguments. In response, we illustrate why POS arguments remain an important source of support for appeal to a priori structure-dependent constraints on the grammars that humans naturally acquire.     

The learnability of abstract syntactic principles

Children acquiring language infer the correct form of syntactic constructions for which they appear to have little or no direct evidence, avoiding simple but incorrect generalizations that would be consistent with the data they receive. These generalizations must be guided by some inductive bias – some abstract knowledge – that leads them to prefer the correct hypotheses even in the absence of directly supporting evidence. What form do these inductive constraints take? It is often argued or assumed that they reflect innately specified knowledge of language. A classic example of such an argument moves from the phenomenon of auxiliary fronting in English interrogatives to the conclusion that children must innately know that syntactic rules are defined over hierarchical phrase structures rather than linear sequences of words (e.g., [Chomsky, 1965] , [Chomsky, 1971] , [Chomsky, 1980] and [Crain and Nakayama, 1987] ). Here we use a Bayesian framework for grammar induction to address a version of this argument and show that, given typical child-directed speech and certain innate domain-general capacities, an ideal learner could recognize the hierarchical phrase structure of language without having this knowledge innately specified as part of the language faculty. We discuss the implications of this analysis for accounts of human language acquisition.     

Adjacent and Non-Adjacent Word Contexts Both Predict Age of Acquisition of English Words: A Distributional Corpus Analysis of Child-Directed Speech

Children show a remarkable degree of consistency in learning some words earlier than others. What patterns of word usage predict variations among words in age of acquisition? We use distributional analysis of a naturalistic corpus of child-directed speech to create quantitative features representing natural variability in word contexts. We evaluate two sets of features: One set is generated from the distribution of words into frames defined by the two adjacent words. These features primarily encode syntactic aspects of word usage. The other set is generated from non-adjacent co-occurrences between words. These features encode complementary thematic aspects of word usage. Regression models using these distributional features to predict age of acquisition of 656 early-acquired English words indicate that both types of features improve predictions over simpler models based on frequency and appearance in salient or simple utterance contexts. Syntactic features were stronger predictors of children's production than comprehension, whereas thematic features were stronger predictors of comprehension. Overall, earlier acquisition was predicted by features representing frames that select for nouns and verbs, and by thematic content related to food and face-to-face play topics; later acquisition was predicted by features representing frames that select for pronouns and question words, and by content related to narratives and object play.     

Learning harmony: the role of serial statistics

How do listeners learn about the statistical regularities underlying musical harmony? In traditional Western music, certain chords predict the occurrence of other chords: Given a particular chord, not all chords are equally likely to follow. In Experiments 1 and 2, we investigated whether adults make use of statistical information when learning new musical structures. Listeners were exposed to a novel musical system containing phrases generated using an artificial grammar. This new system contained statistical structure quite different from Western tonal music. Our results suggest that learners take advantage of the statistical patterning of chords to acquire new musical structures, similar to learning processes previously observed for language learning.     

The probabilistic analysis of language acquisition: theoretical, computational, and experimental analysis

There is much debate over the degree to which language learning is governed by innate language-specific biases, or acquired through cognition-general principles. Here we examine the probabilistic language acquisition hypothesis on three levels: We outline a novel theoretical result showing that it is possible to learn the exact generative model underlying a wide class of languages, purely from observing samples of the language. We then describe a recently proposed practical framework, which quantifies natural language learnability, allowing specific learnability predictions to be made for the first time. In previous work, this framework was used to make learnability predictions for a wide variety of linguistic constructions, for which learnability has been much debated. Here, we present a new experiment which tests these learnability predictions. We find that our experimental results support the possibility that these linguistic constructions are acquired probabilistically from cognition-general principles.     

Acquisition of stimulus control while introducing new stimuli in fading.

After establishing discrimination between a red positive stimulus and a green negative stimulus, the lowest intensity colors that restricted all responding to the positive stimulus were determined. Then, two new white lines differing in terms of line orientation were each superimposed on one of the colors and were increased in intensity. Thereafter, the intensity of the colors was decreased and eventually eliminated. Probe stimuli consisting of the lines presented against dark backgrounds were presented before each change of stimulus intensity, and probe responding was used to assess the control acquired by various dimension of the new stimuli during the course of fading. The lines acquired control of responding while they were being introduced, and control was strengthened as the colors were attenuated. Such a locus of acquisition was attributed to the starting intensity of the original controlling stimuli and was explained in terms of stimulus blocking. Finally, using probes while introducing the new stimuli enhanced the acquisition of control by the new stimuli.     

Word learning is 'smart': evidence that conceptual information affects preschoolers' extension of novel words

We examined electrophysiological correlates of conscious change detection versus change blindness for equivalent displays. Observers had to detect any changes, across a visual interruption, between a pair of successive displays. Each display comprised grey circles on a background of alternate black and white stripes. Foreground changes arose when light-grey circles turned dark-grey and vice-versa. Physically stronger background changes arose when all black stripes turned white and vice-versa. Despite their physical strength, background changes were undetected unless attention was directed to them, whereas foreground changes were invariably seen. Event-related potentials revealed that the P300 component was suppressed for unseen background changes, as compared with the same changes when seen. This effect arose first over frontal sites, and then spread to parietal sites. These results extend recent fMRI findings that fronto-parietal activation is associated with conscious visual change detection, to reveal the timing of these neural correlates.     

Reinforcement of probe responses and acquisition of stimulus control in fading procedures

Stimulus control of pigeons' key pecking was transferred from colors to lines by the method of stimulus fading. Fading was conducted with the addition of probes consisting of the line stimuli presented alone at each fading level. Probe responding was used to measure stimulus-control acquisition by the lines. Effects of reinforcement and nonreinforcement of probe responding upon acquisition of stimulus control were assessed using a single-organism repeated-acquisition design in which three fades were conducted serially. Probe responding was not reinforced in the first and third fade but was in the second. Reinforcement of probe responding substantially reduced the number of fading levels needed to complete fading. The outcome of a control experiment ruled out the possibility of accounting for these results in terms of the specific stimuli used in each fade or in terms of the sequential exposure to the three discriminations. Although probes permitted measurement of stimulus-control acquisition in fading, a measurement/acquisition interaction was also present.     

The evolution of frequency distributions: Relating regularization to inductive biases through iterated learning

The regularization of linguistic structures by learners has played a key role in arguments for strong innate constraints on language acquisition, and has important implications for language evolution. However, relating the inductive biases of learners to regularization behavior in laboratory tasks can be challenging without a formal model. In this paper we explore how regular linguistic structures can emerge from language evolution by iterated learning, in which one person’s linguistic output is used to generate the linguistic input provided to the next person. We use a model of iterated learning with Bayesian agents to show that this process can result in regularization when learners have the appropriate inductive biases. We then present three experiments demonstrating that simulating the process of language evolution in the laboratory can reveal biases towards regularization that might not otherwise be obvious, allowing weak biases to have strong effects. The results of these experiments suggest that people tend to regularize inconsistent word-meaning mappings, and that even a weak bias towards regularization can allow regular languages to be produced via language evolution by iterated learning.     

A Computational Model for the Item‐Based Induction of Construction Networks

According to usage‐based approaches to language acquisition, linguistic knowledge is represented in the form of constructions—form‐meaning pairings—at multiple levels of abstraction and complexity. The emergence of syntactic knowledge is assumed to be a result of the gradual abstraction of lexically specific and item‐based linguistic knowledge. In this article, we explore how the gradual emergence of a network consisting of constructions at varying degrees of complexity can be modeled computationally. Linguistic knowledge is learned by observing natural language utterances in an ambiguous context. To determine meanings of constructions starting from ambiguous contexts, we rely on the principle of cross‐situational learning. While this mechanism has been implemented in several computational models, these models typically focus on learning mappings between words and referents. In contrast, in our model, we show how cross‐situational learning can be applied consistently to learn correspondences between form and meaning beyond such simple correspondences.     

Modeling Statistical Insensitivity: Sources of Suboptimal Behavior

Children acquiring languages with noun classes (grammatical gender) have ample statistical information available that characterizes the distribution of nouns into these classes, but their use of this information to classify novel nouns differs from the predictions made by an optimal Bayesian classifier. We use rational analysis to investigate the hypothesis that children are classifying nouns optimally with respect to a distribution that does not match the surface distribution of statistical features in their input. We propose three ways in which children's apparent statistical insensitivity might arise, and find that all three provide ways to account for the difference between children's behavior and the optimal classifier. A fourth model combines two of these proposals and finds that children's insensitivity is best modeled as a bias to ignore certain features during classification, rather than an inability to encode those features during learning. These results provide insight into children's developing knowledge of noun classes and highlight the complex ways in which statistical information from the input interacts with children's learning processes.     

When global structure "Explains Away" local grammar: a Bayesian account of rule-induction in tone sequences

While many constraints on learning must be relatively experience-independent, past experience provides a rich source of guidance for subsequent learning. Discovering structure in some domain can inform a learner’s future hypotheses about that domain. If a general property accounts for particular sub-patterns, a rational learner should not stipulate separate explanations for each detail without additional evidence, as the general structure has “explained away” the original evidence. In a grammar-learning experiment using tone sequences, manipulating learners’ prior exposure to a tone environment affects their sensitivity to the grammar-defining feature, in this case consecutive repeated tones. Grammar-learning performance is worse if context melodies are “smooth” — when small intervals occur more than large ones — as Smoothness is a general property accounting for a high rate of repetition. We present an idealized Bayesian model as a “best case” benchmark for learning repetition grammars. When context melodies are Smooth, the model places greater weight on the small-interval constraint, and does not learn the repetition rule as well as when context melodies are not Smooth, paralleling the human learners. These findings support an account of abstract grammar-induction in which learners rationally assess the statistical evidence for underlying structure based on a generative model of the environment.