Analyzing the Rate at Which Languages Lose the Influence of a Common Ancestor

Analyzing the rate at which languages change can clarify whether similarities across languages are solely the result of cognitive biases or might be partially due to descent from a common ancestor. To demonstrate this approach, we use a simple model of language evolution to mathematically determine how long it should take for the distribution over languages to lose the influence of a common ancestor and converge to a form that is determined by constraints on language learning. We show that modeling language learning as Bayesian inference of n binary parameters or the ordering of n constraints results in convergence in a number of generations that is on the order of n log n. We relax some of the simplifying assumptions of this model to explore how different assumptions about language evolution affect predictions about the time to convergence; in general, convergence time increases as the model becomes more realistic. This allows us to characterize the assumptions about language learning (given the models that we consider) that are sufficient for convergence to have taken place on a timescale that is consistent with the origin of human languages. These results clearly identify the consequences of a set of simple models of language evolution and show how analysis of convergence rates provides a tool that can be used to explore questions about the relationship between accounts of language learning and the origins of similarities across languages.     

Individual Differences in Learning Abilities Impact Structure Addition: Better Learners Create More Structured Languages

Over the last decade, iterated learning studies have provided compelling evidence for the claim that linguistic structure can emerge from non-structured input, through the process of transmission. However, it is unclear whether individuals differ in their tendency to add structure, an issue with implications for understanding who are the agents of change. Here, we identify and test two contrasting predictions: The first sees learning as a pre-requisite for structure addition, and predicts a positive correlation between learning accuracy and structure addition, whereas the second maintains that it is those learners who struggle with learning and reproducing their input who add structure to it. This prediction is hard to test in standard iterated learning paradigms since each learner is exposed to a different input, and since structure and accuracy are computed using the same test items. Here, we test these contrasting predictions in two experiments using a one-generation artificial language learning paradigm designed to provide independent measures of learning accuracy and structure addition. Adults (N = 48 in each study) were exposed to a semi-regular language (with probabilistic structure) and had to learn it: Learning was assessed using seen items, whereas structure addition was calculated over unseen items. In both studies, we find a strong positive correlation between individuals' ability to learn the language and their tendency to add structure to it: Better learners also produced more structured languages. These findings suggest a strong link between learning and generalization. We discuss the implications of these findings for iterated language models and theories of language change more generally.     

The Utility of Cognitive Plausibility in Language Acquisition Modeling: Evidence From Word Segmentation

The informativity of a computational model of language acquisition is directly related to how closely it approximates the actual acquisition task, sometimes referred to as the model's cognitive plausibility. We suggest that though every computational model necessarily idealizes the modeled task, an informative language acquisition model can aim to be cognitively plausible in multiple ways. We discuss these cognitive plausibility checkpoints generally and then apply them to a case study in word segmentation, investigating a promising Bayesian segmentation strategy. We incorporate cognitive plausibility by using an age‐appropriate unit of perceptual representation, evaluating the model output in terms of its utility, and incorporating cognitive constraints into the inference process. Our more cognitively plausible model shows a beneficial effect of cognitive constraints on segmentation performance. One interpretation of this effect is as a synergy between the naive theories of language structure that infants may have and the cognitive constraints that limit the fidelity of their inference processes, where less accurate inference approximations are better when the underlying assumptions about how words are generated are less accurate. More generally, these results highlight the utility of incorporating cognitive plausibility more fully into computational models of language acquisition.     

Revise and resubmit: How real-time parsing limitations influence grammar acquisition

We present the results from a three-day artificial language learning study on adults. The study examined whether sentence-parsing limitations, in particular, difficulties revising initial syntactic/semantic commitments during comprehension, shape learners’ ability to acquire a language. Findings show that both comprehension and production of morphology pertaining to sentence argument structure are delayed when this morphology consistently appears at the end, rather than at the beginning, of sentences in otherwise identical grammatical systems. This suggests that real-time processing constraints impact acquisition; morphological cues that tend to guide linguistic analyses are easier to learn than cues that revise these analyses. Parallel performance in production and comprehension indicates that parsing constraints affect grammatical acquisition, not just real-time commitments. Properties of the linguistic system (e.g., ordering of cues within a sentence) interact with the properties of the cognitive system (cognitive control and conflict-resolution abilities) and together affect language acquisition.     

Grammatical pattern learning by human infants and cotton-top tamarin monkeys

There is a surprising degree of overlapping structure evident across the languages of the world. One factor leading to cross-linguistic similarities may be constraints on human learning abilities. Linguistic structures that are easier for infants to learn should predominate in human languages. If correct, then (a) human infants should more readily acquire structures that are consistent with the form of natural language, whereas (b) non-human primates' patterns of learning should be less tightly linked to the structure of human languages. Prior experiments have not directly compared laboratory-based learning of grammatical structures by human infants and non-human primates, especially under comparable testing conditions and with similar materials. Five experiments with 12-month-old human infants and adult cotton-top tamarin monkeys addressed these predictions, employing comparable methods (familiarization-discrimination) and materials. Infants rapidly acquired complex grammatical structures by using statistically predictive patterns, failing to learn structures that lacked such patterns. In contrast, the tamarins only exploited predictive patterns when learning relatively simple grammatical structures. Infant learning abilities may serve both to facilitate natural language acquisition and to impose constraints on the structure of human languages.     

Language evolution by iterated learning with bayesian agents

Languages are transmitted from person to person and generation to generation via a process of iterated learning: people learn a language from other people who once learned that language themselves. We analyze the consequences of iterated learning for learning algorithms based on the principles of Bayesian inference, assuming that learners compute a posterior distribution over languages by combining a prior (representing their inductive biases) with the evidence provided by linguistic data. We show that when learners sample languages from this posterior distribution, iterated learning converges to a distribution over languages that is determined entirely by the prior. Under these conditions, iterated learning is a form of Gibbs sampling, a widely-used Markov chain Monte Carlo algorithm. The consequences of iterated learning are more complicated when learners choose the language with maximum posterior probability, being affected by both the prior of the learners and the amount of information transmitted between generations. We show that in this case, iterated learning corresponds to another statistical inference algorithm, a variant of the expectation-maximization (EM) algorithm. These results clarify the role of iterated learning in explanations of linguistic universals and provide a formal connection between constraints on language acquisition and the languages that come to be spoken, suggesting that information transmitted via iterated learning will ultimately come to mirror the minds of the learners.     

Learning a Generative Probabilistic Grammar of Experience: A Process‐Level Model of Language Acquisition

We introduce a set of biologically and computationally motivated design choices for modeling the learning of language, or of other types of sequential, hierarchically structured experience and behavior, and describe an implemented system that conforms to these choices and is capable of unsupervised learning from raw natural‐language corpora. Given a stream of linguistic input, our model incrementally learns a grammar that captures its statistical patterns, which can then be used to parse or generate new data. The grammar constructed in this manner takes the form of a directed weighted graph, whose nodes are recursively (hierarchically) defined patterns over the elements of the input stream. We evaluated the model in seventeen experiments, grouped into five studies, which examined, respectively, (a) the generative ability of grammar learned from a corpus of natural language, (b) the characteristics of the learned representation, (c) sequence segmentation and chunking, (d) artificial grammar learning, and (e) certain types of structure dependence. The model's performance largely vindicates our design choices, suggesting that progress in modeling language acquisition can be made on a broad front—ranging from issues of generativity to the replication of human experimental findings—by bringing biological and computational considerations, as well as lessons from prior efforts, to bear on the modeling approach.     

The evolution of frequency distributions: Relating regularization to inductive biases through iterated learning

The regularization of linguistic structures by learners has played a key role in arguments for strong innate constraints on language acquisition, and has important implications for language evolution. However, relating the inductive biases of learners to regularization behavior in laboratory tasks can be challenging without a formal model. In this paper we explore how regular linguistic structures can emerge from language evolution by iterated learning, in which one person’s linguistic output is used to generate the linguistic input provided to the next person. We use a model of iterated learning with Bayesian agents to show that this process can result in regularization when learners have the appropriate inductive biases. We then present three experiments demonstrating that simulating the process of language evolution in the laboratory can reveal biases towards regularization that might not otherwise be obvious, allowing weak biases to have strong effects. The results of these experiments suggest that people tend to regularize inconsistent word-meaning mappings, and that even a weak bias towards regularization can allow regular languages to be produced via language evolution by iterated learning.     

The Relationship Between Artificial and Second Language Learning

Artificial language learning (ALL) experiments have become an important tool in exploring principles of language and language learning. A persistent question in all of this work, however, is whether ALL engages the linguistic system and whether ALL studies are ecologically valid assessments of natural language ability. In the present study, we considered these questions by examining the relationship between performance in an ALL task and second language learning ability. Participants enrolled in a Spanish language class were evaluated using a number of different measures of Spanish ability and classroom performance, which was compared to IQ and a number of different measures of ALL performance. The results show that success in ALL experiments, particularly more complex artificial languages, correlates positively with indices of L2 learning even after controlling for IQ. These findings provide a key link between studies involving ALL and our understanding of second language learning in the classroom.     

The Persistence of Priming: Exploring Long-lasting Syntactic Priming Effects in Children and Adults

The implicit learning account of syntactic priming proposes that the same mechanism underlies syntactic priming and language development, providing a link between a child and adult language processing. The present experiment tested predictions of this account by comparing the persistence of syntactic priming effects in children and adults. Four-year-olds and adults first described transitive events after hearing transitive primes, constituting an exposure phase that established priming effects for passives. The persistence of this priming effect was measured in a test phase as participants described further transitive events but no longer heard primes. Their production of passives was compared to a baseline group who described the same pictures without any exposure to primes. Neither immediate nor long-term priming effects differed between children and adults but both children and adults showed significant immediate and persistent effects of the priming when the test phase occurred immediately after the exposure phase and when a short delay separated the exposure and test phase. The implications of these results for an implicit learning account of syntactic priming are discussed.     

Early child grammars: qualitative and quantitative analysis of morphosyntactic production

This article reports on a series of 5 analyses of spontaneous production of verbal inflection (tense and person–number agreement) by 2-year-olds acquiring French as a native language. A formal analysis of the qualitative and quantitative results is developed using the unique resources of Optimality Theory (OT; Prince & Smolensky, 2004). It is argued that acquisition of morphosyntax proceeds via overlapping grammars (rather than through abrupt changes), which OT formalizes in terms of partial rather than total constraint rankings. Initially, economy of structure constraints take priority over faithfulness constraints that demand faithful expression of a speaker's intent, resulting in child production of tense that is comparable in level to that of child-directed speech. Using the independent Predominant Length of Utterance measure of syntactic development proposed in Vainikka, Legendre, and Todorova (1999), production of agreement is shown first to lag behind tense then to compete with tense at an intermediate stage of development. As the child's development progresses, faithfulness constraints become more dominant, and the overall production of tense and agreement becomes adult-like.     

A probabilistic computational model of cross-situational word learning

Words are the essence of communication: They are the building blocks of any language. Learning the meaning of words is thus one of the most important aspects of language acquisition: Children must first learn words before they can combine them into complex utterances. Many theories have been developed to explain the impressive efficiency of young children in acquiring the vocabulary of their language, as well as the developmental patterns observed in the course of lexical acquisition. A major source of disagreement among the different theories is whether children are equipped with special mechanisms and biases for word learning, or their general cognitive abilities are adequate for the task. We present a novel computational model of early word learning to shed light on the mechanisms that might be at work in this process. The model learns word meanings as probabilistic associations between words and semantic elements, using an incremental and probabilistic learning mechanism, and drawing only on general cognitive abilities. The results presented here demonstrate that much about word meanings can be learned from naturally occurring child-directed utterances (paired with meaning representations), without using any special biases or constraints, and without any explicit developmental changes in the underlying learning mechanism. Furthermore, our model provides explanations for the occasionally contradictory child experimental data, and offers predictions for the behavior of young word learners in novel situations.     

Persistent structural priming from language comprehension to language production

To examine the relationship between syntactic processes in language comprehension and language production, we compared structural persistence from sentence primes that speakers heard to persistence from primes that speakers produced. [Bock, J. K., & Griffin, Z. M. (2000). The persistence of structural priming: transient activation or implicit learning? Journal of Experimental Psychology: General, 129, 177-192.] showed that the production of target priming structures increased the probability of spontaneously using the same structures to describe events in subsequent pictures that were semantically unrelated to the primes. These priming effects persisted across as many as ten intervening filler trials. The present studies replicated these results using auditorily presented primes to which participants only listened. The results indicated persistence of priming across all lags, with relative magnitudes of priming as large as those observed by Bock and Griffin. The implication is that structural priming is persistent regardless of the modality in which language structures are experienced, underscoring the power of priming as an implicit learning mechanism.     

Acquiring and processing verb argument structure: distributional learning in a miniature language

Adult knowledge of a language involves correctly balancing lexically-based and more language-general patterns. For example, verb argument structures may sometimes readily generalize to new verbs, yet with particular verbs may resist generalization. From the perspective of acquisition, this creates significant learnability problems, with some researchers claiming a crucial role for verb semantics in the determination of when generalization may and may not occur. Similarly, there has been debate regarding how verb-specific and more generalized constraints interact in sentence processing and on the role of semantics in this process. The current work explores these issues using artificial language learning. In three experiments using languages without semantic cues to verb distribution, we demonstrate that learners can acquire both verb-specific and verb-general patterns, based on distributional information in the linguistic input regarding each of the verbs as well as across the language as a whole. As with natural languages, these factors are shown to affect production, judgments and real-time processing. We demonstrate that learners apply a rational procedure in determining their usage of these different input statistics and conclude by suggesting that a Bayesian perspective on statistical learning may be an appropriate framework for capturing our findings.     

How does a word become a message? An illustration on a developmental time-scale

Recent changes in views on cognition underscore its embodied, situated and distributed character. These changes are compatible with the conceptual framework of ecological psychology. However for ecological psychology to propose explanations for a broad range of cognitive phenomena, including language, it needs an account of how to link the dynamics of coupling between the organism and the environment with the apparent symbolicity of informational structures. In this paper it is proposed that a theory of information in biological systems, advocated by Howard Pattee, may help forge this link. By treating informational structures as constraints on dynamics this approach helps to identify which processes, in which systems and on what time-scales are needed for structures to ‘become messages’. I will illustrate how these processes might work on developmental time-scale in the domain of language, building on the work by Edward Reed (1995, 1996) and extending it using the view of linguistic structures as constraints.     

From specific examples to general knowledge in language learning

The extraction of general knowledge from individual episodes is critical if we are to learn new knowledge or abilities. Here we uncover some of the key cognitive mechanisms that characterise this process in the domain of language learning. In five experiments adult participants learned new morphological units embedded in fictitious words created by attaching new affixes (e.g., -afe) to familiar word stems (e.g., “sleepafe is a participant in a study about the effects of sleep”). Participants’ ability to generalise semantic knowledge about the affixes was tested using tasks requiring the comprehension and production of novel words containing a trained affix (e.g., sailafe). We manipulated the delay between training and test (Experiment 1), the number of unique exemplars provided for each affix during training (Experiment 2), and the consistency of the form-to-meaning mapping of the affixes (Experiments 3–5). In a task where speeded online language processing is required (semantic priming), generalisation was achieved only after a memory consolidation opportunity following training, and only if the training included a sufficient number of unique exemplars. Semantic inconsistency disrupted speeded generalisation unless consolidation was allowed to operate on one of the two affix-meanings before introducing inconsistencies. In contrast, in tasks that required slow, deliberate reasoning, generalisation could be achieved largely irrespective of the above constraints. These findings point to two different mechanisms of generalisation that have different cognitive demands and rely on different types of memory representations.     

From domain-generality to domain-sensitivity: 4-Month-olds learn an abstract repetition rule in music that 7-month-olds do not

Learning must be constrained for it to lead to productive generalizations. Although biology is undoubtedly an important source of constraints, prior experience may be another, leading learners to represent input in ways that are more conducive to some generalizations than others, and/or to up- and down-weight features when entertaining generalizations. In two experiments, 4-month-old and 7-month-old infants were familiarized with sequences of musical chords or tones adhering either to an AAB pattern or an ABA pattern. In both cases, the 4-month-olds learned the generalization, but the 7-month-olds did not. The success of the 4-month-olds appears to contradict an account that this type of pattern learning is the provenance of a language-specific rule-learning module. It is not yet clear what drives the age-related change, but plausible candidates include differential experience with language and music, as well as interactions between general cognitive development and stimulus complexity.     

Language Evolution Can Be Shaped by the Structure of the World

Human languages vary in many ways but also show striking cross‐linguistic universals. Why do these universals exist? Recent theoretical results demonstrate that Bayesian learners transmitting language to each other through iterated learning will converge on a distribution of languages that depends only on their prior biases about language and the quantity of data transmitted at each point; the structure of the world being communicated about plays no role (Griffiths & Kalish, 2005 , 2007 ). We revisit these findings and show that when certain assumptions about the relationship between language and the world are abandoned, learners will converge to languages that depend on the structure of the world as well as their prior biases. These theoretical results are supported with a series of experiments showing that when human learners acquire language through iterated learning, the ultimate structure of those languages is shaped by the structure of the meanings to be communicated.     

Grammatik mit Widersprüchen

This paper presents and discusses important aspects of Optimality Theory, which has become one of the dominant models in theoretical linguistics. Optimality Theory is a system of constraints that are potentially in conflict with each other. Conflicts are resolved by ordering the principles in a strict hierarchy. We show that there are many respects in which Optimality Theory is a much more restrictive model of the language faculty than any of its competitors. Furthermore, the implications of the specific way how OT resolves conflicts seem to be more or less correct for natural language. However, modifications may be called for with respect to graded grammaticality and ineffability.     

Infants learn phonotactic regularities from brief auditory experience

Two experiments investigated whether novel phonotactic regularities, not present in English, could be acquired by 16.5-month-old infants from brief auditory experience. Subjects listened to consonant-vowel-consonant syllables in which particular consonants were artificially restricted to either initial or final position (e.g. /baep/ not /paeb/). In a later head-turn preference test, infants listened longer to new syllables that violated the experimental phonotactic constraints than to new syllables that honored them. Thus, infants rapidly learned phonotactic regularities from brief auditory experience and extended them to unstudied syllables, documenting the sensitivity of the infant's language processing system to abstractions over linguistic experience.