Précis of Foundations of language: brain, meaning, grammar, evolution

The goal of this study is to reintegrate the theory of generative grammar into the cognitive sciences. Generative grammar was right to focus on the child's acquisition of language as its central problem, leading to the hypothesis of an innate Universal Grammar. However, generative grammar was mistaken in assuming that the syntactic component is the sole course of combinatoriality, and that everything else is "interpretive." The proper approach is a parallel architecture, in which phonology, syntax, and semantics are autonomous generative systems linked by interface components. The parallel architecture leads to an integration within linguistics, and to a far better integration with the rest of cognitive neuroscience. It fits naturally into the larger architecture of the mind/brain and permits a properly mentalistic theory of semantics. It results in a view of linguistic performance in which the rules of grammar are directly involved in processing. Finally, it leads to a natural account of the incremental evolution of the language capacity.     

A Probabilistic Model of Lexical and Syntactic Access and Disambiguation

The problems of access—retrieving linguistic structure from some mental grammar —and disambiguation—choosing among these structures to correctly parse ambiguous linguistic input—are fundamental to language understanding. The literature abounds with psychological results on lexical access, the access of idioms, syntactic rule access, parsing preferences, syntactic disambiguation, and the processing of garden-path sentences. Unfortunately, it has been difficult to combine models which account for these results to build a general, uniform model of access and disambiguation at the lexical, idiomatic, and syntactic levels. For example, psycholinguistic theories of lexical access and idiom access and parsing theories of syntactic rule access have almost no commonality in methodology or coverage of psycholinguistic data. This article presents a single probabilistic algorithm which models both the access and disambiguation of linguistic knowledge. The algorithm is based on a parallel parser which ranks constructions for access, and interpretations for disambiguation, by their conditional probability. Low-ranked constructions and interpretations are pruned through beam-search; this pruning accounts, among other things, for the garden-path effect. I show that this motivated probabilistic treatment accounts for a wide variety of psycholinguistic results, arguing for a more uniform representation of linguistic knowledge and for the use of probabilistically-enriched grammars and interpreters as models of human knowledge of and processing of language.     

The simpler syntax hypothesis

What roles do syntax and semantics have in the grammar of a language? What are the consequences of these roles for syntactic structure, and why does it matter? We sketch the Simpler Syntax Hypothesis, which holds that much of the explanatory role attributed to syntax in contemporary linguistics is properly the responsibility of semantics. This rebalancing permits broader coverage of empirical linguistic phenomena and promises a tighter integration of linguistic theory into the cognitive scientific enterprise. We suggest that the general perspective of the Simpler Syntax Hypothesis is well suited to approaching language processing and language evolution, and to computational applications that draw upon linguistic insights.     

The minimalist program in syntax

The Minimalist program, a development of earlier work in transformational generative grammar, proposes that the computational system central to human language is a 'perfect' solution to the task of relating sound and meaning. Recent research has investigated the complexities evident in earlier models and attempted to eliminate them, or to show how they are only apparent, following from deeper but simpler properties. Examples of this include the reduction of the number of linguistic levels of representation in the model, and the deduction of constraints on syntactic derivations from general considerations of economy and computational simplicity.     

The semantics of syntactic structures

Over the past 15 years, both linguists and psycholinguists have shown a growing interest in the idea that syntactic structures can carry meanings that are independent of the meanings of specific words. This article considers how this idea relates to traditional notions of compositionality in generative grammar, and examines two modern theories that, although based on different starting assumptions, both readily allow syntactic structures to bear independent meaning. We review work from psycholinguistics suggesting that observation alone is often insufficient to support the efficient learning of word meanings, and that some of the 'slack' left by observation can be picked up by the semantics of the syntactic structures in which words appear. We argue that this convergence between linguistic theory and psycholinguistic experimentation should be no surprise, because language must be learnable.     

Syntactic priming in language production

People have a tendency to repeat the types of sentences they use during language production. Recent experimental work has shown that this phenomenon is at least partly due to ‘syntactic priming’, whereby the act of processing an utterance with a particular form facilitates processing a subsequent utterance with the same or a related form. In this review, we first provide an overview of the evidence for syntactic priming. The review will then explore the implications of this research for three different areas of language theory: the possible functional significance of syntactic priming in coordinating speakers during dialogue, the mechanisms underlying sentence production, and the nature of linguistic representation.     

A Difference of Some Consequence Between Conventions and Rules

Lewis’s view of the way conventions are passed on may have some especially interesting consequences for the study of language. I’ll start by briefly discussing agreements and disagreements that I have with Lewis’s general views on conventions and then turn to how linguistic conventions spread. I’ll compare views of main stream generative linguistics, in particular, Chomsky’s views on how syntactic forms are passed on, with the sort of view of language acquisition and language change advocated by usage-based or construction grammars, which seem to fit better with Lewis’s ideas. Then I will illustrate the interest of Lewis’s perspective on the dissemination of conventions with a variety of linguistic examples.     

Learning first-pass structural attachment preferences with dynamic grammars and recursive neural networks

One of the central problems in the study of human language processing is ambiguity resolution: how do people resolve the extremely pervasive ambiguity of the language they encounter? One possible answer to this question is suggested by experience-based models, which claim that people typically resolve ambiguities in a way which has been successful in the past. In order to determine the course of action that has been "successful in the past" when faced with some ambiguity, it is necessary to generalize over past experience. In this paper, we will present a computational experience-based model, which learns to generalize over linguistic experience from exposure to syntactic structures in a corpus. The model is a hybrid system, which uses symbolic grammars to build and represent syntactic structures, and neural networks to rank these structures on the basis of its experience. We use a dynamic grammar, which provides a very tight correspondence between grammatical derivations and incremental processing, and recursive neural networks, which are able to deal with the complex hierarchical structures produced by the grammar. We demonstrate that the model reproduces a number of the structural preferences found in the experimental psycholinguistics literature, and also performs well on unrestricted text.     

Theories of language acquisition

Prior to the advent of generative grammar, theoretical approaches to language development relied heavily upon the concepts ofdifferential reinforcement andimitation. Current studies of linguistic acquisition are largely dominated by the hypothesis that the child constructs his language on the basis of a primitive grammar which gradually evolves into a more complex grammar. This approach presupposes that the investigator does not impose his own grammatical rules on the utterances of the child; that the sound system of the child and the rules he employs to form sentences are to be described in their own terms, independently of the model provided by the adult linguistic community; and that there is a series of steps or stages through which the child passes on his way toward mastery of the adult grammar in his linguistic environment. This paper attempts to trace the development of human vocalization through prelinguistic stages to the development of what can be clearly recognized as language behavior, and then progresses to transitional phases in which the language of the child begins to approximate that of the adult model. In the view of the authors, the most challenging problems which confront theories of linguistic acquisition arise in seeking to account for structure of sound sequences, in the rules that enable the speaker to go from meaning to sound and which enable the listener to go from sound to meaning. The principal area of concern for the investigator, according to the authors, is the discovery of those rules at various stages of the learning process. The paper concludes with a return to the question of what constitutes an adequate theory of language ontogenesis. It is suggested that such a theory will have to be keyed to theories of cognitive development and will have to include and go beyond a theory which accounts for adult language competence and performance, since these represent only the terminal stage of linguistic ontogenesis.     

Seven principles of surface structure parsing in natural language

In generative grammar there is a traditional distinction between sentence acceptability, having to do with performance, and sentence grammaticality, having to do with competence. The attempt of this paper is to provide a characterization of the notion ‘acceptable sentence’ in English, with some suggestions as to how this characterization might be made universal. The procedure is to outline a set of procedures which are conjectured to be operative in the assignment of a surface structure tree to an input sentence. To some extent, these principles of parsing are modeled on certain parsing techniques formulated by computer scientists for computer languages. These principles account for the high acceptability of right branching structures, outline the role of grammatical function words in sentence perception, describe what seems to be a fixed limit on short-term memory in linguistic processing, and hypothesize the structure of the internal syntactic processing devices. The operation of various classes of transformations with regard to preparing deep structures for input to parsing procedures such as those outlined in the paper is discussed.     

What language says about the psychology of events

Despite the variety of verb meanings, linguistic research on their syntax and semantics has shown that they can be categorized into a finite and surprisingly small number of event types. More recently, research in the psycholinguistics of language acquisition and processing has emphasized the relevance of event type. The wider implication of these findings is that the conceptual fluidity of verbal concepts is confined by the fundamental structures of mental grammar, shedding light on this important interface between cognition and syntactic organization.     

Statistical models of syntax learning and use

This paper shows how to define probability distributions over linguistically realistic syntactic structures in a way that permits us to define language learning and language comprehension as statistical problems. We demonstrate our approach using lexical‐functional grammar (LFG), but our approach generalizes to virtually any linguistic theory. Our probabilistic models are maximum entropy models. In this paper we concentrate on statistical inference procedures for learning the parameters that define these probability distributions. We point out some of the practical problems that make straightforward ways of estimating these distributions infeasible, and develop a “pseudo‐likelihood” estimation procedure that overcomes some of these problems. This method raises interesting questions concerning the nature of the data available to a language learner and the modularity of language learning and processing.     

The relative accessibility of semantic and deep-structure syntactic concepts

Three experiments were conducted to determine the relative accessibility of semantic and deep-structure syntactic concepts. In Experiment 1, which employed a concept-formation task, subjects learned the concept “deep-structure subject” more slowly than the case concept “experiencer.” In Experiments 2 and 3, which employed a new recognition memory procedure, subjects performed more poorly when the sentences to be remembered were differentiated on the basis of deep-structure syntactic relations than when they were differentiated on the basis of semantic relations. These results favor Fillmore’s case grammar, or another semantically based theory, rather than the “standard theory” of Chomsky in a model of linguistic behavior.     

A Probabilistic Constraints Approach to Language Acquisition and Processing

This article provides an overview of a probabilistic constraints framework for thinking about language acquisition and processing. The generative approach attempts to characterize knowledge of language (i.e., competence grammar) and then asks how this knowledge is acquired and used. Our approach is performance oriented: the goal is to explain how people comprehend and produce utterances and how children acquire this skill. Use of language involves exploiting multiple probabilistic constraints over various types of linguistic and nonlinguistic information. Acquisition is the process of accumulating this information, which begins in infancy. The constraint satisfaction processes that are central to language use are the same as the bootstrapping processes that provide entry to language for the child. Framing questions about acquisition in terms of models of adult performance unifies the two topics under a set of common principles and has important consequences for arguments concerning language learnability.     

Learning a Generative Probabilistic Grammar of Experience: A Process‐Level Model of Language Acquisition

We introduce a set of biologically and computationally motivated design choices for modeling the learning of language, or of other types of sequential, hierarchically structured experience and behavior, and describe an implemented system that conforms to these choices and is capable of unsupervised learning from raw natural‐language corpora. Given a stream of linguistic input, our model incrementally learns a grammar that captures its statistical patterns, which can then be used to parse or generate new data. The grammar constructed in this manner takes the form of a directed weighted graph, whose nodes are recursively (hierarchically) defined patterns over the elements of the input stream. We evaluated the model in seventeen experiments, grouped into five studies, which examined, respectively, (a) the generative ability of grammar learned from a corpus of natural language, (b) the characteristics of the learned representation, (c) sequence segmentation and chunking, (d) artificial grammar learning, and (e) certain types of structure dependence. The model's performance largely vindicates our design choices, suggesting that progress in modeling language acquisition can be made on a broad front—ranging from issues of generativity to the replication of human experimental findings—by bringing biological and computational considerations, as well as lessons from prior efforts, to bear on the modeling approach.     

Parsing mit inkonsistenten Grammatiken

A novel approach to the parsing of natural language based on weighted constraints is presented. Wellformedness conditions are exclusively expressed by means of constraints and no generative rule component is available. In general, constraints can be violated and contradictory requirements of a grammar are tolerated. Additional representational levels allow the integration of different linguistic perspectives into a coherent model, thus contributing jointly to the computation of the most plausible structural interpretation. A number of solution procedures is available for such a parsing problem. Their properties are discussed with respect to possible analogies to human language processing, focussing on robustness against deviating utterances and external temporal pressure.     

From conceptual roles to structural relations: bridging the syntactic cleft.

The distinction between underlying and superficial linguistic structure is a staple of modern cognitive psychology. Despite increasingly diverse conceptions of syntactic relations in linguistic theory, the received view in psycholinguistics has remained one in which the entities assigned to underlying relations may assume different surface relations. The present article examines this view in the context of language production and reviews evidence that the disposition to bind animate entities to the surface subject relation is a basic feature of language use, suggesting that mappings from conceptual categories to syntactic relations form a main support of the bridge from conception to language. Proceeding on this assumption, the article also evaluates competing accounts of the mapping process in production. The results argue against syntactic relation-changing operations, but favor a division between meaning- and form-related mechanisms.     

Cortical memory mechanisms and language origins

We have previously proposed that cortical auditory-vocal networks of the monkey brain can be partly homologized with language networks that participate in the phonological loop. In this paper, we suggest that other linguistic phenomena like semantic and syntactic processing also rely on the activation of transient memory networks, which can be compared to active memory networks in the primate. Consequently, short-term cortical memory ensembles that participate in language processing can be phylogenetically tracked to more simple networks present in the primate brain, which became increasingly complex in hominid evolution. This perspective is discussed in the context of two current interpretations of language origins, the "mirror-system hypothesis" and generativist grammar.     

What artificial grammar learning reveals about the neurobiology of syntax

In this paper we examine the neurobiological correlates of syntax, the processing of structured sequences, by comparing FMRI results on artificial and natural language syntax. We discuss these and similar findings in the context of formal language and computability theory. We used a simple right-linear unification grammar in an implicit artificial grammar learning paradigm in 32 healthy Dutch university students (natural language FMRI data were already acquired for these participants). We predicted that artificial syntax processing would engage the left inferior frontal region (BA 44/45) and that this activation would overlap with syntax-related variability observed in the natural language experiment. The main findings of this study show that the left inferior frontal region centered on BA 44/45 is active during artificial syntax processing of well-formed (grammatical) sequence independent of local subsequence familiarity. The same region is engaged to a greater extent when a syntactic violation is present and structural unification becomes difficult or impossible. The effects related to artificial syntax in the left inferior frontal region (BA 44/45) were essentially identical when we masked these with activity related to natural syntax in the same subjects. Finally, the medial temporal lobe was deactivated during this operation, consistent with the view that implicit processing does not rely on declarative memory mechanisms that engage the medial temporal lobe. In the context of recent FMRI findings, we raise the question whether Broca’s region (or subregions) is specifically related to syntactic movement operations or the processing of hierarchically nested non-adjacent dependencies in the discussion section. We conclude that this is not the case. Instead, we argue that the left inferior frontal region is a generic on-line sequence processor that unifies information from various sources in an incremental and recursive manner, independent of whether there are any processing requirements related to syntactic movement or hierarchically nested structures. In addition, we argue that the Chomsky hierarchy is not directly relevant for neurobiological systems.     

二语句法加工的认知机制、影响因素及其神经基础

二语(L2)和母语(L1)在句法加工方面的差异是量的不同还是质的表现？当前, 该问题已成为语言认知领域研究的热点。研究表明, 二语句法加工受到二语习得年龄、语言熟练度、工作记忆、二语习得方式等因素的影响。有关二语句法加工的理论主要有浅层结构假说、D/P模型和统一竞争模型。未来研究应注重探讨二语句法加工中二语熟练度及二语习得年龄的交互效应, 同时也应当加强儿童二语句法加工能力的研究。