Memory requirements and local ambiguities of parsing strategies

We present a method for calculating lower bounds on the space required and local ambiguities entailed by parsing strategies. A fast, compact natural language parser must implement a strategy with low space requirements and few local ambiguities. It is also widely assumed in the psycholinguistics literature that extremely limited short-term space is available to the human parser, and that sentences containing center-embedded constructions are incomprehensible because processing them requires more space than is available. However, we show that the parsing strategies most psycholinguists assume require less space for processing center-embedded constructions than for processing other perfectly comprehensible constructions. We present alternative strategies for which center-embedded constructions do require more space than other constructions.     

The kindergarten-path effect: studying on-line sentence processing in young children

A great deal of psycholinguistic research has focused on the question of how adults interpret language in real time. This work has revealed a complex and interactive language processing system capable of rapidly coordinating linguistic properties of the message with information from the context or situation (e.g. Altmann & Steedman, 1988; Britt, 1994; Tanenhaus, Spivey-Knowlton, Eberhard & Sedivy, 1995; Trueswell & Tanenhaus, 1991). In the study of language acquisition, however, surprisingly little is known about how children process language in real time and whether they coordinate multiple sources of information during interpretation. The lack of child research is due in part to the fact that most existing techniques for studying language processing have relied upon the skill of reading, an ability that young children do not have or are only beginning to acquire. We present here results from a new method for studying children's moment-by-moment language processing abilities, in which a head-mounted eye-tracking system was used to monitor eye movements as participants responded to spoken instructions. The results revealed systematic differences in how children and adults process spoken language: Five Year Olds did not take into account relevant discourse/pragmatic principles when resolving temporary syntactic ambiguities, and showed little or no ability to revise initial parsing commitments. Adults showed sensitivity to these discourse constraints at the earliest possible stages of processing, and were capable of revising incorrect parsing commitments. Implications for current models of sentence processing are discussed.     

SDIS: A sequential data interchange standard

Few general-purpose computer programs are available that analyze sequential categorical data. If there were a sequential data interchange standard—a standard way of representing sequential data—then it would be more attractive to write general-purpose computer programs for such data Moreover, interlaboratory sharing would be facilitated. The present paper defines such a standard, called the sequential data interchange standard, or SDIS. Both the SDIS data language and a parsing program for data that follow SDIS conventions are described. The parsing program will be made available to researchers who wish to develop analysis programs for sequential data     

Revise and resubmit: How real-time parsing limitations influence grammar acquisition

We present the results from a three-day artificial language learning study on adults. The study examined whether sentence-parsing limitations, in particular, difficulties revising initial syntactic/semantic commitments during comprehension, shape learners’ ability to acquire a language. Findings show that both comprehension and production of morphology pertaining to sentence argument structure are delayed when this morphology consistently appears at the end, rather than at the beginning, of sentences in otherwise identical grammatical systems. This suggests that real-time processing constraints impact acquisition; morphological cues that tend to guide linguistic analyses are easier to learn than cues that revise these analyses. Parallel performance in production and comprehension indicates that parsing constraints affect grammatical acquisition, not just real-time commitments. Properties of the linguistic system (e.g., ordering of cues within a sentence) interact with the properties of the cognitive system (cognitive control and conflict-resolution abilities) and together affect language acquisition.     

Modifier Attachment in Sentence Parsing: Evidence from Dutch

Current theories of parsing suggest a wide variety of mechanisms by which modifiers, such as relative clauses, may be related to constituents that offer more than one potential attachment site. Some, like the tuning hypothesis, are based on the premise that people's parsing performance is shaped by prior exposure to language. Others (e.g. garden-path theory and construal theory) play down any potential role of past linguistic experience, stressing instead the varying influences of structural characteristics of the sentence in question. The two views encourage differing expectations about cross-linguistic variation in parsing preference. A questionnaire study and two on-line experiments were carried out to investigate attachment preferences in Dutch. The results pose a number of problems for the majority of the existing parsing models and are clearly inconsistent with some of the traditional theories. In contrast, the findings are compatible with models incorporating parsing mechanisms that are tuned by language experience. The results highlight the need for further corpus studies to subject these accounts to more searching scrutiny.     

Head position and parsing ambiguity

A strongly principle-based model of parsing seeks to employ principles of the competence grammar directly in language processing. Within grammatical theory, the Projection Principle holds that each level of syntactic representation is a uniform projection of the lexical properties of heads. With respect to parsing this suggests that a phrasal node cannot be projected until the occurrence of its head and thus constitutes a strong empirical hypothesis concerning the fundamental nature of human language processing. This paper contrasts some cross-linguistic predictions made by a specific Grammarderived parsing model against those of a well-known top-down model whose functional motivation is decidedly nonlinguistic. This latter Minimal Attachment model is found to predict significant difficulty with respect to the processing of languages such as Japanese, which display rather different surface properties than English. This problem is not encountered in a model which recognizes the crucial role of heads in licensing argument structure with respect to Processing as well as Grammar. Cross-linguistic parsing differences are attributed to the linear and structural positions of licensing heads which constitute the primary locus of the cross-linguistic variation which is therefore ultimately to be ascribed directly to the Projection Principle.     

Disfluencies and human language comprehension

Spoken language contains disfluencies, which include editing terms such as uh and um as well as repeats and corrections. In less than ten years the question of how disfluencies are handled by the human sentence comprehension system has gone from virtually ignored to a topic of major interest in computational linguistics and psycholinguistics. We discuss relevant empirical findings and describe a computational model that captures how disfluencies influence parsing and comprehension. The research reviewed shows that the parser, which presumably evolved to handle conversations, deals with disfluencies in a way that is efficient and linguistically principled. The success of this research program reinforces the current trend in cognitive science to view cognitive mechanisms as adaptations to real-world constraints and challenges.     

The metamorphosis of the statistical segmentation output: Lexicalization during artificial language learning

This study combined artificial language learning (ALL) with conventional experimental techniques to test whether statistical speech segmentation outputs are integrated into adult listeners’ mental lexicon. Lexicalization was assessed through inhibitory effects of novel neighbors (created by the parsing process) on auditory lexical decisions to real words. Both immediately after familiarization and post-one week, ALL outputs were lexicalized only when the cues available during familiarization (transitional probabilities and wordlikeness) suggested the same parsing (Experiments 1 and 3). No lexicalization effect occurred with incongruent cues (Experiments 2 and 4). Yet, ALL differed from chance, suggesting a dissociation between item knowledge and lexicalization. Similarly contrasted results were found when frequency of occurrence of the stimuli was equated during familiarization (Experiments 3 and 4). Our findings thus indicate that ALL outputs may be lexicalized as far as the segmentation cues are congruent, and that this process cannot be accounted for by raw frequency.     

Maturational Constraints on Language Learning

This paper suggests that there are constraints on learning required to explain the acquisition of language, in particular, maturational constraints. First, empirical evidence for this claim is reviewed. The evidence from several studies of both first and second language acquisition suggests that normal language learning occurs only when exposure to the language begins early in life. With exposure beginning later in life, asymptotic performance in the language declines: the effects over age of first exposure are approximately linear through childhood, with a flattening of the function in adulthood. These outcomes argue that some type of constraints ensuring successful language learning exist early in life, and weaken with increasing maturation. Second, two hypotheses are considered as to the nature of these maturational changes. One hypothesis is that constraints on learning particular to language acquisition undergo maturational decay. A second hypothesis, which is considered in more detail, suggests that language learning abilities decline because of the expansion of nonlinguistic cognitive abilities.     

Simple,applied text parsing

Text parsing for use in linguistic or artificial intelligence research can often be ill-suited for use in other lines of investigation in which simple, problem-specific parsing techniques may be used. This paper describes the development of a simple text parser for use in a specific research setting. The resulting parsing programs are compact enough for easy use on most available m inland microcomputers, while still providing adequate results for some applications such as chunking text into small units for later presentation.     

Prediction and substantiation: A new approach to natural language processing

This paper describes a new approach to natural language processing which results in a very robust and efficient system. The approach taken is to integrate the parser with the rest of the system. This enables the parser to benefit from predictions that the rest of the system makes in the course of its processing. These predictions can be invaluable as guides to the parser in such difficult problem areas as resolving referents and selecting meanings of ambiguous words. A program, called FRUMP for Fast Reading Understanding and Memory Program, employs this approach to parsing. FRUMP skims articles rather than reading them for detail. The program works on the relatively unconstrained domain of news articles. It routinely understands stories it has never before seen. The program's success is largely due to its radically different approach to parsing.     

The role of structural context in perception: Syntax in the recognition of algebraic expressions

Two character-identification experiments investigated the function of structural context during the processing of briefly exposed algebraic strings. Neither experiment prodded evidence to support the notion of an algebra-superiority effect, a contextually driven enhancement of the recognition of specific algebraic characters. However, the results of Experiment 2 indicate that the structure of algebra does provide information at the level of a character’s categorical denomination. These findings suggest that the parsing of an algebraic string includes a level of processing in which its structural context places restrictions on the denominations of its symbols. A processing model of algebraic perception is proposed that incorporates these syntactic constraints—constraints that appear to be independent of feature-based character identification processes.     

Performance structures: A psycholinguistic and linguistic appraisal

Two lines of research—one in psycholinguistics and one in linguistics—are combined to deal with a long-standing problem in both fields: why the “performance structures” of sentences (structures based on experimental data, such as pausing and parsing values) are not fully accountable for by linguistic theories of phrase structure. Two psycholinguistic algorithms that have been used to predict these structures are described and their limitations are examined. A third algorithm, based on the prosodic structures of sentences is then proposed and shown to be a far better predictor of performance structures. It is argued that the experimental data reflect aspects of the linguistic cognitive capacity, and that, in turn, linguistic theory can offer an illuminating account of the data. The prosodic model is shown to have a wider domain of application than temporal organization per se, accounting for parsing judgments as well as pausing performance, and reflecting aspects of syntactic and semantic structure as well as purely prosodic structure. Finally, the algorithm is discussed in light of language processing.     

Applying a logical interpretation of semantic nets and graph grammars to natural language parsing and understanding

In this paper a logical interpretation of semantic nets and graph grammars is proposed for modelling natural language understanding and creating language understanding computer systems. An example of parsing a Finnish question by graph grammars and inferring the answer to it by a semantic net representation is provided.     

Some observations on the universality of the late-closure strategy

Two questionnaire studies and a reading time experiment investigated the application of the late-closure principle in Italian, a Romance language which contrasts with English with respect to several linguistic properties. All the studies addressed interpretation preferences in sentences containing a complex NP followed by a relative clause (e.g....the son of the woman who arrived yesterday...). While the questionnaires investigated final preferences, the reading time experiment addressed also the principle governing the initial attachment of a relative clause to a complex NP. Furthermore, through a manipulation of the type of preposition within the complex NP, we investigated the role of the thematic structure of the complex NP in initial and final parsing. The results showed that the late-closure principle applies in Italian to the initial parsing without being affected by the thematic structure of the complex NP. Final interpretation instead shows an effect of pragmatic preference and an effect of thematic structure on syntactic revisions. The results are discussed in terms of a parsing model that adopts syntactic parsing strategies and makes modular use of linguistic information. Some implications for the relationship between syntactic theories and the human parser are also addressed.     

Exposure-based models of human parsing: Evidence for the use of coarse-grained (nonlexical) statistical records

Several current models of human parsing maintain that initial structural decisions are influenced (or tuned) by the listener's or reader's prior contact with language. The precise workings of these models depend upon the grain, or level of detail, at which previous exposures to language are analyzed and used to influence parsing decisions. Some models are premised upon the use of fine-grained records (such as lexical cooccurrence statistics). Others use coarser measures. The present paper considers the viability of models based exclusively on the use of fine-grained lexical records. The results of several studies are reviewed and the evidence suggests that, if they are to account for the data, experience-based parsers must draw upon records or representations that capture statistical regularities beyond the lexical level. This poses problems for several parsing models in the literature.Aspects of this work were supported by ESRC grant No. R0023 4062 to Don Mitchell, by a Spanish Government grant DGICVT No. PB-92-0656-C04-02 to Fernando Cuetos, and by the Belgian National Fonds voor Wetenschappelijk Onderzoek, of which Marc Brysbaert is a Research Associate. We are grateful to Chuck Clifton, Barbara Hemforth, Martin Pickering, Matt Traxler, and an anonymous reviewer-all of whom made helpful comments on an earlier draft of the paper.     

Verbal Integration: The Interaction of Participant Roles and Sentential Argument Structure

This paper explores the interaction between sentence level syntactic information and the semantic information that is carried with the verb during sentence comprehension. A cross-modal integration task was employed to examine whether the number of participant roles (thematic roles associated with the central meaning of the verb) causes an increase in processing load during integration of the verb into on-going sentence comprehension. The effect of preceding sentential structural information (varied with respect to the number of argument and/or adjunct NPs preceding the verb) was also manipulated. Independent of the structural information preceding the verb, verbs with two partici-pant roles were integrated into the sentence faster than verbs with three participant roles. This finding suggests that participant roles are stored with the representation of the verb and made immediately available during integration and comprehension. In addition, the syntactic distinction between arguments and adjuncts is also shown to play an immediate role in parsing and integration of language on-line.     

Sentence processing in an artificial language: Learning and using combinatorial constraints

A study combining artificial grammar and sentence comprehension methods investigated the learning and online use of probabilistic, nonadjacent combinatorial constraints. Participants learned a small artificial language describing cartoon monsters acting on objects. Self-paced reading of sentences in the artificial language revealed comprehenders’ sensitivity to nonadjacent combinatorial constraints, without explicit awareness of the probabilities embedded in the language. These results show that even newly-learned constraints have an identifiable effect on online sentence processing. The rapidity of learning in this paradigm relative to others has implications for theories of implicit learning and its role in language acquisition.