The sausage machine: A new two-stage parsing model

It is proposed that the human sentence parsing device assigns phrase structure to word strings in two steps. The first stage parser assigns lexical and phrasal nodes to substrings of roughly six words. The second stage parser then adds higher nodes to link these phrasal packages together into a complete phrase marker.This model of the parser is compared with ATN models, and with the two-stage models of Kimball (1973) and Fodor, Bever and Garrett (1974). Our assumption that the units which are shunted from the first stage to the second stage are defined by their length, rather than by their syntactic type, explains the effects of constituent length on perceptual complexity in center embedded sentences and in sentences of the kind that fall under Kimball's principle of Right Association. The particular division of labor between the two parsing units allows us to explain, without appeal to any ad hoc parsing strategies, why the parser makes certain ‘shortsighted’ errors even though, in general, it is able to make intelligent use of all the information that is available to it.     

Some observations on the universality of the late-closure strategy

Two questionnaire studies and a reading time experiment investigated the application of the late-closure principle in Italian, a Romance language which contrasts with English with respect to several linguistic properties. All the studies addressed interpretation preferences in sentences containing a complex NP followed by a relative clause (e.g....the son of the woman who arrived yesterday...). While the questionnaires investigated final preferences, the reading time experiment addressed also the principle governing the initial attachment of a relative clause to a complex NP. Furthermore, through a manipulation of the type of preposition within the complex NP, we investigated the role of the thematic structure of the complex NP in initial and final parsing. The results showed that the late-closure principle applies in Italian to the initial parsing without being affected by the thematic structure of the complex NP. Final interpretation instead shows an effect of pragmatic preference and an effect of thematic structure on syntactic revisions. The results are discussed in terms of a parsing model that adopts syntactic parsing strategies and makes modular use of linguistic information. Some implications for the relationship between syntactic theories and the human parser are also addressed.     

The diagnosis and cure of garden paths

We propose that, for the human parser, recovery from garden paths consists in repairing the structure built so far, rather than reparsing the input. The difficulty of a repair is attributable not to the cost of effecting the structural alterations but to the cost of deducing which alterations are needed. The parser must diagnose its error in order to correct it. The error is signaled by an input word that is incompatible with the current structure; this is the symptom from which the diagnosis must be made. If the error is transparently clear from the nature of the symptom, recovery is easy; but sometimes the necessary reasoning is obscure, and then the diagnosis is unsuccessful and the garden path persists. Unlike other repair models, the diagnosis model needs no special mechanism for revising garden path analyses. The garden path recovery device is the same machine as the first-pass parser, merely set into emergency mode. When faced with a breakdown the parser does not stop its normal activities and enter a new mode of reasoning to detect what went wrong. It simply continues to parse, attaching the problematic input item in the least ungrammatical way it can, despite the conflict with previously built structure. This conflict is productive; it provokes adjustments to the existing structure. In successful cases, one adjustment leads to another until a stable state is reached, at which point the original error will have been eliminated. Examples suggest that the parser gives more weight to syntatctic than to pragmatic acceptability; only a syntactic clash between the input and the existing structure sets the adjustment process in motion.     

Memory requirements and local ambiguities of parsing strategies

We present a method for calculating lower bounds on the space required and local ambiguities entailed by parsing strategies. A fast, compact natural language parser must implement a strategy with low space requirements and few local ambiguities. It is also widely assumed in the psycholinguistics literature that extremely limited short-term space is available to the human parser, and that sentences containing center-embedded constructions are incomprehensible because processing them requires more space than is available. However, we show that the parsing strategies most psycholinguists assume require less space for processing center-embedded constructions than for processing other perfectly comprehensible constructions. We present alternative strategies for which center-embedded constructions do require more space than other constructions.     

Use of verb information in syntactic parsing: evidence from eye movements and word-by-word self-paced reading

We investigated whether readers use verb information to aid in their initial parsing of temporarily ambiguous sentences. In the first experiment, subjects' eye movements were recorded. In the second and third experiments, subjects read sentences by using a noncumulative and cumulative word-by-word self-paced paradigm, respectively. The results of the first two experiments supported Frazier and Rayner's (1982) garden-path model of sentence comprehension: Verb information did not influence the initial operation of the parser. The third experiment indicated that the cumulative version of the self-paced paradigm is not appropriate for studying on-line parsing. We conclude that verb information is not used by the parser to modify its initial parsing strategies, although it may be used to guide subsequent reanalysis.     

When Does Gender Constrain Parsing? Evidence from ERPs

We review the implications of recent ERP evidence for when and how grammatical gender agreement constrains sentence parsing. In some theories of parsing, gender is assumed to immediately and categorically block gender-incongruent phrase structure alternatives from being pursued. In other theories, the parser initially ignores gender altogether. The ERP evidence we discuss suggests an intermediate position, in which grammatical gender does not immediately block gender-incongruent phrase structures from being considered, but is used to dispose of them shortly thereafter.     

Predicting garden path sentences

This work is an investigation into part of the human sentence parsing mechanism (HSPM). The major test of the psychological validity of any model of the HSPM is that it fail on precisely those sentences that humans find to be garden paths. It is hypothesized that the HSPM consists of at least two processes. We call the first process the syntactic processor, and the second will be known as the semantic processor. It is hypothesized that the syntactic processor is unconscious, deterministic and fast, but limited. While most ambiguities are resolved on the basis of syntactic information, when the syntactic processor can no longer guarantee a correct analysis, semantic information is used to help resolve the ambiguity. This model leads to a better prediction and explanation of which sentences will cause people to garden path.     

Parsing to Learn

Learning a language by parameter setting is almost certainly less onerous than composing a grammar from scratch. But recent computational modeling of how parameters are set has shown that it is not at all the simple mechanical process sometimes imagined. Sentences must be parsed to discover the properties that select between parameter values. But the sentences that drive learning cannot be parsed with the learner's current grammar. And there is not much point in parsing them with just one new grammar. They must apparently be parsed with all possible grammars, in order to find out which one is most successful at licensing the language. The research task is to reconcile this with the fact that the human sentence parsing mechanism, even in adults, has only very limited parallel parsing capacity. I have proposed that all possible grammars can be folded into one, if parameter values are fragments of sentential tree structures that the parser can make use of where necessary to assign a structure to an input sentence. However, the problem of capacity limitations remains. The combined grammar will afford multiple analyses for some sentences, too many to be computed on-line. I propose that the parser computes only one analysis per sentence but can detect ambiguity, and that the learner makes use of unambiguous input only. This provides secure information but relatively little of it, particularly at early stages of learning where few grammars have been excluded and ambiguity is rife. I consider three solutions: improving the parser's ability to extract unambiguous information from partially ambiguous sentences, assuming default parameter values to temporarily eliminate ambiguity, reconfiguring the parameters so that some are subordinate to others and do not present themselves to the learner until the others have been set. A more radical alternative is to give up the quest for error-free learning and permit parameters to be set without regard for whether the parser may have overlooked an alternative analysis of the sentence. If it can be assumed that the human parser keeps a running tally of the parameter values it has accessed, then the learner would do nothing other than parse sentences for comprehension, as adults do. The most useful parameter values would become more and more easily accessed; the noncontributors would drop out of the running. There would be no learning mechanism at all, over and above the parser. But how accurate this system would be remains to be established.     

Syntactic analysis in sentence comprehension: Effects of dependency types and grammatical constraints

This paper presents three experiments on the parsing of Italian wh-questions that manipulate the wh-type (whovs. which-N)and the whextraction site (main clause, dependent clause with or without complementizer). The aim of these manipulations is to see whether the parser is sensitive to the type of dependencies being processed and whether the processing effects can be explained by a unique processing principle, the minimal chain principle (MCP; De Vincenzi, 1991). The results show that the parser, following the MCP, prefers structures with fewer and less complex chains. In particular: (1) There is a processing advantage for the wh-subject extractions, the structures with less complex chains; (2) there is a processing dissociation between the whoand which questions; (3) the parser respects the principle that governs the well-formedness of the empty categories (ECP).     

A Multiple‐Channel Model of Task‐Dependent Ambiguity Resolution in Sentence Comprehension

Traxler, Pickering, and Clifton (1998) found that ambiguous sentences are read faster than their unambiguous counterparts. This so‐called ambiguity advantage has presented a major challenge to classical theories of human sentence comprehension (parsing) because its most prominent explanation, in the form of the unrestricted race model (URM), assumes that parsing is non‐deterministic. Recently, Swets, Desmet, Clifton, and Ferreira (2008) have challenged the URM. They argue that readers strategically underspecify the representation of ambiguous sentences to save time, unless disambiguation is required by task demands. When disambiguation is required, however, readers assign sentences full structure—and Swets et al. provide experimental evidence to this end. On the basis of their findings, they argue against the URM and in favor of a model of task‐dependent sentence comprehension. We show through simulations that the Swets et al. data do not constitute evidence for task‐dependent parsing because they can be explained by the URM. However, we provide decisive evidence from a German self‐paced reading study consistent with Swets et al.'s general claim about task‐dependent parsing. Specifically, we show that under certain conditions, ambiguous sentences can be read more slowly than their unambiguous counterparts, suggesting that the parser may create several parses, when required. Finally, we present the first quantitative model of task‐driven disambiguation that subsumes the URM, and we show that it can explain both Swets et al.'s results and our findings.     

Competition and recency in a hybrid network model of syntactic disambiguation

The competitive attachment model of human parsing is a hybrid connectionist architecture consisting of a distributed feature passing method for establishing syntactic relations within the network, and a numeric competition mechanism for resolving ambiguities, which applies to all syntactic relations. Because the approach employs a uniform mechanism for establishing syntactic relations, and a single competition mechanism for disambiguation, the model can capture general behaviors of the human parser that hold across a range of syntactic constructions. In particular, attachment and binding relations are similarly processed and are therefore subject to the very same influences of disambuguation and processing over time. An important influence on the competitive disambiguation process is distance within the network. Decay of numeric activation, along with distributed feature passing through the network structure, has an unavoidable effect on the outcome of attachment and binding competitions. Inherent properties of the model thus lead to a principled explanation of recency effects in the human parsing of both attachment and filler/gap ambiguities.     

Parsing as a Cue-Based Retrieval Model

This paper develops a novel psycholinguistic parser and tests it against experimental and corpus reading data. The parser builds on the recent research into memory structures, which argues that memory retrieval is content-addressable and cue-based. It is shown that the theory of cue-based memory systems can be combined with transition-based parsing to produce a parser that, when combined with the cognitive architecture ACT-R, can model reading and predict online behavioral measures (reading times and regressions). The parser's modeling capacities are tested against self-paced reading experimental data (Grodner & Gibson, 2005), eye-tracking experimental data (Staub, 2011), and a self-paced reading corpus (Futrell et al., 2018).     

Anomaly Detection: Eye Movement Patterns

The symptom of a garden path in sentence processing is an apparent anomaly in the input string. This anomaly signals to the parser that an error has occurred, and provides cues for how to repair it. Anomaly detection is thus an important aspect of sentence processing. In the present study, we investigated how the parser responds to unambiguous sentences that contain syntactic anomalies and pragmatic anomalies, examining records of eye movement during reading. While sensitivity to the two kinds of anomaly was very rapid and essentially simultaneous, qualitative differences existed in the patterns of first-pass reading times and eye regressions. The results are compatible with the proposal that syntactic information and pragmatic information are used differently in garden-path recovery.     

Prediction and substantiation: A new approach to natural language processing

This paper describes a new approach to natural language processing which results in a very robust and efficient system. The approach taken is to integrate the parser with the rest of the system. This enables the parser to benefit from predictions that the rest of the system makes in the course of its processing. These predictions can be invaluable as guides to the parser in such difficult problem areas as resolving referents and selecting meanings of ambiguous words. A program, called FRUMP for Fast Reading Understanding and Memory Program, employs this approach to parsing. FRUMP skims articles rather than reading them for detail. The program works on the relatively unconstrained domain of news articles. It routinely understands stories it has never before seen. The program's success is largely due to its radically different approach to parsing.     

Learning To Parse?

A strong claim about human sentence comprehension is that the processing mechanism is fully innate and applies differently to different languages only to the extent that their grammars differ. If so, there is hope for an explanatory project which attributes all parsing strategies to fundamental design characteristics of the parsing device. However, the whole explanatory program is in peril because of the discovery (Cuetos & Mitchell, 1988) that Late Closure is not universal: Spanish, and also Dutch and other languages, favor Early Closure (high attachment) where English favors Late Closure flow attachment). I argue that the universal parser can weather this storm. Exceptions to Late Closure in Spanish and other languages are observed only in one construction (a relative clause attaching into a complex noun phrase [NP]), which is borderline in English too. For other constructions, low attachment is preferred in all languages tested. I propose that what differentiates the complex NP construction is the heaviness of the attachee compared to that of the host configuration. A relative clause is a heavy attachee, and the lower NP alone is small as a host; the relative is therefore better balanced if the whole complex NP is its host. A wide range of facts is accounted for by the principle that a constituent likes to have a sister of its own size. Light constituents will tend to attach low, and heavy ones to attach high, since larger constituents are dominated by higher nodes. A preference for balanced weight is familiar from work on prosodic phrasing. I suggest, therefore, that prosodic processing occurs in parallel with syntactic processing (even in reading) and influences structural ambiguity resolution. Height of attachment ambiguities are resolved by the prosodically motivated same-size-sister constraint. The exceptional behavior of English may be due to its prosodic packaging of a relative pronoun with the adjacent noun, overriding the balance tendency. If this explanation is correct, it is possible that all cross-language variations in parsing preferences are due to cross-language variations in the prosodic component of the competence grammar.     

Syntactic structure assembly in human parsing: a computational model based on competitive inhibition and a lexicalist grammar

We present the design, implementation and simulation results of a psycholinguistic model of human syntactic processing that meets major empirical criteria. The parser operates in conjunction with a lexicalist grammar and is driven by syntactic information associated with heads of phrases. The dynamics of the model are based on competition by lateral inhibition ('competitive inhibition'). Input words activate lexical frames (i.e. elementary trees anchored to input words) in the mental lexicon, and a network of candidate 'unification links' is set up between frame nodes. These links represent tentative attachments that are graded rather than all-or-none. Candidate links that, due to grammatical or 'treehood' constraints, are incompatible, compete for inclusion in the final syntactic tree by sending each other inhibitory signals that reduce the competitor's attachment strength. The outcome of these local and simultaneous competitions is controlled by dynamic parameters, in particular by the Entry Activation and the Activation Decay rate of syntactic nodes, and by the Strength and Strength Build-up rate of Unification links. In case of a successful parse, a single syntactic tree is returned that covers the whole input string and consists of lexical frames connected by winning Unification links. Simulations are reported of a significant range of psycholinguistic parsing phenomena in both normal and aphasic speakers of English: (i) various effects of linguistic complexity (single versus double, center versus right-hand self-embeddings of relative clauses; the difference between relative clauses with subject and object extraction; the contrast between a complement clause embedded within a relative clause versus a relative clause embedded within a complement clause); (ii) effects of local and global ambiguity, and of word-class and syntactic ambiguity (including recency and length effects); (iii) certain difficulty-of-reanalysis effects (contrasts between local ambiguities that are easy to resolve versus ones that lead to serious garden-path effects); (iv) effects of agrammatism on parsing performance, in particular the performance of various groups of aphasic patients on several sentence types.     

The influence of contextual contrast on syntactic processing: evidence for strong-interaction in sentence comprehension

The present study compares the processing of unambiguous restrictive and non-restrictive relative clauses (RCs) within both a null context and a supportive discourse using a self-paced reading methodology. Individuals read restrictive RCs more slowly than non-restrictive RCs in a null context, but processed restrictive RCs faster than non-restrictive RCs in supportive context, resulting in an interaction between context and RC type. These results provide evidence for two theoretical points. First, principles analogous to those in referential theory [Altmann G. T. M., & Steedman, M. (1988). Interaction with context during human sentence processing. Cognition, 30, 191-238; Crain, S., & Steedman, M. (1985). On not being led up the garden path: The use of context by the psychological parser. In D. Dowty, L. Karttunnen, A. Zwicky (Eds.), Natural language parsing. Cambridge, UK: Cambridge University Press] apply not only in resolving ambiguity but also in processing unambiguous sentences. Second, the discourse context can guide and facilitate interpretive processing. This result suggests that intrasentential factors such as syntax are not autonomous from contextual processing, contrary to the modularity hypothesis [Fodor, J. A. (1983). Modularity of mind. Cambridge, MA: MIT Press].     

TSUNAMI: Simultaneous Understanding,Answering, and Memory Interaction for Questions

Question processing involves parsing, memory retrieval, question categorization, initiation of appropriate answer-retrieval heuristics, answer formulation, and output. Computational and psychological models have traditionally treated these processes as separate, sequential, independent, and in pursuit of a single answer type at a time. Here this view is challenged and the implications of a theory in which question processes operate simultaneously on multiple question interpretations are explored. A highly interactive model is described in which an expectation-driven parser generates multiple question candidates, including partially-specified candidates. Question candidates act as constraints for a matcher which activates memory items. An answer retrieval process examines question candidates and the active portions of memory in an attempt to generate answer candidates. Answer candidates are examined by an output process that derives the final answer. These processes run simultaneously and interact. Three experiments on human question answering are also described which provide evidence that working memory load during question reading is affected by processes related to answer retrieval.     

Simple,applied text parsing

Text parsing for use in linguistic or artificial intelligence research can often be ill-suited for use in other lines of investigation in which simple, problem-specific parsing techniques may be used. This paper describes the development of a simple text parser for use in a specific research setting. The resulting parsing programs are compact enough for easy use on most available m inland microcomputers, while still providing adequate results for some applications such as chunking text into small units for later presentation.