Constraint satisfaction as a theory of sentence processing

Various problems with the constraint satisfaction model are discussed. It is argued that the empirical evidence presented in support of the model does not concern predictions of the model that diverge from those of depth-first (one analysis at a time) models. Several methodological problems are also noted. As a theory of sentence processing, the model is inadequate. It fails to account for the assignment of local structure, global structure, structure involving discontinuous dependencies, long-distance dependencies, and adjunct phrases. It makes incorrect predictions about the timing of syntactic analysis. Further, because syntactic structure is available only through activation of syntactic projections stored in the lexical entry of words, the model leaves entirely unexplained the myriad psycholinguistic findings demonstrating independence of lexical and syntactic structure (in Event Related Potential studies, code-switching, pure syntactic priming, etc). Finally, the model is not restrictive or explanatory, providing an account that largely consists ofpost hoc correlations between frequency counts or subjects ratings of sentences and processing time data for the same sentences.The present paper is an excerpt from Sentence (Re-)Analysis presented at the 1994 CUNY Conference. It was supported by NIH Grant HD18708 to Clifton and Frazier and DBS9121375 to Rayner and Frazier. I am very grateful to Chuck Clifton and to an anonymous reviewer for comments on an earlier draft of this paper.     

A probabilistic corpus-based model of syntactic parallelism

Work in experimental psycholinguistics has shown that the processing of coordinate structures is facilitated when the two conjuncts share the same syntactic structure [Frazier, L., Munn, A., & Clifton, C. (2000). Processing coordinate structures. Journal of Psycholinguistic Research, 29(4) 343–370]. In the present paper, we argue that this parallelism effect is a specific case of the more general phenomenon of syntactic priming—the tendency to repeat recently used syntactic structures. We show that there is a significant tendency for structural repetition in corpora, and that this tendency is not limited to syntactic environments involving coordination, though it is greater in these environments. We present two different implementations of a syntactic priming mechanism in a probabilistic parsing model and test their predictions against experimental data on NP parallelism in English. Based on these results, we argue that a general purpose priming mechanism is preferred over a special mechanism limited to coordination. Finally, we show how notions of activation and decay from ACT-R can be incorporated in the model, enabling it to account for a set of experimental data on sentential parallelism in German.     

Constituent order and semantic parallelism in online comprehension: Eye-tracking evidence from German

Reading times for the second conjunct of and-coordinated clauses are faster when the second conjunct parallels the first conjunct in its syntactic or semantic (animacy) structure than when its structure differs (Frazier, Munn, & Clifton, 2000; Frazier, Taft, Roeper, & Clifton, 1984). What remains unclear, however, is the time course of parallelism effects, their scope, and the kinds of linguistic information to which they are sensitive. Findings from the first two eye-tracking experiments revealed incremental constituent order parallelism across the board—both during structural disambiguation (Experiment 1) and in sentences with unambiguously case-marked constituent order (Experiment 2), as well as for both marked and unmarked constituent orders (Experiments 1 and 2). Findings from Experiment 3 revealed effects of both constituent order and subtle semantic (noun phrase similarity) parallelism. Together our findings provide evidence for an across-the-board account of parallelism for processing and-coordinated clauses, in which both constituent order and semantic aspects of representations contribute towards incremental parallelism effects. We discuss our findings in the context of existing findings on parallelism and priming, as well as mechanisms of sentence processing.     

Construal: Overview, Motivation, and Some New Evidence

Is there underspecification in the syntactic phrase marker constructed during on-line sentence analysis? According to the construal hypothesis (Frazier &; Clifton, 1996), a very limited amount and type of structural underspecification is available to the human sentence parsing mechanism. Here we present the basic definitions of construal, illustrating the theory with some already published evidence. We also discuss several new pieces of evidence, from our laboratory and elsewhere, that support the construal hypothesis. We end by raising the question of what kind of mechanism operates in the process of interpreting a nonprimary phrase (a phrase that receives an underspecified syntactic analysis), and conclude that it is not a process of competition between multiple activated possible analyses but instead is a process in which the sheer existence of ambiguity need not result in increased processing cost.     

Interaction with context during human sentence processing

Psychological theories of natural language processing have usually assumed that the sentence processor resolves local syntactic ambiguities by selecting a single analysis on the basis of structural criteria such as Frazier's (1978) “minimal attachment.” According to such theories, alternative analyses will only be attempted if the initial analysis subsequently proves inconsistent with the context. (See also Ferreira & Clifton, 1986; Ford, Bresnan, & Kaplan, 1982; Rayner, Carlson, & Frazier, 1983). An alternative hypothesis exists, however: If sentences are understood incrementally, more or less word-by-word (Marlsen-Wilson, 1973, 1975), then syntactic processing can in principle exploit the fact that interpretations are available, using them “interactively” to select among alternative syntactic analyses on the basis of their plausibility with respect to the context. The present paper considers possible architectures for such incremental and interactive sentence processors, and argues for an architecture     

Children's comprehension of sentences with focus particles

We report three studies investigating children's and adults' comprehension of sentences containing the focus particle only. In Experiments 1 and 2, four groups of participants (6-7 years, 8-10 years, 11-12 years and adult) compared sentences with only in different syntactic positions against pictures that matched or mismatched events described by the sentence. Contrary to previous findings (Crain, S., Ni, W., & Conway, L. (1994). Learning, parsing and modularity. In C. Clifton, L. Frazier, & K. Rayner (Eds.), Perspectives on sentence processing. Hillsdale, NJ: Lawrence Erlbaum; Philip, W., & Lynch, E. (1999). Felicity, relevance, and acquisition of the grammar of every and only. In S. C. Howell, S. A. Fish, & T. Keith-Lucas (Eds.), Proceedings of the 24th annual Boston University conference on language development. Somerville, MA: Cascadilla Press) we found that young children predominantly made errors by failing to process contrast information rather than errors in which they failed to use syntactic information to restrict the scope of the particle. Experiment 3 replicated these findings with pre-schoolers.     

Redundancy and reduction: Speakers manage syntactic information density

A principle of efficient language production based on information theoretic considerations is proposed: Uniform Information Density predicts that language production is affected by a preference to distribute information uniformly across the linguistic signal. This prediction is tested against data from syntactic reduction. A single multilevel logit model analysis of naturally distributed data from a corpus of spontaneous speech is used to assess the effect of information density on complementizer that-mentioning, while simultaneously evaluating the predictions of several influential alternative accounts: availability, ambiguity avoidance, and dependency processing accounts. Information density emerges as an important predictor of speakers’ preferences during production. As information is defined in terms of probabilities, it follows that production is probability-sensitive, in that speakers’ preferences are affected by the contextual probability of syntactic structures. The merits of a corpus-based approach to the study of language production are discussed as well.     

Jerry Fodor on non-conceptual content

Proponents of non-conceptual content have recruited it for various philosophical jobs. Some epistemologists have suggested that it may play the role of “the given” that Sellars is supposed to have exorcised from philosophy. Some philosophers of mind (e.g., Dretske) have suggested that it plays an important role in the project of naturalizing semantics as a kind of halfway between merely information bearing and possessing conceptual content. Here I will focus on a recent proposal by Jerry Fodor. In a recent paper he characterizes non-conceptual content in a particular way and argues that it is plausible that it plays an explanatory role in accounting for certain auditory and visual phenomena. So he thinks that there is reason to believe that there is non-conceptual content. On the other hand, Fodor thinks that non-conceptual content has a limited role. It occurs only in the very early stages of perceptual processing prior to conscious awareness. My paper is examines Fodor’s characterization of non-conceptual content and his claims for its explanatory importance. I also discuss if Fodor has made a case for limiting non-conceptual content to non-conscious, sub-personal mental states. This paper has grown out of comments I made on Fodor’s paper “Revenge of the Given,” delivered at The Steven Humphrey Excellence in Philosophy Conference: “Content and Concepts: A Conference on the Philosophy of Mind” at the University of California, Santa Barbara, February 14, 2004. Thanks to Jerry Fodor for useful feedback, and to Barry Loewer for discussing with me the ideas that went into this paper.     

Differences in the Timing of Implausibility Detection for Recipient and Instrument Prepositional Phrases

We conducted two word-by-word reading experiments to investigate the timing of implausibility detection for recipient and instrument prepositional phrases (PPs). These PPs differ in thematic role, relative frequency, and possibly in argument status. The results showed a difference in the timing of garden path effects such that the detection of implausible dative recipients (which are clearly arguments) was delayed relative to the detection of implausible instruments (which may not be arguments). They also demonstrated that commitments to syntactic structure were made at the preposition for both dative and instrument PPs. While these results refute delay models of parsing (e.g., Britt, 1994) and syntax-first accounts of PP-attachment (e.g., Frazier, 1978; Frazier & Clifton, 1996), they support constraint-based lexicalist models that enable verb bias and plausibility information to compete (Garnsey, Pearlmutter, Myers, & Lotocky, 1997).     

Learning first-pass structural attachment preferences with dynamic grammars and recursive neural networks

One of the central problems in the study of human language processing is ambiguity resolution: how do people resolve the extremely pervasive ambiguity of the language they encounter? One possible answer to this question is suggested by experience-based models, which claim that people typically resolve ambiguities in a way which has been successful in the past. In order to determine the course of action that has been "successful in the past" when faced with some ambiguity, it is necessary to generalize over past experience. In this paper, we will present a computational experience-based model, which learns to generalize over linguistic experience from exposure to syntactic structures in a corpus. The model is a hybrid system, which uses symbolic grammars to build and represent syntactic structures, and neural networks to rank these structures on the basis of its experience. We use a dynamic grammar, which provides a very tight correspondence between grammatical derivations and incremental processing, and recursive neural networks, which are able to deal with the complex hierarchical structures produced by the grammar. We demonstrate that the model reproduces a number of the structural preferences found in the experimental psycholinguistics literature, and also performs well on unrestricted text.     

Studying the grammatical aspects of word recognition: lexical priming,parsing, and syntactic ambiguity resolution

Two experiments are reported examining the relationship between lexical and syntactic processing during language comprehension, combining techniques common to the on-line study of syntactic ambiguity resolution with priming techniques common to the study of lexical processing. By manipulating grammatical properties of lexical primes, we explore how lexically based knowledge is activated and guides combinatory sentence processing. Particularly, we find that nouns (like verbs, see Trueswell & Kim, 1998) can activate detailed lexically specific syntactic information and that these representations guide the resolution of relevant syntactic ambiguities pertaining to verb argument structure. These findings suggest that certain principles of knowledge representation common to theories of lexical knowledge—such as overlapping and distributed representations—also characterize grammatical knowledge. Additionally, observations from an auditory comprehension study suggest similar conclusions about the lexical nature of parsing in spoken language comprehension. They also suggest that thematic role and syntactic preferences are activated during word recognition and that both influence combinatory processing.     

De l’impossibilité d’un traitement syntaxique parafovéal à longue portée : une réplique à Foucambert (2008)

Based on results with a new method, Foucambert (2008) argued that syntactic processing can occur as far as 27 characters to the right of the fixation point. In this reply, we demonstrate that the method used by Foucambert was not appropriate. In addition, Foucambert's results did not support his conclusion. We replicate Foucambert's study while monitoring eye movements. Results revealed that subjects’ eye movements were biased to the right of the fixation point. In addition, subjects were unable to engage in syntactic processing of words far in periphery. Taken together, results fit well with current models of eye movements and of the missing-letter effect.     

A probabilistic model of semantic plausibility in sentence processing

Experimental research shows that human sentence processing uses information from different levels of linguistic analysis, for example, lexical and syntactic preferences as well as semantic plausibility. Existing computational models of human sentence processing, however, have focused primarily on lexico-syntactic factors. Those models that do account for semantic plausibility effects lack a general model of human plausibility intuitions at the sentence level. Within a probabilistic framework, we propose a wide-coverage model that both assigns thematic roles to verb–argument pairs and determines a preferred interpretation by evaluating the plausibility of the resulting ( verb , role , argument ) triples. The model is trained on a corpus of role-annotated language data. We also present a transparent integration of the semantic model with an incremental probabilistic parser. We demonstrate that both the semantic plausibility model and the combined syntax/semantics model predict judgment and reading time data from the experimental literature.     

Sentence Complexity and Working Memory Effects in Ambiguity Resolution

Two self-paced reading experiments using a paraphrase decision task paradigm were performed to investigate how sentence complexity contributed to the relative clause (RC) attachment preferences of speakers of different working memory capacities (WMCs). Experiment 1 (English) showed working memory effects on relative clause processing in both offline RC attachment preferences and in online reading time measures, but no effects of syntactic complexity. In Experiment 2 (Korean), syntactic complexity due to greater distance between integrating heads, as measured by the dependency locality theory (Gibson in Cognition 68:1–76, 1998), significantly increased the proportion of attachment to NP1. However, no effects of working memory were found. The difference in results between English and Korean is proposed to be due to head-directionality effects. The results of our study support the conclusion that working memory-based accounts provide a better explanation than previous language-dependent accounts for differences in RC attachment preferences. We propose that previous language dependent-accounts of cross-linguistic differences in RC processing have overlooked the interaction between individual WMC and a language’s general structure, which is a central factor in RC attachment.     

Constituent Length Affects Prosody and Processing for a Dative NP Ambiguity in Korean

Two sentence processing experiments on a dative NP ambiguity in Korean demonstrate effects of phrase length on overt and implicit prosody. Both experiments controlled non-prosodic length factors by using long versus short proper names that occurred before the syntactically critical material. Experiment 1 found that long phrases induce different prosodic phrasing than short phrases in a read-aloud task and change the preferred interpretation of globally ambiguous sentences. It also showed that speakers who have been told of the ambiguity can provide significantly different prosody for the two interpretations, for both lengths. Experiment 2 verified that prosodic patterns found in first-pass pronunciations predict self-paced reading patterns for silent reading. The results extend the coverage of the Implicit Prosody Hypothesis [Fodor, J Psycholinguist Res 27:285–319, 1998; Prosodic disambiguation in silent reading. In M. Hirotani (Ed.), NELS 32 (pp. 113–132). Amherst, MA: GLSA Publications, 2002] to another construction and to Korean. They further indicate that strong syntactic biases can have rapid effects on the formulation of implicit prosody.     

Connectionist semantic systematicity

Fodor and Pylyshyn [Fodor, J. A., & Pylyshyn, Z. W. (1988). Connectionism and cognitive architecture: A critical analysis. Cognition, 28, 3–71] argue that connectionist models are not able to display systematicity other than by implementing a classical symbol system. This claim entails that connectionism cannot compete with the classical approach as an alternative architectural framework for human cognition. We present a connectionist model of sentence comprehension that does not implement a symbol system yet behaves systematically. It consists in a recurrent neural network that maps sentences describing situations in a microworld, onto representations of these situations. After being trained on particular sentence–situation pairs, the model can comprehend new sentences, even if these describe new situations. We argue that this systematicity arises robustly and in a psychologically plausible manner because it depends on structure inherent in the world.     

Cognition without classical architecture

Fodor and Pylyshyn (1988) argue that any successful model of cognition must use classical architecture; it must depend upon rule-based processing sensitive to constituent structure. This claim is central to their defense of classical AI against the recent enthusiasm for connectionism. Connectionist nets, they contend, may serve as theories of the implementation of cognition, but never as proper theories of psychology. Connectionist models are doomed to describing the brain at the wrong level, leaving the classical view to account for the mind.This paper considers whether recent results in connectionist research weigh against Fodor and Pylyshyn's thesis. The investigation will force us to develop criteria for determining exactly when a net is capable of systematic processing. Fodor and Pylyshyn clearly intend their thesis to affect the course of research in psychology. I will argue that when systematicity is defined in a way that makes the thesis relevant in this way, the thesis is challenged by recent progress in connectionism.     

Verbal working memory and sentence comprehension

This target article discusses the verbal working memory system used in sentence comprehension. We review the concept of working memory as a short-duration system in which small amounts of information are simultaneously stored and manipulated in the service of accomplishing a task. We summarize the argument that syntactic processing in sentence comprehension requires such a storage and computational system. We then ask whether the working memory system used in syntactic processing is the same as that used in verbally mediated tasks that involve conscious controlled processing. Evidence is brought to bear from various sources: the relationship between individual differences in working memory and individual differences in the efficiency of syntactic processing; the effect of concurrent verbal memory load on syntactic processing; and syntactic processing in patients with poor short-term memory, patients with poor working memory, and patients with aphasia. Experimental results from these normal subjects and patients with various brain lesions converge on the conclusion that there is a specialization in the verbal working memory system for assigning the syntactic structure of a sentence and using that structure in determining sentence meaning that is separate from the working memory system underlying the use of sentence meaning to accomplish other functions. We present a theory of the divisions of the verbal working memory system and suggestions regarding its neural basis.     

Working memory constraints on the processing of syntactic ambiguity.

We propose a model that explains how the working-memory capacity of a comprehender can constrain syntactic parsing and thereby affect the processing of syntactic ambiguities. The model's predictions are examined in four experiments that measure the reading times for two constructions that contain a temporary syntactic ambiguity. An example of the syntactic ambiguity is The soldiers warned about the dangers . . . ; the verb warned may either be the main verb, in which case soldiers is the agent; or the verb warned may introduce a relative clause, in which case soldiers is the patient of warned rather than the agent, as in The soldiers warned about the dangers conducted the midnight raid. The model proposes that both alternative interpretations of warned are initially activated. However, the duration for which both interpretations are maintained depends, in part, on the reader's working-memory capacity, which can be assessed by the Reading Span task (Daneman & Carpenter, 1980). The word-by-word reading times indicate that all subjects do additional processing after encountering an ambiguity, suggesting that they generate both representations. Furthermore, readers with larger working-memory capacities maintain both representations for some period of time (several words), whereas readers with smaller working-memory capacities revert to maintaining only the more likely representation.