The Value of Symbolic Computation

Standard generative linguistic theory, which uses discrete symbolic models of cognition, has some strengths and weaknesses. It is strong on providing a network of outposts that make scientific travel in the jungles of natural language feasible. It is weak in that it currently depends on the elaborate and unformalized use of intuition to develop critical supporting assumptions about each data point. In this regard, it is not in a position to characterize natural language systems in the lawful terms that ecological psychologists strive for. Connectionist learning models offer some help: They define lawful relations between linguistic environments and language systems. But our understanding of them is currently weak, especially when it comes to natural language syntax. Fortunately, symbolic linguistic analysis can help connectionism if the two meet via dynamical systems theory. I discuss a case in point: Insights from linguistic explorations of natural language syntax appear to have identified information structures that are particularly relevant to understanding ecologically appealing but analytically mysterious connectionist learning models.     

Are rules and modules really necessary for explaining language?

Over the past few years Steven Pinker has argued that although some aspects of language may be more associational, and therefore properly modeled in connectionist networks, for the most part human language is still best characterized as a modularized set of rulesymbol systems. In support of his claim, Pinker garners a broad array of clinical, experimental, and observational data from neurology, psychology, and linguistics. Those data, unfortunately, are not compelling because they do not support his position uniquely. In this paper, I show how each of his arguments is compatible with alternative interpretations. I argue, moreover, that in focusing on certain details of connectionist models Pinker and his colleagues actually overlooked both the most serious deficiencies of the connectionist approach and its most significant theoretical contribution. I conclude by sketching briefly some emerging alternatives to connectionism which avoid those deficiencies while retaining its strengths over the rule-symbol systems of linguistic theory.This research was funded in part by University of Idaho Seed Grant 681-Y304.     

Connectionist representation,semantic compositionality,and the instability of concept structure

Summary In the present paper connectionist approaches to the problem of internal representation and the nature of concepts are discussed. In the first part the concept of representation that underlies connectionist modeling is made explicit. It is argued that the connectionist view of representation relies on a correlational theory of semantic content- i.e., the covariation between internal and external states is taken as the basis for ascribing meaning to internal states. The problems and virtues of such a correlational approach to internal representation are addressed. The second part of the paper is concerned with whether connectionism is capable of accounting for the apparent productivity and systematicity of language and thought. There is an evaluation of the recent arguments of Fodor and Pylyshyn, who claim that systematicity can only be explained if one conceives of mental representations as structured symbols composed of context-free constituents. There is a review of empirical evidence that strongly suggests that concepts are not fixed memory structures and that the meaning of constituent symbols varies, depending on the context in which they are embedded. On the basis of this review it is concluded that the meaning of a complex expression is not computed from the context-free meanings of the constituents, and that strong compositionality, as endorsed by Fodor and Pylyshyn (1988), seems implausible as a process theory for the comprehension of complex concepts. Instead, the hypothesis is endorsed that constraint satisfaction in distributed connectionist networks may allow for an alternative account of weak compositionality compatible with the context sensitivity of meaning. In the final section, it is argued that neither mere implementation of a language of thought in connectionist networks nor radical elimination of symbol systems seems to be a fruitful research strategy, but that it might be more useful to discuss how connectionist systems can develop the capacity to use external symbol systems like language or logic without instantiating symbol systems themselves.     

Grammar-based Connectionist Approaches to Language

This article describes an approach to connectionist language research that relies on the development of grammar formalisms rather than computer models. From formulations of the fundamental theoretical commitments of connectionism and of generative grammar, it is argued that these two paradigms are mutually compatible. Integrating the basic assumptions of the paradigms results in formal theories of grammar that centrally incorporate a certain degree of connectionist computation. Two such grammar formalisms—Harmonic Grammar and Optimality Theory —are briefly introduced to illustrate grammar-based approaches to connectionist language research. The strengths and weaknesses of grammar-based research and more traditional model-based research are argued to be complementary, suggesting a significant role for both strategies in the spectrum of connectionist language research.     

Connectionism and three levels of nativism

Along with the increasing popularity of connectionist language models has come a number of provocative suggestions about the challenge these models present to Chomsky's arguments for nativism. The aim of this paper is to assess these claims. We begin by reconstructing Chomsky's argument from the poverty of the stimulus and arguing that it is best understood as three related arguments, with increasingly strong conclusions. Next, we provide a brief introduction to connectionism and give a quick survey of recent efforts to develop networks that model various aspects of human linguistic behavior. Finally, we explore the implications of this research for Chomsky's arguments. Our claim is that the relation between connectionism and Chomsky's views on innate knowledge is more complicated than many have assumed, and that even if these models enjoy considerable success the threat they pose for linguistic nativism is small.     

Implicit and explicit memory: Implications for the symbol-manipulation versus connectionism controversy

Summary This paper deals with three issues. First, the symbol-manipulation versus connectionism controversy is discussed briefly. Both can be seen as formalisms or languages for describing human behavior, but it is argued that these languages are not fully equivalent. The importance of the ability of connectionist models to learn autonomously, often underestimated, allows them to perform tasks that may defy formalization. Secondly, the structural and functional characteristics of a new connectionist learning model are presented. It avoids some of the psychological and biological implausibilities of currently popular models and solves some of the problems and shortcomings of these models. Thirdly, some simulation results are presented. The model successfully simulates the dissociation between explicit and implicit memory performance as found in patients with anterograde amnesia. This simulation provides an example of the advantages of using the connectionist language in the domain of memory psychology. It is also a good example of how psychological evidence can be used to improve connectionist models.The second author is supported by the Dutch Organization for Scientific Research (NWO)     

Systematicity redux

One of the main challenges that Jerry Fodor and Zenon Pylyshyn (Cognition 28:3–71, 1988) posed for any connectionist theory of cognitive architecture is to explain the systematicity of thought without implementing a Language of Thought (LOT) architecture. The systematicity challenge presents a dilemma: if connectionism cannot explain the systematicity of thought, then it fails to offer an adequate theory of cognitive architecture; and if it explains the systematicity of thought by implementing a LOT architecture, then it fails to offer an alternative to the LOT hypothesis. Given that thought is systematic, connectionism can offer an adequate alternative to the LOT hypothesis only if it can meet the challenge. Although some critics tried to meet the challenge, others argued that it need not be met since thought is not in fact systematic; and some claimed not to even understand the claim that thought is systematic. I do not here examine attempts to answer the challenge. Instead, I defend the challenge itself by explicating the notion of systematicity in a way that I hope makes clear that thought is indeed systematic, and so that to offer an adequate alternative to the LOT hypothesis, connectionism must meet the challenge.     

The relationship between connectionist models and a dynamic data-oriented theory of concept formation

In this paper I shall compare two models of concept formation, both inspired by basic convictions of philosophical empiricism. The first, the connectionist model, will be exemplified by Kohonen maps, and the second will be my own dynamic theory of concept formation. Both can be understood in probabilistic terms, both use a notion of convergence or stabilization in modelling how concepts are built up. Both admit destabilization of concepts and conceptual change. Both do not use a notion of representation in some pregiven language, such as a language of thought or some logical language. Representation in a formal language only plays a role on the meta-level, namely within the theory about concept formation.A short version of this paper was given at the Fachtagung der Gesellschaft für Kognitionswissenschaft. Freiburg, October 12–15th, 1994.     

Classical and connectionist models: Levels of description

To begin, I introduce an analysis of interlevel relations that allows us to offer an initial characterization of the debate about the way classical and connectionist models relate. Subsequently, I examine a compatibility thesis and a conditional claim on this issue. With respect to the compatibility thesis, I argue that, even if classical and connectionist models are not necessarily incompatible, the emergence of the latter seems to undermine the best arguments for the Language of Thought Hypothesis, which is essential to the former. I attack the conditional claim of connectionism to eliminativism, presented by Ramsey et al. (1990), by discrediting their discrete characterization of common-sense psychological explanations and pointing to the presence of a moderate holistic constraint. Finally, I conclude that neither of the arguments considered excludes the possibility of viewing connectionist models as forming a part of a representational theory of cognition that dispenses with the Language of Thought Hypothesis.     

Tools for connectionist modeling: The dynamical systems methodology

Some understanding of dynamical systems is essential to achieving competency in connectionist models. This mathematical background can be acquired either through a rigorous set of upper undergraduate and/or graduate formal courses or via disciplined self-teaching. As part of developing a course in connectionism, we feel that although certain very basic mathematical tools are most appropriately learned in their “pure” form (i.e., from mathematics textbooks and courses), more advanced exposure to dynamical systems theory can be given in the context of an introduction to connectionism. Students thus learn to write connectionist simulations by first writing programs for simulating arbitrary dynamical systems, then using them to learn some aspects of dynamical systems in general by simulating some special cases, and finally applying this technique to connectionist models of increasing complexity.     

心理表征哲学及其联结主义诠释

对心理表征问题进行自然的解释是至关重要的,但这方面的研究都是建立在有关概念和命题表征的预设的基础之上的,所以有必要弄清概念表征和命题表征的哲学解释;联结主义也对心理表征问题作了解释,根据被表征的是什么以及通过什么来表征的原则,联结主义网络主要通过四种方式对概念和命题进行表征。总的来说,联结主义对概念和命题的表征与它们的哲学解释并不是一一对应的,其方法论基础是费耶阿本德的多元方法论,即"怎么都行"。     

Representations without rules,connectionism and the syntactic argument

Terry Horgan and John Tienson have suggested that connectionism might provide a framework within which to articulate a theory of cognition according to which there are mental representations without rules (RWR) (Horgan and Tienson 1988, 1989, 1991, 1992). In essence, RWR states that cognition involves representations in a language of thought, but that these representations are not manipulated by the sort of rules that have traditionally been posited. In the development of RWR, Horgan and Tienson attempt to forestall a particular line of criticism, theSyntactic Argument, which would show RWR to be inconsistent with connectionism. In essence, the argument claims that the node-level rules of connectionist networks, along with the semantic interpretations assigned to patterns of activation, serve to determine a set of representation-level rules incompatible with the RWR conception of cognition. The present paper argues that the Syntactic Argument can be made to show that RWR is inconsistent with connectionism.The present paper has been improved by conversations with Terry Horgan and John Tienson. Thanks are also due to Gary Fuller, John Heil, Terry Horgan and Bob Stecker for comments on earlier drafts of this paper.     

A symbolic-connectionist theory of relational inference and generalization

The authors present a theory of how relational inference and generalization can be accomplished within a cognitive architecture that is psychologically and neurally realistic. Their proposal is a form of symbolic connectionism: a connectionist system based on distributed representations of concept meanings, using temporal synchrony to bind fillers and roles into relational structures. The authors present a specific instantiation of their theory in the form of a computer simulation model, Learning and Inference with Schemas and Analogies (LISA). By using a kind of self-supervised learning, LISA can make specific inferences and form new relational generalizations and can hence acquire new schemas by induction from examples. The authors demonstrate the sufficiency of the model by using it to simulate a body of empirical phenomena concerning analogical inference and relational generalization.     

Theories of language acquisition

Prior to the advent of generative grammar, theoretical approaches to language development relied heavily upon the concepts ofdifferential reinforcement andimitation. Current studies of linguistic acquisition are largely dominated by the hypothesis that the child constructs his language on the basis of a primitive grammar which gradually evolves into a more complex grammar. This approach presupposes that the investigator does not impose his own grammatical rules on the utterances of the child; that the sound system of the child and the rules he employs to form sentences are to be described in their own terms, independently of the model provided by the adult linguistic community; and that there is a series of steps or stages through which the child passes on his way toward mastery of the adult grammar in his linguistic environment. This paper attempts to trace the development of human vocalization through prelinguistic stages to the development of what can be clearly recognized as language behavior, and then progresses to transitional phases in which the language of the child begins to approximate that of the adult model. In the view of the authors, the most challenging problems which confront theories of linguistic acquisition arise in seeking to account for structure of sound sequences, in the rules that enable the speaker to go from meaning to sound and which enable the listener to go from sound to meaning. The principal area of concern for the investigator, according to the authors, is the discovery of those rules at various stages of the learning process. The paper concludes with a return to the question of what constitutes an adequate theory of language ontogenesis. It is suggested that such a theory will have to be keyed to theories of cognitive development and will have to include and go beyond a theory which accounts for adult language competence and performance, since these represent only the terminal stage of linguistic ontogenesis.     

The past and future of the past tense

What is the interaction between storage and computation in language processing? What is the psychological status of grammatical rules? What are the relative strengths of connectionist and symbolic models of cognition? How are the components of language implemented in the brain? The English past tense has served as an arena for debates on these issues. We defend the theory that irregular past-tense forms are stored in the lexicon, a division of declarative memory, whereas regular forms can be computed by a concatenation rule, which requires the procedural system. Irregulars have the psychological, linguistic and neuropsychological signatures of lexical memory, whereas regulars often have the signatures of grammatical processing. Furthermore, because regular inflection is rule-driven, speakers can apply it whenever memory fails.     

多备择决策的联结网络模型

关于选择偏好可变性的心理机制,新联结主义框架下的联结网络模型强调决策是动态的随机的过程.其中,多备择决策场理论强调由心理场距离决定水平抑制强度的线性关系,渗漏竞争累积模型强调主观价值评价过程中非线性的规避损失偏向.     

Was Heidegger an Externalist?

To address the question posed in the title, I focus on Heidegger's conception of linguistic communication developed in the sections on Rede and Gerede of Being and Time. On the basis of a detailed analysis of these sections I argue that Heidegger was a social externalist but semantic internalist. To make this claim, however, I first need to clarify some key points that have led critics to assume Heidegger's commitment to social externalism automatically commits him to semantic externalism regarding concept use. I begin by explaining the independence of those positions, arguing that social externalism answers the question of whose concepts in a linguistic community are properly individuated, whereas semantic externalism makes a claim about what it takes for concepts to be properly individuated. Once these issues are distinguished, it is possible to see that Heidegger's intersubjectivist conception of language commits him to social externalism, while his conception of the ontological difference commits him to semantic internalism.     

Connectionism: Is it a paradigm shift for psychology?

Connectionism is a method of modeling cognition as the interaction of neuron-like units. Connectionism has received a gread deal of interest and may represent a paradigm shift for psychology. The nature of a paradigm shift (Kuhn, 1970) is reviewed with respect to connectionism. The reader is provided an overview on connectionism including: an introduction to connectionist modeling, new issues it emphasizes, a brief history, its developing sociopolitical impact, theoretical impact, and empirical impact. Cautions, concerns, and enthusiasm for connectionism are expressed.     

What does it mean to ‘live’ and ‘die’? A cross‐linguistic analysis of parent–child conversations in English and Indonesian

Previous work on children's intuitive knowledge about the natural world has documented their difficulty in acquiring an overarching concept of biological life that includes plants as well as humans and non‐human animals. It has also suggested that the acquisition of fundamental biological concepts like alive and die may be influenced by the language used to describe them, as evidenced by differences between English‐ and Indonesian‐speaking children's performance in tasks involving these concepts. Here, we examine one particularly important source of linguistic information available to children during this acquisition process: everyday conversations with their parents. We take a cross‐linguistic approach in analysing the evidence available to English‐ and Indonesian‐speaking children as they acquire meanings for words corresponding to the concepts alive and die . Our analysis illustrates that young children acquiring English and Indonesian are faced with distinct problems, but that parental input in both languages does little to support the acquisition of broad, inclusive biological concepts.     

Challenging the importance of vision for the development of an extrinsic spatial framework: evidence from the blind and sighted

Although connectionism is advocated by its proponents as an alternative to the classical computational theory of mind, doubts persist about its computational credentials. Our aim is to dispel these doubts by explaining how connectionist networks compute. We first develop a generic account of computation-no easy task, because computation, like almost every other foundational concept in cognitive science, has resisted canonical definition. We opt for a characterisation that does justice to the explanatory role of computation in cognitive science. Next we examine what might be regarded as the "conventional" account of connectionist computation. We show why this account is inadequate and hence fosters the suspicion that connectionist networks are not genuinely computational. Lastly, we turn to the principal task of the paper: the development of a more robust portrait of connectionist computation. The basis of this portrait is an explanation of the representational capacities of connection weights, supported by an analysis of the weight configurations of a series of simulated neural networks.