Connectionist semantic systematicity

Fodor and Pylyshyn [Fodor, J. A., & Pylyshyn, Z. W. (1988). Connectionism and cognitive architecture: A critical analysis. Cognition, 28, 3–71] argue that connectionist models are not able to display systematicity other than by implementing a classical symbol system. This claim entails that connectionism cannot compete with the classical approach as an alternative architectural framework for human cognition. We present a connectionist model of sentence comprehension that does not implement a symbol system yet behaves systematically. It consists in a recurrent neural network that maps sentences describing situations in a microworld, onto representations of these situations. After being trained on particular sentence–situation pairs, the model can comprehend new sentences, even if these describe new situations. We argue that this systematicity arises robustly and in a psychologically plausible manner because it depends on structure inherent in the world.     

Connectionism and the Philosophical Foundations of Cognitive Science

This is an overview of recent philosophical discussion about connectionism and the foundations of cognitive science. Connectionist modeling in cognitive science is described. Three broad conceptions of the mind are characterized, and their comparative strengths and weaknesses are discussed: (1) the classical computation conception in cognitive science; (2) a popular foundational interpretation of connectionism that John Tienson and I call "non-sentential computationalism"; and (3) an alternative interpretation of connectionism we call "dynamical cognition." Also discussed are two recent philosophical attempts to enlist connectionism in defense of eliminativism about folk psychology.     

Compositionality: A Connectionist Variation on a Classical Theme

Connectionism has been attacked on the grounds that it does not employ compositionally structured representations (e.g., Fodor & Pylyshyn, 1988). This article develops the response that Connectionist models can, and in fact sometimes do, employ compositionally structured representations without, thereby, simply implementing a Classical “Language of Thought.” Focusing on the mode of combination employed in constructing representations, it distinguishes concatenative compositionality, essential to the Classical approach, from a merely functional counterpart increasingly common in Connectionist research. On the basis of this distinction it is possible to demonstrate that Connectionist representations can be compositional without being Classical, and further, that Fodor and Pylyshyn's supposedly conclusive arguments in favor of the Classical approach do not in fact support that approach over the Connectionist alternative (as opposed to an “ossociotionist” straw man).     

Connectionism and the problem of systematicity: why Smolensky's solution doesn't work

In two recent papers, Paul Smolensky responds to a challenge Jerry Fodor and Zenon Pylyshyn posed for connectionist theories of cognition: to explain the existence of systematic relations among cognitive capacities without assuming that mental processes are causally sensitive to the constituent structure of mental representations. Smolensky thinks connectionists can explain systematicity if they avail themselves of "distributed" mental representation. In facts, Smolensky offers two accounts of distributed mental representation, corresponding to his notions of "weak" and "strong" compositional structure. We argue that weak compositional structure is irrelevant to the systematicity problem and of dubious internal coherence. We then argue that strong compositional (tensor product) representations fail to explain systematicity because they fail to exhibit the sort of constituents that can provide domains for structure sensitive mental processes.     

Directions in Connectionist Research: Tractable Computations Without Syntactically Structured Representations

Should connectionists abandon the quest for tractably computable cognitive transition functions while retaining syntactically structured mental representations? We argue, in opposition to Horgan, that it should not. We argue that the case against tractably computable functions, based upon the claimed isotropic and Quinean character of cognition, fails since cognition is not as isotropic and Quinean as Fodor and Horgan contend. Moreover, we illustrate how current research in both connectionism and cognitive neuroscience suggests that tractability can be preserved through division of overall computations into modular sub-processes, each of which is tractable. As to syntactically structured representations, we argue that they are unneeded for most cognitive tasks organisms confront, and that when they are needed, they may be provided by external representational media such as natural language. Moreover, we note that increasingly cognitive linguistics has become the ally of connectionism and that the research program of cognitive linguistics suggests that abilities to use natural languages may be developed without requiring syntactically structured mental representations to exist prior to natural language.     

Systematicity redux

One of the main challenges that Jerry Fodor and Zenon Pylyshyn (Cognition 28:3–71, 1988) posed for any connectionist theory of cognitive architecture is to explain the systematicity of thought without implementing a Language of Thought (LOT) architecture. The systematicity challenge presents a dilemma: if connectionism cannot explain the systematicity of thought, then it fails to offer an adequate theory of cognitive architecture; and if it explains the systematicity of thought by implementing a LOT architecture, then it fails to offer an alternative to the LOT hypothesis. Given that thought is systematic, connectionism can offer an adequate alternative to the LOT hypothesis only if it can meet the challenge. Although some critics tried to meet the challenge, others argued that it need not be met since thought is not in fact systematic; and some claimed not to even understand the claim that thought is systematic. I do not here examine attempts to answer the challenge. Instead, I defend the challenge itself by explicating the notion of systematicity in a way that I hope makes clear that thought is indeed systematic, and so that to offer an adequate alternative to the LOT hypothesis, connectionism must meet the challenge.     

Classical and connectionist models: Levels of description

To begin, I introduce an analysis of interlevel relations that allows us to offer an initial characterization of the debate about the way classical and connectionist models relate. Subsequently, I examine a compatibility thesis and a conditional claim on this issue. With respect to the compatibility thesis, I argue that, even if classical and connectionist models are not necessarily incompatible, the emergence of the latter seems to undermine the best arguments for the Language of Thought Hypothesis, which is essential to the former. I attack the conditional claim of connectionism to eliminativism, presented by Ramsey et al. (1990), by discrediting their discrete characterization of common-sense psychological explanations and pointing to the presence of a moderate holistic constraint. Finally, I conclude that neither of the arguments considered excludes the possibility of viewing connectionist models as forming a part of a representational theory of cognition that dispenses with the Language of Thought Hypothesis.     

Three-concept monte: Explanation,implementation and systematicity

Fodor and Pylyshyn (1988), Fodor and McLaughlin (1990) and McLaughlin (1993) challenge connectionists to explain systematicity without simply implementing a classical architecture. In this paper I argue that what makes the challenge difficult for connectionists to meet has less to do with what is to be explained than with what is to count as an explanation. Fodor et al. are prepared to admit as explanatory, accounts of a sort that only classical models can provide. If connectionists are to meet the challenge, they are going to have to insist on the propriety of changing what counts as an explanation of systematicity. Once that is done, there would seem to be as yet no reason to suppose that connectionists are unable to explain systematicity.Shorter versions of this paper were presented at the 1993 annual meetings of both the Southern Society for Philosophy and Psychology and the Society for Philosophy and Psychology. The paper has benefited from the comments and criticisms of Frances Egan, David Chalmers, Terry Horgan, Brian McLaughlin, Tim van Gelder and Ted Warfield.     

Minds as Connoting Systems: Logic and the Language of Thought

The principal aim of this essay is to discuss some logical features of the so-called Classical model of cognitive architecture as it is advocated by J. Fodor and Z. Pylyshyn in their much discussed article 'Connectionism and Cognitive Architecture: A Critical Analysis'. It is pointed out that their structural assumptions have consequences of a logical kind which call into question the view that the Classical architecture (in their sense) can be employed to model human cognition. It seems that the consequences have escaped Fodor and Pylyshyn's notice, or else they have failed to appreciate them, since some of their claims evidently conflict with them. It is also investigated whether the human mind can be characterized as being logical in some weaker sense of logic. Furthermore, it is argued that J. H. Fetzer's view that it is a semiotic system is more realistic than the Classical model, but the distinction he suggests between human cognition and other kinds may be problematic.     

Different is good: connectionism and dynamic systems theory are complementary emergentist approaches to development

Connectionist and dynamic systems approaches to development are similar in that they are both emergentist theories that take a very different perspective from more traditional symbolic systems. Moreover, they are both based on similar mathematical principles. Nevertheless, connectionism and dynamic systems differ in the approach they take to the study of development. We argue that differences between connectionist and dynamic systems approaches in terms of the basic components of the models, what they see as the object of study, how they view the nature of knowledge and their notions of developmental change mean that they each stand to make different and unique contributions to a more complete theory of development. We present an example from our work on how children learn to learn words that illustrates the complementary nature of connectionist and dynamic systems theories.     

信息加工认知心理学的发展和面临的挑战

本文认为当代信息加工认知心理学发展的特点是：（１）与高新科学技术结合．在基础理论的研究方面有较大的突破;（２）将自然认知与社会认知的研究结合起来,促使信息加工的研究能更接近人类实际的认知过程：（３）将基础理论研究与应用研究相结合,不断扩大认知心理学的研究领域和应用范围．为了迎接面临的挑战和适应社会发展的需要,信息加工认知心理学应更充分地体现时代精神,坚持走实验研究和现象学研究相结合、定量研究与定性研究相结合、微观研究与宏观研究相结合,以及文理互补、多科渗透的发展道路。     

The computational account of belief

Fodor and others who think that scientific, computational psychology will vindicate commonsense belief-desire psychology have maintained that belief can be identified with the explicit storage of a token with appropriate content. I review and develop problems for the explicit storage view and show that a more plausible account identifies belief with the disposition to use a token with appropriate content in explicit reasoning and planning processes and as a basis for action. I argue that this type of inner disposition account will also apply to most other common sense attitudes. The result is a realism about commonsense belief-desire psychology that is more modest than Fodor's: While such inner dispositions probably do exist, these states will probably not be the main focus of scientific psychological theories.     

A connectionist theory of phenomenal experience

When cognitive scientists apply computational theory to the problem of phenomenal consciousness, as many have been doing recently, there are two fundamentally distinct approaches available. Consciousness is to be explained either in terms of the nature of the representational vehicles the brain deploys or in terms of the computational processes defined over these vehicles. We call versions of these two approaches vehicle and process theories of consciousness, respectively. However, although there may be space for vehicle theories of consciousness in cognitive science, they are relatively rare. This is because of the influence exerted, on the one hand, by a large body of research that purports to show that the explicit representation of information in the brain and conscious experience are dissociable, and on the other, by the classical computational theory of mind--the theory that takes human cognition to be a species of symbol manipulation. Two recent developments in cognitive science combine to suggest that a reappraisal of this situation is in order. First, a number of theorists have recently been highly critical of the experimental methodologies used in the dissociation studies--so critical, in fact, that it is no longer reasonable to assume that the dissociability of conscious experience and explicit representation has been adequately demonstrated. Second, classicism, as a theory of human cognition, is no longer as dominant in cognitive science as it once was. It now has a lively competitor in the form of connectionism; and connectionism, unlike classicism, does have the computational resources to support a robust vehicle theory of consciousness. In this target article we develop and defend this connectionist vehicle theory of consciousness. It takes the form of the following simple empirical hypothesis: phenomenal experience consists of the explicit representation of information in neurally realized parallel distributed processing (PDP) networks. This hypothesis leads us to reassess some common wisdom about consciousness, but, we argue, in fruitful and ultimately plausible ways.     

Connectionist Natural Language Processing: The State of the Art

This Special Issue on Connectionist Models of Human Language Processing provides an opportunity for an appraisal both of specific connectionist models and of the status and utility of connectionist models of language in general. This introduction provides the background for the papers in the Special Issue. The development of connectionist models of language is traced, from their intellectual origins, to the state of current research. Key themes that arise throughout different areas of connectionist psycholinguistics are highlighted, and recent developments in speech processing, morphology, sentence processing, language production, and reading are described. We argue that connectionist psycholinguistics has already had a significant impact on the psychology of language, and that connectionist models are likely to have an important influence on future research.     

Asymmetry in Action

In The Elm and The Expert (Fodor 1994), Jerry Fodor claims that in order to solve the mind/body problem (consciousness excluded), a computational psychology needs to be combined with a naturalistic theory of content such as the asymmetric dependence theory put forward in 'A Theory of Content II' (in Fodor 1990, pp. 89-136). However, since this theory was first proposed, it has been reproached for a number of failings, perhaps the most significant of which is the objection that it simply begs the question. In this paper I provide an outline of the story so far, covering Fodor's objectives, the theory he proposes to meet them and the objection in question, before going on to argue that a promising attempt to answer this objection will not bear fruit given the way in which Fodor tackles the original problems. I then argue that the seeming intractability of this objection is due to Fodor's striving to solve all the problems that a naturalistic theory presents in one fell swoop, and conclude by suggesting how, given a careful choice of targets and a well organised methodology, asymmetric dependence might yet prove to be an attractive theory of content.     

Connectionist representation,semantic compositionality,and the instability of concept structure

Summary In the present paper connectionist approaches to the problem of internal representation and the nature of concepts are discussed. In the first part the concept of representation that underlies connectionist modeling is made explicit. It is argued that the connectionist view of representation relies on a correlational theory of semantic content- i.e., the covariation between internal and external states is taken as the basis for ascribing meaning to internal states. The problems and virtues of such a correlational approach to internal representation are addressed. The second part of the paper is concerned with whether connectionism is capable of accounting for the apparent productivity and systematicity of language and thought. There is an evaluation of the recent arguments of Fodor and Pylyshyn, who claim that systematicity can only be explained if one conceives of mental representations as structured symbols composed of context-free constituents. There is a review of empirical evidence that strongly suggests that concepts are not fixed memory structures and that the meaning of constituent symbols varies, depending on the context in which they are embedded. On the basis of this review it is concluded that the meaning of a complex expression is not computed from the context-free meanings of the constituents, and that strong compositionality, as endorsed by Fodor and Pylyshyn (1988), seems implausible as a process theory for the comprehension of complex concepts. Instead, the hypothesis is endorsed that constraint satisfaction in distributed connectionist networks may allow for an alternative account of weak compositionality compatible with the context sensitivity of meaning. In the final section, it is argued that neither mere implementation of a language of thought in connectionist networks nor radical elimination of symbol systems seems to be a fruitful research strategy, but that it might be more useful to discuss how connectionist systems can develop the capacity to use external symbol systems like language or logic without instantiating symbol systems themselves.     

How to stay safe while extending the mind

According to the extended mind thesis, cognitive processes are not confined to the nervous system but can extend beyond skin and skull to notebooks, iPhones, computers and such. The extended mind thesis is a metaphysical thesis about the material basis of our cognition. As such, whether the thesis is true can have implications for epistemological issues. Carter has recently argued that safety-based theories of knowledge are in tension with the extended mind hypothesis, since the safety condition implies that there is an epistemic difference between subjects who form their beliefs via their biological capacities and between subjects who have extended their cognition. Kelp, on the other hand, has argued that a safety-based theory of knowledge can be correct only if the extended mind thesis is true. While these claims are not logically inconsistent, they do leave the safety theorist in an uncomfortable position. I will argue that safety-based theories of knowledge are not hostage to the truth of the extended mind thesis, and that once the safety condition is properly understood it is not in tension with the extended mind thesis.

     

Representations gone mental

Many philosophers and psychologists have attempted to elucidate the nature of mental representation by appealing to notions like isomorphism or abstract structural resemblance. The ‘structural representations’ that these theorists champion are said to count as representations by virtue of functioning as internal models of distal systems. In his 2007 book, Representation Reconsidered, William Ramsey endorses the structural conception of mental representation, but uses it to develop a novel argument against representationalism, the widespread view that cognition essentially involves the manipulation of mental representations. Ramsey argues that although theories within the ‘classical’ tradition of cognitive science once posited structural representations, these theories are being superseded by newer theories, within the tradition of connectionism and cognitive neuroscience, which rarely if ever appeal to structural representations. Instead, these theories seem to be explaining cognition by invoking so-called ‘receptor representations’, which, Ramsey claims, aren’t genuine representations at all—despite being called representations, these mechanisms function more as triggers or causal relays than as genuine stand-ins for distal systems. I argue that when the notions of structural and receptor representation are properly explicated, there turns out to be no distinction between them. There only appears to be a distinction between receptor and structural representations because the latter are tacitly conflated with the ‘mental models’ ostensibly involved in offline cognitive processes such as episodic memory and mental imagery. While structural representations might count as genuine representations, they aren’t distinctively mental representations, for they can be found in all sorts of non-intentional systems such as plants. Thus to explain the kinds of offline cognitive capacities that have motivated talk of mental models, we must develop richer conceptions of mental representation than those provided by the notions of structural and receptor representation.     

Probabilistic single function dual process theory and logic programming as approaches to non-monotonicity in human vs. artificial reasoning

In this paper, it is argued that single function dual process theory is a more credible psychological account of non-monotonicity in human conditional reasoning than recent attempts to apply logic programming (LP) approaches in artificial intelligence to these data. LP is introduced and among other critiques, it is argued that it is psychologically unrealistic in a similar way to hash coding in the classicism vs. connectionism debate. Second, it is argued that causal Bayes nets provide a framework for modelling probabilistic conditional inference in System 2 that can deal with patterns of inference LP cannot. Third, we offer some speculations on how the cognitive system may avoid problems for System 1 identified by Fodor in 1983. We conclude that while many problems remain, the probabilistic single function dual processing theory is to be preferred over LP as an account of the non-monotonicity of human reasoning.     

Can Compositionality Solve the Thought-or-Language Problem?

Abstract

Jerry Fodor has claimed to have a solution to the traditional problem of what comes first, thought or language. Compositionality, he says, will give us the answer, for at least one must be compositional, and if only one of them is, that is the one that has underived semantic content. He argues that natural languages are not compositional, and therefore that the content of language is derived from the content of thought. I will argue that the idea that language is not compositional conflicts with his productivity and systematicity arguments for the existence of a language of thought. I will also show that Fodor’s solution to the problem fails, as his main argument is circular. Finally, I suggest that Fodor’s argument against the compositionality of language is not decisive, and that we can still attribute at least some degree of compositionality to language.