Representations without rules,connectionism and the syntactic argument

Terry Horgan and John Tienson have suggested that connectionism might provide a framework within which to articulate a theory of cognition according to which there are mental representations without rules (RWR) (Horgan and Tienson 1988, 1989, 1991, 1992). In essence, RWR states that cognition involves representations in a language of thought, but that these representations are not manipulated by the sort of rules that have traditionally been posited. In the development of RWR, Horgan and Tienson attempt to forestall a particular line of criticism, theSyntactic Argument, which would show RWR to be inconsistent with connectionism. In essence, the argument claims that the node-level rules of connectionist networks, along with the semantic interpretations assigned to patterns of activation, serve to determine a set of representation-level rules incompatible with the RWR conception of cognition. The present paper argues that the Syntactic Argument can be made to show that RWR is inconsistent with connectionism.The present paper has been improved by conversations with Terry Horgan and John Tienson. Thanks are also due to Gary Fuller, John Heil, Terry Horgan and Bob Stecker for comments on earlier drafts of this paper.     

Connectionist and Memory-Array Models of Artificial Grammar Learning

Subjects exposed to strings of letters generated by a finite state grammar can later classify grammatical and nongrammatical test strings, even though they cannot adequately say what the rules of the grammar are (e.g., Reber, 1989). The MINERVA 2 (Hintzman, 1986) and Medin and Schaffer (1978) memory-array models and a number of connectionist outoassociator models are tested against experimental data by deriving mainly parameter-free predictions from the models of the rank order of classification difficulty of test strings. The importance of different assumptions regarding the coding of features (How should the absence of a feature be coded? Should single letters or digrams be coded?), the learning rule used (Hebb rule vs. delta rule), and the connectivity (Should features be predicted only by previous features in the string, or by all features simultaneously?) is investigated by determining the performance of the models with and without each assumption. Only one class of connectionist model (the simultaneous delta rule) passes all the tests. It is shown that this class of model can be regarded by abstracting a set of representative but incomplete rules of the grammar.     

A Distributed Connectionist Production System

DCPS is a connectionist production system interpreter that uses distributed representations. As a connectionist model it consists of many simple, richly interconnected neuron-like computing units that cooperate to solve problems in parallel. One motivation for constructing DCPS was to demonstrate that connectionist models are capable of representing and using explicit rules. A second motivation was to show how "coarse coding" or "distributed representations" can be used to construct a working memory that requires far fewer units than the number of different facts that can potentially be stored. The simulation we present is intended as a detailed demonstration of the feasibility of certain ideas and should not be viewed as a full implementation of production systems. Our current model only has a few of the many interesting emergent properties that we eventually hope to demonstrate: It is damage-resistant, it performs matching and variable binding by massively parallel constraint satisfaction, and the capacity of its working memory is dependent on the similarity of the items being stored.     

Cognition without classical architecture

Fodor and Pylyshyn (1988) argue that any successful model of cognition must use classical architecture; it must depend upon rule-based processing sensitive to constituent structure. This claim is central to their defense of classical AI against the recent enthusiasm for connectionism. Connectionist nets, they contend, may serve as theories of the implementation of cognition, but never as proper theories of psychology. Connectionist models are doomed to describing the brain at the wrong level, leaving the classical view to account for the mind.This paper considers whether recent results in connectionist research weigh against Fodor and Pylyshyn's thesis. The investigation will force us to develop criteria for determining exactly when a net is capable of systematic processing. Fodor and Pylyshyn clearly intend their thesis to affect the course of research in psychology. I will argue that when systematicity is defined in a way that makes the thesis relevant in this way, the thesis is challenged by recent progress in connectionism.     

Knowledge of Rules,Causal Systematicity,and the Language of Thought

Martin Davies' criterion for the knowledge of implicit rules, viz. the causal systematicity of cognitive processes, is first exposed. Then the inference from causal systematicity of a process to syntactic properties of the input states is examined. It is argued that Davies' notion of a syntactic property is too weak to bear the conclusion that causal systematicity implies a language of thought as far as the input states are concerned. Next, it is shown that Davies' criterion leads to a counterintuitive consequence: it groups together distributed connectionist systems with look-up tables. To avoid this consequence, a modified construal of causal systematicity is proposed and Davies' argument for the causal systematicity of thought is shown to be question-begging. It is briefly sketched how the modified construal links up with multiple dispositions of the same categorical base. Finally, the question of the causal efficacy of single rules is distinguished from the question of their psychological reality: implicit rules might be psychologically real without being causally efficacious. This revised version was published online in June 2006 with corrections to the Cover Date.     

Connectionist natural language parsing

The key developments of two decades of connectionist parsing are reviewed. Connectionist parsers are assessed according to their ability to learn to represent syntactic structures from examples automatically, without being presented with symbolic grammar rules. This review also considers the extent to which connectionist parsers offer computational models of human sentence processing and provide plausible accounts of psycholinguistic data. In considering these issues, special attention is paid to the level of realism, the nature of the modularity, and the type of processing that is to be found in a wide range of parsers.     

The relationship between connectionist models and a dynamic data-oriented theory of concept formation

In this paper I shall compare two models of concept formation, both inspired by basic convictions of philosophical empiricism. The first, the connectionist model, will be exemplified by Kohonen maps, and the second will be my own dynamic theory of concept formation. Both can be understood in probabilistic terms, both use a notion of convergence or stabilization in modelling how concepts are built up. Both admit destabilization of concepts and conceptual change. Both do not use a notion of representation in some pregiven language, such as a language of thought or some logical language. Representation in a formal language only plays a role on the meta-level, namely within the theory about concept formation.A short version of this paper was given at the Fachtagung der Gesellschaft für Kognitionswissenschaft. Freiburg, October 12–15th, 1994.     

Rule following and rule use in the balance-scale task

Quinlan et al. [Quinlan, p., van der Mass, H., Jansen, B., Booij, O., & Rendell, M. (this issue). Re-thinking stages of cognitive development: An appraisal of connectionist models of the balance scale task. Cognition, doi:10.1016/j.cognition.2006.02.004] use Latent Class Analysis (LCA) to criticize a connectionist model of development on the balance-scale task, arguing that LCA shows that this model fails to capture a torque rule and exhibits rules that children do not. In this rejoinder we focus on the latter problem, noting the tendency of LCA to find small, unreliable, and difficult-to-interpret classes. This tendency is documented in network and synthetic simulations and in psychological research, and statistical reasons for finding such unreliable classes are discussed. We recommend that LCA should be used with care, and argue that its small and unreliable classes should be discounted. Further, we note that a preoccupation with diagnosing rules ignores important phenomena that rules do not account for. Finally, we conjecture that simple extensions of the network model should be able to achieve torque-rule performance.     

Exploring the dynamics of design fluency in children with and without ADHD using artificial neural networks

The neuropsychology of attention deficit/hyperactivity disorder (ADHD) has been extensively studied, with a general focus on global performance measures of executive function. In this study, we compared how global (i.e., endpoint) versus process (i.e., dynamic) measures of performance may help characterize children with and without ADHD using a design fluency task as a case study. The secondary goal was to compare the sensitivity of standard versus connectionist statistical models to group differences in cognitive data. Thirty-four children diagnosed with ADHD and 37 children without ADHD aged 8–11 years old were tested on the Five-Point Test. The continuous process measure of performance, indexed as the number of produced designs at each consecutive 1 minute interval during 5 minutes, was analyzed against the discrete process measure, that is, the number of designs between first and last intervals and the standard global performance measure of total number of produced designs. Results show that the continuous process measure distinguished the two groups better than the two other measures. The detailed observation of production patterns revealed a decreasing linear trajectory in children without ADHD that contrasts with the flat, but fluctuating productivity pattern of children with ADHD. With regards to the second goal, results show that the connectionist and standard methods are equally sensitive to group differences for the three types of measures. This illustrates the utility of quantitative process measures together with the connectionist method in neuropsychological research and suggests great potential for a dynamical approach to cognition.     

Connectionist representation,semantic compositionality,and the instability of concept structure

Summary In the present paper connectionist approaches to the problem of internal representation and the nature of concepts are discussed. In the first part the concept of representation that underlies connectionist modeling is made explicit. It is argued that the connectionist view of representation relies on a correlational theory of semantic content- i.e., the covariation between internal and external states is taken as the basis for ascribing meaning to internal states. The problems and virtues of such a correlational approach to internal representation are addressed. The second part of the paper is concerned with whether connectionism is capable of accounting for the apparent productivity and systematicity of language and thought. There is an evaluation of the recent arguments of Fodor and Pylyshyn, who claim that systematicity can only be explained if one conceives of mental representations as structured symbols composed of context-free constituents. There is a review of empirical evidence that strongly suggests that concepts are not fixed memory structures and that the meaning of constituent symbols varies, depending on the context in which they are embedded. On the basis of this review it is concluded that the meaning of a complex expression is not computed from the context-free meanings of the constituents, and that strong compositionality, as endorsed by Fodor and Pylyshyn (1988), seems implausible as a process theory for the comprehension of complex concepts. Instead, the hypothesis is endorsed that constraint satisfaction in distributed connectionist networks may allow for an alternative account of weak compositionality compatible with the context sensitivity of meaning. In the final section, it is argued that neither mere implementation of a language of thought in connectionist networks nor radical elimination of symbol systems seems to be a fruitful research strategy, but that it might be more useful to discuss how connectionist systems can develop the capacity to use external symbol systems like language or logic without instantiating symbol systems themselves.     

NETWORKS AND THEORIES: The Place of Connectionism to Cognitive Science

This article considers how connectionist modeling can contribute to understanding of human cognition. I argue that connectionist networks should not be thought of as theories or simulations of theories, hut may nevertheless contribute to the development of theories.     

The Case for Rules in Reasoning

A number of theoretical positions in psychology—including variants of case-based reasoning, instance-based analogy, and connectionist models—maintain that abstract rules are not involved in human reasoning, or at best play a minor role. Other views hold that the use of abstract rules is a core aspect of human reasoning. We propose eight criteria for determining whether or not people use abstract rules in reasoning, and examine evidence relevant to each criterion for several rule systems. We argue that there is substantial evidence that several different inferential rules, including modus ponens, contractual rules, causal rules, and the law of large numbers, are used in solving everyday problems. We discuss the implications for various theoretical positions and consider hybrid mechanisms that combine aspects of instance and rule models.     

Connectionist models and their properties

Much of the progress in the fields constituting cognitive science has been based upon the use of explicit information processing models, almost exclusively patterned after conventional serial computers. An extension of these ideas to massively parallel, connectionist models appears to offer a number of advantages. After a preliminary discussion, this paper introduces a general connectionist model and considers how it might be used in cognitive science. Among the issues addressed are: stability and noise-sensitivity, distributed decision-making, time and sequence problems, and the representation of complex concepts.     

Classical and connectionist models: Levels of description

To begin, I introduce an analysis of interlevel relations that allows us to offer an initial characterization of the debate about the way classical and connectionist models relate. Subsequently, I examine a compatibility thesis and a conditional claim on this issue. With respect to the compatibility thesis, I argue that, even if classical and connectionist models are not necessarily incompatible, the emergence of the latter seems to undermine the best arguments for the Language of Thought Hypothesis, which is essential to the former. I attack the conditional claim of connectionism to eliminativism, presented by Ramsey et al. (1990), by discrediting their discrete characterization of common-sense psychological explanations and pointing to the presence of a moderate holistic constraint. Finally, I conclude that neither of the arguments considered excludes the possibility of viewing connectionist models as forming a part of a representational theory of cognition that dispenses with the Language of Thought Hypothesis.     

Analogy as relational priming: a developmental and computational perspective on the origins of a complex cognitive skill

The development of analogical reasoning has traditionally been understood in terms of theories of adult competence. This approach emphasizes structured representations and structure mapping. In contrast, we argue that by taking a developmental perspective, analogical reasoning can be viewed as the product of a substantially different cognitive ability - relational priming. To illustrate this, we present a computational (here connectionist) account where analogy arises gradually as a by-product of pattern completion in a recurrent network. Initial exposure to a situation primes a relation that can then be applied to a novel situation to make an analogy. Relations are represented as transformations between states. The network exhibits behaviors consistent with a broad range of key phenomena from the developmental literature, lending support to the appropriateness of this approach (using low-level cognitive mechanisms) for investigating a domain that has normally been the preserve of high-level models. Furthermore, we present an additional simulation that integrates the relational priming mechanism with deliberative controlled use of inhibition to demonstrate how the framework can be extended to complex analogical reasoning, such as the data from explicit mapping studies in the literature on adults. This account highlights how taking a developmental perspective constrains the theory construction and cognitive modeling processes in a way that differs substantially from that based purely on adult studies, and illustrates how a putative complex cognitive skill can emerge out of a simple mechanism.     

Re-thinking stages of cognitive development: an appraisal of connectionist models of the balance scale task

The present paper re-appraises connectionist attempts to explain how human cognitive development appears to progress through a series of sequential stages. Models of performance on the Piagetian balance scale task are the focus of attention. Limitations of these models are discussed and replications and extensions to the work are provided via the Cascade-Correlation algorithm. An application of multi-group latent class analysis for examining performance of the networks is described and these results reveal fundamental functional characteristics of the networks. Evidence is provided that strongly suggests that the networks are unable to acquire a mastery of torque and, although they do recover certain rules of operation that humans do, they also show a propensity to acquire rules never previously seen.     

The allure of connectionism reexamined

There is currently a debate over whether cognitive architecture is classical or connectionist in nature. One finds the following three comparisons between classical architecture and connectionist architecture made in the pro-connectionist literature in this debate: (1) connectionist architecture is neurally plausible and classical architecture is not; (2) connectionist architecture is far better suited to model pattern recognition capacities than is classical architecture; and (3) connectionist architecture is far better suited to model the acquisition of pattern recognition capacities by learning than is classical architecture. If true, (1)–(3) would yield a compelling case against the view that cognitive architecture is classical, and would offer some reason to think that cognitive architecture may be connectionist. We first present the case for (1)–(3) in the very words of connectionist enthusiasts. We then argue that the currently available evidence fails to support any of (1)–(3).     

Implicit and explicit memory: Implications for the symbol-manipulation versus connectionism controversy

Summary This paper deals with three issues. First, the symbol-manipulation versus connectionism controversy is discussed briefly. Both can be seen as formalisms or languages for describing human behavior, but it is argued that these languages are not fully equivalent. The importance of the ability of connectionist models to learn autonomously, often underestimated, allows them to perform tasks that may defy formalization. Secondly, the structural and functional characteristics of a new connectionist learning model are presented. It avoids some of the psychological and biological implausibilities of currently popular models and solves some of the problems and shortcomings of these models. Thirdly, some simulation results are presented. The model successfully simulates the dissociation between explicit and implicit memory performance as found in patients with anterograde amnesia. This simulation provides an example of the advantages of using the connectionist language in the domain of memory psychology. It is also a good example of how psychological evidence can be used to improve connectionist models.The second author is supported by the Dutch Organization for Scientific Research (NWO)     

Challenging the importance of vision for the development of an extrinsic spatial framework: evidence from the blind and sighted

Although connectionism is advocated by its proponents as an alternative to the classical computational theory of mind, doubts persist about its computational credentials. Our aim is to dispel these doubts by explaining how connectionist networks compute. We first develop a generic account of computation-no easy task, because computation, like almost every other foundational concept in cognitive science, has resisted canonical definition. We opt for a characterisation that does justice to the explanatory role of computation in cognitive science. Next we examine what might be regarded as the "conventional" account of connectionist computation. We show why this account is inadequate and hence fosters the suspicion that connectionist networks are not genuinely computational. Lastly, we turn to the principal task of the paper: the development of a more robust portrait of connectionist computation. The basis of this portrait is an explanation of the representational capacities of connection weights, supported by an analysis of the weight configurations of a series of simulated neural networks.