A Connectionist Approach to Knowledge Representation and Limited Inference

Although the connectionist approach has lead to elegant solutions to a number of problems in cognitive science and artificial intelligence, its suitability for dealing with problems in knowledge representation and inference has often been questioned. This paper partly answers this criticism by demonstrating that effective solutions to certain problems in knowledge representation and limited inference can be found by adopting a connectionist approach. The paper presents a connectionist realization of semantic networks, that is, it describes how knowledge about concepts, their properties, and the hierarchical relationship between them may be encoded as an interpreter-free massively parallel network of simple processing elements that can solve an interesting class of inheritance and recognition problems extremely fast—in time proportional to the depth of the conceptual hierarchy. The connectionist realization is based on an evidential formulation that leads to principled solutions to the problems of exceptions and conflicting multiple inheritance situations during inheritance, and the best-match or partial-match computation during recognition. The paper also identifies constraints that must be satisfied by the conceptual structure in order to arrive at an efficient parallel realization.     

A connectionist theory of phenomenal experience

When cognitive scientists apply computational theory to the problem of phenomenal consciousness, as many have been doing recently, there are two fundamentally distinct approaches available. Consciousness is to be explained either in terms of the nature of the representational vehicles the brain deploys or in terms of the computational processes defined over these vehicles. We call versions of these two approaches vehicle and process theories of consciousness, respectively. However, although there may be space for vehicle theories of consciousness in cognitive science, they are relatively rare. This is because of the influence exerted, on the one hand, by a large body of research that purports to show that the explicit representation of information in the brain and conscious experience are dissociable, and on the other, by the classical computational theory of mind--the theory that takes human cognition to be a species of symbol manipulation. Two recent developments in cognitive science combine to suggest that a reappraisal of this situation is in order. First, a number of theorists have recently been highly critical of the experimental methodologies used in the dissociation studies--so critical, in fact, that it is no longer reasonable to assume that the dissociability of conscious experience and explicit representation has been adequately demonstrated. Second, classicism, as a theory of human cognition, is no longer as dominant in cognitive science as it once was. It now has a lively competitor in the form of connectionism; and connectionism, unlike classicism, does have the computational resources to support a robust vehicle theory of consciousness. In this target article we develop and defend this connectionist vehicle theory of consciousness. It takes the form of the following simple empirical hypothesis: phenomenal experience consists of the explicit representation of information in neurally realized parallel distributed processing (PDP) networks. This hypothesis leads us to reassess some common wisdom about consciousness, but, we argue, in fruitful and ultimately plausible ways.     

Learning Łukasiewicz logic

The integration between connectionist learning and logic-based reasoning is a longstanding foundational question in artificial intelligence, cognitive systems, and computer science in general. Research into neural-symbolic integration aims to tackle this challenge, developing approaches bridging the gap between sub-symbolic and symbolic representation and computation. In this line of work the core method has been suggested as a way of translating logic programs into a multilayer perceptron computing least models of the programs. In particular, a variant of the core method for three valued Łukasiewicz logic has proven to be applicable to cognitive modelling among others in the context of Byrne’s suppression task. Building on the underlying formal results and the corresponding computational framework, the present article provides a modified core method suitable for the supervised learning of Łukasiewicz logic (and of a closely-related variant thereof), implements and executes the corresponding supervised learning with the backpropagation algorithm and, finally, constructs a rule extraction method in order to close the neural-symbolic cycle. The resulting system is then evaluated in several empirical test cases, and recommendations for future developments are derived.     

Concepts as Semantic Pointers: A Framework and Computational Model

The reconciliation of theories of concepts based on prototypes, exemplars, and theory‐like structures is a longstanding problem in cognitive science. In response to this problem, researchers have recently tended to adopt either hybrid theories that combine various kinds of representational structure, or eliminative theories that replace concepts with a more finely grained taxonomy of mental representations. In this paper, we describe an alternative approach involving a single class of mental representations called “semantic pointers.” Semantic pointers are symbol‐like representations that result from the compression and recursive binding of perceptual, lexical, and motor representations, effectively integrating traditional connectionist and symbolic approaches. We present a computational model using semantic pointers that replicates experimental data from categorization studies involving each prior paradigm. We argue that a framework involving semantic pointers can provide a unified account of conceptual phenomena, and we compare our framework to existing alternatives in accounting for the scope, content, recursive combination, and neural implementation of concepts.     

Knowledge Representation: Two Kinds Of Emergence

Two different but closely related issues in current cognitive science will be considered in this essay. One is the controversial and extensively discussed question of how connectionist and symbolic representations of knowledge are related to each other. The other concerns the notion of connectionist learning and its relevance for the understanding of the distinction between propositional and nonpropositional knowledge. More specifically, I shall give an overview of a result in Rantala and Vadén (1994) establishing a limiting case correspondence between symbolic and connectionist representations and, on the other hand, study the problem, preliminarily investigated in Rantala (1998), of how propositional knowledge may arise from nonpropositional knowledge. I shall also try to point out that on some more or less plausible assumptions, often made by cognitive scientists, these results may have some significance when we try to comprehend the nature of human knowledge representation. Some of these assumptions are rather hypothethical and debatable for the time being and they will become justified in the future only if there will be more progress in the empirical and theoretical research on the brain and on artificial networks. The assumptions concern, besides some questions of the behavior of neural networks, such things as the relevance of pattern recognition for modelling human cognition, in particular, knowledge acquisition, and the relation between emergence and reduction.     

Rule following and rule use in the balance-scale task

Quinlan et al. [Quinlan, p., van der Mass, H., Jansen, B., Booij, O., & Rendell, M. (this issue). Re-thinking stages of cognitive development: An appraisal of connectionist models of the balance scale task. Cognition, doi:10.1016/j.cognition.2006.02.004] use Latent Class Analysis (LCA) to criticize a connectionist model of development on the balance-scale task, arguing that LCA shows that this model fails to capture a torque rule and exhibits rules that children do not. In this rejoinder we focus on the latter problem, noting the tendency of LCA to find small, unreliable, and difficult-to-interpret classes. This tendency is documented in network and synthetic simulations and in psychological research, and statistical reasons for finding such unreliable classes are discussed. We recommend that LCA should be used with care, and argue that its small and unreliable classes should be discounted. Further, we note that a preoccupation with diagnosing rules ignores important phenomena that rules do not account for. Finally, we conjecture that simple extensions of the network model should be able to achieve torque-rule performance.     

Multiple levels of inquiry in cognitive science

Summary Gibsonian ecological psychology, symbolic information processing, and connectionist information processing are frequently construed as three competing paradigms or research traditions, each seeking dominance in experimental psychology and in cognitive science generally. There is an important element of truth in this perspective, and any adequate account of the development of experimental psychology over the past 30 years would have to examine seriously how the various conceptual frameworks, experimental endeavors, and social institutions have figured in this conflict. But the goal of this paper is not to characterize the historical dynamics within experimental psychology and cognitive science; rather, it is to consider what sorts of rapprochement is possible. Rapprochement, however, is not sought simply for its own sake or out of an a priori conviction that scientific enterprises should be unified. Spirited controversy between competing traditions is often an important component of progess (Laudan, 1977). Rapprochement has a purpose when alternative theoretical traditions have reached a point when each confronts serious shortcomings that can best be overcome by incorporating alternative perspectives. In this paper I try to show that this is the situation that exists in experimental psychology and cognitve science generally with respect to the three traditions enumerated above. I first explore how cognitive inquiry directed at internal procedures for processing information could benefit from a detailed study of the context of cognition, including insights provided by the Gibsonian tradition. Second, I examine the current controversy between symbolic and connectionist approaches and address the question of what contributions each offers to the other. Finally, I offer a framework in which multiple levels of inquiry in cognitve science can be related.     

The tractable cognition thesis

The recognition that human minds/brains are finite systems with limited resources for computation has led some researchers to advance the Tractable Cognition thesis : Human cognitive capacities are constrained by computational tractability. This thesis, if true, serves cognitive psychology by constraining the space of computational-level theories of cognition. To utilize this constraint, a precise and workable definition of "computational tractability" is needed. Following computer science tradition, many cognitive scientists and psychologists define computational tractability as polynomial-time computability, leading to the P-Cognition thesis . This article explains how and why the P-Cognition thesis may be overly restrictive, risking the exclusion of veridical computational-level theories from scientific investigation. An argument is made to replace the P-Cognition thesis by the FPT-Cognition thesis as an alternative formalization of the Tractable Cognition thesis (here, FPT stands for fixed-parameter tractable). Possible objections to the Tractable Cognition thesis, and its proposed formalization, are discussed, and existing misconceptions are clarified.     

The Default Mode Network and Recurrent Depression: A Neurobiological Model of Cognitive Risk Factors

A neurobiological account of cognitive vulnerability for recurrent depression is presented based on recent developments of resting state neural networks. We propose that alterations in the interplay between task positive (TP) and task negative (TN) elements of the Default Mode Network (DMN) act as a neurobiological risk factor for recurrent depression mediated by cognitive mechanisms. In the framework, depression is characterized by an imbalance between TN-TP components leading to an overpowering of TP by TN activity. The TN-TP imbalance is associated with a dysfunctional internally-focused cognitive style as well as a failure to attenuate TN activity in the transition from rest to task. Thus we propose the TN-TP imbalance as overarching neural mechanism involved in crucial cognitive risk factors for recurrent depression, namely rumination, impaired attentional control, and cognitive reactivity. During remission the TN-TP imbalance persists predisposing to vulnerability of recurrent depression. Empirical data to support this model is reviewed. Finally, we specify how this framework can guide future research efforts.     

"认知革命"与"第二代认知科学"刍议

以计算隐喻为核心假设的传统认知心理学以及联结主义心理学均不能克服离身心智（disembodied mind）的根本缺陷,当代认知心理学正面临着新的范型转换．以具身性和情境性为重要特征的第二代认知科学将日受重视,并促使认知神经科学进入新的发展阶段。作者认为在身心关系上应该坚持生理只是心理的必要条件,而非充分条件的赢场,克服生理还原论的危险;应该重新审视基于二元论的生理机制这种说法;心理学传统中的科学主义和人文主义有可能在第二代认知科学强调认知情境性的基础上达成某种融合;第一代认知科学对意识的研究是不成功的,因为对知觉、注意、记忆、思维等心理过程的研究不能代替意识的研究,同时还应避免以意识内容的研究取代心理学研究的倾向、第二代认知科学中的动力系统理论关于变量（因素）之间的偶合（coupling）关系完全不同于变差分析巾的变量之间的交互作用关系,其动力系统模式可能更有助于破解意识的产生（涌现）之谜,并引发心理学研究的方法论的变革新潮,第二代认知科学的兴起将启发人们对身心关系、生理还原论、意识研究在心理学中的地位、人工智能对心智完全模拟的可能性等重夫问题重新思考。     

Learning Continuous Probability Distributions with Symmetric Diffusion Networks

In this article we present symmetric diffusion networks, a family of networks that instantiate the principles of continuous, stochastic, adaptive and interactive propagation of information. Using methods of Markovion diffusion theory, we formalize the activation dynamics of these networks and then show that they can be trained to reproduce entire multivariate probability distributions on their outputs using the contrastive Hebbion learning rule (CHL). We show that CHL performs gradient descent on an error function that captures differences between desired and obtained continuous multivariate probability distributions. This allows the learning algorithm to go beyond expected values of output units and to approximate complete probability distributions on continuous multivariate activation spaces. We argue that learning continuous distributions is an important task underlying a variety of real-life situations that were beyond the scope of previous connectionist networks. Deterministic networks, like back propagation, cannot learn this task because they are limited to learning average values of independent output units. Previous stochastic connectionist networks could learn probability distributions but they were limited to discrete variables. Simulations show that symmetric diffusion networks can be trained with the CHL rule to approximate discrete and continuous probability distributions of various types.     

The role of representation in computation

Reformers urge that representation no longer earns its explanatory keep in cognitive science, and that it is time to discard this troublesome concept. In contrast, we hold that without representation cognitive science is utterly bereft of tools for explaining natural intelligence. In order to defend the latter position, we focus on the explanatory role of representation in computation. We examine how the methods of digital and analog computation are used to model a relatively simple target system, and show that representation plays an in-eliminable explanatory role in both cases. We conclude that, to the extent that biologic systems engage in computation, representation is destined to play an explanatory role in cognitive science.

Nested incremental modeling in the development of computational theories: the CDP+ model of reading aloud

At least 3 different types of computational model have been shown to account for various facets of both normal and impaired single word reading: (a) the connectionist triangle model, (b) the dual-route cascaded model, and (c) the connectionist dual process model. Major strengths and weaknesses of these models are identified. In the spirit of nested incremental modeling, a new connectionist dual process model (the CDP+ model) is presented. This model builds on the strengths of 2 of the previous models while eliminating their weaknesses. Contrary to the dual-route cascaded model, CDP+ is able to learn and produce graded consistency effects. Contrary to the triangle and the connectionist dual process models, CDP+ accounts for serial effects and has more accurate nonword reading performance. CDP+ also beats all previous models by an order of magnitude when predicting individual item-level variance on large databases. Thus, the authors show that building on existing theories by combining the best features of previous models--a nested modeling strategy that is commonly used in other areas of science but often neglected in psychology--results in better and more powerful computational models.     

Dispelling the "mystery" of computational cognitive science

H. Crowther-Heyck (1999) argued that early advocates of computational cognitive science, especially George Miller, aimed to bring about a revival of traditional mentalism, including the issues of consciousness and free will. He therefore found it inexplicable, and even "ironic," that they selected the computer as their main research tool because computers seem no more conscious and no more free than, for instance, the telephone switchboard that was one of the behaviorists' key metaphors. I argue, by contrast, that this misunderstands the main thrust of cognitive science, which was not to bring back all of traditional mentalism, but was rather only to give a rigorous account of intentionality. Once this is recognized, Crowther-Heyck's "mystery" of cognitive science is dispelled because, as is well known, computers use symbolic representations, and thus were seen by the early cognitive scientists as being prime mechanical models of intentional processes.     

A neural cognitive model of argumentation with application to legal inference and decision making

Formal models of argumentation have been investigated in several areas, from multi-agent systems and artificial intelligence (AI) to decision making, philosophy and law. In artificial intelligence, logic-based models have been the standard for the representation of argumentative reasoning. More recently, the standard logic-based models have been shown equivalent to standard connectionist models. This has created a new line of research where (i) neural networks can be used as a parallel computational model for argumentation and (ii) neural networks can be used to combine argumentation, quantitative reasoning and statistical learning. At the same time, non-standard logic models of argumentation started to emerge. In this paper, we propose a connectionist cognitive model of argumentation that accounts for both standard and non-standard forms of argumentation. The model is shown to be an adequate framework for dealing with standard and non-standard argumentation, including joint-attacks, argument support, ordered attacks, disjunctive attacks, meta-level attacks, self-defeating attacks, argument accrual and uncertainty. We show that the neural cognitive approach offers an adequate way of modelling all of these different aspects of argumentation. We have applied the framework to the modelling of a public prosecution charging decision as part of a real legal decision making case study containing many of the above aspects of argumentation. The results show that the model can be a useful tool in the analysis of legal decision making, including the analysis of what-if questions and the analysis of alternative conclusions. The approach opens up two new perspectives in the short-term: the use of neural networks for computing prevailing arguments efficiently through the propagation in parallel of neuronal activations, and the use of the same networks to evolve the structure of the argumentation network through learning (e.g. to learn the strength of arguments from data).     

Connectionist psycholinguistics: capturing the empirical data

Connectionist psycholinguistics is an emerging approach to modeling empirical data on human language processing using connectionist computational architectures. For almost 20 years, connectionist models have increasingly been used to model empirical data across many areas of language processing. We critically review four key areas: speech processing, sentence processing, language production, and reading aloud, and evaluate progress against three criteria: data contact, task veridicality, and input representativeness. Recent connectionist modeling efforts have made considerable headway toward meeting these criteria, although it is by no means clear whether connectionist (or symbolic) psycholinguistics will eventually provide an integrated model of full-scale human language processing.     

Hybrid preference machines based on inspiration from neuroscience

In the past, a variety of computational problems have been tackled with different connectionist network approaches. However, very little research has been done on a framework which connects neuroscience-inspired models with connectionist models and higher level symbolic processing. In this paper, we outline a preference machine framework which focuses on a hybrid integration of various neural and symbolic techniques in order to address how we may process higher level concepts based on concepts from neuroscience. It is a first hybrid framework which allows a link between spiking neural networks, connectionist preference machines and symbolic finite state machines. Furthermore, we present an example experiment on interpreting a neuroscience-inspired network by using preferences which may be connected to connectionist or symbolic interpretations.     

Computationalism

What counts as a computation and how it relates to cognitive function are important questions for scientists interested in understanding how the mind thinks. This paper argues that pragmatic aspects of explanation ultimately determine how we answer those questions by examining what is needed to make rigorous the notion of computation used in the (cognitive) sciences. It (1) outlines the connection between the Church-Turing Thesis and computational theories of physical systems, (2) differentiates merely satisfying a computational function from true computation, and finally (3) relates how we determine a true computation to the functional methodology in cognitive science. All of the discussion will be directed toward showing that the only way to connect formal notions of computation to empirical theory will be in virtue of the pragmatic aspects of explanation.     

Analogy as relational priming: a developmental and computational perspective on the origins of a complex cognitive skill

The development of analogical reasoning has traditionally been understood in terms of theories of adult competence. This approach emphasizes structured representations and structure mapping. In contrast, we argue that by taking a developmental perspective, analogical reasoning can be viewed as the product of a substantially different cognitive ability - relational priming. To illustrate this, we present a computational (here connectionist) account where analogy arises gradually as a by-product of pattern completion in a recurrent network. Initial exposure to a situation primes a relation that can then be applied to a novel situation to make an analogy. Relations are represented as transformations between states. The network exhibits behaviors consistent with a broad range of key phenomena from the developmental literature, lending support to the appropriateness of this approach (using low-level cognitive mechanisms) for investigating a domain that has normally been the preserve of high-level models. Furthermore, we present an additional simulation that integrates the relational priming mechanism with deliberative controlled use of inhibition to demonstrate how the framework can be extended to complex analogical reasoning, such as the data from explicit mapping studies in the literature on adults. This account highlights how taking a developmental perspective constrains the theory construction and cognitive modeling processes in a way that differs substantially from that based purely on adult studies, and illustrates how a putative complex cognitive skill can emerge out of a simple mechanism.