Connectionist representation,semantic compositionality,and the instability of concept structure

Summary In the present paper connectionist approaches to the problem of internal representation and the nature of concepts are discussed. In the first part the concept of representation that underlies connectionist modeling is made explicit. It is argued that the connectionist view of representation relies on a correlational theory of semantic content- i.e., the covariation between internal and external states is taken as the basis for ascribing meaning to internal states. The problems and virtues of such a correlational approach to internal representation are addressed. The second part of the paper is concerned with whether connectionism is capable of accounting for the apparent productivity and systematicity of language and thought. There is an evaluation of the recent arguments of Fodor and Pylyshyn, who claim that systematicity can only be explained if one conceives of mental representations as structured symbols composed of context-free constituents. There is a review of empirical evidence that strongly suggests that concepts are not fixed memory structures and that the meaning of constituent symbols varies, depending on the context in which they are embedded. On the basis of this review it is concluded that the meaning of a complex expression is not computed from the context-free meanings of the constituents, and that strong compositionality, as endorsed by Fodor and Pylyshyn (1988), seems implausible as a process theory for the comprehension of complex concepts. Instead, the hypothesis is endorsed that constraint satisfaction in distributed connectionist networks may allow for an alternative account of weak compositionality compatible with the context sensitivity of meaning. In the final section, it is argued that neither mere implementation of a language of thought in connectionist networks nor radical elimination of symbol systems seems to be a fruitful research strategy, but that it might be more useful to discuss how connectionist systems can develop the capacity to use external symbol systems like language or logic without instantiating symbol systems themselves.     

Connectionist modeling of speech perception.

Connectionist models of perception and cognition, including the process of deducing meaningful messages from patterns of acoustic waves emitted by vocal tracts, are developed and refined as human understanding of brain function, psychological processes, and the properties of massively parallel architectures advances. The present article presents several important contributions from diverse points of view in the area of connectionist modeling of speech perception and discusses their relative merits with respect to specific theoretical issues and empirical findings. TRACE, the Elman/Norris net, and Adaptive Resonance Theory constitute pivotal points exemplifying overall modeling success, progress in temporal representation, and plausible modeling of learning, respectively. Other modeling efforts are presented for the specific insights they offer, and the article concludes with a discussion of computational versus dynamic modeling of phonological processes.     

Connectionist psycholinguistics: capturing the empirical data

Connectionist psycholinguistics is an emerging approach to modeling empirical data on human language processing using connectionist computational architectures. For almost 20 years, connectionist models have increasingly been used to model empirical data across many areas of language processing. We critically review four key areas: speech processing, sentence processing, language production, and reading aloud, and evaluate progress against three criteria: data contact, task veridicality, and input representativeness. Recent connectionist modeling efforts have made considerable headway toward meeting these criteria, although it is by no means clear whether connectionist (or symbolic) psycholinguistics will eventually provide an integrated model of full-scale human language processing.     

Varieties of working memory as seen in biology and in connectionist/control architectures

Biological and computational concepts that underlie the nature working memory are briefly reviewed. The conceptualization of working memory has changed dramatically in the last 30 years. Current biological work has monitored several aspects of memory, including activation decay, sustained activation, long-term connection change, and differential structures for episodic (hippocampal formation) and procedural learning. Current connectionist modeling has identified factors including multiple-region-based processing, control processing as well as data storage, tradeoffs between fast- and slow-connection-change learning effects, and the speeding of acquisition via multiple levels of learning. The need to relate the biological, behavioral, and computational constraints into models of working memory is discussed. Finally, conceptualizations of working memory must acknowledge the need for human learning systems to be robust enough to operate in a dynamic world.     

Connectionist models of development

How have connectionist models informed the study of development? This paper considers three contributions from specific models. First, connectionist models have proven useful for exploring nonlinear dynamics and emergent properties, and their role in nonlinear developmental trajectories, critical periods and developmental disorders. Second, connectionist models have informed the study of the representations that lead to behavioral dissociations. Third, connectionist models have provided insight into neural mechanisms, and why different brain regions are specialized for different functions. Connectionist and dynamic systems approaches to development have differed, with connectionist approaches focused on learning processes and representations in cognitive tasks, and dynamic systems approaches focused on mathematical characterizations of physical elements of the system and their interactions with the environment. The two approaches also share much in common, such as their emphasis on continuous, nonlinear processes and their broad application to a range of behaviors.     

Computing with connections in visual recognition of origami objects

This paper summarizes our initial foray in tackling Artificial Intelligence problems using a connectionist approach. The particular task chosen was the visual recognition of objects in the Origami world as defined by. The two major questions answered were how to construct a connectionist network to represent and recognize projected line drawings of Origami objects and what advantages such an approach would have. The structure of the resulting connectionist network can be described as a hierarchy of parameter or feature spaces with each node in each of the feature spaces representing a hypothesis about the possible existence of a specific geometric feature of an Origami object. The dynamic behavior of the network is a form of iterative refinement or relaxation whose major characteristic is to prefer more globally interesting interpretations of the input over locally pleasing ones. Examples from the implementation illustrate the system's ability to deal with forms of noise, occlusion and missing information. Other benefits are an inherently parallel approach to vision, limitation of explicit ordering of the search involved in matching model to instance and the elimination of backtracking due to the sharing of partial results as the search progresses. Extensions and problems are also discussed.     

Connectionism and Self: James,Mead, and the Stream of Enculturated Consciousness

William James conceptualized I, the self as subject as a stream of consciousness. When this conception is augmented with George Herbert Mead's view of self as a radically socialized and enculturated process, a result is the James-Mead model of dynamic self as a stream of enculturated consciousness. In this paper, we argue that connectionism is best suited to theorize this challenging notion. Based on the view that a connectionist model should describe psychological processes that carry out psychological functions grounded in a biological living system, we propose the I-SELF (Imitative and Sequence Learning Functional) model, which is designed to capture the temporal dynamics of a stream of consciousness whose content can be acquired via symbolically mediated social interaction with others in society. We identify four implications of the James-Mead model of dynamic self (embodiment, narrative and self, individual and collective self, and culture and self), and report computer simulations to show the utility of I-SELF in conceptualizing the dynamic self-processes in the contemporary social psychological literature. Theoretical and metatheoretical implications of the connectionist approach to self are discussed.     

Nested incremental modeling in the development of computational theories: the CDP+ model of reading aloud

At least 3 different types of computational model have been shown to account for various facets of both normal and impaired single word reading: (a) the connectionist triangle model, (b) the dual-route cascaded model, and (c) the connectionist dual process model. Major strengths and weaknesses of these models are identified. In the spirit of nested incremental modeling, a new connectionist dual process model (the CDP+ model) is presented. This model builds on the strengths of 2 of the previous models while eliminating their weaknesses. Contrary to the dual-route cascaded model, CDP+ is able to learn and produce graded consistency effects. Contrary to the triangle and the connectionist dual process models, CDP+ accounts for serial effects and has more accurate nonword reading performance. CDP+ also beats all previous models by an order of magnitude when predicting individual item-level variance on large databases. Thus, the authors show that building on existing theories by combining the best features of previous models--a nested modeling strategy that is commonly used in other areas of science but often neglected in psychology--results in better and more powerful computational models.     

NETWORKS AND THEORIES: The Place of Connectionism to Cognitive Science

This article considers how connectionist modeling can contribute to understanding of human cognition. I argue that connectionist networks should not be thought of as theories or simulations of theories, hut may nevertheless contribute to the development of theories.     

A computational analysis of conservation

An approach to modeling cognitive development with a generative connectionist algorithm is described and illustrated with a new model of conservation acquisition. Among the conservation phenomena captured with this model are acquisition, the problem size effect, the length bias effect, and the screening effect. The simulations suggest novel explanations for sudden jumps in conservation performance (based on new representations of conservation transformations) and for the problem size effect (based on an analog representation of number). The simulations support the correlation-learning explanation of length bias (that length correlates with number during number altering transformations). Some conservation phenomena that so far elude computational modeling attempts are also discussed along with their prospects for capture. Suggestions are made for theorizing about cognitive development as well as about conservation acquisition. A variety of classic puzzles about cognitive development are addressed in the light of this model and similar models of other aspects of cognitive development.     

Group impressions as dynamic configurations: the tensor product model of group impression formation and change

Group impressions are dynamic configurations. The tensor product model (TPM), a connectionist model of memory and learning, is used to describe the process of group impression formation and change, emphasizing the structured and contextualized nature of group impressions and the dynamic evolution of group impressions over time. TPM is first shown to be consistent with algebraic models of social judgment (the weighted averaging model; N. Anderson, 1981) and exemplar-based social category learning (the context model; E. R. Smith & M. A. Zárate, 1992), providing a theoretical reduction of the algebraic models to the present connectionist framework. TPM is then shown to describe a common process that underlies both formation and change of group impressions despite the often-made assumption that they constitute different psychological processes. In particular, various time-dependent properties of both group impression formation (e.g., time variability, response dependency, and order effects in impression judgments) and change (e.g., stereotype change and group accentuation) are explained, demonstrating a hidden unity beneath the diverse array of empirical findings. Implications of the model for conceptualizing stereotype formation and change are discussed.     

The facets of anxiety sensitivity represented in the childhood anxiety sensitivity index: confirmatory analyses of factor models from past studies

Results of past factor analytic studies of the Childhood Anxiety Sensitivity Index and Anxiety Sensitivity Index were used to formulate hypotheses about factor models of anxiety sensitivity. Using a nonclinical sample of 767 children and adolescents and confirmatory factor analysis, hypothesized models with 2, 3, and 4 lower order factors (facets) were tested. Goodness-of-fit criteria indicated that a model with 4 facets fits these data well. Support was found for factorial invariance of the 4 facets across age and gender, using nonclinical and clinical samples. Results support a hierarchical factor model in that there was a strong general factor, explaining 71% of the variance. Findings are discussed in the context of anxiety sensitivity theory and research with children and adolescents.     

Generative connectionist networks and constructivist cognitive development

This article presents a novel computational framework for modeling cognitive development. The new modeling paradigm provides a language with which to compare and contrast radically different facets of children's knowledge. Concepts from the study of machine learning are used to explore the power of connectionist networks that construct their own architectures during learning. These so-called generative algorithms are shown to escape from Fodor's (1980) critique of Constructivist development. We describe one generative connectionist algorithm (cascade-correlation) in detail. We report on the successful use of the algorithm to model cognitive development on balance scale phenomena; seriation; the integration of velocity, time, and distance cues; prediction of effect sizes from magnitudes of causal potencies and effect resistances; and the acquisition of English personal pronouns. The article demonstrates that computer models are invaluable for illuminating otherwise obscure discussions.     

Cognitive processing of facial affect: connectionist model of deviations in schizophrenia

Connectionist-model simulations of competing hypotheses of cognition in schizophrenia were constructed and tested. Emphasis was placed on judgment of affect, a prominent area of disturbance in this disorder with potential implications for social impairment. Participants with paranoid or nonparanoid schizophrenia and control participants provided judgments of affect as expressed in photographic faces. Schizophrenia groups were less accurate than control groups, and the paranoid group had greater latencies than did other groups. Model predictions simultaneously addressed judgment content and latencies for each trial. Results provide a connectionist extension of an account of deficits in schizophrenia that originated at the computational (stochastic modeling) level of analysis. This account postulates extra stages of item encoding but no reduction in formally defined processing capacity. It also provides for abnormalities in both judgment patterns and duration and is consistent with biological accounts of schizophrenia deficits. The substantive findings are supported by strategic innovations in the construction and testing of connectionist models.     

Challenging the widespread assumption that connectionism and distributed representations go hand-in-hand

One of the central claims associated with the parallel distributed processing approach popularized by D.E. Rumelhart, J.L. McClelland and the PDP Research Group is that knowledge is coded in a distributed fashion. Localist representations within this perspective are widely rejected. It is important to note, however, that connectionist networks can learn localist representations and many connectionist models depend on localist coding for their functioning. Accordingly, a commitment to distributed representations should be considered a specific theoretical claim regarding the structure of knowledge rather than a core principle, as often assumed. In this paper, it is argued that there are fundamental computational and empirical challenges that have not yet been addressed by distributed connectionist theories that are readily accommodated within localist approaches. This is highlighted in the context of modeling word and nonword naming, the domain in which some of the strongest claims have been made. It is shown that current PDP models provide a poor account of naming monosyllable items, and that distributed representations make it difficult for these models to scale up to more complex language phenomena. At the same time, models that learn localist representations are shown to hold promise in supporting many of the core reading and language functions on which PDP models fail. It is concluded that the common rejection of localist coding schemes within connectionist architectures is premature.     

Implicit and explicit memory: Implications for the symbol-manipulation versus connectionism controversy

Summary This paper deals with three issues. First, the symbol-manipulation versus connectionism controversy is discussed briefly. Both can be seen as formalisms or languages for describing human behavior, but it is argued that these languages are not fully equivalent. The importance of the ability of connectionist models to learn autonomously, often underestimated, allows them to perform tasks that may defy formalization. Secondly, the structural and functional characteristics of a new connectionist learning model are presented. It avoids some of the psychological and biological implausibilities of currently popular models and solves some of the problems and shortcomings of these models. Thirdly, some simulation results are presented. The model successfully simulates the dissociation between explicit and implicit memory performance as found in patients with anterograde amnesia. This simulation provides an example of the advantages of using the connectionist language in the domain of memory psychology. It is also a good example of how psychological evidence can be used to improve connectionist models.The second author is supported by the Dutch Organization for Scientific Research (NWO)     

Exploring the dynamics of design fluency in children with and without ADHD using artificial neural networks

The neuropsychology of attention deficit/hyperactivity disorder (ADHD) has been extensively studied, with a general focus on global performance measures of executive function. In this study, we compared how global (i.e., endpoint) versus process (i.e., dynamic) measures of performance may help characterize children with and without ADHD using a design fluency task as a case study. The secondary goal was to compare the sensitivity of standard versus connectionist statistical models to group differences in cognitive data. Thirty-four children diagnosed with ADHD and 37 children without ADHD aged 8–11 years old were tested on the Five-Point Test. The continuous process measure of performance, indexed as the number of produced designs at each consecutive 1 minute interval during 5 minutes, was analyzed against the discrete process measure, that is, the number of designs between first and last intervals and the standard global performance measure of total number of produced designs. Results show that the continuous process measure distinguished the two groups better than the two other measures. The detailed observation of production patterns revealed a decreasing linear trajectory in children without ADHD that contrasts with the flat, but fluctuating productivity pattern of children with ADHD. With regards to the second goal, results show that the connectionist and standard methods are equally sensitive to group differences for the three types of measures. This illustrates the utility of quantitative process measures together with the connectionist method in neuropsychological research and suggests great potential for a dynamical approach to cognition.     

Different is good: connectionism and dynamic systems theory are complementary emergentist approaches to development

Connectionist and dynamic systems approaches to development are similar in that they are both emergentist theories that take a very different perspective from more traditional symbolic systems. Moreover, they are both based on similar mathematical principles. Nevertheless, connectionism and dynamic systems differ in the approach they take to the study of development. We argue that differences between connectionist and dynamic systems approaches in terms of the basic components of the models, what they see as the object of study, how they view the nature of knowledge and their notions of developmental change mean that they each stand to make different and unique contributions to a more complete theory of development. We present an example from our work on how children learn to learn words that illustrates the complementary nature of connectionist and dynamic systems theories.     

A recurrent connectionist model of group biases

Major biases and stereotypes in group judgments are reviewed and modeled from a recurrent connectionist perspective. These biases are in the areas of group impression formation (illusory correlation), group differentiation (accentuation), stereotype change (dispersed vs. concentrated distribution of inconsistent information), and group homogeneity. All these phenomena are illustrated with well-known experiments, and simulated with an autoassociative network architecture with linear activation update and delta learning algorithm for adjusting the connection weights. All the biases were successfully reproduced in the simulations. The discussion centers on how the particular simulation specifications compare with other models of group biases and how they may be used to develop novel hypotheses for testing the connectionist modeling approach and, more generally, for improving theorizing in the field of social biases and stereotype change.