Hemispheric asymmetries in cognitive modeling: connectionist modeling of unilateral visual neglect

Neglect is an acquired cognitive disorder characterized by a lack of processing of one side of a stimulus or representational space. There are hemispheric asymmetries in its cause and in its effects, but implemented computational models of neglect have tended not to incorporate this fact. The authors report a series of neural network simulations of the line-bisection task. They test the hypothesis that simple, neuroanatomically realistic principles of connectivity in the nervous system can produce emergent behaviors that capture a wide range of quantitative and qualitative data observed in neglect patients presenting with general visuospatial neglect. They demonstrate that exploring low-level architectural principles in implemented computational models is both a productive avenue of research and offers the most parsimonious explanations of behaviors observed in patients.     

Consciousness: converging insights from connectionist modeling and neuroscience

Over the past decade, many findings in cognitive neuroscience have resulted in the view that selective attention, working memory and cognitive control involve competition between widely distributed representations. This competition is biased by top-down projections (notably from prefrontal cortex), which can selectively enhance some representations over others. This view has now been implemented in several connectionist models. In this review, we emphasize the relevance of these models to understanding consciousness. Interestingly, the models we review have striking similarities to others directly aimed at implementing 'global workspace theory'. All of these models embody a fundamental principle that has been used in many connectionist models over the past twenty years: global constraint satisfaction.     

Tools for connectionist modeling: The dynamical systems methodology

Some understanding of dynamical systems is essential to achieving competency in connectionist models. This mathematical background can be acquired either through a rigorous set of upper undergraduate and/or graduate formal courses or via disciplined self-teaching. As part of developing a course in connectionism, we feel that although certain very basic mathematical tools are most appropriately learned in their “pure” form (i.e., from mathematics textbooks and courses), more advanced exposure to dynamical systems theory can be given in the context of an introduction to connectionism. Students thus learn to write connectionist simulations by first writing programs for simulating arbitrary dynamical systems, then using them to learn some aspects of dynamical systems in general by simulating some special cases, and finally applying this technique to connectionist models of increasing complexity.     

Natural deduction in connectionist systems

The relation between logic and thought has long been controversial, but has recently influenced theorizing about the nature of mental processes in cognitive science. One prominent tradition argues that to explain the systematicity of thought we must posit syntactically structured representations inside the cognitive system which can be operated upon by structure sensitive rules similar to those employed in systems of natural deduction. I have argued elsewhere that the systematicity of human thought might better be explained as resulting from the fact that we have learned natural languages which are themselves syntactically structured. According to this view, symbols of natural language are external to the cognitive processing system and what the cognitive system must learn to do is produce and comprehend such symbols. In this paper I pursue that idea by arguing that ability in natural deduction itself may rely on pattern recognition abilities that enable us to operate on external symbols rather than encodings of rules that might be applied to internal representations. To support this suggestion, I present a series of experiments with connectionist networks that have been trained to construct simple natural deductions in sentential logic. These networks not only succeed in reconstructing the derivations on which they have been trained, but in constructing new derivations that are only similar to the ones on which they have been trained.     

Parameterization of connectionist models

We present a method for estimating parameters of connectionist models that allows the model’s output to fit as closely as possible to empirical data. The method minimizes a cost function that measures the difference between statistics computed from the model’s output and statistics computed from the subjects’ performance. An optimization algorithm finds the values of the parameters that minimize the value of this cost function. The cost function also indicates whether the model’s statistics are significantly different from the data’s. In some cases, the method can find the optimal parameters automatically. In others, the method may facilitate the manual search for optimal parameters. The method has been implemented in Matlab, is fully documented, and is available for free download from the Psychonomic Society Web archive atwww.psychonomic.org/archive/.     

Logic programs and connectionist networks

One facet of the question of integration of Logic and Connectionist Systems, and how these can complement each other, concerns the points of contact, in terms of semantics, between neural networks and logic programs. In this paper, we show that certain semantic operators for propositional logic programs can be computed by feedforward connectionist networks, and that the same semantic operators for first-order normal logic programs can be approximated by feedforward connectionist networks. Turning the networks into recurrent ones allows one also to approximate the models associated with the semantic operators. Our methods depend on a well-known theorem of Funahashi, and necessitate the study of when Funahashi's theorem can be applied, and also the study of what means of approximation are appropriate and significant.     

Parameterization of connectionist models.

We present a method for estimating parameters of connectionist models that allows the model's output to fit as closely as possible to empirical data. The method minimizes a cost function that measures the difference between statistics computed from the model's output and statistics computed from the subjects' performance. An optimization algorithm finds the values of the parameters that minimize the value of this cost function. The cost function also indicates whether the model's statistics are significantly different from the data's. In some cases, the method can find the optimal parameters automatically. In others, the method may facilitate the manual search for optimal parameters. The method has been implemented in Matlab, is fully documented, and is available for free download from the Psychonomic Society Web archive at www.psychonomic.org/archive/.     

Associative change in connectionist networks: an addendum

The results of a recent study have provided direct support for the suggestion that conditional learning in rats is best characterized by a 3-layer connectionist network (M. J. Allman, J. Ward-Robinson, & R. C. Honey, 2004). In the 2 experiments reported here, rats were used to investigate the nature of the changes that occur when a stimulus compound is presented, whose components activate hidden units associated with food and no food, and either food or no food is presented. The results of both experiments, while controlling for the possible contribution of associations between these hidden units (within-layer links), provide evidence that the distribution of associative change between units in the hidden layer that are activated by the stimulus compound and those in the output layer (between-layer links) are unequal. They also indicate that associative change is more marked on trials on which no food was presented than on trials on which food was presented.     

Prejudiced learning: A connectionist account

Connectionist simulation was employed to investigate processes that may underlie the relationships between prior expectancies or prejudices and the acquisition of attitudes, under conditions where learners can only discover the valence of attitude objects through directly experiencing them. We compared contexts analogous to learners holding either false negative expectancies (‘prejudices’) about a subclass of objects that were actually good or false positive expectancies about objects that were actually bad. We introduced expectancy‐related bias either by altering the probability of approach, or by varying the rate of learning following experience with good or bad objects. Where feedback was contingent on approach, the false positive expectancies were corrected by experience, but negative prejudices resisted change, since the network avoided objects deemed to be bad, and so received less corrective feedback. These findings are discussed in relation to the effects of intergroup contact and expectancy‐confirmation processes in reducing or sustaining prejudice.     

A connectionist theory of phenomenal experience

When cognitive scientists apply computational theory to the problem of phenomenal consciousness, as many have been doing recently, there are two fundamentally distinct approaches available. Consciousness is to be explained either in terms of the nature of the representational vehicles the brain deploys or in terms of the computational processes defined over these vehicles. We call versions of these two approaches vehicle and process theories of consciousness, respectively. However, although there may be space for vehicle theories of consciousness in cognitive science, they are relatively rare. This is because of the influence exerted, on the one hand, by a large body of research that purports to show that the explicit representation of information in the brain and conscious experience are dissociable, and on the other, by the classical computational theory of mind--the theory that takes human cognition to be a species of symbol manipulation. Two recent developments in cognitive science combine to suggest that a reappraisal of this situation is in order. First, a number of theorists have recently been highly critical of the experimental methodologies used in the dissociation studies--so critical, in fact, that it is no longer reasonable to assume that the dissociability of conscious experience and explicit representation has been adequately demonstrated. Second, classicism, as a theory of human cognition, is no longer as dominant in cognitive science as it once was. It now has a lively competitor in the form of connectionism; and connectionism, unlike classicism, does have the computational resources to support a robust vehicle theory of consciousness. In this target article we develop and defend this connectionist vehicle theory of consciousness. It takes the form of the following simple empirical hypothesis: phenomenal experience consists of the explicit representation of information in neurally realized parallel distributed processing (PDP) networks. This hypothesis leads us to reassess some common wisdom about consciousness, but, we argue, in fruitful and ultimately plausible ways.     

Word recognition in developmental dyslexia: A connectionist interpretation

We present a study of the accuracy, consistency, and speed of word naming in a dyslexic boy, JM, who has severe impairments in the ability to use sub-lexical, phonological reading strategies. For words that he can recognise, JM's naming latencies do not differ from those of control subjects matched for reading age, and he is generally consistent from one occasion to the next. He can also match printed homophones with their definitions--a skill that requires access to well-specified orthographic representations. The data are interpreted as evidence for the creation of efficient recognition devices for words within JM's sight vocabulary. However, he shows a continuing inability to use phonological decoding strategies to deal with words that he cannot recognize by sight. Overall we argue our results pose problems for stage models of reading development, and that they may best be interpreted within a connectionist framework of the development of word recognition skills.     

A recurrent connectionist model of group biases

Major biases and stereotypes in group judgments are reviewed and modeled from a recurrent connectionist perspective. These biases are in the areas of group impression formation (illusory correlation), group differentiation (accentuation), stereotype change (dispersed vs. concentrated distribution of inconsistent information), and group homogeneity. All these phenomena are illustrated with well-known experiments, and simulated with an autoassociative network architecture with linear activation update and delta learning algorithm for adjusting the connection weights. All the biases were successfully reproduced in the simulations. The discussion centers on how the particular simulation specifications compare with other models of group biases and how they may be used to develop novel hypotheses for testing the connectionist modeling approach and, more generally, for improving theorizing in the field of social biases and stereotype change.     

Sensitivity analysis of structural equation models

The main purpose of this paper is to investigate the sensitivity analysis of structural equation model when minor perturbation is introduced. Some influence measure that based on the general case weight perturbation is derived for the generalized least squares estimation. An influence measure that related to the Cook's distance is also developed for the special case deletion perturbation scheme. Using the proposed methodology, the influential observation in a data set can be detected. Moreover, the general theory can be applied to detect the influential parameters in a model. Finally, some illustrative artificial and real examples are presented. The research of the first author was supported by a Hong Kong UPGC grant. The authors are greatly indebted to two reviewers for some very valuable comments for improvement of the paper.     

Generative connectionist networks and constructivist cognitive development

This article presents a novel computational framework for modeling cognitive development. The new modeling paradigm provides a language with which to compare and contrast radically different facets of children's knowledge. Concepts from the study of machine learning are used to explore the power of connectionist networks that construct their own architectures during learning. These so-called generative algorithms are shown to escape from Fodor's (1980) critique of Constructivist development. We describe one generative connectionist algorithm (cascade-correlation) in detail. We report on the successful use of the algorithm to model cognitive development on balance scale phenomena; seriation; the integration of velocity, time, and distance cues; prediction of effect sizes from magnitudes of causal potencies and effect resistances; and the acquisition of English personal pronouns. The article demonstrates that computer models are invaluable for illuminating otherwise obscure discussions.     

Acquiring contextualized concepts: a connectionist approach

Conceptual knowledge is acquired through recurrent experiences, by extracting statistical regularities at different levels of granularity. At a fine level, patterns of feature co-occurrence are categorized into objects. At a coarser level, patterns of concept co-occurrence are categorized into contexts. We present and test CONCAT, a connectionist model that simultaneously learns to categorize objects and contexts. The model contains two hierarchically organized CALM modules (Murre, Phaf, & Wolters, 1992). The first module, the Object Module, forms object representations based on co-occurrences between features. These representations are used as input for the second module, the Context Module, which categorizes contexts based on object co-occurrences. Feedback connections from the Context Module to the Object Module send activation from the active context to those objects that frequently occur within this context. We demonstrate that context feedback contributes to the successful categorization of objects, especially when bottom-up feature information is degraded or ambiguous.     

Forgetting curves: implications for connectionist models

Forgetting in long-term memory, as measured in a recall or a recognition test, is faster for items encoded more recently than for items encoded earlier. Data on forgetting curves fit a power function well. In contrast, many connectionist models predict either exponential decay or completely flat forgetting curves. This paper suggests a connectionist model to account for power-function forgetting curves by using bounded weights and by generating the learning rates from a monotonically decreasing function. The bounded weights introduce exponential forgetting in each weight and a power-function forgetting results when weights with different learning rates are averaged. It is argued that these assumptions are biologically reasonable. Therefore power-function forgetting curves are a property that may be expected from biological networks. The model has an analytic solution, which is a good approximation of a power function displaced one lag in time. This function fits better than any of the 105 suggested two-parameter forgetting-curve functions when tested on the most precise recognition memory data set collected by. Unlike the power-function normally used, the suggested function is defined at lag zero. Several functions for generating learning rates with a finite integral yield power-function forgetting curves; however, the type of function influences the rate of forgetting. It is shown that power-function forgetting curves cannot be accounted for by variability in performance between subjects because it requires a distribution of performance that is not found in empirical data. An extension of the model accounts for intersecting forgetting curves found in massed and spaced repetitions. The model can also be extended to account for a faster forgetting rate in item recognition (IR) compared to associative recognition in short but not long retention intervals.     

A connectionist account of asymmetric category learning in early infancy

Young infants show unexplained asymmetries in the exclusivity of categories formed on the basis of visually presented stimuli. A connectionist model is described that shows similar exclusivity asymmetries when categorizing the same stimuli presented to infants. The asymmetries can be explained in terms of an associative learning mechanism, distributed internal representations, and the statistics of the feature distributions in the stimuli. The model was used to explore the robustness of this asymmetry. The model predicts that the asymmetry will persist when a category is acquired in the presence of mixed category exemplars. An experiment with 3-4-month-olds showed that asymmetric exclusivity persisted in the presence of mixed-exemplar familiarization, thereby confirming the model's prediction.     

Classical and connectionist models: Levels of description

To begin, I introduce an analysis of interlevel relations that allows us to offer an initial characterization of the debate about the way classical and connectionist models relate. Subsequently, I examine a compatibility thesis and a conditional claim on this issue. With respect to the compatibility thesis, I argue that, even if classical and connectionist models are not necessarily incompatible, the emergence of the latter seems to undermine the best arguments for the Language of Thought Hypothesis, which is essential to the former. I attack the conditional claim of connectionism to eliminativism, presented by Ramsey et al. (1990), by discrediting their discrete characterization of common-sense psychological explanations and pointing to the presence of a moderate holistic constraint. Finally, I conclude that neither of the arguments considered excludes the possibility of viewing connectionist models as forming a part of a representational theory of cognition that dispenses with the Language of Thought Hypothesis.     

A connectionist perspective on the development of reading skills in children

The development of decoding skills has traditionally been viewed as a stage-like process during which children's reading strategies change as a consequence of the acquisition of phonological awareness. More explicit accounts of the mechanisms involved in learning to read are provided by recent connectionist models in which children learn mappings initially between orthography and phonology, and later between orthography, phonology and semantics. Evidence from studies of reading development suggests that learning to read is determined primarily by the status of a child's phonological representations and is therefore compromised in dyslexic children who have phonological deficits. Children who have language impairments encompassing deficits in semantic representations have qualitatively different reading problems centring on difficulties with reading comprehension and in learning to read exception words.