SUSTAIN: a network model of category learning

SUSTAIN (Supervised and Unsupervised STratified Adaptive Incremental Network) is a model of how humans learn categories from examples. SUSTAIN initially assumes a simple category structure. If simple solutions prove inadequate and SUSTAIN is confronted with a surprising event (e.g., it is told that a bat is a mammal instead of a bird), SUSTAIN recruits an additional cluster to represent the surprising event. Newly recruited clusters are available to explain future events and can themselves evolve into prototypes-attractors-rules. SUSTAIN's discovery of category substructure is affected not only by the structure of the world but by the nature of the learning task and the learner's goals. SUSTAIN successfully extends category learning models to studies of inference learning, unsupervised learning, category construction, and contexts in which identification learning is faster than classification learning.     

Supervised versus unsupervised categorization: two sides of the same coin?

Supervised and unsupervised categorization have been studied in separate research traditions. A handful of studies have attempted to explore a possible convergence between the two. The present research builds on these studies, by comparing the unsupervised categorization results of Pothos et al. ( 2011 ; Pothos et al., 2008 ) with the results from two procedures of supervised categorization. In two experiments, we tested 375 participants with nine different stimulus sets and examined the relation between ease of learning of a classification, memory for a classification, and spontaneous preference for a classification. After taking into account the role of the number of category labels (clusters) in supervised learning, we found the three variables to be closely associated with each other. Our results provide encouragement for researchers seeking unified theoretical explanations for supervised and unsupervised categorization, but raise a range of challenging theoretical questions.     

Long-term persistence of sort strategy in free classification

Two free classification experiments that investigate the persistence of sort strategy are reported. Participants tend to persist with their initial categorization type (family resemblance or unidimensional) for the remaining sorts, overriding the effects of otherwise influential stimulus properties. Sort type was found to persist even after a one-week delay. Stimulus-driven models of free classification (e.g., the SUSTAIN model, [Love, B. C., Medin, D. L., & Gureckis, T. M. (2004). SUSTAIN: A network model of category learning. Psychological Review, 111, 309-332]) cannot predict the sort type persistence effects we observe, but they are naturally accounted for by theories that posit strategic selection of a problem-solving strategy (e.g., Hypothesis theory, [Levine, M. (1971). Hypothesis theory and nonlearning despite ideal S-R-reinforcement contingencies. Psychological Review, 78, 130-140]).     

Unsupervised categorization and category learning.

When people categorize a set of items in a certain way they often change their perceptions for these items so that they become more compatible with the learned categorization. In two experiments we examined whether such changes are extensive enough to change the unsupervised categorization for the items-that is, the categorization of the items that is considered more intuitive or natural without any learning. In Experiment 1 we directly employed an unsupervised categorization task; in Experiment 2 we collected similarity ratings for the items and inferred unsupervised categorizations using Pothos and Chater's (2002) model of unsupervised categorization. The unsupervised categorization for the items changed to resemble more the learned one when this was specified by the suppression of a stimulus dimension (both experiments), but less so when it was almost specified by the suppression of a stimulus dimension (Experiment 1, nonsignificant trend in Experiment 2). By contrast, no changes in the unsupervised categorization were observed when participants were taught a classification that was specified by a more fine tuning of the relative salience of the two dimensions.     

The divergent autoencoder (DIVA) model of category learning

A novel theoretical approach to human category learning is proposed in which categories are represented as coordinated statistical models of the properties of the members. Key elements of the account are learning to recode inputs as task-constrained principle components and evaluating category membership in terms of model fit-that is, the fidelity of the reconstruction after recoding and decoding the stimulus. The approach is implemented as a computational model called DIVA (for DIVergent Autoencoder), an artificial neural network that uses reconstructive learning to solve N-way classification tasks. DIVA shows good qualitative fits to benchmark human learning data and provides a compelling theoretical alternative to established models.     

Geometric and featural representations in semantic concepts

We explore the adequacy of two types of similarity representation in the context of semantic concepts. To this end, we evaluate different categorization models, assuming either a geometric or a featural representation, using categorization decisions involving familiar and unfamiliar foods and animals. The study aims to assess the optimal stimulus representation as a function of the familiarity of the stimuli. For the unfamiliar stimuli, the geometric categorization models provide the best account of the categorization data, whereas for the familiar stimuli, the featural categorization models provide the best account. This pattern of results suggests that people rely on perceptual information to assign an unfamiliar stimulus to a category but rely on more elaborate conceptual knowledge when assigning a familiar stimulus.     

The impact of category separation on unsupervised categorization

Most previous research on unsupervised categorization has used unconstrained tasks in which no instructions are provided about the underlying category structure or in which the stimuli are not clustered into categories. Few studies have investigated constrained tasks in which the goal is to learn predefined stimulus clusters in the absence of feedback. These studies have generally reported good performance when the stimulus clusters could be separated by a one-dimensional rule. In the present study, we investigated the limits of this ability. Results suggest that even when two stimulus clusters are as widely separated, as in previous studies, performance is poor if within-category variance on the relevant dimension is nonnegligible. In fact, under these conditions, many participants failed even to identify the single relevant stimulus dimension. This poor performance is generally incompatible with all current models of unsupervised category learning.     

非监控类别学习的分类策略与表征

类别学习是人类对不同类别加以归类的过程。类别信息的表征、分类策略运用的特点一直是类别学习研究的重点。非监控类别学习可分为直接的非监控类别学习和间接的非监控类别学习。直接的非监控类别学习(非限制任务, 限制任务)中被试的分类策略具有分类“单维度倾向”策略特点,类别变异程度会影响类别表征; 间接的非监控类别学习更倾向形成相似性表征, 直接的非监控类别学习则为基于规则表征。现有的非监控类别学习的理论对分类策略和表征的解释仍显薄弱, 不同学习任务下类别迁移和知识效应的研究还存在不足, 未来研究还需要进一步验证知识效应对非监控类别学习的认知加工过程的影响、探索影响类别表征形成的因素等问题。     

Learning and transfer of category knowledge in an indirect categorization task

Knowledge representations acquired during category learning experiments are ‘tuned’ to the task goal. A useful paradigm to study category representations is indirect category learning. In the present article, we propose a new indirect categorization task called the “same”–“different” categorization task. The same–different categorization task is a regular same–different task, but the question asked to the participants is about the stimulus category membership instead of stimulus identity. Experiment 1 explores the possibility of indirectly learning rule-based and information-integration category structures using the new paradigm. The results suggest that there is little learning about the category structures resulting from an indirect categorization task unless the categories can be separated by a one-dimensional rule. Experiment 2 explores whether a category representation learned indirectly can be used in a direct classification task (and vice versa). The results suggest that previous categorical knowledge acquired during a direct classification task can be expressed in the same–different categorization task only when the categories can be separated by a rule that is easily verbalized. Implications of these results for categorization research are discussed.     

Prototypes in category learning: the effects of category size, category structure, and stimulus complexity

Although research in categorization has sometimes been motivated by prototype theory, recent studies have favored exemplar theory. However, some of these studies focused on small, poorly differentiated categories composed of simple, 4-dimensional stimuli. Some analyzed the aggregate data of entire groups. Some compared powerful multiplicative exemplar models to less powerful additive prototype models. Here, comparable prototype and exemplar models were fit to individual-participant data in 4 experiments that sampled category sets varying in size, level of category structure, and stimulus complexity (dimensionality). The prototype model always fit the observed data better than the exemplar model did. Prototype-based processes seemed especially relevant when participants learned categories that were larger or contained more complex stimuli.     

Classification in well-defined and ill-defined categories: evidence for common processing strategies

Early work in perceptual and conceptual categorization assumed that categories had criterial features and that category membership could be determined by logical rules for the combination of features. More recent theories have assumed that categories have an ill-defined structure and have prosposed probabilistic or global similarity models for the verification of category membership. In the experiments reported here, several models of categorization were compared, using one set of categories having criterial features and another set having an ill-defined structure. Schematic faces were used as exemplars in both cases. Because many models depend on distance in a multidimensional space for their predictions, in Experiment 1 a multidimensional scaling study was performed using the faces of both sets as stimuli, In Experiment 2, subjects learned the category membership of faces for the categories having criterial features. After learning, reaction times for category verification and typicality judgments were obtained. Subjects also judged the similarity of pairs of faces. Since these categories had characteristic as well as defining features, it was possible to test the predictions of the feature comparison model (Smith et al.), which asserts that reaction times and typicalities are affected by characteristic features. Only weak support for this model was obtained. Instead, it appeared that subjects developed logical rules for the classification of faces. A characteristic feature affected reaction times only when it was part of the rule system devised by the subject. The procedure for Experiment 3 was like that for Experiment 2, but with ill-defined rather than well-defined categories. The obtained reaction times had high correlations with some of the models for ill-defined categories. However, subjects' performance could best be described as one of feature testing based on a logical rule system for classification. These experiments indicate that whether or not categories have criterial features, subjects attempt to develop a set of feature tests that allow for exemplar classification. Previous evidence supporting probabilistic or similarity models may be interpreted as resulting from subjects' use of the most efficient rules for classification and the averaging of responses for subjects using different sets of rules.     

Strategy development and learning differences in supervised and unsupervised categorization

The processes that determine unsupervised categorization, the task of classifying stimuli without guidance or feedback, are poorly understood. Two experiments examined the emergence and plasticity of unsupervised strategies using perceptual stimuli that varied along two separable dimensions. In the first experiment, participants either classified stimuli into any two categories of their choice or learned identical classifications by supervised categorization. Irrespective of the complexity of classification, supervised and unsupervised learning rates differed little when both modes of learning were maximally comparable. The second experiment examined the plasticity of unsupervised classifications by introducing novel stimuli halfway through training. Whether or not people altered their strategies, they responded to novel stimuli in a gradual manner. The gradual and continuous evolution and adaptation of strategies suggests that unsupervised categorization involves true learning which shares many properties of supervised category learning. We also show that the choice of unsupervised strategy cannot be predicted from the properties of early learning trials, but is best understood as a function of the initial distribution of dimensional attention.     

One or two dimensions in spontaneous classification: a simplicity approach

When participants are asked to spontaneously categorize a set of items, they typically produce unidimensional classifications, i.e., categorize the items on the basis of only one of their dimensions of variation. We examine whether it is possible to predict unidimensional vs. two-dimensional classification on the basis of the abstract stimulus structure, by employing Pothos and Chater's simplicity model of spontaneous categorization [Pothos, E. M., & Chater, N. (2002). A simplicity principle in unsupervised human categorization. Cognitive Science, 26, 303-343]. The simplicity model provides a quantitative measure of how intuitive a particular classification is. With objects represented in two dimensions, we propose that a unidimensional classification will be preferred if it is more intuitive than all possible two-dimensional ones, and vice versa. Empirical results supporting this proposal are reported. Implications for Goodman's paradox are discussed.     

Environment and Goals Jointly Direct Category Acquisition

Abstract— Developing categorization schemes involves discovering structures in the world that support a learner's goals. Existing models of category learning, such as exemplar and prototype models, neglect the role of goals in shaping conceptual organization. Here, a clustering approach is discussed that reflects the joint influences of the environment and goals in directing category acquisition. Clusters are a flexible representational medium that exhibits properties of exemplar, prototype, and rule-based models. Clusters reflect the natural bundles of correlated features present in our environment. The clustering model Supervised and Unsupervised Stratified Incremental Adaptive Network (SUSTAIN) operates by assuming the world has a simple structure and adding complexity (i.e., clusters) when existing clusters fail to satisfy the learner's goals and thus elicit surprise. Although simple, this operation is sufficient to address findings from numerous laboratory and cross-cultural categorization studies.     

Formation of category representations

Many formal models of categorization assume, implicitly or explicitly, that categorization results in the formation of direct associations from representations of the presented stimuli to representations of the experimentally provided category labels. In three categorization experiments employing a polymorphous classification structure (Dennis, Hampton, & Lea, 1973) and a partial reversal,optional shift procedure (Kendler, Kendler, & Wells, 1960), we provide evidence consistent with the hypothesis that learning a new classification problem results in the creation of category representations that mediate between representations of the stimulus and the label. This hypothesis can be instantiated through the AMBRY model (Kruschke, 1996).     

类别学习的SUSTAIN模型

SUSTAIN模型是关于类别学习的类群或群集规则的选择模型,它强调多层次的类别子结构和模型的结构搜索功能.模型的运行以类别相似性为基础,从最简单的规则开始,有灵活的参数及其运算过程,与其他主要的类别学习模型相比有更多的优势.因此,SUSTAIN模型是迄今解释人类类别学习的最优模型.     

Unsupervised category learning with integral-dimension stimuli

Despite the recent surge in research on unsupervised category learning, the majority of studies have focused on unconstrained tasks in which no instructions are provided about the underlying category structure. Relatively little research has focused on constrained tasks in which the goal is to learn predefined stimulus clusters in the absence of feedback. The few studies that have addressed this issue have focused almost exclusively on stimuli for which it is relatively easy to attend selectively to the component dimensions (i.e., separable dimensions). In the present study, we investigated the ability of participants to learn categories constructed from stimuli for which it is difficult, if not impossible, to attend selectively to the component dimensions (i.e., integral dimensions). The experiments demonstrate that individuals are capable of learning categories constructed from the integral dimensions of brightness and saturation, but this ability is generally limited to category structures requiring selective attention to brightness. As might be expected with integral dimensions, participants were often able to integrate brightness and saturation information in the absence of feedback—an ability not observed in previous studies with separable dimensions. Even so, there was a bias to weight brightness more heavily than saturation in the categorization process, suggesting a weak form of selective attention to brightness. These data present an important challenge for the development of models of unsupervised category learning.     

Models in search of a brain

Mental localization efforts tend to stress the where more than the what. We argue that the proper targets for localization are well-specified cognitive models. We make this case by relating an existing cognitive model of category learning to a learning circuit involving the hippocampus, perirhinal, and prefrontal cortices. Results from groups varying in function along this circuit (e.g., infants, amnesics, and older adults) are successfully simulated by reducing the model’s ability to form new clusters in response to surprising events, such as an error in supervised learning or an unfamiliar stimulus in unsupervised learning. Clusters in the model are akin to conjunctive codes that are rooted in an episodic experience (the surprising event) yet can develop to resemble abstract codes as they are updated by subsequent experiences. Thus, the model holds that the line separating episodic and semantic information can become blurred. Dissociations (categorization vs. recognition) are explained in terms of cluster recruitment demands.     

Response-time tests of logical-rule models of categorization

A recent resurgence in logical-rule theories of categorization has motivated the development of a class of models that predict not only choice probabilities but also categorization response times (RTs; Fifi?, Little, & Nosofsky, 2010). The new models combine mental-architecture and random-walk approaches within an integrated framework and predict detailed RT-distribution data at the level of individual participants and individual stimuli. To date, however, tests of the models have been limited to validation tests in which participants were provided with explicit instructions to adopt particular processing strategies for implementing the rules. In the present research, we test conditions in which categories are learned via induction over training exemplars and in which participants are free to adopt whatever classification strategy they choose. In addition, we explore how variations in stimulus formats, involving either spatially separated or overlapping dimensions, influence processing modes in rule-based classification tasks. In conditions involving spatially separated dimensions, strong evidence is obtained for application of logical-rule strategies operating in a serial-self-terminating processing mode. In conditions involving spatially overlapping dimensions, preliminary evidence is obtained that a mixture of serial and parallel processing underlies the application of rule-based classification strategies. The logical-rule models fare considerably better than major extant alternative models in accounting for the categorization RTs.     

Central tendencies, extreme points, and prototype enhancement effects in ill-defined perceptual categorization

In three perceptual classification experiments involving ill-defined category structures, extreme prototype enhancement effects were observed in which prototypes were classified more accurately than other category instances. Such empirical findings can prove theoretically challenging to exemplar-based models of categorization if prototypes are psychological central tendencies of category instances. We found instead that category prototypes were sometimes better characterized as psychological extreme points relative to contrast categories. Extending a classic and widely cited study (Posner & Keele, 1968), participants learned categories created from distortions of dot patterns arranged in familiar shapes. Participants then made pairwise similarity judgements of the patterns. Multidimensional scaling (MDS) analyses of the similarity data revealed the prototypes to be psychological extreme points, not central tendencies. Evidence for extreme point representations was also found for novel prototype patterns displaying a symmetry structure and for prototypes of grid patterns used in recent studies by McLaren and colleagues (McLaren, Bennet, Guttman-Nahir, Kim, & M ackintosh, 1995). When used in combination with the derived M DS solutions, an exemplar-based model of categorization, the Generalized Context Model (Nosofsky, 1986), provided good fits to the observed categorization data in all three experiments.