Goal‐Proximity Decision‐Making

Reinforcement learning (RL) models of decision‐making cannot account for human decisions in the absence of prior reward or punishment. We propose a mechanism for choosing among available options based on goal‐option association strengths, where association strengths between objects represent previously experienced object proximity. The proposed mechanism, Goal‐Proximity Decision‐making (GPD), is implemented within the ACT‐R cognitive framework. GPD is found to be more efficient than RL in three maze‐navigation simulations. GPD advantages over RL seem to grow as task difficulty is increased. An experiment is presented where participants are asked to make choices in the absence of prior reward. GPD captures human performance in this experiment better than RL.     

Heterogeneity in generalized reinforcement learning and its relation to cognitive ability

In this paper, we study the connections between working memory capacity (WMC) and learning in the context of economic guessing games. We apply a generalized version of reinforcement learning, popularly known as the experience-weighted attraction (EWA) learning model, which has a connection to specific cognitive constructs, such as memory decay, the depreciation of past experience, counterfactual thinking, and choice intensity. Through the estimates of the model, we examine behavioral differences among individuals due to different levels of WMC. In accordance with ‘Miller’s magic number’, which is the constraint of working memory capacity, we consider two different sizes (granularities) of strategy space: one is larger (finer) and one is smaller (coarser). We find that constraining the EWA models by using levels (granules) within the limits of working memory allows for a better characterization of the data based on individual differences in WMC. Using this level-reinforcement version of EWA learning, also referred to as the EWA rule learning model, we find that working memory capacity can significantly affect learning behavior. Our likelihood ratio test rejects the null that subjects with high WMC and subjects with low WMC follow the same EWA learning model. In addition, the parameter corresponding to ‘counterfactual thinking ability’ is found to be reduced when working memory capacity is low.     

大鼠注意定势转移任务模型的深入研究：种系和检测程序的影响

注意定势转移任务(attentional set-shifting task, AST)可用于特异性检测啮齿类动物前额叶皮层及其皮层下神经通路介导的认知灵活性, 是目前研究认知灵活性及其障碍神经基础的重要模型。本研究系统调查了大鼠种系和检测程序差异对AST结果的影响。通过比较Wistar和Sprague Dawley (SD)两个种系大鼠在七阶段和五阶段两种AST检测程序中的认知表现, 研究发现：(1) SD和Wistar大鼠前额叶认知功能存在差异, 后者的总体认知表现优于前者。尤其是Wistar大鼠在逆反学习阶段的达标训练次数及错误率显著低于SD大鼠, 表明Wistar大鼠具有更高的策略转换灵活性。(2)在AST测试中逆反学习和外维度定势转移是认知灵活性评价的核心指标。这两种认知转换过程分别以前期策略和注意定势建立为基础。结果显示在两种AST检测程序中Wistar和SD大鼠在逆反学习和/或外维度定势转移等复杂学习阶段的达标训练次数和错误率均高于其它简单关联学习阶段, 表明在目前实验条件下大鼠均表现出定势形成和转换困难的反应模式, 不同认知反应间的结构关系具有稳定性。这些结果提示大鼠前额叶皮质介导的认知灵活性存在种系差异, AST各阶段认知反应间的结构效度不受目前使用的大鼠种系和检测程序差异的影响, 扩展了对AST模型的认识。     

An associative account of the development of word learning

Word learning is a notoriously difficult induction problem because meaning is underdetermined by positive examples. How do children solve this problem? Some have argued that word learning is achieved by means of inference: young word learners rely on a number of assumptions that reduce the overall hypothesis space by favoring some meanings over others. However, these approaches have difficulty explaining how words are learned from conversations or text, without pointing or explicit instruction. In this research, we propose an associative mechanism that can account for such learning. In a series of experiments, 4-year-olds and adults were presented with sets of words that included a single nonsense word (e.g. dax). Some lists were taxonomic (i.,e., all items were members of a given category), some were associative (i.e., all items were associates of a given category, but not members), and some were mixed. Participants were asked to indicate whether the nonsense word was an animal or an artifact. Adults exhibited evidence of learning when lists consisted of either associatively or taxonomically related items. In contrast, children exhibited evidence of word learning only when lists consisted of associatively related items. These results present challenges to several extant models of word learning, and a new model based on the distinction between syntagmatic and paradigmatic associations is proposed.     

The influence of depression symptoms on exploratory decision-making

People with symptoms of depression show impairments in decision-making. One explanation is that they have difficulty maintaining rich representations of the task environment. We test this hypothesis in the context of exploratory choice. We analyze depressive and non-depressive participants’ exploration strategies by comparing their choices to two computational models: (1) an “Ideal Actor” model that reflectively updates beliefs and plans ahead, employing a rich representation of the environment and (2) a “Naïve Reinforcement Learning” (RL) model that updates beliefs reflexively utilizing a minimal task representation. Relative to non-depressive participants, we find that depressive participants’ choices are better described by the simple RL model. Further, depressive participants were more exploratory than non-depressives in their decision-making. Depressive symptoms appear to influence basic mechanisms supporting choice behavior by reducing use of rich task representations and hindering performance during exploratory decision-making.     

A temporal common ground for learning: The moderating effect of shared mental models on the relation between team learning behaviours and performance improvement

In this longitudinal study, we integrated a team process and a learning curve perspective on team learning and empirically analysed whether team learning processes lead to performance improvement. In addition, we tested whether this relation is moderated by the similarity of team members’ task, team, and temporal mental models. We tested our model on a sample of 67 teams (314 individuals) competing in a management simulation over five consecutive time periods, using random coefficient modelling (RCM). Our findings suggest that team learning behaviours do not have a direct effect on the team learning curve, but temporal and task mental models are crucial for the translation of team learning behaviours into performance improvement. We found that when teams have similar task and temporal mental models, engaging in team learning processes is beneficial, whereas, when teams have dissimilar task and temporal mental models, it is detrimental to performance improvement. We did not find a significant effect for the moderating role of team mental model similarity. Our study emphasizes the importance of integrating different perspectives on team learning and provides support for the role of team cognition as a catalyst for team learning.     

Heterogeneity of strategy use in the Iowa gambling task: A comparison of win-stay/lose-shift and reinforcement learning models

The Iowa gambling task (IGT) has been used in numerous studies, often to examine decision-making performance in different clinical populations. Reinforcement learning (RL) models such as the expectancy valence (EV) model have often been used to characterize choice behavior in this work, and accordingly, parameter differences from these models have been used to examine differences in decision-making processes between different populations. These RL models assume a strategy whereby participants incrementally update the expected rewards for each option and probabilistically select options with higher expected rewards. Here we show that a formal model that assumes a win-stay/lose-shift (WSLS) strategy—which is sensitive only to the outcome of the previous choice—provides the best fit to IGT data from about half of our sample of healthy young adults, and that a prospect valence learning (PVL) model that utilizes a decay reinforcement learning rule provides the best fit to the other half of the data. Further analyses suggested that the better fits of the WSLS model to many participants’ data were not due to an enhanced ability of the WSLS model to mimic the RL strategy assumed by the PVL and EV models. These results suggest that WSLS is a common strategy in the IGT and that both heuristic-based and RL-based models should be used to inform decision-making behavior in the IGT and similar choice tasks.     

Motivated learning for the development of autonomous systems

A new machine learning approach known as motivated learning (ML) is presented in this work. Motivated learning drives a machine to develop abstract motivations and choose its own goals. ML also provides a self-organizing system that controls a machine’s behavior based on competition between dynamically-changing pain signals. This provides an interplay of externally driven and internally generated control signals. It is demonstrated that ML not only yields a more sophisticated learning mechanism and system of values than reinforcement learning (RL), but is also more efficient in learning complex relations and delivers better performance than RL in dynamically-changing environments. In addition, this paper shows the basic neural network structures used to create abstract motivations, higher level goals, and subgoals. Finally, simulation results show comparisons between ML and RL in environments of gradually increasing sophistication and levels of difficulty.     

类别学习的SUSTAIN模型

SUSTAIN模型是关于类别学习的类群或群集规则的选择模型,它强调多层次的类别子结构和模型的结构搜索功能.模型的运行以类别相似性为基础,从最简单的规则开始,有灵活的参数及其运算过程,与其他主要的类别学习模型相比有更多的优势.因此,SUSTAIN模型是迄今解释人类类别学习的最优模型.     

A Computational Model for the Item‐Based Induction of Construction Networks

According to usage‐based approaches to language acquisition, linguistic knowledge is represented in the form of constructions—form‐meaning pairings—at multiple levels of abstraction and complexity. The emergence of syntactic knowledge is assumed to be a result of the gradual abstraction of lexically specific and item‐based linguistic knowledge. In this article, we explore how the gradual emergence of a network consisting of constructions at varying degrees of complexity can be modeled computationally. Linguistic knowledge is learned by observing natural language utterances in an ambiguous context. To determine meanings of constructions starting from ambiguous contexts, we rely on the principle of cross‐situational learning. While this mechanism has been implemented in several computational models, these models typically focus on learning mappings between words and referents. In contrast, in our model, we show how cross‐situational learning can be applied consistently to learn correspondences between form and meaning beyond such simple correspondences.     

The integrative action of the cerebral hemispheres

In four experiments, right-handed subjects were asked to identify uppercase letters presented tachistoscopically in the right (R) or left (L) visual field or in both visual fields simultaneously (RL). When R, L, and RL trials were randomized together and RL trials consisted of the same letter in each visual field (Experiments 1 and 2), 11 of 21 male subjects (Experiment 1) and 9 of 18 female subjects (Experiment 2) showed a strong left-visual-field advantage, and the remainder in each experiment showed an equivalently strong right-visual-field advantage. When the RL trials consisted of a different letter in the two visual fields (Experiment 3), a consistent right-visual-field advantage was observed. It is argued that these results reflect predominantly analytical (left-hemisphere) processing of “different” pairs and relatively holistic processing of “same” pairs, which induces a shift toward a right-hemisphere advantage in some subjects. The main purpose of three of the four experiments was to test five probability models of hemispheric integration: (1) statistical summation, which assumes that the hemispheres operate independently; (2) redundancy, which assumes that RL decisions reflect only processing by the more specialized hemisphere; (3) complete dependency, according to which RL decisions are the mean of Rand L decisions; (4) integration, which assumes that R and L decisions can be represented as vectors in a joint RL space; (5) correlation, according to which the Rand L decisions are assumed to be statistically dependent. Whether R, L, and RL trials were randomized together or RL trials were presented in separate blocks, models 1 and 4 could be clearly rejected. In general, the best predictor of RL performance was model 5. It is argued that the hemispheres function as an integrated system in letter identification.     

Does learning behavior augment cognitive ability as an indicator of academic achievement?

Measures of cognitive ability have a rich history of accounting for meaningful levels of achievement variance. In contrast to other student characteristics, however, they are somewhat limited in terms of their intervention relevance and treatment validity. Alternatively, children's observable learning behaviors are believed to enhance both treatment validity and the predictions afforded by cognitive ability. This study examined student learning behaviors in the context of cognitive ability and academic achievement. Three structural models were evaluated on a sample of 1304 students ranging in age from 6 to 17 years. Results supported the unique relationship between learning behavior and academic achievement, beyond cognitive ability. Multi-group structural equation modeling (SEM) analysis revealed that these findings were invariant across groups differing by gender and ethnicity. These findings are consistent with previous observations that children's behavioral features in learning situations will supplement the outcome-oriented standardized tests of cognitive ability.     

A computational model of perception and action for cognitive robotics

Robots are increasingly expected to perform tasks in complex environments. To this end, engineers provide them with processing architectures that are based on models of human information processing. In contrast to traditional models, where information processing is typically set up in stages (i.e., from perception to cognition to action), it is increasingly acknowledged by psychologists and robot engineers that perception and action are parts of an interactive and integrated process. In this paper, we present HiTEC, a novel computational (cognitive) model that allows for direct interaction between perception and action as well as for cognitive control, demonstrated by task-related attentional influences. Simulation results show that key behavioral studies can be readily replicated. Three processing aspects of HiTEC are stressed for their importance for cognitive robotics: (1) ideomotor learning of action control, (2) the influence of task context and attention on perception, action planning, and learning, and (3) the interaction between perception and action planning. Implications for the design of cognitive robotics are discussed.     

The past,present, and future of computational models of cognitive development

Does modeling matter? We address this question by providing a broad survey of the computational models of cognitive development that have been proposed and studied over the last three decades. We begin by noting the advantages and limitations of computational models. We then describe four key dimensions across which models of development can be organized and classified. With this taxonomy in hand, we focus on how the modeling enterprise has evolved over time. In particular, we separate the timeline into three overlapping historical waves and highlight how each wave of models has not only been shaped by developmental theory and behavioral research, but in return also provided valuable insights and innovations to the study of cognitive development.     

Emotional facial expressions affect visual rule learning in 7- to 8-month-old infants

Rule learning (RL) is an implicit learning mechanism that allows infants to detect and generalize rule-like repetition-based patterns (such as ABB and ABA) from a sequence of elements. Increasing evidence shows that RL operates both in the auditory and the visual domain and is modulated by the perceptual expertise with the to-be-learned stimuli. Yet, whether infants’ ability to detect a high-order rule from a sequence of stimuli is affected by affective information remains a largely unexplored issue. Using a visual habituation paradigm, we investigated whether the presence of emotional expressions with a positive and a negative value (i.e., happiness and anger) modulates 7- to 8-month-old infants’ ability to learn a rule-like pattern from a sequence of faces of different identities. Results demonstrate that emotional facial expressions (either positive and negative) modulate infants’ visual RL mechanism, even though positive and negative facial expressions affect infants’ RL in a different manner: while anger disrupts infants’ ability to learn the rule-like pattern from a face sequence, in the presence of a happy face infants show a familiarity preference, thus maintaining their learning ability. These findings show that emotional expressions exert an influence on infants’ RL abilities, contributing to the investigation on how emotion and cognition interact in face processing during infancy.     

A learning theory account of depression

Learning theory provides a foundation for understanding and deriving treatment principles for impacting a spectrum of functional processes relevant to the construct of depression. While behavioral interventions have been commonplace in the cognitive behavioral tradition, most often conceptualized within a cognitive theoretical framework, recent years have seen renewed interest in more purely behavioral models. These modern learning theory accounts of depression focus on the interchange between behavior and the environment, mainly in terms of lack of reinforcement, extinction of instrumental behavior, and excesses of aversive control, and include a conceptualization of relevant cognitive and emotional variables. These positions, drawn from extensive basic and applied research, cohere with biological theories on reduced reward learning and reward responsiveness and views of depression as a heterogeneous, complex set of disorders. Treatment techniques based on learning theory, often labeled Behavioral Activation (BA) focus on activating the individual in directions that increase contact with potential reinforcers, as defined ideographically with the client. BA is considered an empirically well‐established treatment that generalizes well across diverse contexts and populations. The learning theory account is discussed in terms of being a parsimonious model and ground for treatments highly suitable for large scale dissemination.     

Proposal and evaluation of deep exploitation-oriented learning under multiple reward environment

Recently, deep reinforcement learning (DRL) has attracted considerable attention. The well-known deep Q-network (DQN) architecture successfully combines deep learning and Q-learning which is a representative reinforcement learning (RL) method. In general, RL and DRL require many trial-and-error searches. To overcome this limitation, alternative approaches called exploitation-oriented learning (XoL) and deep exploitation-oriented learning (DXoL) have been proposed.Although the effectiveness of DXoL for DQNs has been verified, its effectiveness in an environment where multiple types of rewards are present remains unclear. In this study, we apply the DXoL method to two applications with multiple reward types: the driver drowsiness determination system and the decision-making system. Our experimental results show that DXoL is more suitable for learning priorities among multiple rewards than DQNs in these applications.     

Conditional discrimination learning: A critique and amplification

Carter and Werner recently reviewed the literature on conditional discrimination learning by pigeons, which consists of studies of matching-to-sample and oddity-from-sample. They also discussed three models of such learning: the “multiple-rule” model (learning of stimulus-specific relations), the “configuration” model, and the “single-rule” model (concept learning). Although their treatment of the multiple-rule model, which seems most applicable to the pigeon data, is generally excellent, their discussion of the other two models is incomplete and sometimes inaccurate. Potential problems of terminology are discussed in the present paper, as are additional lines of research that deserve consideration by those interested in further work in this area. The issue of response versus stimulus selection (configuration versus compound-cue learning) is discussed in connection with the configuration model. Particular attention is given to Carter and Werner's criticism of the application, in studies with other species, of the learning set procedure in testing for single-rule learning. Some of the important related issues are: the bias for improvement on new problems in a series, the adequacy of a multiple-rule model to explain learning set formation, and evidence in favor of the single-rule model, at least in primates. Consideration of these additional contributions to the study of conditional discrimination learning emphasizes the usefulness of this task in the comparative study of cognitive processes.