Regulating recognition decisions through incremental reinforcement learning

Does incremental reinforcement learning influence recognition memory judgments? We examined this question by subtly altering the relative validity or availability of feedback in order to differentially reinforce old or new recognition judgments. Experiment 1 probabilistically and incorrectly indicated that either misses or false alarms were correct in the context of feedback that was otherwise accurate. Experiment 2 selectively withheld feedback for either misses or false alarms in the context of feedback that was otherwise present. Both manipulations caused prominent shifts of recognition memory decision criteria that remained for considerable periods even after feedback had been altogether removed. Overall, these data demonstrate that incremental reinforcement-learning mechanisms influence the degree of caution subjects exercise when evaluating explicit memories.     

A crossed random effects diffusion model for speeded semantic categorization decisions

Choice reaction times (RTs) are often used as a proxy measure of typicality in semantic categorization studies. However, other item properties have been linked to choice RTs as well. We apply a tailored process model of choice RT to a speeded semantic categorization task in order to deconfound different sources of variability in RT. Our model is based on a diffusion model of choice RT, extended to include crossed random effects (of items and participants). This model retains the interesting process interpretation of the diffusion model’s parameters, but it can be applied to choice RTs even in the case where there are few or no repeated measurements of each participant-item combination. Different aspects of the response process are then linked to different types of item properties. A typicality measure turns out to predict the rate of information uptake, while a lexicographic measure predicts the stimulus encoding time. Accessibility measures cannot reliably predict any component of the decision process.     

A mathematical model of learning under schedules of interresponse time reinforcement

A model is proposed for free responding under schedules of interresponse time reinforcement. Each response generates a pattern of stimuli that changes in an orderly way over time. At any time the changing pattern may be sampled, and the state of conditioning of the sampled pattern determines whether or not a response actually occurs. The prediction of this model for the asymptotic distribution of interresponse times is derived. The predictions are tested against data from ratio, interval, and DRL schedules of reinforcement. The fit of the model is sufficiently good to make it worth-while continuing investigation of the model.     

Anatomy of a decision: striato-orbitofrontal interactions in reinforcement learning, decision making, and reversal

The authors explore the division of labor between the basal ganglia-dopamine (BG-DA) system and the orbitofrontal cortex (OFC) in decision making. They show that a primitive neural network model of the BG-DA system slowly learns to make decisions on the basis of the relative probability of rewards but is not as sensitive to (a) recency or (b) the value of specific rewards. An augmented model that explores BG-OFC interactions is more successful at estimating the true expected value of decisions and is faster at switching behavior when reinforcement contingencies change. In the augmented model, OFC areas exert top-down control on the BG and premotor areas by representing reinforcement magnitudes in working memory. The model successfully captures patterns of behavior resulting from OFC damage in decision making, reversal learning, and devaluation paradigms and makes additional predictions for the underlying source of these deficits.     

Bayesian analysis of the piecewise diffusion decision model

Most past research on sequential sampling models of decision-making have assumed a time homogeneous process (i.e., parameters such as drift rates and boundaries are constant and do not change during the deliberation process). This has largely been due to the theoretical difficulty in testing and fitting more complex models. In recent years, the development of simulation-based modeling approaches matched with Bayesian fitting methodologies has opened the possibility of developing more complex models such as those with time-varying properties. In the present work, we discuss a piecewise variant of the well-studied diffusion decision model (termed pDDM) that allows evidence accumulation rates to change during the deliberation process. Given the complex, time-varying nature of this model, standard Bayesian parameter estimation methodologies cannot be used to fit the model. To overcome this, we apply a recently developed simulation-based, hierarchal Bayesian methodology called the probability density approximation (PDA) method. We provide an analysis of this methodology and present results of parameter recovery experiments to demonstrate the strengths and limitations of this approach. With those established, we fit pDDM to data from a perceptual experiment where information changes during the course of trials. This extensible modeling platform opens the possibility of applying sequential sampling models to a range of complex non-stationary decision tasks.     

Understanding motivation for implementing cooperative learning methods: a value-based approach

Social Psychology of Education - The implementation of cooperative learning methods remains disparate in primary schools despite their widely recognised benefits. To explain this paradox, we first...     

A reinforcement learning approach to personalized learning recommendation systems

Personalized learning refers to instruction in which the pace of learning and the instructional approach are optimized for the needs of each learner. With the latest advances in information technology and data science, personalized learning is becoming possible for anyone with a personal computer, supported by a data-driven recommendation system that automatically schedules the learning sequence. The engine of such a recommendation system is a recommendation strategy that, based on data from other learners and the performance of the current learner, recommends suitable learning materials to optimize certain learning outcomes. A powerful engine achieves a balance between making the best possible recommendations based on the current knowledge and exploring new learning trajectories that may potentially pay off. Building such an engine is a challenging task. We formulate this problem within the Markov decision framework and propose a reinforcement learning approach to solving the problem.     

Curiosity-driven recommendation strategy for adaptive learning via deep reinforcement learning

The design of recommendation strategies in the adaptive learning systems focuses on utilizing currently available information to provide learners with individual-specific learning instructions. As a critical motivate for human behaviours, curiosity is essentially the drive to explore knowledge and seek information. In a psychologically inspired view, we propose a curiosity-driven recommendation policy within the reinforcement learning framework, allowing for an efficient and enjoyable personalized learning path. Specifically, a curiosity reward from a well-designed predictive model is generated to model one's familiarity with the knowledge space. Given such curiosity rewards, we apply the actor–critic method to approximate the policy directly through neural networks. Numerical analyses with a large continuous knowledge state space and concrete learning scenarios are provided to further demonstrate the efficiency of the proposed method.     

A non-linear multi-criteria model for strategic fms selection decisions

The strategic decision of selecting an optimal flexible manufacturing system (FMS) configuration is a complicated question which involves evaluating trade-offs between a number of different, potentially conflicting criteria such as annual production volume, flexibility, production and investment costs and average throughput of the system. Recently, several structured multicriteria approaches have been proposed to aid management in the FMS selection process. While acknowledging the non-linear nature of a number of the relationships in the model, notably between batch size and the number of batches produced of each part, these studies used linear simplifications to illustrate the decision dynamics of the problem. These linear models were shown to offer useful analytical tools in the FMS pre-design process. Owing to the non-linearities of the true relationships, however, the trade-offs between the criteria could not fully be explored within the linear framework. This paper builds on the two-phase decision support framework proposed by Stam and Kuula (1991) and uses a modified non-linear multi-criteria formulation to solve the problem. The software used in the illustration can easily be implemented, is user-interactive and menu-driven. The methodology is applied to real data from a Finnish metal product company and the results are compared with those obtained in previous studies.     

A cognitive probability model for learning

A quantitative model for the behavior of albino rats in choice-making situations is presented. The model, which is based upon a cognitive conceptualization of the learning process, is shown to yield predictions which are equivalent to those produced by the linear operator stochastic models at the asymptotic limit but which differ from these during early trials in the learning situation.     

A unified model for perceptual learning

Perceptual learning in adult humans and animals refers to improvements in sensory abilities after training. These improvements had been thought to occur only when attention is focused on the stimuli to be learned (task-relevant learning) but recent studies demonstrate performance improvements outside the focus of attention (task-irrelevant learning). Here, we propose a unified model that explains both task-relevant and task-irrelevant learning. The model suggests that long-term sensitivity enhancements to task-relevant or irrelevant stimuli occur as a result of timely interactions between diffused signals triggered by task performance and signals produced by stimulus presentation. The proposed mechanism uses multiple attentional and reinforcement systems that rely on different underlying neuromodulators. Our model provides insights into how neural modulators, attentional and reinforcement learning systems are related.     

A model for serial verbal learning

A model for analyzing the learning process with a special emphasis on serial-position effect is proposed. This model consists of two analyses, one being an analysis of the learning process of each item in a list by a stochastic method, and the other being an analysis of serial-position effect in terms of pro- and retroactive inhibitions, and of forgetting. The model is experimentally verified, and moreover, it is found that the model permits prediction of the results of many experiments with lists of various lengths and varying difficulty.The author wishes to acknowledge help received during discussion with Prof. T. Indow.     

Impaired social learning in patients with major depressive disorder revealed by a reinforcement learning model

Background/objectivePatients with major depressive disorder (MDD) have altered learning rates for rewards and losses in non-social learning paradigms. However, it is not well understood whether the ability to learn from social interactions is altered in MDD patients. Using reinforcement learning during the repeated Trust Game (rTG), we investigated how MDD patients learn to trust newly-met partners in MDD patients.MethodSixty-eight MDD patients and fifty-four controls each played as ‘investor’ and interacted with ten different partners. We manipulated both the level of trustworthiness by varying the chance of reciprocity (10, 30, 50, 70 and 90%) and reputation disclosure, where partners’ reputation was either pre-disclosed or hidden.ResultsOur reinforcement learning model revealed that MDD patients had significantly higher learning rates for losses than the controls in both the reputation disclosure and non-disclosure condition. The difference was larger when reputation was not disclosed than disclosed. We observed no difference in learning rates for gains in either condition.ConclusionsOur findings highlight that abnormal learning for losses underlies the social learning process in MDD patients. This abnormality is higher when situational unpredictability is high versus low. Our findings provide novel insights into social rehabilitation of MDD.     

A vigilance model for latent learning

The author proposes a heuristic model for latent learning. It is concluded that to regard academic learning as qualitatively different from other forms of learning is to deny evolutionary continuity. Academic learning is not a unitary process governed by a single set of parameters. In addition, it is observed that the problem of student motivation may very well turn out to be purely academic. The instructional technique for a captive audience of a class may be so structured as to make the direction of attention irresistible, the performance of a response, when needed, compelling, and the acquisition of knowledge inevitable. Vigilance is an instance of innate foundation. Its most striking characteristics are its universality in the animal world, its ready evocation by a wide range of stimuli, and its apparent behavioral and physiological manifestations. The last two are the natural resources for objective investigation, and the first may well be the basis of broad and valid generalizations.     

A markov model for discrimination learning

A theory for discrimination learning which incorporates the concept of an observing response is presented. The theory is developed in detail for experimental procedures in which two stimuli are employed and two responses are available to the subject. Applications of the model to cases involving probabilistic and nonprobabilistic schedules of reinforcement are considered; some predictions are derived and compared with experimental results.This research was supported by a grant from the National Science Foundation.