Emotion-based learning and central executive resources: an investigation of intuition and the Iowa Gambling Task

The role of emotion in complex decision-making can be assessed on the Iowa Gambling Task (IGT), a widely used neuropsychological measure that may tap a different aspect of executive function than that assessed by conventional measures. Most notably, the 'feeling' about which decks are good or bad, often described in relation to IGT performance, seems reminiscent of decision-making based on intuition-linked to a long history of research in decision-making contrasting the 'intuition' versus 'reasoning' styles of problem solving. To test the claim that the performance on the IGT relies more on emotion-based learning than conventional executive resources for normal performance, a group of participants completed the IGT simultaneously with one of two secondary-tasks, one of which (random number generation) is known to load executive resources. A third group performed the IGT with no secondary-task. If performance on the IGT requires the properties associated with intuitive operations, then participants should either show no disruption when completing a secondary-task, or at least show no selective disruption on a secondary-task that loads for executive function. The rate of learning in the three groups was not significantly different. This suggests that the sorts of cognitive resources loaded by traditional executive tasks such as random number generation do not overlap, in the cognitive architecture, with the emotion-based learning skills that are required for Iowa Gambling Task performance. The findings of the present study are also consistent with a previous claim of the Iowa group that emotion-based learning and working memory resources are doubly dissociable.     

Effects of time-pressure on decision-making under uncertainty: changes in affective state and information processing strategy

An experiment is reported that investigated the extent to which affective state, information processing strategy and task structure determine the effects of time-pressure on decision-making. Research participants were presented with risk scenarios involving a choice between safe and risky actions. The scenarios were systematically varied in terms of outcome valence (positive or negative) and effort associated with taking the safe action (high or low). Half the participants were given unlimited time to make their decision, the other half were required to choose within a deadline. The findings showed that time-pressured participants were more anxious and energetic and used a number of different strategies to cope with the deadline. These effects, as well as changes in risk-taking, were shown to vary systematically with task structure, particularly the effort manipulation. The findings are discussed in terms of how they contribute to theories of time-pressure and the methodological implications they have for future research in this area.     

The influence of depression symptoms on exploratory decision-making

People with symptoms of depression show impairments in decision-making. One explanation is that they have difficulty maintaining rich representations of the task environment. We test this hypothesis in the context of exploratory choice. We analyze depressive and non-depressive participants’ exploration strategies by comparing their choices to two computational models: (1) an “Ideal Actor” model that reflectively updates beliefs and plans ahead, employing a rich representation of the environment and (2) a “Naïve Reinforcement Learning” (RL) model that updates beliefs reflexively utilizing a minimal task representation. Relative to non-depressive participants, we find that depressive participants’ choices are better described by the simple RL model. Further, depressive participants were more exploratory than non-depressives in their decision-making. Depressive symptoms appear to influence basic mechanisms supporting choice behavior by reducing use of rich task representations and hindering performance during exploratory decision-making.     

Proposal and evaluation of deep exploitation-oriented learning under multiple reward environment

Recently, deep reinforcement learning (DRL) has attracted considerable attention. The well-known deep Q-network (DQN) architecture successfully combines deep learning and Q-learning which is a representative reinforcement learning (RL) method. In general, RL and DRL require many trial-and-error searches. To overcome this limitation, alternative approaches called exploitation-oriented learning (XoL) and deep exploitation-oriented learning (DXoL) have been proposed.Although the effectiveness of DXoL for DQNs has been verified, its effectiveness in an environment where multiple types of rewards are present remains unclear. In this study, we apply the DXoL method to two applications with multiple reward types: the driver drowsiness determination system and the decision-making system. Our experimental results show that DXoL is more suitable for learning priorities among multiple rewards than DQNs in these applications.     

Deterrence, Opportunity, and Choice

Traditional studies of deterrence have focused on punishment with less regard for the rewards of both crime and noncrime. Influenced by work in economics and behavioral psychology, more recent studies have departed from tradition by incorporating rewards for crime. To this extent, they should be regarded as advances over the more traditional approaches. Notwithstanding these advances, variations in both the probability and magnitude of reward for noncrime have not been systematically included in these more recent theories of choice. In an attempt to determine whether opportunities for noncrime are either central or trivial to the criminal decision-making process, the present study fitted two alternative models to experimental data involving risk-taking: (a) the economic utility model employed by Piliavin, Gartner, Thornton, and Matsueda (1986) in their study of criminal choice; and (b) the satisfaction balance model developed by Gray and Tallman (1984). Results showed that while both models explained significant amounts of variation in the dependent variable, the Gray-Tallman model provided a substantially better fit of the data. Despite limitations inherent in experimental studies as, for example, limitations surrounding the issue of external validity, the findings strongly suggest that opportunities for noncrime are as important as rewards and costs for crime in the process by which criminal decisions are made.     

Ratio and difference comparisons of expected reward in decision-making tasks

Several models of choice compute the probability of selecting a given option by comparing the expected value (EV) of each option. However, a subtle but important difference between two common rules used to compute the action probability is often ignored. Specifically, one common rule type, the exponential rule, compares EVs via a difference operation, whereas another rule type, the power rule, uses a ratio operation. We tested the empirical validity of each rule type by having human participants perform a choice task in which either the difference or the ratio between the reward values was altered relative to a control condition. Results indicated that participants can compare expected rewards by either ratio or difference operations but that altering the ratio between EVs produces the most dramatic changes in behavior. We discuss implications for several related research fields.     

With age comes wisdom: decision making in younger and older adults

In two experiments, younger and older adults performed decision-making tasks in which reward values available were either independent of or dependent on the previous sequence of choices made. The choice-independent task involved learning and exploiting the options that gave the highest rewards on each trial. In this task, the stability of the expected reward for each option was not influenced by the previous choices participants made. The choice-dependent task involved learning how each choice influenced future rewards for two options and making the best decisions based on that knowledge. Younger adults performed better when rewards were independent of choice, whereas older adults performed better when rewards were dependent on choice. These findings suggest a fundamental difference in the way in which younger adults and older adults approach decision-making situations. We discuss the results in the context of prominent decision-making theories and offer possible explanations based on neurobiological and behavioral changes associated with aging.     

A formal cognitive model of the go/no-go discrimination task: evaluation and implications

This article proposes and tests a formal cognitive model for the go/no-go discrimination task. In this task, the performer chooses whether to respond to stimuli and receives rewards for responding to certain stimuli and punishments for responding to others. Three cognitive models were evaluated on the basis of data from a longitudinal study involving 400 adolescents. The results show that a cue-dependent model presupposing that participants can differentiate between cues was the most accurate and parsimonious. This model has 3 parameters denoting the relative impact of rewards and punishments on evaluations, the rate that contingent payoffs are learned, and the consistency between learning and responding. Commission errors were associated with increased attention to rewards; omission errors were associated with increased attention to punishments. Both error types were associated with low choice consistency. The parameters were also shown to have external validity: Attention to rewards was associated with externalizing behavior problems on the Achenbach scale, and choice consistency was associated with low Welsh anxiety. The present model can thus potentially improve the sensitivity of the task to differences between clinical populations.     

A mixed-effects expectancy-valence model for the Iowa gambling task

The Iowa gambling task (IGT; Bechara, Damasio, Damasio, & Anderson, 1994) was developed to simulate real-life decision making under uncertainty. The task has been widely used to examine possible neurocognitive deficits in normal and clinical populations. Busemeyer and Stout (2002) proposed the expectancy-valence (EV) model to explicitly account for individual participants’ repeated choices in the IGT. Parameters of the EV model presumably measure different psychological processes that underlie performance on the task, and their values may be used to differentiate individuals across different populations. In the present article, the EV model is extended to include both fixed effects and subject-specific random effects. The mixed-effects EV model fits the nested structure of observations in the IGT naturally and provides a unified procedure for parameter estimation and comparisons among groups of populations. We illustrate the utility of the mixed-effects approach with an analysis of gender differences using a real data set. A simulation study was conducted to verify the advantages of this approach.     

The Temporal Dynamics of Cooperation

Parties in real-world conflicts often attempt to punish each other's behavior. If this strategy fails to produce mutual cooperation, they may increase punishment magnitude. The present experiment investigated whether delay-reduction - potentially less harmful than magnitude increase - would generate mutual cooperation as interactions are repeated. Participants played a prisoner's dilemma game against a computer that played a tit-for-tat strategy, cooperating after a participant cooperated, defecting after a participant defected. For half of the participants, the delay between their choice and the computer's next choice was long relative to the delay between the computer's choice and their next choice. For the other half, long and short delays were reversed. The tit-for-tat contingency reinforces the other player's cooperation (by cooperating) and punishes the other player's defection (by defecting). Both rewards and punishers are discounted by delay. Consistent with delay discounting, participants cooperated more when the delay between their choice and the computer's cooperation (reward) or defection (punishment) was relatively short. These results suggest that, in real-world tit-for-tat conflicts, decreasing delay of reciprocation or retaliation may foster mutual cooperation as effectively as (or more effectively than) the more usual tactic of increasing magnitude of reciprocation or retaliation.     

Similar Processes Despite Divergent Behavior in Two Commonly Used Measures of Risky Decision Making

Performance on complex decision-making tasks may depend on a multitude of processes. Two such tasks, the Iowa Gambling Task (IGT) and Balloon Analog Risk Task (BART), are of particular interest because they are associated with real world risky behavior, including illegal drug use. We used cognitive models to disentangle underlying processes in both tasks. Whereas behavioral measures from the IGT and BART were uncorrelated, cognitive models revealed two reliable cross-task associations. Results suggest that the tasks similarly measure loss aversion and decision-consistency processes, but not necessarily the same learning process. Additionally, substance-using individuals (and especially stimulant users) performed worse on the IGT than healthy controls did, and this pattern could be explained by reduced decision consistency.     

Why Higher Working Memory Capacity May Help You Learn: Sampling,Search, and Degrees of Approximation

Algorithms for approximate Bayesian inference, such as those based on sampling (i.e., Monte Carlo methods), provide a natural source of models of how people may deal with uncertainty with limited cognitive resources. Here, we consider the idea that individual differences in working memory capacity (WMC) may be usefully modeled in terms of the number of samples, or “particles,” available to perform inference. To test this idea, we focus on two recent experiments that report positive associations between WMC and two distinct aspects of categorization performance: the ability to learn novel categories, and the ability to switch between different categorization strategies (“knowledge restructuring”). In favor of the idea of modeling WMC as a number of particles, we show that a single model can reproduce both experimental results by varying the number of particles—increasing the number of particles leads to both faster category learning and improved strategy‐switching. Furthermore, when we fit the model to individual participants, we found a positive association between WMC and best‐fit number of particles for strategy switching. However, no association between WMC and best‐fit number of particles was found for category learning. These results are discussed in the context of the general challenge of disentangling the contributions of different potential sources of behavioral variability.     

厌恶和恐惧情绪对跨期选择的不同影响：基于ERP的研究

本研究运用事件相关电位技术(ERPs)考察厌恶和恐惧情绪对跨期选择的影响。其中,厌恶和恐惧情绪采用情绪面孔图片进行启动,分析跨期选择任务中评估阶段所诱发的ERP成分。行为结果发现,与中性面孔相比,厌恶面孔启动使个体倾向于选择立即奖赏。ERP结果发现,在选项评估阶段,厌恶面孔比中性和恐惧面孔启动诱发更大的P2、P3和LPP波幅。这说明,厌恶情绪促使个体投入更多的注意和动机资源对跨期选项进行评估,进而使个体倾向即时满足。     

厌恶和恐惧情绪对跨期选择的不同影响：基于ERP的研究

本研究运用事件相关电位技术(ERPs)考察厌恶和恐惧情绪对跨期选择的影响。其中,厌恶和恐惧情绪采用情绪面孔图片进行启动,分析跨期选择任务中评估阶段所诱发的ERP成分。行为结果发现,与中性面孔相比,厌恶面孔启动使个体倾向于选择立即奖赏。ERP结果发现,在选项评估阶段,厌恶面孔比中性和恐惧面孔启动诱发更大的P2、P3和LPP波幅。这说明,厌恶情绪促使个体投入更多的注意和动机资源对跨期选项进行评估,进而使个体倾向即时满足。     

海洛因戒除者的行为冲动性： 基于DDT和IGT任务反应模式的探讨

以61名男性海洛因戒除者和32名正常成年男性为被试,探讨了海洛因戒除者在延迟折扣任务和Iowa赌博任务两种决策条件下的冲动性反应模式及其关系。结果发现,海洛因戒除者的决策行为受其对即时强化的超敏感性或“即时收益优先”模式控制,其高冲动性特质既表现为对强化物长远价值的非敏感性,又表现为对强化物潜在风险的耐受性;该类人群上述两方面的特征虽有一定关联,但二者分别反应了其行为冲动性的两个相对独立的维度,且具有跨戒除时相的相对稳定性     

Emotionally controlled decision-making and a gene variant related to serotonin synthesis in women with borderline personality disorder

The Iowa Gambling Task (IGT) was used to examine (i) social decision-making in women with borderline personality disorder (BPD), and (ii) the relationship between impaired decision-making and the tryptophan hydroxylase-1 (TPH-1) gene, involved in serotonin synthesis. Forty-two women with BPD and a history of suicide attempts were genotyped, and the frequency of a TPH-1 haplotype previously uniquely associated with BPD was calculated. The BPD group scored significantly lower than a control group in the IGT. Furthermore, the TPH-1 haplotype displayed a significantly higher frequency in BPD participants with impaired decision making, compared to BPD participants with normal scores. These findings suggest that impaired decision-making as determined by the IGT is a feature of BPD and may be (i) associated with serotonin dysfunction, and (ii) possibly relevant for suicidal behavior.     

Better late than never? The effect of feedback delay on ERP indices of reward processing

The feedback negativity (FN), an early neural response that differentiates rewards from losses, appears to be generated in part by reward circuits in the brain. A prominent model of the FN suggests that it reflects learning processes by which environmental feedback shapes behavior. Although there is evidence that human behavior is more strongly influenced by rewards that quickly follow actions, in nonlaboratory settings, optimal behaviors are not always followed by immediate rewards. However, it is not clear how the introduction of a delay between response selection and feedback impacts the FN. Thus, the present study used a simple forced choice gambling task to elicit the FN, in which feedback about rewards and losses was presented after either 1 or 6?s. Results suggest that, at short delays (1?s), participants clearly differentiated losses from rewards, as evidenced in the magnitude of the FN. At long delays (6?s), on the other hand, the difference between losses and rewards was negligible. Results are discussed in terms of eligibility traces and the reinforcement learning model of the FN.     

Alexithymia, Cumulative Feedback, and Differential Response Patterns on the Iowa Gambling Task

ABSTRACT Although the role of emotional processing is central to contemporary models of risky decision making, to date the role of trait emotional understanding has not been explored experimentally in this context. The current experiment ( N =326) explores the role of alexithymia with respect to performance on the standard Iowa Gambling Task (IGT) and a version where cumulative financial feedback is obscured. Standard learning on the IGT was observed for those low in alexithymia. Those high in alexithymia learned to avoid disadvantageous decks over the first half of the task. However, over the later trials they showed a change in performance, shifting from advantageous to disadvantageous and back to advantageous decks again (termed an "explore-learn-change-return" strategy). It is argued that this is due to an inability to fully consolidate earlier learning and reduced sensitivity to losses. The absence of cumulative feedback independently resulted in reduced performance.     

A simple computational algorithm of model-based choice preference

A broadly used computational framework posits that two learning systems operate in parallel during the learning of choice preferences—namely, the model-free and model-based reinforcement-learning systems. In this study, we examined another possibility, through which model-free learning is the basic system and model-based information is its modulator. Accordingly, we proposed several modified versions of a temporal-difference learning model to explain the choice-learning process. Using the two-stage decision task developed by Daw, Gershman, Seymour, Dayan, and Dolan (2011), we compared their original computational model, which assumes a parallel learning process, and our proposed models, which assume a sequential learning process. Choice data from 23 participants showed a better fit with the proposed models. More specifically, the proposed eligibility adjustment model, which assumes that the environmental model can weight the degree of the eligibility trace, can explain choices better under both model-free and model-based controls and has a simpler computational algorithm than the original model. In addition, the forgetting learning model and its variation, which assume changes in the values of unchosen actions, substantially improved the fits to the data. Overall, we show that a hybrid computational model best fits the data. The parameters used in this model succeed in capturing individual tendencies with respect to both model use in learning and exploration behavior. This computational model provides novel insights into learning with interacting model-free and model-based components.