Heterogeneity of strategy use in the Iowa gambling task: A comparison of win-stay/lose-shift and reinforcement learning models

The Iowa gambling task (IGT) has been used in numerous studies, often to examine decision-making performance in different clinical populations. Reinforcement learning (RL) models such as the expectancy valence (EV) model have often been used to characterize choice behavior in this work, and accordingly, parameter differences from these models have been used to examine differences in decision-making processes between different populations. These RL models assume a strategy whereby participants incrementally update the expected rewards for each option and probabilistically select options with higher expected rewards. Here we show that a formal model that assumes a win-stay/lose-shift (WSLS) strategy—which is sensitive only to the outcome of the previous choice—provides the best fit to IGT data from about half of our sample of healthy young adults, and that a prospect valence learning (PVL) model that utilizes a decay reinforcement learning rule provides the best fit to the other half of the data. Further analyses suggested that the better fits of the WSLS model to many participants’ data were not due to an enhanced ability of the WSLS model to mimic the RL strategy assumed by the PVL and EV models. These results suggest that WSLS is a common strategy in the IGT and that both heuristic-based and RL-based models should be used to inform decision-making behavior in the IGT and similar choice tasks.     

Secondary psychopathy,but not primary psychopathy,is associated with risky decision-making in noninstitutionalized young adults

Although risky decision-making has been posited to contribute to the maladaptive behavior of individuals with psychopathic tendencies, the performance of psychopathic groups on a common task of risky decision-making, the Iowa Gambling Task (IGT; Bechara, Damasio, Damasio, & Anderson, 1994), has been equivocal. Different aspects of psychopathy (personality traits, antisocial deviance) and/or moderating variables may help to explain these inconsistent findings. In a sample of college students (N = 129, age 18–27), we examined the relationship between primary and secondary psychopathic features and IGT performance. A measure of impulsivity was included to investigate its potential as a moderator. In a joint model including main effects and interactions between primary psychopathy, secondary psychopathy and impulsivity, only secondary psychopathy was significantly related to risky IGT performance, and this effect was not moderated by the other variables. This finding supports the growing literature suggesting that secondary psychopathy is a better predictor of decision-making problems than the primary psychopathic personality traits of lack of empathy and remorselessness.     

Some decks are better than others: the effect of reinforcer type and task instructions on learning in the Iowa Gambling Task

The Iowa Gambling Task (Bechara, Damasio, Damasio, & Anderson, 1994) has become widely used as a laboratory test of "real-life" decision-making. However, aspects of its administration that have been varied by researchers may differentially affect performance and the conclusions researchers can draw. Some researchers have used facsimile money reinforcers while others have used real money reinforcers. More importantly, the instructions participants receive have also been varied. While no differences have been reported in performance dependent on reinforcer type, no previous comparison of participants' instructions has been conducted. This is despite one set of instructions giving participants a clear hint about the nature of the task. Additionally, in previous research one set of instructions have not been used exclusively with one reinforcer type making any differential or cumulative effects of these factors difficult to interpret. The present study compared the effects of instruction and reinforcer type on IGT performance. When participants received instructions without a hint performance was affected by reinforcer type. This was not the case when the instructions included a hint. In a second IGT session performance was improved in participants who had received the hint instructions compared with those who had not.     

Somatic markers,working memory,and decision making

The somatic marker hypothesis formulated by Damasio (e.g., 1994; Damasio, Tranel, & Damasio, 1991) argues that affective reactions ordinarily guide and simplify decision making. Although originally intended to explain decision-making deficits in people with specific frontal lobe damage, the hypothesis also applies to decision-making problems in populations without brain injury. Subsequently, the gambling task was developed by Bechara (Bechara, Damasio, Damasio, & Anderson, 1994) as a diagnostic test of decision-making deficit in neurological populations. More recently, the gambling task has been used to explore implications of the somatic marker hypothesis, as well as to study suboptimal decision making in a variety of domains. We examined relations among gambling task decision making, working memory (WM) load, and somatic markers in a modified version of the gambling task. Increased WM load produced by secondary tasks led to poorer gambling performance. Declines in gambling performance were associated with the absence of the affective reactions that anticipate choice outcomes and guide future decision making. Our experiments provide evidence that WM processes contribute to the development of somatic markers. If WM functioning is taxed, somatic markers may not develop, and decision making may thereby suffer.     

Evaluating the Iowa Gambling Task as a direct assessment of impulsivity with low-income children

This study examined performance on the Iowa Gambling Task (IGT; Bechara, Damasio, Damasio, & Anderson, 1994) as a measure of low-income school-aged children’s affective decision-making and considered its utility as a direct indicator of impulsivity. One hundred and ninety-three 8–11 year olds performed a computerized version of the Iowa Gambling Task, a validated measure of decision-making. Multi-level modeling was used to examine children’s performance over the course of the task, with age, gender, and teachers’ ratings of child impulsivity (BIS-11; Patton, Stanford, & Barratt, 1995) used to predict children’s Iowa Gambling performance. Higher impulsivity scores predicted a decrease in slope of Iowa Gambling performance, indicating students rated higher on impulsivity chose more disadvantageously across the task blocks. Results support evidence of the validity of the Iowa Gambling Task as a measure of impulsivity in low-income minority children.     

爱荷华博弈任务(IGT)与决策的认知神经机制

爱荷华博弈任务(IGT)是一项检查情感性决策机制的常用实验范式。据此, Damasio等人提出了躯体标记假设(SMH)解释情绪影响决策的神经生理机制。近期, 大量研究在IGT究竟是模糊决策还是风险决策、与情绪和认知的关系、与工作记忆和陈述性记忆的关系以及IGT的神经网络与分子遗传机制等方面积累了丰富资料。结果显示, IGT加工的早期由模糊决策主导, 情绪性躯体信号对引导决策选项的偏好可能起关键作用, 后期则倾向是一种风险决策, 认知评价和预期对选项偏向逐渐占优势; IGT与工作记忆的加工成分有相互重叠, 也需陈述性记忆的参与; IGT的加工不仅依赖于杏仁核、腹内侧前额皮层、眶额皮层等组成的情绪脑网络的活动, 还与背外侧前额皮层、海马、腹侧纹状体、岛叶皮层、辅助运动前区、扣带回皮层等许多脑区的活动有关; COMT和5-HTT的基因多态性会调节IGT相关的决策加工。总之, IGT是一项需要多重神经系统协同活动的决策加工任务, 且模糊与风险决策可能具有不同的遗传基础。     

Adolescents' performance on the Iowa Gambling Task: implications for the development of decision making and ventromedial prefrontal cortex

Healthy adolescents (79 girls, 66 boys), ages 9-17, completed the Iowa Gambling Task (IGT; A. Bechara, A. R. Damasio, H. Damasio, & S. W. Anderson, 1994) as well as working memory (digit span) and behavioral inhibition (go/no-go) tasks. Cross-sectional age-related changes were seen on all 3 tasks. Gender differences were seen in IGT deck preference and attentional variables (i.e., go/no-go hit rate and forward digit span). After age, gender, and general intellectual abilities were controlled for, IGT performance was not predicted by working memory or behavioral inhibition scores. Findings suggest that the ventromedial prefrontal cortex or its connections are functionally maturing during adolescence in a manner that can be distinguished from maturation of other prefrontal regions. Development of these functions may continue into young adulthood.     

Testing Damasio's somatic marker hypothesis with psychopathic individuals: risk takers or risk averse?

Damasio and colleagues (A. R. Damasio, 1994; A. R. Damasio, D. Tranel, & H. Damasio, 1990) have theorized about a possible relationship between somatic markers and the behavior of psychopathic individuals (Ps), but, to date, there are no published data regarding the proposed relationship. The authors assessed 86 Caucasian and 71 African American male offenders using R. D. Hare's (1991) Psychopathy Checklist--Revised and used a modified version of Bechara and colleagues' (A. Bechara, A. R. Damasio, H. Damasio, & S. W. Anderson, 1994; A. Bechara, H. Damasio, D. Tranel, & A. R. Damasio, 1997) gambling task to test the hypothesis that Ps would, consistent with the somatic marker hypothesis, fail to become risk averse. Results indicated that level of anxiety, but not psychopathy, was predictive of response choices. Several limitations and implications of the study are noted.     

Older adults as adaptive decision makers: evidence from the Iowa Gambling Task

Older adults process emotional information differently than younger adults and may demonstrate less of a negativity bias on cognitive tasks. The Iowa Gambling Task designed by A. Bechara, H. Damasio, D. Tranel, and A. R. Damasio (1997) has been used to examine the integration of emotion and cognition in a risky-choice decision task and may give insight into differences in the decision-making strategies in younger and older adults. Eighty-eight younger adults (18-34 years) and 67 older adults (65-88 years) completed the Iowa Gambling Task. Using a theoretical decomposition of the task designed by J. R. Busemeyer and J. C. Stout (2002), the authors found that both groups were successful at solving the task but used very different strategies that reflected each group's strength. For younger adults, that strength was learning and memory. For older adults, that strength was an accurate representation of wins and losses (valence).     

Cognitive modeling analysis of decision-making processes in cocaine abusers

This article examines the theoretical basis of decision-making deficits exhibited by cocaine abusers in a laboratory decision-making task first described by Bechara, Damasio, Damasio, and Anderson (1994). A total of 12 male cocaine abusers and 14 comparison subjects performed the task, and the cocaine group performed significantly worse than the comparison group. A cognitive modeling analysis (Busemeyer & Stout, 2002) was used to estimate three parameters that measure importance of the cognitive, motivational, and response processes for determining the observed performance deficit. The results of this analysis indicated, for the first time, that motivational and choice consistency factors, but not learning/ memory were mainly responsible for the decision-making deficit of the cocaine abusers in this task.     

Stronger autonomic response accompanies better learning: A test of Damasio's somatic marker hypothesis

According to Damasio's (1994, 1999) somatic marker hypothesis (SMH), positive and negative events are “marked” by bodily feelings. These markers aid future adaptive learning by producing signals to warn against or affirm behavioural options. An implicit assumption of the SMH is that the stronger the signal is, the stronger the resultant behaviour will be. In this study, we tested 30 healthy women on a gambling task (Bechara, Damasio, Damasio, & Anderson, 1994), while measuring skin conductance responses. For half of the participants, real money was used, and for half, fake money was used. Success on the task was positively correlated with anticipatory autonomic response, with no difference in pattern between real and fake money conditions. The results show clear support for Damasio's SMH, and suggest that it can be used to predict learning performance within a healthy population.     

Working memory involvement in emotion-based processes underlying choosing advantageously

The Iowa Gambling Task (IGT) is widely used to assess decision making under conditions of uncertainty in clinical as well as in nonclinical populations. However, there is still debate as to whether normal performance at this task relies on implicit, emotion-based processes that are independent of working memory. To clarify the role of working memory on normal performance on the IGT, participants performed the task under low or high working memory load. We used a modified version of the original task, in which the position of the four decks was randomized between trials. Results showed that only participants performing under low memory load significantly chose more advantageously halfway through the task. In addition, when comparing the number of cards chosen from the two decks with frequent losses, one advantageous and one disadvantageous, only participants performing under low memory load chose more cards from the advantageous deck. The present findings indicate that the processes underlying optimal advantageous performance on the IGT rely on working memory functions.     

Impulsive decision-making: learning to gamble wisely?

A decade ago, Bechara et al. [Bechara, A., Damasio, A. R., Damasio, H., & Anderson, S. W. (1994). Insensitivity to future consequences following damage to human prefrontal cortex. Cognition, 50, 7-15] published a paper in Cognition, introducing a Gambling Task which was designed to mimic everyday decision-making. Since then, the task has been computerized, its decision-making components have been scrutinized, and it has been employed in various patient groups, such as pathological gamblers, substance abusers, and so forth. Time and again it has been shown that while normal controls perform well on the task, it is the various target populations under scrutiny who exhibit poor performance. Yet, as we show in this paper, a total of 46% of normal individuals exhibited poor performance on the task, indicating a lack of learning. We argue that while so much importance has been placed on contrasting patients with controls, perhaps the real emphasis should lie in distinguishing among different individual profiles of performance on the task, irrespective of clinical classification. Our basic recommendation is to look at the individual data, pool when you can, and only then to contrast groups.     

Age-related changes in decision making: comparing informed and noninformed situations

Advantageous decision making progressively develops into early adulthood, most specifically in complex and motivationally salient decision situations in which direct feedback on gains and losses is provided (Figner & Weber, 2011). However, the factors that underlie this developmental improvement in decision making are still not well understood. The current study therefore investigates 2 potential factors, long-term memory and working memory, by assigning a large developmental sample (7-29 years of age) to a condition with either high or low demands on long-term and working memory. The first condition featured an age-adapted version of the Iowa Gambling Task (IGT; Bechara, Damasio, Damasio, & Anderson, 1994; i.e., a noninformed situation), whereas the second condition provided an external store where explicit information on gains, losses, and probabilities per choice option was presented (i.e., an informed situation). Consistent with previous developmental IGT studies, children up to age 12 did not learn to prefer advantageous options in the noninformed condition. In contrast, all age groups learned to prefer the advantageous options in the informed conditions, although a slight developmental increase in advantageous decision making remained. These results indicate that lowering dependence on long-term and working memory improves children's advantageous decision making. The results additionally suggest that other factors, like inhibitory control processes, may play an additional role in the development of advantageous decision making.     

Comparison of deck- and trial-based approaches to advantageous decision making on the Iowa Gambling Task

We compared the original deck-based model of advantageous decision making assessed with the Iowa Gambling Task (IGT) with a trial-based approach across behavioral and physiological outcomes in 33 younger adults (15 men, 18 women; 22.2 ± 3.7 years of age). One administration of the IGT with simultaneous measurement of skin conductance responses (SCRs) was performed and the two methods applied: (a) the original approach of subtracting disadvantageous picks of Decks A and B from advantageous picks of Decks C and D and (b) a trial-based approach focused on the financial outcome for each deck leading up to the trial in question. When directly compared, the deck-based approach resulted in a more advantageous behavioral profile than did the trial-based approach. Analysis of SCR data revealed no significant differences between methods for physiological measurements of SCR fluctuations or anticipatory responses to disadvantageous picks. Post hoc investigation of the trial-based method revealed Deck B contributed to both advantageous and disadvantageous decision making for the majority of participants. When divided by blocks of 20, the number of advantageous to disadvantageous choices reversed as the task progressed despite the total number of picks from Deck B remaining high. SCR fluctuations for Deck B, although not significantly different from the other decks, did show a sharp decline after the first block of 20 and remained below levels for Decks C and D toward the end of the task, suggesting that participants may have gained knowledge of the frequency of loss for this deck.     

Neurocognitive deficits related to poor decision making in people behind bars

Using a novel quantitative model of repeated choice behavior, we investigated the cognitive processes of criminal offenders incarcerated for various crimes. Eighty-one criminals, including violent offenders, drug and sex offenders, drivers operating a vehicle while impaired, and 18 matched controls were tested. The results were also contrasted with those obtained from neurological patients with focal brain lesions in the orbitofrontal cortex and from drug abusers. Participants performed the computerized version of the Iowa gambling task (Bechara, Damasio, Damasio, & Anderson, 1994), and the results were decomposed into specific component processes, using the expectancy valence model (Busemeyer & Stout, 2002). The findings indicated that whereas all the criminal groups tended to select disadvantageously, the cognitive profiles exhibited by different groups were considerably different. Certain subpopulations--most significantly, drug and sex offenders--overweighted potential gains as compared with losses, similar to chronic cocaine abusers. In contrast, assault/murder criminals tended to make less consistent choices and to focus on immediate outcomes and, in these respects, weremore similar to patients with orbitofrontal damage. The present cognitive model provides a novel way for building a bridge between cognitive neuroscience and complex human behaviors.     

Somatic Markers and Response Reversal: Is There Orbitofrontal Cortex Dysfunction in Boys with Psychopathic Tendencies?

This study investigated the performance of boys with psychopathic tendencies and comparison boys, aged 9 to 17 years, on two tasks believed to be sensitive to amygdala and orbitofrontal cortex functioning. Fifty-one boys were divided into two groups according to the Psychopathy Screening Device (PSD, P. J. Frick & R. D. Hare, in press) and presented with two tasks. The tasks were the gambling task (A. Bechara, A. R. Damasio, H. Damasio, & S. W. Anderson, 1994) and the Intradimensional/Extradimensional (ID/ED) shift task (R. Dias, T. W. Robbins, & A. C. Roberts, 1996). The boys with psychopathic tendencies showed impaired performance on the gambling task. However, there were no group differences on the ID/ED task either for response reversal or extradimensional set shifting. The implications of these results for models of psychopathy are discussed.     

IGT-Open: An open-source,computerized version of the Iowa Gambling Task

The Iowa Gambling Task (IGT) is commonly used to understand the processes involved in decision-making. Though the task was originally run without a computer, using a computerized version of the task has become typical. These computerized versions of the IGT are useful, because they can make the task more standardized across studies and allow for the task to be used in environments where a physical version of the task may be difficult or impossible to use (e.g., while collecting brain imaging data). Though these computerized versions of the IGT have been useful for experimentation, having multiple software implementations of the task could present reliability issues. We present an open-source software version of the Iowa Gambling Task (called IGT-Open) that allows for millisecond visual presentation accuracy and is freely available to be used and modified. This software has been used to collect data from human subjects and also has been used to run model-based simulations with computational process models developed to run in the ACT-R architecture.     

Adapted to explore: Reinforcement learning in Autistic Spectrum Conditions

Recent studies have recorded a tendency of individuals with Autism Spectrum Conditions (ASC) to continually change their choices in repeated choice tasks. In the current study we examine if this finding implies that ASC individuals have a cognitive style that facilitates exploration and discovery. Six decision tasks were administered to adolescents with ASC and matched controls. Significant differences in shifting between choice options appeared in the Iowa Gambling task (Bechara, Damasio, Damasio, & Anderson, 1994). A formal cognitive modeling analysis demonstrated that for about half of the ASC participants the adaptation process did not conform to the standard reinforcement learning model. These individuals were only coarsely affected by choice-outcomes, and were more influenced by the exploratory value of choices, being attracted to previously un-explored alternatives. An examination of the five simpler decision tasks where the advantageous option was easier to determine showed no evidence of this pattern, suggesting that the shifting choice pattern is not an uncontrollable tendency independent of task outcomes. These findings suggest that ASC individuals have a unique adaptive learning style, which may be beneficial is some learning environment but maladaptive in others, particularly in social contexts.     

Estimation of nonlinear mixed-effects continuous-time models using the continuous-discrete extended Kalman filter

Many intensive longitudinal measurements are collected at irregularly spaced time intervals, and involve complex, possibly nonlinear and heterogeneous patterns of change. Effective modelling of such change processes requires continuous-time differential equation models that may be nonlinear and include mixed effects in the parameters. One approach of fitting such models is to define random effect variables as additional latent variables in a stochastic differential equation (SDE) model of choice, and use estimation algorithms designed for fitting SDE models, such as the continuous-discrete extended Kalman filter (CDEKF) approach implemented in the dynr R package, to estimate the random effect variables as latent variables. However, this approach's efficacy and identification constraints in handling mixed-effects SDE models have not been investigated. In the current study, we analytically inspect the identification constraints of using the CDEKF approach to fit nonlinear mixed-effects SDE models; extend a published model of emotions to a nonlinear mixed-effects SDE model as an example, and fit it to a set of irregularly spaced ecological momentary assessment data; and evaluate the feasibility of the proposed approach to fit the model through a Monte Carlo simulation study. Results show that the proposed approach produces reasonable parameter and standard error estimates when some identification constraint is met. We address the effects of sample size, process noise variance, and data spacing conditions on estimation results.