Resolution of outcome-induced response conflict by humans after extended training

Studies of incongruent discrimination learning, where the outcome event of one response acts as the discriminative stimulus for the opposite response, suggest that humans rely on habitual stimulus–response (S–R) associations when outcome–response (O–R) associations would cause response conflict. Here, two experiments were conducted to investigate the robustness of this habitual strategy. In Experiment 1, we found that extensive instrumental discrimination training supported learning about the incongruent R → O contingencies, as assessed by an outcome devaluation test. Differential representations of the stimulus and the (associatively retrieved) outcome may have allowed for goal-directed incongruent performance. Experiment 2 failed to provide evidence for this possibility; direct presentation as well as associative retrieval of the incongruent events (by Pavlovian stimuli) activated the response that was associated with each event in its role of stimulus as opposed to outcome. We did find that participants successfully acquired explicit knowledge of the incongruent contingencies, which raises the possibility that propositional encoding allowed them to overcome the response conflict caused by O–R associations. Alternative associative and propositional accounts of successful goal-directed incongruent performance with extensive training will be discussed.     

Impaired goal-directed behavioural control in human impulsivity

Two dissociable learning processes underlie instrumental behaviour. Whereas goal-directed behaviour is controlled by knowledge of the consequences, habitual behaviour is elicited directly by antecedent Pavlovian stimuli without knowledge of the consequences. Predominance of habitual control is thought to underlie psychopathological conditions associated with corticostriatal abnormalities, such as impulsivity and drug dependence. To explore this claim, smokers were assessed for nicotine dependence, impulsivity, and capacity for goal-directed control over instrumental performance in an outcome devaluation procedure. Reduced goal-directed control was selectively associated with the Motor Impulsivity factor of Barrett's Impulsivity Scale (BIS), which reflects propensity for action without thought. These data support the claim that human impulsivity is marked by impaired use of causal knowledge to make adaptive decisions. The predominance of habit learning may play a role in psychopathological conditions that are associated with trait impulsivity.     

Stimulus control of habits: Evidence for both stimulus specificity and devaluation insensitivity in a dual-response task

Goal-directed and habitual actions are clearly defined by their associative relations. Whereas goal-directed control can be confirmed via tests of outcome devaluation and contingency-degradation sensitivity, a comparable criterion for positively detecting habits has not been established. To confirm habitual responding, a test of control by the stimulus–response association is required while also ruling out goal-directed control. Here we describe an approach to developing such a test in rats using two discriminative stimuli that set the occasion for two different responses that then earn the same outcome. Performance was insensitive to outcome devaluation and showed stimulus–response specificity, indicative of stimulus-controlled behavior. The reliance of stimulus–response associations was further supported by a lack of sensitivity during the single extinction test session used here. These results demonstrate that two concurrently trained responses can come under habitual control when they share a common outcome. By reducing the ability of one stimulus to signal its corresponding response–outcome association, we found evidence for goal-directed control that can be dissociated from habits. Overall, these experiments provide evidence that tests assessing specific stimulus–response associations can be used to investigate habits.     

A normative perspective on motivation

Understanding the effects of motivation on instrumental action selection, and specifically on its two main forms, goal-directed and habitual control, is fundamental to the study of decision making. Motivational states have been shown to 'direct' goal-directed behavior rather straightforwardly towards more valuable outcomes. However, how motivational states can influence outcome-insensitive habitual behavior is more mysterious. We adopt a normative perspective, assuming that animals seek to maximize the utilities they achieve, and viewing motivation as a mapping from outcomes to utilities. We suggest that habitual action selection can direct responding properly only in motivational states which pertained during behavioral training. However, in novel states, we propose that outcome-independent, global effects of the utilities can 'energize' habitual actions.     

Age-Friendly Communities Matter for Older People’s Well-Being

This study aims to identify relationships between age-friendly environments (in terms of social and physical neighborhood attributes) and older people’s overall well-being, as well as the underlying instrumental goals to achieve overall well-being. A sample of 945 community-dwelling older adults living in Rotterdam’s districts Lombardijen, Lage Land/Prinsenland, Oude Westen, and Vreewijk was asked to complete a questionnaire in 2013. A total of 588 (62%) responded. The majority (56%) of respondents was female, 19% had low educational levels, 35% were married, and 85% were born in the Netherlands. Mean age was 77.1 ± 5.3 (range 70–93) years. Levels of age-friendliness and older people’s ability to realize the instrumental goals to achieve overall well-being varied tremendously among neighborhoods, with older people living in less age-friendly communities reporting lower levels of well-being. These differences in well-being resulted especially from differences in affection, behavioral confirmation, and comfort. Higher-educated older persons were more critical regarding the domains civic participation, transportation, and communication and information in their neighborhoods, suggesting a socioeconomic gradient in the perceived lack of neighborhood attributes facilitating aging in place. Currently, physical and social neighborhood attributes enabling aging in place seem to satisfy the needs for affection, behavioral confirmation, and comfort in some, but not all, neighborhoods. Levels of age-friendliness in neighborhoods did not explain differences in opportunities for older people to realize the instrumental goals of status and stimulation.     

State-based versus reward-based motivation in younger and older adults

Recent decision-making work has focused on a distinction between a habitual, model-free neural system that is motivated toward actions that lead directly to reward and a more computationally demanding goal-directed, model-based system that is motivated toward actions that improve one’s future state. In this article, we examine how aging affects motivation toward reward-based versus state-based decision making. Participants performed tasks in which one type of option provided larger immediate rewards but the alternative type of option led to larger rewards on future trials, or improvements in state. We predicted that older adults would show a reduced preference for choices that led to improvements in state and a greater preference for choices that maximized immediate reward. We also predicted that fits from a hybrid reinforcement-learning model would indicate greater model-based strategy use in younger than in older adults. In line with these predictions, older adults selected the options that maximized reward more often than did younger adults in three of the four tasks, and modeling results suggested reduced model-based strategy use. In the task where older adults showed similar behavior to younger adults, our model-fitting results suggested that this was due to the utilization of a win-stay–lose-shift heuristic rather than a more complex model-based strategy. Additionally, within older adults, we found that model-based strategy use was positively correlated with memory measures from our neuropsychological test battery. We suggest that this shift from state-based to reward-based motivation may be due to age related declines in the neural structures needed for more computationally demanding model-based decision making.     

Age differences in the neural response to negative feedback

Affective processing is one domain that remains relatively intact in healthy aging. Investigations into the neural responses associated with reward anticipation have revealed that older and younger adults recruit the same midbrain reward regions, but other evidence suggests this recruitment may differ depending on the valence (gain, loss) of the incentive cue. The goal of the current study was to examine functional covariance during gain and loss feedback in younger and healthy older adults. A group of 15 older adults (mean age = 68.5) and 16 younger adults (mean age = 25.4) completed a revised Monetary Incentive Delay task (rMID; Knutson, Westdorp, Kaiser, &; Hommer, 2000) while in the fMRI scanner. The rMID is a reaction time task where successful performance, either gaining a reward or avoiding a loss, is defined by hitting a button during the brief presentation of a visual target. Participants receive gain and loss anticipation cues before each trial and feedback after each trial with four possible outcomes: +$5.00, +0.00, -$5.00, and -Affective processing is one domain that remains relatively intact in healthy aging. Investigations into the neural responses associated with reward anticipation have revealed that older and younger adults recruit the same midbrain reward regions, but other evidence suggests this recruitment may differ depending on the valence (gain, loss) of the incentive cue. The goal of the current study was to examine functional covariance during gain and loss feedback in younger and healthy older adults. A group of 15 older adults (mean age = 68.5) and 16 younger adults (mean age = 25.4) completed a revised Monetary Incentive Delay task (rMID; Knutson, Westdorp, Kaiser, & Hommer, 2000) while in the fMRI scanner. The rMID is a reaction time task where successful performance, either gaining a reward or avoiding a loss, is defined by hitting a button during the brief presentation of a visual target. Participants receive gain and loss anticipation cues before each trial and feedback after each trial with four possible outcomes: +$5.00, +0.00, -$5.00, and -$0.00. Using seed-voxel partial least squares analyses, with seed voxels in the caudate and ventromedial prefrontal cortex, whole-brain functional covariance revealed that younger and older adults engage the same network of regions to support general feedback processing. However, older adults engaged two additional networks to support processing of negative feedback, gain_miss (+0), loss_miss (-$5), and loss_hit (?0), specifically. These findings are in line with theories of a positivity effect in aging and may have implications for reward-stimulus learning and decision making following performance-contingent negative feedback.     

The spacing effect in older and younger adults: does context matter?

Age-related memory change has been a topic of much investigation in recent years, including spacing benefits and reliance on contextual cues. We manipulated the spacing schedule and the context of learning and observed the effects on long-term recall ability in healthy older and younger adults. After learning Swahili–English word pairs, half practiced immediately (massed) and half practiced 24 h later (spaced) either in the same room or a different room (context) from the initial session. A final recall test 10 days after the practice session occurred in the same room as the first session. Participants in the spaced condition remembered more than those in the massed condition 10 days later. Younger adults remembered more word pairs than older adult participants. Context change eliminated the spacing benefit for both age groups.     

More expertise for a better perspective: Task and strategy-driven adaptive neurofunctional reorganization for word production in high-performing older adults

The suggestion that neurofunctional reorganization may contribute to preserved language abilities is still emerging in aging studies. Some of these abilities, such as verbal fluency (VF), are not unitary but instead rely on different strategic processes that are differentially changed with age. Younger (n = 13) and older adults (n = 13) carried out an overt self-paced semantic and orthographic VF tasks within mixed fMRI design. Our results suggest that patterns of brain activation sustaining equivalent performances could be underpinned by different strategies facing brain changes during healthy aging. These main findings suggest that temporally mediated semantic clustering and frontally mediated orthographic switching were driven by evolutive neurofunctional resources in high-performing older adults. These age-related activation changes can appear to be compatible with the idea that unique neural patterns expressing distinctive cognitive strategies are necessary to support older adults’ performance on VF tasks.     

Multiple bases for young and older adults' judgments of learning in multitrial learning

Prior work has suggested that participants use a memory-for-past-tests (MPT) heuristic for judgments of learning (JOLs) in a multitrial learning scenario. That is, when learning the same material in multiple sessions, previous memory performance can be used as a basis for later memory predictions. We explored this issue by evaluating the impact of healthy aging on the use of MPT across trials. Young adults and healthy older adults learned pairs of words, made JOLs, and received a memory test in three study-test trials on the same material. Results indicated that both young and older adults relied on MPT as a basis for JOLs and changes in MPT across trials were nominal. Further, only the most-recent past test influenced JOLs, whereas earlier tests were unrelated to later judgments. JOLs were also influenced by prior-trial JOLs and were related to subsequent memory performance on the same trial. We suggest that these data support both indirect- and direct-memory mechanisms as the bases for the MPT heuristic. Further, in a multitrial learning scenario, in which the same information was being learned, young and older adults used the same bases for their JOLs.     

Contributions of dorsal striatal subregions to spatial alternation behavior

Considerable evidence has shown a clear dissociation between the dorsomedial (DMS) and the dorsolateral (DLS) striatum in instrumental conditioning. In particular, DMS activity is necessary to form action-outcome associations, whereas the DLS is required for developing habitual behavior. However, few studies have investigated whether a similar dissociation exists in more complex goal-directed learning processes. The present study examined the role of the two structures in such complex learning by analyzing the effects of excitotoxic DMS and DLS lesions during the acquisition and extinction of spatial alternation behavior, in a continuous alternation T-maze task. We demonstrate that DMS and DLS lesions have opposite effects, the former impairing and the latter improving animal performance during learning and extinction. DMS lesions may impair the acquisition of spatial alternation behavior by disrupting the signal necessary to link a goal with a specific spatial sequence. In contrast, DLS lesions may accelerate goal-driven strategies by minimizing the influence of external stimuli on the response, thus increasing the impact of action-reward contingencies. Taken together, these results suggest that DMS- and DLS-mediated learning strategies develop in parallel and compete for the control of the behavioral response early in learning.     

Aging affects the balance between goal-guided and habitual spatial attention

Visual clutter imposes significant challenges to older adults in everyday tasks and often calls on selective processing of relevant information. Previous research has shown that both visual search habits and task goals influence older adults’ allocation of spatial attention, but has not examined the relative impact of these two sources of attention when they compete. To examine how aging affects the balance between goal-driven and habitual attention, and to inform our understanding of different attentional subsystems, we tested young and older adults in an adapted visual search task involving a display laid flat on a desk. To induce habitual attention, unbeknownst to participants, the target was more often placed in one quadrant than in the others. All participants rapidly acquired habitual attention toward the high-probability quadrant. We then informed participants where the high-probability quadrant was and instructed them to search that screen location first—but pitted their habit-based, viewer-centered search against this instruction by requiring participants to change their physical position relative to the desk. Both groups prioritized search in the instructed location, but this effect was stronger in young adults than in older adults. In contrast, age did not influence viewer-centered search habits: the two groups showed similar attentional preference for the visual field where the target was most often found before. Aging disrupted goal-guided but not habitual attention. Product, work, and home design for people of all ages––but especially for older individuals––should take into account the strong viewer-centered nature of habitual attention.     

Effects of age simulation and age on motor sequence learning: Interaction of age-related cognitive and motor decline

Aging is known to lead to decrements in sensory and cognitive functioning and motor performance. The purpose of the present experiment was twofold: a) We assessed the influence of wearing an age simulation suit on motor sequence learning, cognitive speed tasks and far visual acuity in healthy, younger adults. b) We evaluated the interaction of cognitive aging and declining motor sequence learning in older adults. In a between-subjects design we tested 11 younger adults (M_age = 23.6 years) without the age suit, 12 younger adults wearing the age suit (M_age = 23.2 years), and 23 older adults (M_age = 72.6 years). All participants learned a simple, spatial-temporal movement sequence on two consecutive days, and we assessed perceptual processing speed (Digit Symbol Substitution test and Figural Speed test) and far visual acuity. Wearing an age simulation suit neither affected the learning of the simple motor sequence nor the performance at the cognitive speed tasks in younger adults. However, far visual acuity suffered from wearing the suit. Younger adults with and without the suit showed better motor sequence learning compared to older adults. The significant correlations between the cognitive speed tests and the motor learning performance in older adults indicated that cognitive aging partially explains some of the variance in age-related motor learning deficits.     

Chronic motivational state interacts with task reward structure in dynamic decision-making

Research distinguishes between a habitual, model-free system motivated toward immediately rewarding actions, and a goal-directed, model-based system motivated toward actions that improve future state. We examined the balance of processing in these two systems during state-based decision-making. We tested a regulatory fit hypothesis (Maddox & Markman, 2010) that predicts that global trait motivation affects the balance of habitual- vs. goal-directed processing but only through its interaction with the task framing as gain-maximization or loss-minimization. We found support for the hypothesis that a match between an individual’s chronic motivational state and the task framing enhances goal-directed processing, and thus state-based decision-making. Specifically, chronic promotion-focused individuals under gain-maximization and chronic prevention-focused individuals under loss-minimization both showed enhanced state-based decision-making. Computational modeling indicates that individuals in a match between global chronic motivational state and local task reward structure engaged more goal-directed processing, whereas those in a mismatch engaged more habitual processing.     

Investigating ToM in aging with the MASC: from accuracy to error type

Growing evidence indicates that Theory of Mind (ToM) declines in normal aging. However, the majority of this research has used classic and static verbal tasks that present scenarios, which are very different from real life. The present study was designed to fill this gap by administering the Movie for the Assessment of Social Cognition (MASC) to young and older participants. It allows one to analyze not only the accuracy, but also the typology of error in mental states attribution distinguishing between iper-ToM (over-mentalization), ipo-ToM (insufficient mentalization), and no-ToM (lack of mentalization). We recruited 30 young (20–29 years), 39 young-old (65–74 years), and 31 old-old (75–86 years) participants. Along with the MASC, we administered a classic ToM task, the Strange Stories, and several measures of cognitive functioning. Results showed that older adults were less accurate in mental state attribution than young adults in the MASC, but not in the Strange Stories. In addition, compared to young adults, older adults committed more errors of both ipo- and no-ToM, while young adults committed more often iper-ToM errors. Additionally, older adults, but not young adults, did not show a difference between iper-ToM and ipo-ToM errors, which were equally frequent in this age group. Globally, results indicated that older adults’ failure in classic ToM tasks may be due to both ipo- and iper-ToM and provide needed evidence for the MASC as a suitable measure of ToM in aging.     

Parallel goal-directed and habitual control of human drug-seeking: implications for dependence vulnerability

Dual-process theories of learning and addiction propose that whereas freely elected drug/reward-seeking is goal-directed in being mediated by the expected value of the outcome, cue-elicited drug/reward-seeking is habitual in being elicited directly by antecedent stimuli, without retrieving a representation of outcome value. To substantiate this claim, the current study conducted a human devaluation-transfer procedure in which young adult smokers were first trained on a concurrent choice task to earn tobacco and chocolate points before one outcome was devalued by specific satiety or health warnings against consumption of that outcome. When choice was again tested in extinction, the selective reduction in performance of the action associated with the devalued outcome indicated that choice was controlled by an expectation of outcome value, that is, was goal-directed. Moreover, the presentation of tobacco and chocolate cues enhanced selection of the response associated with that outcome, indicating that transfer was also mediated by the retrieval of the outcome representation. Paradoxically, however, the magnitude of this transfer effect was unaffected by devaluation, indicating that the stimulus retrieved a representation of outcome identity but not current incentive value. Individual differences in tobacco dependence in the young adult sample were associated with tobacco preference in the concurrent choice task but not with the devaluation or transfer effects. These data accord with dual-process theories in suggesting that drug/reward-seeking are mediated by goal-directed and habitual controllers under freely elected and cued conditions, respectively, and that initial uptake of drug use is associated with hyper-valuation of the drug as an outcome of goal-directed drug-seeking rather than with accelerated habit formation.     

Episodic memory function is affected by lifestyle factors: a 14-year follow-up study in an elderly population

Understanding the relationship between memory function and lifestyle offers great opportunities for promoting beneficial lifestyle choices to foster healthy cognitive aging and for the development of intervention programs for older adults. We studied a cohort of older adults (age 65 and older) enrolled in the Longitudinal Aging Study Amsterdam, an ongoing prospective population-based research project. A total of 1,966 men and women participated in an episodic memory test every 3 years over a period of 14 years. Lifestyle habits were repeatedly assessed using self-report measures. Physical activity, light-to-moderate alcohol consumption, difficulties staying asleep, and social engagement were associated with better memory function over the course of 14 years. In contrast, smoking and long sleep duration were associated with worse memory function. These findings suggest that certain lifestyle factors can have long-term protective or harmful effects on memory function in aging individuals.     

The temporal structure of goal-directed and habitual operant behavior

Operant behavior can reflect the influence of goal-directed and habitual processes. These can be distinguished by changes to response rate following devaluation of the reinforcing outcome. Whether a response is goal directed or habitual depends on whether devaluation affects response rate. Response rate can be decomposed into frequencies of bouts and pauses by analyzing the distribution of interresponse times. This study sought to characterize goal-directed and habitual behaviors in terms of bout-initiation rate, within-bout response rate, bout length, and bout duration. Data were taken from three published studies that compared sensitivity to devaluation following brief and extended training with variable-interval schedules. Analyses focused on goal-directed and habitual responding, a comparison of a habitual response to a similarly trained response that had been converted back to goal-directed status after a surprising event, and a demonstration of contextual control of habit and goal direction in the same subjects. Across experiments and despite responses being clearly distinguished as goal directed and habitual by total response rate, analyses of bout-initiation rate, within-bout rate, bout length, and bout duration did not reveal a pattern that distinguished goal-directed from habitual responding.     

Analysis of Narrative Discourse Structure as an Ecologically Relevant Measure of Executive Function in Adults

This study examined the narrative discourse production and executive function (EF) abilities of 46 neuro-typical adults (18–98 years old). Two questions were addressed: Is the analysis of narrative structure sensitive to changes associated with aging? & What is the relationship between measures of narrative structure and EF? Narratives were elicited under two conditions and narrative structure was analyzed for the presence of organizing story grammar elements. Narrative structure was significantly correlated with age as well as linguistic and non-linguistic measures of EF. Factor analysis of story structure and EF variables yielded two factors reflecting constructs of output-fluidity and organizational-efficiency. These data suggest that narrative structure and EF represent aspects of goal-directed knowledge that are not bound by a traditional linguistic and non-linguistic division. Thus, narrative structure may represent a global and ecologically valid measure of goal-directed executive function knowledge that is also sensitive to changes associated with typical aging.