Reconciling reinforcement learning models with behavioral extinction and renewal: implications for addiction, relapse, and problem gambling

Because learned associations are quickly renewed following extinction, the extinction process must include processes other than unlearning. However, reinforcement learning models, such as the temporal difference reinforcement learning (TDRL) model, treat extinction as an unlearning of associated value and are thus unable to capture renewal. TDRL models are based on the hypothesis that dopamine carries a reward prediction error signal; these models predict reward by driving that reward error to zero. The authors construct a TDRL model that can accommodate extinction and renewal through two simple processes: (a) a TDRL process that learns the value of situation-action pairs and (b) a situation recognition process that categorizes the observed cues into situations. This model has implications for dysfunctional states, including relapse after addiction and problem gambling.     

Dreading the pain of others? Altruistic responses to others' pain underestimate dread

A dislike of waiting for pain, aptly termed ‘dread’, is so great that people will increase pain to avoid delaying it. However, despite many accounts of altruistic responses to pain in others, no previous studies have tested whether people take delay into account when attempting to ameliorate others' pain. We examined the impact of delay in 2 experiments where participants (total N = 130) specified the intensity and delay of pain either for themselves or another person. Participants were willing to increase the experimental pain of another participant to avoid delaying it, indicative of dread, though did so to a lesser extent than was the case for their own pain. We observed a similar attenuation in dread when participants chose the timing of a hypothetical painful medical treatment for a close friend or relative, but no such attenuation when participants chose for a more distant acquaintance. A model in which altruism is biased to privilege pain intensity over the dread of pain parsimoniously accounts for these findings. We refer to this underestimation of others' dread as a ‘Dread Empathy Gap’.     

Social discounting of pain

Impatience can be formalized as a delay discount rate, describing how the subjective value of reward decreases as it is delayed. By analogy, selfishness can be formalized as a social discount rate, representing how the subjective value of rewarding another person decreases with increasing social distance. Delay and social discount rates for reward are correlated across individuals. However no previous work has examined whether this relationship also holds for aversive outcomes. Neither has previous work described a functional form for social discounting of pain in humans. This is a pertinent question, since preferences over aversive outcomes formally diverge from those for reward. We addressed this issue in an experiment in which healthy adult participants (N = 67) chose the timing and intensity of hypothetical pain for themselves and others. In keeping with previous studies, participants showed a strong preference for immediate over delayed pain. Participants showed greater concern for pain in close others than for their own pain, though this hyperaltruism was steeply discounted with increasing social distance. Impatience for pain and social discounting of pain were weakly correlated across individuals. Our results extend a link between impatience and selfishness to the aversive domain.