Predicting similarity judgments in intertemporal choice with machine learning

Similarity models of intertemporal choice are heuristics that choose based on similarity judgments of the reward amounts and time delays. Yet, we do not know how these judgments are made. Here, we use machine-learning algorithms to assess what factors predict similarity judgments and whether decision trees capture the judgment outcomes and process. We find that combining small and large values into numerical differences and ratios and arranging them in tree-like structures can predict both similarity judgments and response times. Our results suggest that we can use machine learning to not only model decision outcomes but also model how decisions are made. Revealing how people make these important judgments may be useful in developing interventions to help them make better decisions.     

Weighing Outcomes by Time or Against Time? Evaluation Rules in Intertemporal Choice

Models of intertemporal choice draw on three evaluation rules, which we compare in the restricted domain of choices between smaller sooner and larger later monetary outcomes. The hyperbolic discounting model proposes an alternative‐based rule, in which options are evaluated separately. The interval discounting model proposes a hybrid rule, in which the outcomes are evaluated separately, but the delays to those outcomes are evaluated in comparison with one another. The tradeoff model proposes an attribute‐based rule, in which both outcomes and delays are evaluated in comparison with one another: People consider both the intervals between the outcomes and the compensations received or paid over those intervals. We compare highly general parametric functional forms of these models by means of a Bayesian analysis, a method of analysis not previously used in intertemporal choice. We find that the hyperbolic discounting model is outperformed by the interval discounting model, which, in turn, is outperformed by the tradeoff model. Our cognitive modeling is among the first to offer quantitative evidence against the conventional view that people make intertemporal choices by discounting the value of future outcomes, and in favor of the view that they directly compare options along the time and outcome attributes.     

Evidence for a magnitude effect in temporal discounting with pigeons

A magnitude effect in human intertemporal choice is well established-larger rewards or outcomes are discounted over time at a lower rate than are smaller rewards. However, many recent studies have failed to find a corresponding effect in nonhuman animals. Here we report a magnitude effect in temporal discounting for pigeons' choices involving a tradeoff between reward delay and amount. Pigeons chose between a small reward (1-s access to food) after a 2-s delay, and a large reward (4.5-s access to food) after a 28-s delay. Across conditions, the delays to the small and large rewards were increased or decreased, respectively. Temporal discounting functions obtained through a value-estimation procedure showed clear evidence of a magnitude effect: The value of the large reward decreased more slowly with increasing delay than the value of the small reward. We linked this result to a nonlinear relationship between choice and the delays associated with the small and large rewards. The nonlinearity was contrary to the generalized matching law but was predicted by the contextual choice model. Our results confirm the existence of a magnitude effect in nonhuman temporal discounting, showing that this adaptation is not unique to humans.     

跨期选择的认知机制与神经基础

跨期选择是指个体对发生在不同时间的成本与收益进行权衡的决策过程。跨期选择的计算模型从经济学的角度用数学模型来建构时间折扣函数,而认知成分模型则从心理学的角度来研究跨期选择中的心理效应与认知成分。跨期选择的神经基础有三种不同的研究取向：双机制加工取向、单机制加工取向、自我控制取向。未来研究应该在跨期选择的认知机制、神经通路及运行机制、跨期选择的应用,以及从进化的角度对人与动物的跨期选择行为进行更深入的研究。     

Changes in intertemporal choices in deviant behaviors

Delay discounting is the process of devaluing results that happen in the future. We present a comprehensive literature review of changes on intertemporal choices in deviant behaviors, namely in (a) substance-related and addictive disorders, (b) disruptive, impulse-control, and conduct disorders, and (c) eating disorders. We also present studies focused on differences in demographic characteristics of the populations by gender, age, and education/social class. Delay discounting is presented as a process of studying intertemporal choices, resulting from decades of empirical research. Studies indicate that this process may provide explanation as to why individuals will sometimes choose a smaller reward, available sooner, instead of a larger reward available later. When studying populations with the above-mentioned problems, they tend to exhibit more pronounced discounting functions than control groups. The association between discounting and gender is not clear. The relationship between delay discounting and age is relatively clear, where older individuals discount less markedly than younger individuals. Studies suggest that shallower discounting gradients are associated with higher levels of intelligence and academic success. We emphasize the need for more empirical research on delay discounting, especially in regard to deviant behavior that may be associated with impulse-control disorders.     

PREFERENCE REVERSALS DUE TO MYOPIC DISCOUNTING OF DELAYED REWARD

Abstract— A basic stationarity axiom of economic theory assumes stable preference between two deferred goods separated by a fixed time. To test this assumption, we offered subjects choices between delayed rewards, while manipulating the delays to those rewards. Preferences typically reversed with changes in delay, as predicted by hyperbolic discounting models of impulsiveness. Of 36 subjects, 34 reversed preference from a larger, later reward to a smaller, earlier reward as the delays to both rewards decreased. We conclude that the stationarity axiom is not appropriate in models of human choice.     

Time discounting and time preference in animals: A critical review

Animals are an important model for studies of impulsivity and self-control. Many studies have made use of the intertemporal choice task, which pits small rewards available sooner against larger rewards available later (typically several seconds), repeated over many trials. Preference for the sooner reward is often taken to indicate impulsivity and/or a failure of self-control. This review shows that very little evidence supports this assumption; on the contrary, ostensible discounting behavior may reflect a boundedly rational but not necessarily impulsive reward-maximizing strategy. Specifically, animals may discount weakly, or even adopt a long-term rate-maximizing strategy, but fail to fully incorporate postreward delays into their choices. This failure may reflect learning biases. Consequently, tasks that measure animal discounting may greatly overestimate the true discounting and may be confounded by processes unrelated to time preferences. If so, animals may be much more patient than is widely believed; human and animal intertemporal choices may reflect unrelated mental operations; and the shared hyperbolic shape of the human and animal discount curves, which is used to justify cross-species comparisons, may be coincidental. The discussion concludes with a consideration of alternative ways to measure self-control in animals.     

Temporal discounting when the choice is between two delayed rewards

The present experiments extend the temporal discounting paradigm from choice between an immediate and a delayed reward to choice between 2 delayed rewards: a smaller amount of money available sooner and a larger amount available later. Across different amounts and delays, the data were consistently well described by a hyperbola-like discounting function, and the degree of discounting decreased systematically as the delay to the sooner reward increased. Three theoretical models (the elimination-by-aspects, present-value comparison, and common-aspect attenuation hypotheses) were evaluated. The best account of the data was provided by the common-aspect attenuation hypothesis, according to which the common aspect of the choice alternatives (i.e., the time until the sooner reward is available) receives less weight in the decision-making process.     

适应性时间管理：死亡意识对时间知觉和跨期决策的影响

死亡意识是个体对自身死亡必然性的认识; 基于进化适应性的考量, 作者认为具有前瞻性的死亡意识作为人类特有的认知能力和个体生命史的终极坐标, 不仅可能引起恐惧感和防御反应, 更重要的是可以帮助人们做出适应性的时间管理。据此推论, 死亡意识的启动会加强人们对时间有限性的关注, 体验出更强的时间流逝感, 影响人们对时间的评估, 以及人们对不同时间点的成本与收益进行权衡而做出的跨期决策。本研究采用死亡凸显范式启动死亡意识, 通过两个实验探讨了死亡意识、时间知觉和跨期决策三者之间的关系。结果发现, 死亡意识启动使被试低估时间距离, 并表现出较低的跨期决策延迟折扣率, 从而在较小的即时获益与更大的延迟的获益之间更为偏好未来导向的选项。进一步的分析显示, 时间知觉在死亡意识和跨期决策延迟折扣率之间起到部分中介变量的作用。以上的研究结果初步揭示了死亡意识在时间管理上的适应性价值。     

不确定性容忍度对跨期选择的影响及其情景依赖性

采用2(不确定性容忍度：高、低)×2(跨期日期：今天/14天、今天/180天)×2(延迟奖赏值：200元、1000元)混合实验设计,探讨不同任务特征下不确定性容忍度对跨期选择的影响。结果表明：跨期日期为180天时,不确定性容忍度主效应边缘显著;不确定性容忍度与延迟奖赏值交互作用显著：在200元时低容忍度个体对延迟奖赏的折扣程度大于高容忍度个体,在1000元时无此效应;跨期日期为14天时,不确定性容忍度的主效应及其与延迟奖赏值交互作用均不显著。这表明,不确定性容忍度对跨期选择存在影响,但这种影响受到跨期日期和延迟奖赏值的调节,具有情景依赖性。     

Building blocks of self-control: increased tolerance for delay with bundled rewards

Impulsive choice can be defined as temporary preference for a smaller-sooner reward (SS) over a larger-later reward (LL). Hyperbolic discounting implies that impulsive choices will occur less when organisms choose between a series of SSs versus LLs all at once than when they choose between single SS versus LL pairs. Eight rats were exposed to two conditions of an intertemporal choice paradigm using sucrose solution as reward. In both conditions, the LL was 150 microl delayed by 3 s, while the SS was an immediate reward that ranged from 25-150 microl across sessions. Preference for the LL was greater when the chosen reward was automatically delivered three times in succession (bundled) than when it was chosen singly and delivered after each choice. For each of the 8 rats, the estimated SS amount that produced indifference was higher in the bundled condition than in the single condition. Because bundling in humans may be based on the perception that one's current choice is predictive of future choices, the data presented here may demonstrate an important building block of self-control.     

跨期决策中的负折扣现象及其机制

主流跨期决策模型认为, 跨期决策存在一个时间折扣过程。时间折扣是指人们会根据将来获益或损失的延迟时间对其效用进行折扣, 折扣后的效用小于原来的效用, 而负折扣现象违背了时间折扣过程。负折扣现象出现在金钱、非金钱的获益与损失领域, 其可能的解释机制为预期情绪。目前关于负折扣现象的研究, 较少验证其影响机制, 未来的研究可以结合眼动追踪和fMRI技术探讨其机制, 并丰富对负折扣现象的影响因素研究。     

时间洞察力对跨期选择偏好的预测

利用津巴多时间洞察力问卷,采用跨期选择经典范式,考察“现在享乐型”和“未来定向型”两类时间洞察力个体在跨期选择上的偏好,从人格特质角度来探究跨期选择的认知机制。结果发现：(1)现在享乐定向组的延迟折扣率显著高于未来定向组;(2)未来定向组在时间洞察力量表“未来维度”得分与延迟折扣率显著负相关。这说明时间洞察力对个体延迟折扣率具有预测作用,支持了跨期选择的时间感知基础模型。     

Tests of an indifference rule in variable‐delay and double‐reward choice procedures with humans

Four hundred and fifty participants were recruited from Amazon Mechanical Turk across 3 experiments to test the predictions of a hyperbolic discounting equation in accounting for human choices involving variable delays or multiple rewards (Mazur, 1984, 1986). In Experiment 1, participants made hypothetical choices between 2 monetary alternatives, 1 consisting of a fixed delay and another consisting of 2 delays of equal probability (i.e., a variable‐delay procedure). In Experiment 2, participants made hypothetical monetary choices between a single, immediate reward and 2 rewards, 1 immediate and 1 delayed (i.e., a double‐reward procedure). Experiment 3 also used a double‐reward procedure, but with 2 delayed rewards. Participants in all 3 experiments also completed a standard delay‐discounting task. Finally, 3 reward amounts were tested in each type of task ($100, $1000, and $5000). In the double‐reward conditions (Experiments 2 and 3), the results were in good qualitative and quantitative agreement with Mazur's model (1984, 1986). In contrast, when participants made choices involving variable delays (Experiment 1), there was relatively poor qualitative and quantitative agreement with this model. These results, along with our previous findings, suggest the structure of questions in hypothetical tasks with humans can be a strong determinant of the choice pattern.     

DISCOUNTING OF DELAYED FOOD REWARDS IN PIGEONS AND RATS: IS THERE A MAGNITUDE EFFECT?

Temporal discounting refers to the decrease in the present, subjective value of a reward as the time to its receipt increases. Results from humans have shown that a hyperbola-like function describes the form of the discounting function when choices involve hypothetical monetary rewards. In addition, magnitude effects have been reported in which smaller reward amounts are discounted more steeply than larger amounts. The present research examines the cross-species generality of these findings using real rewards, namely food pellets, with both pigeons and rats. As with humans, an adjusting amount procedure was used to estimate the amount of immediate reward judged equal in value to a delayed reward. Different amounts of delayed food rewards (ranging from 5 to 32 pellets in pigeons and from 5 to 20 pellets in rats) were studied at delays varying from 1 s to 32 s. A simple hyperbola, similar to the hyperbola-like mathematical function that describes the discounting of hypothetical monetary rewards in humans, described the discounting of food rewards in both pigeons and rats. These results extend the generality of the mathematical model of discounting. Rates of discounting delayed food rewards were higher for pigeons than for rats. Unlike humans, however, neither pigeons nor rats showed a reliable magnitude effect: Rate of discounting did not vary systematically as a function of the amount of the delayed reward.     

Temporal discounting for monetary and close relationship outcomes

The present study extended research on intertemporal choice—in which individuals choose between outcomes that may be received immediately or after a delay—to close relationships. In Experiment 1, New Zealand university students aged 18 to 25 made decisions about hypothetical monetary and relationship outcomes, and in Experiment 2, about relationship outcomes which emphasized companionate or sexual aspects (Ns = 64). Both experiments found effects of delay and magnitude on temporal discounting rates, and domain independence for choices about close relationships, similar to previous studies with monetary outcomes. There were no significant gender differences. Overall, results suggest that people make intertemporal choices about relationships according to a similar process used to make decisions involving other types of outcomes.     

厌恶和恐惧情绪对跨期选择的不同影响：基于ERP的研究

本研究运用事件相关电位技术(ERPs)考察厌恶和恐惧情绪对跨期选择的影响。其中,厌恶和恐惧情绪采用情绪面孔图片进行启动,分析跨期选择任务中评估阶段所诱发的ERP成分。行为结果发现,与中性面孔相比,厌恶面孔启动使个体倾向于选择立即奖赏。ERP结果发现,在选项评估阶段,厌恶面孔比中性和恐惧面孔启动诱发更大的P2、P3和LPP波幅。这说明,厌恶情绪促使个体投入更多的注意和动机资源对跨期选项进行评估,进而使个体倾向即时满足。     

厌恶和恐惧情绪对跨期选择的不同影响：基于ERP的研究

本研究运用事件相关电位技术(ERPs)考察厌恶和恐惧情绪对跨期选择的影响。其中,厌恶和恐惧情绪采用情绪面孔图片进行启动,分析跨期选择任务中评估阶段所诱发的ERP成分。行为结果发现,与中性面孔相比,厌恶面孔启动使个体倾向于选择立即奖赏。ERP结果发现,在选项评估阶段,厌恶面孔比中性和恐惧面孔启动诱发更大的P2、P3和LPP波幅。这说明,厌恶情绪促使个体投入更多的注意和动机资源对跨期选项进行评估,进而使个体倾向即时满足。     

Discounting: A practical guide to multilevel analysis of choice data

Multilevel modeling provides the ability to simultaneously evaluate the discounting of individuals and groups by examining choices between smaller sooner and larger later rewards. A multilevel logistic regression approach is advocated in which sensitivity to relative reward magnitude and relative delay are considered as separate contributors to choice. Examples of how to fit choice data using multilevel logistic models are provided to help researchers in the adoption of these methods.     

Transitional and steady-state choice behavior under an adjusting-delay schedule

Twelve rats made repeated choices on an adjusting-delay schedule between a smaller reinforcer (A) that was delivered immediately after a response and a larger reinforcer (B) that was delivered after a delay which increased or decreased by 20% depending on the subject's choices in successive blocks of trials. In two phases of the experiment (100 sessions and 40 sessions), reinforcer sizes were selected which enabled theoretical parameters expressing the rate of delay discounting and sensitivity to reinforcer size to be estimated from the ratio of the indifference delays obtained in the two phases. Indifference delays, calculated from adjusting delays in the last 10 sessions of each phase, were shorter when the sizes of A and B were 14 and 25 μl of a 0.6 M sucrose solution than when they were 25 and 100 μl of the same solution. The ratio of the indifference delays was significantly smaller than that predicted on the basis of an assumed linear relation between reinforcer size and instantaneous reinforcer value, consistent with a previous proposal that this relation may be hyperbolic in form. Estimates of the rate of delay discounting based on the ratio of the two indifference delays (mean, 0.08 s(-1)) were similar to values obtained previously using different intertemporal choice protocols. Estimates of the size-sensitivity parameter (mean 113 μl) were similar to estimates recently derived from performance on progressive-ratio schedules. In both phases of the experiment, adjusting delays in successive blocks of trials were analyzed using the Fourier transform. The power spectrum obtained from individual rats had a dominant frequency that corresponded to a period of oscillation of the adjusting delay between 30 and 100 trial blocks (mean, 78). Power in the dominant frequency band was highest in the early sessions of the first phase and declined with extended training. It is suggested that this experimental protocol may have utility in neurobehavioral studies of intertemporal choice.