Delay discounting as impaired valuation: Delayed rewards in an animal obesity model

Obesity is a major public health problem, which, like many forms of addiction, is associated with an elevated tendency to choose smaller immediate rather than larger delayed rewards, a response pattern often referred to as excessive delay discounting. Although some accounts of delay discounting conceptualize this process as impulsivity (placing the emphasis on overvaluing the smaller immediate reward), others have conceptualized delay discounting as an executive function (placing the emphasis on delayed rewards failing to retain their value). The present experiments used a popular animal model of obesity that has been shown to discount delayed rewards at elevated rates (i.e., obese Zucker rats) to test two predictions that conceptualize delay discounting as executive function. In the first experiment, acquisition of lever pressing with delayed rewards was compared in obese versus lean Zucker rats. Contrary to predictions based on delay discounting as executive function, obese Zucker rats learned to press the lever more quickly than controls. In the second experiment, progressive ratio breakpoints (a measure of reward efficacy) with delayed rewards were compared in obese versus lean Zucker rats. Contrary to the notion that obese rats fail to value delayed rewards, the obese Zucker rats' breakpoints were (at least) as high as those of the lean Zucker rats.     

Behavioral effects of delayed timeouts from reinforcement

Timeouts are sometimes used in applied settings to reduce target responses, and in some circumstances delays are unavoidably imposed between the onset of a timeout and the offset of the response that produces it. The present study examined the effects of signaled and unsignaled timeouts in rats exposed to concurrent fixed‐ratio 1 fixed‐ratio 1 schedules of food delivery, where each response on one lever, the location of which changed across conditions, produced both food and a delayed 10‐s timeout. Delays of 0 to 38 s were examined. Delayed timeouts often, but not always, substantially reduced the number of responses emitted on the lever that produced timeouts relative to the number emitted on the lever that did not produce timeouts. In general, greater sensitivity was observed to delayed timeouts when they were signaled. These results demonstrate that delayed timeouts, like other delayed consequences, can affect behavior, albeit less strongly than immediate consequences.     

The effects of 100 dB 1‐kHz and 22‐kHz tones as punishers on lever pressing in rats

Aversive control is an important yet understudied process of learning. One reason aversive control may be relatively understudied is ethical concerns about painful stimuli (e.g., electric shock). High decibel broad‐band noise and 22‐kHz vocalizations both demonstrably affect rodent behavior while not necessarily being painful. The goal of this study was to determine if 100‐dB 22‐kHz‐pure tones were differentially more effective in reducing operant response rates in rats. We examined whether 22‐kHz pure tones would function as aversive stimuli, specifically as positive punishers. The effects of response‐dependent as well as continuously presented 22‐kHz and 1‐kHz tones on rate of response maintained by variable interval 30‐s food deliveries were assessed across several conditions. We found that response rates were lower when tones were presented response dependently than when tones were presented continuously throughout a session. We also found that the lower response rates obtained with response‐dependent 22‐kHz tones were not significantly different from response rates obtained with response‐dependent 1‐kHz tones. The primary conclusion of this experiment is that both 1‐kHz and 22‐kHz tones functioned as punishers, but that the 22‐kHz tones were not differentially more effective in reducing response rate.     

Modulating effects in learned helplessness of dyadic dominance-submission relations

In this experiment, learned helplessness was studied from an ethological perspective by examining individual differences in social dominance and its influence on the effects of helplessness. Ninety animals were used, 30 randomly selected and 60 selected because of their clear dominance or submission. Each condition (dominant, submissive, and random) was distributed in three subgroups corresponding to the triadic design. The test consisted of an escape/avoidance task. The results showed that the animals in the uncontrollable condition performed worse than those in the controllable and no treatment conditions. Social submission and dominance reduced vulnerability of the subjects against learned helplessness. Submission had a facilitating effect on subsequent learning, independently of whether pretreatment was controllability or uncontrollability. Learned mastery was observed in the submissive condition, because submission benefited the subjects in the controllable condition in comparison with the untreated subjects, and dominance impaired the subjects in the controllable condition.     

ISOLATING BEHAVIORAL MECHANISMS OF INTERTEMPORAL CHOICE: NICOTINE EFFECT ON DELAY DISCOUNTING AND AMOUNT SENSITIVITY

Many drugs of abuse produce changes in impulsive choice, that is, choice for a smaller—sooner reinforcer over a larger—later reinforcer. Because the alternatives differ in both delay and amount, it is not clear whether these drug effects are due to the differences in reinforcer delay or amount. To isolate the effects of delay, we used a titrating delay procedure. In phase 1, 9 rats made discrete choices between variable delays (1 or 19 s, equal probability of each) and a delay to a single food pellet. The computer titrated the delay to a single food pellet until the rats were indifferent between the two options. This indifference delay was used as the starting value for the titrating delay for all future sessions. We next evaluated the acute effects of nicotine (subcutaneous 1.0, 0.3, 0.1, and 0.03 mg/kg) on choice. If nicotine increases delay discounting, it should have increased preference for the variable delay. Instead, nicotine had very little effect on choice. In a second phase, the titrated delay alternative produced three food pellets instead of one, which was again produced by the variable delay (1 s or 19 s) alternative. Under this procedure, nicotine increased preference for the one pellet alternative. Nicotine‐induced changes in impulsive choice are therefore likely due to differences in reinforcer amount rather than differences in reinforcer delay. In addition, it may be necessary to include an amount sensitivity parameter in any mathematical model of choice when the alternatives differ in reinforcer amount.     

OBSERVING RESPONSES AND SERIAL STIMULI: SEARCHING FOR THE REINFORCING PROPERTIES OF THE S−

The control exerted by a stimulus associated with an extinction component (S−) on observing responses was determined as a function of its temporal relation with the onset of the reinforcement component. Lever pressing by rats was reinforced on a mixed random-interval extinction schedule. Each press on a second lever produced stimuli associated with the component of the schedule in effect. In Experiment 1 a response-dependent clock procedure that incorporated different stimuli associated with an extinction component of a variable duration was used. When a single S− was presented throughout the extinction component, the rate of observing remained relatively constant across this component. In the response-dependent clock procedure, observing responses increased from the beginning to the end of the extinction component. This result was replicated in Experiment 2, using a similar clock procedure but keeping the number of stimuli per extinction component constant. We conclude that the S− can function as a conditioned reinforcer, a neutral stimulus or as an aversive stimulus, depending on its temporal location within the extinction component.     

Compatibilism &; desert: critical comments on <Emphasis Type="Italic">four views on free will</Emphasis>

In this paper I offer from a source compatibilist’s perspective a critical discussion of Four Views on Free Will by John Martin Fischer, Robert Kane, Derk Pereboom, and Manuel Vargas. Sharing Fischer’s semi-compatibilist view, I propose modifications to his arguments while resisting his coauthors’ objections. I argue against Kane that he should give up the requirement that a free and morally responsible agent be able to do otherwise (in relevant cases). I argue against Pereboom that his famed manipulation argument be resisted by contending that the agents in it are free and responsible. And I also argue against Vargas by challenging the sense in which his revisionist thesis differs from a position like Fischer’s and mine. I close by reflecting on the nature of desert. All seem to assume it is central to the debate, but what is it?     

Effects of qualitatively different reinforcers on the parameters of the response-strength equation.

This experiment examined the relationship between two qualitatively different reinforcers and the parameters of a quantitative model of reinforced responding, referred to as the response-strength equation or the Herrnstein equation. A group of rats was first food deprived and later water deprived. An 11.5% sucrose solution served as the reinforcer in the food-deprivation condition, and water was the reinforcer in the water-deprivation condition. Each experimental session consisted of a series of seven variable-interval schedules, providing reinforcement rates that varied between 20 and 1,200 reinforcers per hour. The response rates increased in a negatively accelerating function in a manner consistent with the response-strength equation. This equation has two fitted parameters, k and Re. According to one theory, the k parameter is a measure of motor performance, and Re is indicative of the relative reinforcement efficacy of the background uncontrollable sources of reinforcement in relation to the experimentally arranged reinforcer. In this study, k did not change as a result of the different reinforcers, but Re was significantly larger in the sucrose-reinforcement condition. These results are consistent with the interpretation that k and Re measure two independent and experimentally distinguishable parameters and provide further evidence that absolute response rate is a function of relative reinforcement rate, as implied by the derivation of the response-strength equation based on the matching law.     

Reinforcer magnitude (sucrose concentration) and the matching law theory of response strength

This experiment investigated the relationship between reinforcer magnitude (sucrose concentration) and response rate. The purpose was to evaluate the behavior of two parameters of an equation that predicts absolute response rate as a function of reinforcement rate and two free parameters. According to Herrnstein's (1970) theory of reinforced behavior, one parameter of this "response-strength equation" measures the efficacy of the reinforcer maintaining responding and the other parameter measures motoric components of response rate, such as response duration. Seven rats served as subjects. Experimental sessions consisted of a series of five different variable-interval schedules of reinforcement, each in effect for 5 minutes. Within each session, obtained reinforcement rates varied over more than a 30-fold range, from about 20 per hour to 700 per hour. The reinforcer was sucrose solution, and, between sessions, its concentration was varied from 0.0 to 0.64 molar (0 to 21.9%). For sucrose concentrations of 0.16 to 0.64 m, response rate was a negatively accelerated function of reinforcement rate. Increases in sucrose concentration increased response rates maintained by low but not high reinforcement rates. This pattern of changes corresponds to a change in the reinforcement-efficacy parameter of the response-strength equation. In contrast, the motor-performance parameter did not change as a function of sucrose concentration. These findings are inconsistent with the results of a similar study (Bradshaw, Szabadi, & Bevan, 1978) but support Herrnstein's theory of reinforced behavior.     

An eight-alternative concurrent schedule: foraging in a radial maze.

In two experiments conducted in an eight-arm radial maze, food pellets were delivered when a photocell beam was broken at the end of each arm via a nose poke, according to either fixed-interval or random-interval schedules of reinforcement, with each arm providing a different frequency of reinforcement. The behavior of rats exposed to these procedures was well described by the generalized matching law; that is, the relationships between log behavior ratios and log pellet ratios were approximated by linear functions. The slopes of these log-log functions, an index of sensitivity to reinforcement frequency, were greatest for nose pokes, intermediate for time spent in an arm, and least for arm entries. Similar results were obtained with both fixed-interval and random-interval schedules. Addition of a 10-s changeover delay in both experiments eliminated the slope differentials between nose pokes and time spent in an arm by reducing the slopes of the nose-poke functions. These results suggest that different aspects of foraging may be differentially sensitive to reinforcement frequency. With concurrent fixed-interval schedules, the degree of temporal control exerted by individual fixed-interval schedules was directly related to reinforcement frequency.