The role of dopamine in reinforcement: changes in reinforcement sensitivity induced by D1-type, D2-type, and nonselective dopamine receptor agonists

Dose-dependent changes in sensitivity to reinforcement were found when rats were treated with low, moderate, and high doses of the partial dopamine D1-type receptor agonist SKF38393 and with the nonselective dopamine agonist apomorphine, but did not change when rats were treated with similar doses of the selective dopamine D2-type receptor agonist quinpirole. Estimates of bias did not differ significantly across exposure to SKF38393 or quinpirole, but did change significantly at the high dose of apomorphine. Estimates of goodness of fit (r2) did not change significantly during quinpirole exposure. Poor goodness of fit was obtained for the high doses of SKF38393 and apomorphine. Decrements in absolute rates of responding were observed at the high dose of quinpirole and at the moderate and high doses of SKF38393 and apomorphine. Changes in r2 and absolute responding may be due to increases in stereotyped behavior during SKF38393 and apomorphine exposure that, in contrast to quinpirole, were distant from the response lever. The present data provide evidence that sensitivity to reward is affected more strongly by dopamine D1-like receptors rather than D2-like receptors, consistent with evidence from other studies investigating consummatory dopamine behavior and the tonic/phasic dopamine hypothesis.     

Pilocarpine seizures cause age-dependent impairment in auditory location discrimination

Children who have status epilepticus have continuous or rapidly repeating seizures that may be life-threatening and may cause life-long changes in brain and behavior. The extent to which status epilepticus causes deficits in auditory discrimination is unknown. A naturalistic auditory location discrimination method was used to evaluate this question using an animal model of status epilepticus. Male Sprague-Dawley rats were injected with saline on postnatal day (P) 20, or a convulsant dose of pilocarpine on P20 or P45. Pilocarpine on either day induced status epilepticus; status epilepticus at P45 resulted in CA3 cell loss and spontaneous seizures, whereas P20 rats had no cell loss or spontaneous seizures. Mature rats were trained with sound-source location and sound-silence discriminations. Control (saline P20) rats acquired both discriminations immediately. In status epilepticus (P20) rats, acquisition of the sound-source location discrimination was moderately impaired. Status epilepticus (P45) rats failed to acquire either sound-source location or sound-silence discriminations. Status epilepticus in rat causes an age-dependent, long-term impairment in auditory discrimination. This impairment may explain one cause of impaired auditory location discrimination in humans.     

Earning and obtaining reinforcers under concurrent interval scheduling

Contingencies of reinforcement specify how reinforcers are earned and how they are obtained. Ratio contingencies specify the number of responses that earn a reinforcer, and the response satisfying the ratio requirement obtains the earned reinforcer. Simple interval schedules specify that a certain time earns a reinforcer, which is obtained by the first response after the interval. The earning of reinforcers has been overlooked, perhaps because simple schedules confound the rates of earning reinforcers with the rates of obtaining reinforcers. In concurrent variable-interval schedules, however, spending time at one alternative earns reinforcers not only at that alternative, but at the other alternative as well. Reinforcers earned for delivery at the other alternative are obtained after changing over. Thus the rates of earning reinforcers are not confounded with the rate of obtaining reinforcers, but the rates of earning reinforcers are the same at both alternatives, which masks their possibly differing effects on preference. Two experiments examined the separate effects of earning reinforcers and of obtaining reinforcers on preference by using concurrent interval schedules composed of two pairs of stay and switch schedules (MacDonall, 2000). In both experiments, the generalized matching law, which is based on rates of obtaining reinforcers, described responding only when rates of earning reinforcers were the same at each alternative. An equation that included both the ratio of the rates of obtaining reinforcers and the ratio of the rates of earning reinforcers described the results from all conditions from each experiment.     

Effects of adding a second reinforcement alternative: implications for Herrnstein's interpretation of r(e)

Herrnstein's hyperbola describes the relation between response rate and reinforcer rate on variable-interval (VI) schedules. According to Herrnstein's (1970) interpretation, the parameter r(e) represents the reinforcer rate extraneous to the alternative to which the equation is fitted (the target alternative). The hyperbola is based on an assumption that extraneous reinforcer rate remains constant with changes in reinforcer rate on the target alternative (the constant-r(e) assumption) and that matching with no bias and perfect sensitivity occurs between response and reinforcer ratios. In the present experiment, 12 rats pressed levers for food on a series of 10 VI schedules arranged on the target alternative. Across conditions, six VI values and extinction were arranged on a second alternative. Reinforcer rate on the second alternative, r2, negatively covaried with reinforcer rate on the target alternative for five of the six VI values on the second alternative, and significant degrees of bias and undermatching occurred in response ratios. Given covariation of reinforcer rate on the second and target alternatives, the constant-r(e) assumption can be maintained only by assuming that reinforcer rate from unmeasured background sources, rb, covaries with reinforcer rate on the second alternative such that their sum, r(e), remains constant. In a single-schedule arrangement, however, r(e) equals rb and thus rb is assumed to remain constant, forcing a conceptual inconsistency between single- and concurrent-schedule arrangements. Furthermore, although an alternative formulation of the hyperbola can account for variations in bias and sensitivity, the modified equation also is based on the constant-r(e) assumption and therefore suffers from the same logical problem as the hyperbola when reinforcer rate on the second alternative covaries with reinforcer rate on the target alternative.     

Effects of reinforcer probability, delay, and response requirements on the choices of rats and pigeons: possible species differences

In Experiment 1 with rats, a left lever press led to a 5-s delay and then a possible reinforcer. A right lever press led to an adjusting delay and then a certain reinforcer. This delay was adjusted over trials to estimate an indifference point, or a delay at which the two alternatives were chosen about equally often. Indifference points increased as the probability of reinforcement for the left lever decreased. In some conditions with a 20% chance of food, a light above the left lever was lit during the 5-s delay on all trials, but in other conditions, the light was only lit on those trials that ended with food. Unlike previous results with pigeons, the presence or absence of the delay light on no-food trials had no effect on the rats' indifference points. In other conditions, the rats showed less preference for the 20% alternative when the time between trials was longer. In Experiment 2 with rats, fixed-interval schedules were used instead of simple delays, and the presence or absence of the fixed-interval requirement on no-food trials had no effect on the indifference points. In Experiment 3 with rats and Experiment 4 with pigeons, the animals chose between a fixed-ratio 8 schedule that led to food on 33% of the trials and an adjusting-ratio schedule with food on 100% of the trials. Surprisingly, the rats showed less preference for the 33% alternative in conditions in which the ratio requirement was omitted on no-food trials. For the pigeons, the presence or absence of the ratio requirement on no-food trials had little effect. The results suggest that there may be differences between rats and pigeons in how they respond in choice situations involving delayed and probabilistic reinforcers.     

Within-subject testing of the signaled-reinforcement effect on operant responding as measured by response rate and resistance to change

Response rates under random-interval schedules are lower when a brief (500 ms) signal accompanies reinforcement than when there is no signal. The present study examined this signaled-reinforcement effect and its relation to resistance to change. In Experiment 1, rats responded on a multiple random-interval 60-s random-interval 60-s schedule, with signaled reinforcement in only one component. Response resistance to alternative reinforcement, prefeeding, and extinction was compared between these components. Lower response rates, and greater resistance to change, occurred in the component with the reinforcement signal. In Experiment 2, response rates and resistance to change were compared after training on a multiple random-interval 60-s random-interval 60-s schedule in which reinforcer delivery was unsignaled in one component and a response-produced uncorrelated stimulus was presented in the other component. Higher response rates and greater resistance to change occurred with the uncorrelated stimulus. These results highlight the significance of considering the effects of an uncorrelated signal when used as a control condition, and challenge accounts of resistance to change that depend solely on reinforcer rate.     

Building blocks of self-control: increased tolerance for delay with bundled rewards

Impulsive choice can be defined as temporary preference for a smaller-sooner reward (SS) over a larger-later reward (LL). Hyperbolic discounting implies that impulsive choices will occur less when organisms choose between a series of SSs versus LLs all at once than when they choose between single SS versus LL pairs. Eight rats were exposed to two conditions of an intertemporal choice paradigm using sucrose solution as reward. In both conditions, the LL was 150 microl delayed by 3 s, while the SS was an immediate reward that ranged from 25-150 microl across sessions. Preference for the LL was greater when the chosen reward was automatically delivered three times in succession (bundled) than when it was chosen singly and delivered after each choice. For each of the 8 rats, the estimated SS amount that produced indifference was higher in the bundled condition than in the single condition. Because bundling in humans may be based on the perception that one's current choice is predictive of future choices, the data presented here may demonstrate an important building block of self-control.     

Drug discrimination using a conditioned taste-aversion paradigm in rhesus monkeys.

The development of drug discrimination was assessed in rhesus monkeys using the conditioned taste-aversion paradigm. Monkeys were initially trained to respond under a fixed-ratio 30-response schedule of food-pellet delivery to assess the rate-decreasing effects of alprazolam (0.03 to 3 mg/kg, i.m., 60 min presession). Alprazolam decreased responding at doses greater than 0.1 mg/kg. Discriminative stimulus effects of alprazolam were then assessed by giving 0.03 mg/kg before sessions in which 1.8 mEq/kg lithium chloride was given immediately after the session (alprazolam/lithium session). On intervening days, saline was given before and after the session (saline/saline session). Rates of responding decreased over successive alprazolam/lithium sessions and also during the saline/saline session that immediately followed an alprazolam/lithium session. During subsequent saline/saline sessions, rates of responding returned to levels near baseline rates within two to four sessions. The discriminative stimulus effects of alprazolam were then assessed by giving 0.1 mg/kg before sessions in which 1 mg/kg d-amphetamine was given immediately after the session (alprazolam/d-amphetamine session). Rates of responding decreased during subsequent alprazolam/d-amphetamine sessions in drug-experienced monkeys, but did not decrease during intervening saline/saline sessions. These findings demonstrate that drug stimuli associated with postsession drug injections can rapidly develop control over behavior and suggest that similar methods be explored in the assessment of drug discrimination.     

A quantitative interresponse-time analysis of DRL performance differentiates similar effects of the antidepressant desipramine and the novel anxiolytic gepirone.

We describe an interresponse-time analysis of performance on a differential-reinforcement-of-low-rate 72-s schedule. This analysis compares the obtained interresponse-time distribution of individual rats to a corresponding random interresponse-time distribution. The random interresponse-time distribution is a negative exponential probability function; it predicts the relative distribution of interresponse times if the rat emitted the same number of responses randomly (i.e., with a constant probability) with respect to time. The analysis provides quantitative measures of peak location and dispersion of the interresponse times toward random performance. In Experiment 1, an unexpected outcome of this analysis was that the rats would have obtained more reinforcers had they responded at the same rate but randomly. Based on the interresponse-time analysis in Experiment 1, it was shown that rats trained on the differential-reinforcement-of-low-rate 72-s schedule could increase the number of reinforcers obtained in two ways: first, by a coherent shift of the interresponse-time distribution toward longer durations and, second, by dispersal of the interresponse times toward a random interresponse-time distribution. Experiment 2 applied the analysis described in Experiment 1 to the effects of desipramine and gepirone. Both drugs decreased response rate and increased reinforcement rate, but their effects on the distribution of interresponse times were different. The increase in reinforcement rate observed with desipramine was accompanied by a coherent shift of the reinforcement rate observed with gepirone was accompanied by dispersal of the interresponse-time distribution toward the random negative exponential prediction.     

Response acquisition with delayed reinforcement: a comparison of two-lever procedures.

Groups of 8 experimentally naive rats were exposed during 8-hr sessions to resetting delay procedures in which responses on one lever (the reinforcement lever) produced water after a delay of 8, 16, 32, or 64 s. For rats in one condition, responses on a second (no-consequences) lever had no programmed consequences. For rats in another condition, responses on a second (cancellation) lever during a delay initiated by a response on the reinforcement lever prevented delivery of the scheduled reinforcer; responses on the cancellation lever at other times had no programmed consequences. Under both conditions and at all delays, most subjects emitted more responses on the reinforcement lever than did control rats that never received water emitted on either lever. At 8-s delays, both conditions engendered substantially more responding on the reinforcement lever than on the other lever, and performance closely resembled that of immediate-reinforcement controls. At delays of 16 and 32 s, however, there was clear differential responding on the two levers under the cancellation condition but not under the other condition. When the delay was 64 s, differential responding on the two levers did not occur consistently under either condition. These findings provide strong evidence that the behavior of rats is sensitive to consequences delayed by 8, 16, and 32 s, but only equivocal evidence of such sensitivity to consequences delayed 64 s. They also indicate that acquisition depends, in part, on the measure of performance used to index it.