The effects of behavioral history on response acquisition with immediate and delayed reinforcement

Effects of prior exposure to the experimental chamber with levers present or absent and variable-time (VT) 60-s water deliveries arranged during one, five, or no 1-hr sessions were examined in rats during a 6-hr response-acquisition session in which presses on one lever produced water delivery immediately or after a 15-s resetting delay, and presses on the other lever canceled scheduled water deliveries. Response acquisition was (a) slower to occur when water deliveries were delayed, (b) most consistent in groups that had received five VT sessions, and (c) impaired by the presence of levers only when there had been five VT sessions and water deliveries were delayed during the acquisition session.     

Pilocarpine seizures cause age-dependent impairment in auditory location discrimination

Children who have status epilepticus have continuous or rapidly repeating seizures that may be life-threatening and may cause life-long changes in brain and behavior. The extent to which status epilepticus causes deficits in auditory discrimination is unknown. A naturalistic auditory location discrimination method was used to evaluate this question using an animal model of status epilepticus. Male Sprague-Dawley rats were injected with saline on postnatal day (P) 20, or a convulsant dose of pilocarpine on P20 or P45. Pilocarpine on either day induced status epilepticus; status epilepticus at P45 resulted in CA3 cell loss and spontaneous seizures, whereas P20 rats had no cell loss or spontaneous seizures. Mature rats were trained with sound-source location and sound-silence discriminations. Control (saline P20) rats acquired both discriminations immediately. In status epilepticus (P20) rats, acquisition of the sound-source location discrimination was moderately impaired. Status epilepticus (P45) rats failed to acquire either sound-source location or sound-silence discriminations. Status epilepticus in rat causes an age-dependent, long-term impairment in auditory discrimination. This impairment may explain one cause of impaired auditory location discrimination in humans.     

Earning and obtaining reinforcers under concurrent interval scheduling

Contingencies of reinforcement specify how reinforcers are earned and how they are obtained. Ratio contingencies specify the number of responses that earn a reinforcer, and the response satisfying the ratio requirement obtains the earned reinforcer. Simple interval schedules specify that a certain time earns a reinforcer, which is obtained by the first response after the interval. The earning of reinforcers has been overlooked, perhaps because simple schedules confound the rates of earning reinforcers with the rates of obtaining reinforcers. In concurrent variable-interval schedules, however, spending time at one alternative earns reinforcers not only at that alternative, but at the other alternative as well. Reinforcers earned for delivery at the other alternative are obtained after changing over. Thus the rates of earning reinforcers are not confounded with the rate of obtaining reinforcers, but the rates of earning reinforcers are the same at both alternatives, which masks their possibly differing effects on preference. Two experiments examined the separate effects of earning reinforcers and of obtaining reinforcers on preference by using concurrent interval schedules composed of two pairs of stay and switch schedules (MacDonall, 2000). In both experiments, the generalized matching law, which is based on rates of obtaining reinforcers, described responding only when rates of earning reinforcers were the same at each alternative. An equation that included both the ratio of the rates of obtaining reinforcers and the ratio of the rates of earning reinforcers described the results from all conditions from each experiment.     

Effects of adding a second reinforcement alternative: implications for Herrnstein's interpretation of r(e)

Herrnstein's hyperbola describes the relation between response rate and reinforcer rate on variable-interval (VI) schedules. According to Herrnstein's (1970) interpretation, the parameter r(e) represents the reinforcer rate extraneous to the alternative to which the equation is fitted (the target alternative). The hyperbola is based on an assumption that extraneous reinforcer rate remains constant with changes in reinforcer rate on the target alternative (the constant-r(e) assumption) and that matching with no bias and perfect sensitivity occurs between response and reinforcer ratios. In the present experiment, 12 rats pressed levers for food on a series of 10 VI schedules arranged on the target alternative. Across conditions, six VI values and extinction were arranged on a second alternative. Reinforcer rate on the second alternative, r2, negatively covaried with reinforcer rate on the target alternative for five of the six VI values on the second alternative, and significant degrees of bias and undermatching occurred in response ratios. Given covariation of reinforcer rate on the second and target alternatives, the constant-r(e) assumption can be maintained only by assuming that reinforcer rate from unmeasured background sources, rb, covaries with reinforcer rate on the second alternative such that their sum, r(e), remains constant. In a single-schedule arrangement, however, r(e) equals rb and thus rb is assumed to remain constant, forcing a conceptual inconsistency between single- and concurrent-schedule arrangements. Furthermore, although an alternative formulation of the hyperbola can account for variations in bias and sensitivity, the modified equation also is based on the constant-r(e) assumption and therefore suffers from the same logical problem as the hyperbola when reinforcer rate on the second alternative covaries with reinforcer rate on the target alternative.     

Effects of reinforcer probability, delay, and response requirements on the choices of rats and pigeons: possible species differences

In Experiment 1 with rats, a left lever press led to a 5-s delay and then a possible reinforcer. A right lever press led to an adjusting delay and then a certain reinforcer. This delay was adjusted over trials to estimate an indifference point, or a delay at which the two alternatives were chosen about equally often. Indifference points increased as the probability of reinforcement for the left lever decreased. In some conditions with a 20% chance of food, a light above the left lever was lit during the 5-s delay on all trials, but in other conditions, the light was only lit on those trials that ended with food. Unlike previous results with pigeons, the presence or absence of the delay light on no-food trials had no effect on the rats' indifference points. In other conditions, the rats showed less preference for the 20% alternative when the time between trials was longer. In Experiment 2 with rats, fixed-interval schedules were used instead of simple delays, and the presence or absence of the fixed-interval requirement on no-food trials had no effect on the indifference points. In Experiment 3 with rats and Experiment 4 with pigeons, the animals chose between a fixed-ratio 8 schedule that led to food on 33% of the trials and an adjusting-ratio schedule with food on 100% of the trials. Surprisingly, the rats showed less preference for the 33% alternative in conditions in which the ratio requirement was omitted on no-food trials. For the pigeons, the presence or absence of the ratio requirement on no-food trials had little effect. The results suggest that there may be differences between rats and pigeons in how they respond in choice situations involving delayed and probabilistic reinforcers.     

Conditioned ethanol aversion in rats induced by voluntary wheel running,forced swimming,and electric shock: An implication for aversion therapy of alcoholism

This study was planned to demonstrate rats’ acquisition of aversion to ethanol solution consumed before voluntary running, forced swimming, or electric shock delivery. Wistar rats under water deprivation were allotted to four groups of eight rats each, and all rats were allowed to drink 5% ethanol solution for 15 min. Immediately after the ethanol drinking, rats of Group Run were put into the individual running wheels for 15 min, those of Group Swim were put into the individual swimming pools for 15 min, those of Group Shock received electric shocks for 15 min (15 0.45-mA shocks of 0.7s with the intershock interval of 1 min) in the individual small chambers, and those of Group Control were directly returned back to the home cages. This procedure was repeated for six days, followed by a two-day choice test of ethanol aversion where a bottle containing 5% ethanol solution and a bottle of tap water were simultaneously presented for 15 min. In the test, Groups Run, Swim, and Shock drank ethanol solution significantly less than tap water, while Group Control drank both fluids equally. The effects of running, swimming, and shock were equivalent. The successful demonstration of acquired ethanol aversion induced by exercise (running and swimming) or shock in rats suggests an avenue for clinical application of exercise and shock treatments for human alcoholics, though there are many issues to be resolved before the practical use.     

Building blocks of self-control: increased tolerance for delay with bundled rewards

Impulsive choice can be defined as temporary preference for a smaller-sooner reward (SS) over a larger-later reward (LL). Hyperbolic discounting implies that impulsive choices will occur less when organisms choose between a series of SSs versus LLs all at once than when they choose between single SS versus LL pairs. Eight rats were exposed to two conditions of an intertemporal choice paradigm using sucrose solution as reward. In both conditions, the LL was 150 microl delayed by 3 s, while the SS was an immediate reward that ranged from 25-150 microl across sessions. Preference for the LL was greater when the chosen reward was automatically delivered three times in succession (bundled) than when it was chosen singly and delivered after each choice. For each of the 8 rats, the estimated SS amount that produced indifference was higher in the bundled condition than in the single condition. Because bundling in humans may be based on the perception that one's current choice is predictive of future choices, the data presented here may demonstrate an important building block of self-control.     

Facilitation of Sodium Aversion Learning in Sodium-Deprived Rats

Two experiments with rats explored the effects of sodium deprivation induced by furosemide injections upon acquisition of taste aversion to sodium chloride (NaCl). In Experiment 1, rats under either sodium deprivation or a balanced nutrition condition were given access to a limited amount of NaCl solution prior to poisoning. When all rats were tested under sodium repletion, the previously sodium-deprived rats consumed less NaCl than did the nondeprived rats. This finding was replicated in Experiment 2A in which ingestion of a compound solution of NaCl and hydrochloric acid (HCl) was followed by poisoning. Consumption of HCl, however, showed the opposite pattern: the sodium-deprived rats drank more HCl than did the nondeprived rats, a result that was replicated in Experiment 2B. These results suggest that sodium deprivation strengthens the salience of NaCl, thereby facilitating acquisition of aversion to this taste and strongly overshadowing that to a simultaneously presented taste.     

A quantitative interresponse-time analysis of DRL performance differentiates similar effects of the antidepressant desipramine and the novel anxiolytic gepirone.

We describe an interresponse-time analysis of performance on a differential-reinforcement-of-low-rate 72-s schedule. This analysis compares the obtained interresponse-time distribution of individual rats to a corresponding random interresponse-time distribution. The random interresponse-time distribution is a negative exponential probability function; it predicts the relative distribution of interresponse times if the rat emitted the same number of responses randomly (i.e., with a constant probability) with respect to time. The analysis provides quantitative measures of peak location and dispersion of the interresponse times toward random performance. In Experiment 1, an unexpected outcome of this analysis was that the rats would have obtained more reinforcers had they responded at the same rate but randomly. Based on the interresponse-time analysis in Experiment 1, it was shown that rats trained on the differential-reinforcement-of-low-rate 72-s schedule could increase the number of reinforcers obtained in two ways: first, by a coherent shift of the interresponse-time distribution toward longer durations and, second, by dispersal of the interresponse times toward a random interresponse-time distribution. Experiment 2 applied the analysis described in Experiment 1 to the effects of desipramine and gepirone. Both drugs decreased response rate and increased reinforcement rate, but their effects on the distribution of interresponse times were different. The increase in reinforcement rate observed with desipramine was accompanied by a coherent shift of the reinforcement rate observed with gepirone was accompanied by dispersal of the interresponse-time distribution toward the random negative exponential prediction.     

Response acquisition with delayed reinforcement: a comparison of two-lever procedures.

Groups of 8 experimentally naive rats were exposed during 8-hr sessions to resetting delay procedures in which responses on one lever (the reinforcement lever) produced water after a delay of 8, 16, 32, or 64 s. For rats in one condition, responses on a second (no-consequences) lever had no programmed consequences. For rats in another condition, responses on a second (cancellation) lever during a delay initiated by a response on the reinforcement lever prevented delivery of the scheduled reinforcer; responses on the cancellation lever at other times had no programmed consequences. Under both conditions and at all delays, most subjects emitted more responses on the reinforcement lever than did control rats that never received water emitted on either lever. At 8-s delays, both conditions engendered substantially more responding on the reinforcement lever than on the other lever, and performance closely resembled that of immediate-reinforcement controls. At delays of 16 and 32 s, however, there was clear differential responding on the two levers under the cancellation condition but not under the other condition. When the delay was 64 s, differential responding on the two levers did not occur consistently under either condition. These findings provide strong evidence that the behavior of rats is sensitive to consequences delayed by 8, 16, and 32 s, but only equivocal evidence of such sensitivity to consequences delayed 64 s. They also indicate that acquisition depends, in part, on the measure of performance used to index it.