Effects of reinforcer delays on choice as a function of income level

Three rats earned their daily food ration by responding during individual trials either on a lever that delivered one food pellet immediately or on a second lever that delivered three pellets after a delay that was continuously adjusted to ensure substantial responding to both alternatives. Choice of the delayed reinforcer increased when the number of trials per session was reduced. This result suggests that models seeking closure on choice effects must include a parameter reflecting how preference changes with sessionwide income. Moreover, models positing that reinforcer probability and immediacy (1/delay) function equivalently in choice are called into question by the finding that probability and immediacy produce opposing effects when income level is changed.     

An animal model of excessive eating: schedule-induced hyperphagia in food-satiated rats.

Nineteen rats were maintained throughout the experiment on ad libitum wet mash and water and were trained to press a lever on fixed-interval or fixed-ratio schedules of reinforcement with electrical brain stimulation. Fourteen rats ate at least 150% more mash during intermittent reinforcement sessions than during baseline, massed reinforcement control, and/or extinction sessions. In a 3-hr session, 11 of those 14 consumed more than 22 g of wet mash (13 g dry weight), the equivalent of nearly half an animal's daily food intake. In subsequent control sessions, the electrodes did not support stimulus-bound eating despite attempts to make stimulation parameters optimal. These results indicate that the eating was schedule induced or adjunctive, and suggest that the procedure may provide an animal model of excessive nonregulatory eating that contributes to obesity in humans.     

Food and water intake as functions of resource consumption costs in a closed economy.

In two experiments, rats living in a closed economy were offered continuous, concurrent access to four resources: food, water, a nest, and a running wheel. Costs of consuming food and water were imposed with bar-press requirements, and the price of either one or both resources was raised. As the consumption cost increased, less was consumed in each bout of resource use. Bout frequency increased, but not sufficiently to compensate for the fall in bout size, and total intake fell. Food and water tended to be complementary resources, in that as intake of one fell with its price, intake of the other also decreased. This interaction was accounted for by the defense of the ratio of body water to lean body mass. As amount consumed decreased, increases in feed efficiency (weight gain per unit of food ingested) and the use of stored calories compensated for the reduced energy intake. There was evidence of competition between feeding and drinking at the higher costs: When both commodities were expensive, the decline in the intake of each one was greater than when only one commodity was expensive. Although the time spent nesting, running, and in unmonitored activity was adjusted when feeding or drinking took more of the rat's day, there was no particular activity that was sacrificed.     

Preference for signaled over unsignaled shock schedules: Ruling out asymmetry and response fixation as factors

Experiment 1 tested whether a “symmetrical” choice procedure yields results different from those previously reported using the “unidirectional” standard changeover procedure (e.g., Badia & Culbertson, 1972). Subjects could change at any time from unsignaled to signaled shock by pressing a lever and from signaled to unsignaled shock by pressing a second lever. Results were identical to those of the standard procedure and showed that the standard procedure is fully adequate. Experiment 2 tested whether choice of high density signaled shock over low-density unsignaled shock (Badia, Coker, & Harsh, 1973) resulted from initial training with equal-density schedules. Subjects were trained and tested with signaled shock twice as dense as unsignaled shock. Three of four subjects strongly preferred the signaled condition, thus ruling out carry-over and “response fixation” as alternative explanations.     

Signal modality and choice between signaled and unsignaled food

Choice between signaled and unsignaled response-independent food schedules was assessed in three experiments using a commitment procedure. In Experiment 1, subjects tested with a 5-s visual signal consistently changed from the signaled to the unsignaled schedule. Changing from the unsignaled to the signaled schedule was observed only occasionally and only at low levels. The same outcome was observed in Experiment 2 with different types of visual signals and with different stimulus combinations identifying the signal period, the signal-absent period, and the unsignaled schedule. In Experiment 3 the visual signal was replaced with an auditory signal for four of the subjects tested in Experiment 2. The subjects then changed from the unsignaled to the signaled schedule or showed a substantial reduction in choice for the unsignaled schedule. The data were assessed using a conditioned-reinforcement interpretation of choice.     

Schedule-induced polydipsia in rats living in an operant environment

The effect of variations in interreinforcement interval on the temporal and distributional relation between feeding and drinking was continuously monitored. Rats were housed continuously in an operant chamber in which water was freely available, but lever pressing was required to obtain food (45-mg pellets). Initially, pellets were delivered on a fixed-ratio 1 schedule of reinforcement, which was followed by testing on response-initiated fixed-interval 15-, 30-, and 60-second schedules. The total number of discrete, daily meals (a period in which several pellets were earned in succession) was slightly higher during the fixed-interval schedules than during the fixed-ratio 1, but there was no systematic effect of fixed-interval length on meal frequency. Total water consumption, in contrast, increased dramatically as the interval was lengthened: both subjects consumed two to three times as much water on the fixed-interval 60-second schedule as on the fixed-ratio 1. The increased water consumption was the result of an alteration in the distribution of drinking relative to eating. During the fixed-ratio 1 condition, drinking occurred infrequently following individual food pellets and represented the smallest percentage of total drinking; drinking occurred predominantly just before or after a meal. As the fixed interval was lengthened, however, the frequency of postpellet drinking gradually increased and eventually comprised the largest proportion of daily drinking.     

The effects of d-amphetamine on the temporal control of operant responding in rats during a preshock stimulus.

The operant behavior of six rats was maintained by a random-interval schedule of reinforcement. Three-minute periods of noise were superimposed on this behavior, each period ending with the delivery of an unavoidable shock. Overall rates of responding were generally lower during the periods of noise than in its absence (conditioned suppression). These suppressed response rates also exhibited temporal patterning, with responding becoming less frequent as each noise period progressed. The effects of d-amphetamine on this behavioral baseline were then assessed. In four animals the relative response rates during the noise and in its absence suggested that the drug produced a dose-related decrease in the amount of conditioned suppression. However, this effect was often due to a decrease in the rates of responding in the absence of the preshock stimulus, rather than to an increase in response rates during the stimulus. Temporal patterning in response rates during the preshock stimulus was abolished, an effect that was interpreted in terms of rate-dependent effect of d-amphetamine. This study thus extends rate-dependent analyses of the effects of amphetamines to the patterns of operant behavior that occur during a preshock stimulus, and which have been discussed in terms of the disrupting effects of anxiety on operant behavior.     

Acquisition and maintenance of postshock response pattern in nondiscriminated avoidance with rats

Five experimentally naive albino rats were placed under a nondiscriminated lever-press avoidance schedule in which the delay to the next shock for responses after a shock was longer than the delay for responses after a response. Four rats acquired the postshock response pattern and maintained it for a prolonged period. The results revealed that postshock responding was under operant control and was not purely shock-elicited. It was suggested that the two kinds of response-shock interval, i.e. the shock-response-shock interval and the response-response-shock interval, could and should be independently controlled in nondiscriminated avoidance schedules.     

Choice behavior of rats in a concurrent-chains schedule: Amount and delay of reinforcement

Rats were exposed to concurrent-chains schedules in which the terminal links were equal fixed-interval schedules terminating in one or three food pellets. Choice proportions for large reward increased with increases in delay intervals programmed on fixed-interval schedules and supported the predictions derived from a general choice model originally formulated by Fantino and later developed by Navarick and Fantino. In addition, a functional equivalence of two alternatives was established by increasing delay intervals with large reward, whereas delay intervals for small reward were held constant. Functionally equivalent delay intervals with large reward, for each delay interval with small reward, can be described by a power function with exponent smaller than 1.0. A better prediction of choice proportions resulted when this function was used to derive predicted choice proportions.     

Social reinforcement of operant behavior in rats: a methodological note.

An apparatus was developed to study social reinforcement in the rat. Four Long-Evans female rats were trained to press a lever via shaping, with the reinforcer being access to a castrated male rat. Responding under a fixed-ratio schedule and in extinction was also observed. Social access was found to be an effective reinforcer. When social reinforcement was compared with food reinforcement under similar conditions of deprivation and reinforcer duration, no significant differences were observed.     

Sequential reacquisition as a function of timeout from avoidance

Rats learned to reacquire four similar three-member response sequences. Each sequence member was associated with a different response lever, and the correct sequence of levers (i.e., 3-1-2, 2-1-3, 1-3-2, and 2-3-1) changed each session. The first two correct responses of each sequence postponed shock for a fixed period of time. The third correct response initiated a signalled timeout from avoidance. Incorrect responses did not affect the shock interval or reset the sequence. The effects of manipulating timeout duration on the sequential reacquisition baseline were investigated. All subjects displayed biphasic reacquisition performances similar to those controlled by food. The phases were characterized by an initial increase in accuracy, which reached a stable level during the latter portion of each session. Timeout duration affected rate of sequence completion and shock density, but not percentage of errors. Rate of sequence completion was fastest with intermediate timeouts (15 to 60 sec), and slowest with extreme durations (1 or 120 sec). Shock densities peaked with extreme durations and were at minimum with intermediate timeout values. The percentage of errors was the same across timeout durations. These data extend the generality of sequential reacquisition as a procedure for studying learning, and demonstrate timeout from avoidance to be a controlling variable.     

Low-response-rate conditioning history and fixed-interval responding in rats.

Bar presses by one group of rats were conditioned under a differential-reinforcement-of-low-rate reinforcement schedule immediately prior to conditioning under a fixed-interval schedule. In a second group of rats, bar presses were conditioned first under a differential-reinforcement-of-low-rate schedule and then under a fixed-ratio schedule prior to conditioning under a fixed-interval schedule. Low response rates occurred under the fixed-interval schedule only when it was immediately preceded by low-rate conditioning. Otherwise, fixed-interval responding was similar to responding under the fixed-ratio schedule. This finding suggests that responses of laboratory animals are sensitive to immediate history, and, unlike human responses, are relatively insensitive to a history of low-rate conditioning when it is followed by high-rate conditioning.     

Choice and the rate of punishment in concurrent schedules

Rats' responses on two levers were reinforced according to independent random-interval 1.5-min food schedules. In addition, both lever presses were intermittently punished according to several concurrent random-interval random-interval shock schedules. For the left, the scheduled rate of punishment was kept constant according to a random-interval 6-min schedule. For the right, the rate of punishment varied. As the frequency of punishment for the right lever press increased, its rate decreased. The rate of the left punished lever press increased, however, even though its scheduled reinforcement rate and punishment rate remained unchanged.     

Changes in feeding and foraging patterns as an antipredator defensive strategy: a laboratory simulation using aversive stimulation in a closed economy.

The effects of the risk of electric shock on the meal patterns of rats living in an operant chamber were investigated. Rats could obtain food by working on a response lever that provided reinforcement according to chained fixed-ratio continuous reinforcement schedules that allowed the animals control over meal size. Using a two-compartment operant chamber with a safe nesting area and manipulanda area with a grid floor, shock could be correlated with responding on the schedule. Shocks (less than or equal to 1.25 per hour) were scheduled to occur randomly throughout the day, independent of the rat's behavior. Shock caused a reorganization of meal patterns such that the animals took less frequent but larger meals. This pattern reduced the time the animals spent at risk without compromising caloric balance. Similar changes in feeding pattern were obtained in both hooded and albino rats. Exposure to shock in a separate chamber did not produce these behavioral modifications. The magnitude of shock-induced alterations of meal patterns was greater with chained fixed-ratio 90 continuous reinforcement than with chained fixed-ratio 10 continuous reinforcement. Additionally, the rats seemed to be able to reduce food intake but increase caloric efficiency, such that the reduced food intake did not have deleterious effects on maintenance of body weight. These behavioral modifications reduced the number of shocks received from that which would have been expected if meal pattern changes had not occurred. We suggest that this technique may provide a useful laboratory simulation of the impact that the risk of predation has on foraging behavior.     

Acquisition of lever-press responding in rats with delayed reinforcement: A comparison of three procedures

The present study examined the acquisition of lever pressing in rats under three procedures in which food delivery was delayed by 4, 8, and 16 seconds relative to the response. Under the nonresetting delay procedure, food followed the response selected for reinforcement after a specified interval elapsed; responses during this interval had no programmed effect. Under the resetting procedure, the response selected for reinforcement initiated an interval to food delivery that was reset by each subsequent response. Under the stacked delay procedure, every response programmed delivery of food t seconds after its occurrence. Two control groups were studied, one that received food immediately after each lever press and another that never received food. With the exception of the group that did not receive food, responding was established with every procedure at every delay value without autoshaping or shaping. Although responding was established under the resetting delay procedure, response rates were generally not as high as under the other two procedures. These findings support the results of other recent investigations in demonstrating that a response not previously reinforced can be brought to strength by delayed reinforcement in the absence of explicit training.     

Effects of reinforcement rate and delay on the acquisition of lever pressing by rats.

The acquisition of lever pressing by naive rats, in the absence of shaping, was studied as a function of different rates and unsignaled delays of reinforcement. Groups of 3 rats were each exposed to tandem schedules that differed in either the first or the second component. First-component schedules were either continuous reinforcement or random-interval 15, 30, 60 or 120 s; second-component schedules were fixed-time 0, 1, 3, 6, 12, or 24 s. Rate of responding was low under continuous immediate reinforcement and higher under random-interval 15 s. Random interval 30-s and 60-s schedules produced lower rates that were similar to each other. Random-interval 120 s controlled the lowest rate in the immediate-reinforcement condition. Adding a constant 12-s delay to each of the first-component schedule parameters controlled lower response rates that did not vary systematically with reinforcement rate. The continuous and random-interval 60-s schedules of immediate reinforcement controlled higher global and first-component response rates than did the same schedules combined with longer delays, and first-component rates showed some graded effects of delay duration. In addition, the same schedules controlled higher second-component response rates in combination with a 1-s delay than in combination with longer delays. These results were related to those from previous studies on acquisition with delayed reinforcement as well as to those from similar reinforcement procedures used during steady-state responding.     

Choice as a function of local versus molar reinforcement contingencies.

Rats were trained on a discrete-trial probability learning task. In Experiment 1, the molar reinforcement probabilities for the two response alternatives were equal, and the local contingencies of reinforcement differentially reinforced a win-stay, lose-shift response pattern. The win-stay portion was learned substantially more easily and appeared from the outset of training, suggesting that its occurrence did not depend upon discrimination of the local contingencies but rather only upon simple strengthening effects of individual reinforcements. Control by both types of local contingencies decreased with increases in the intertrial interval, although some control remained with intertrial intervals as long as 30 s. In Experiment 2, the local contingencies always favored win-shift and lose-shift response patterns but were asymmetrical for the two responses, causing the molar reinforcement rates for the two responses to differ. Some learning of the alternation pattern occurred with short intertrial intervals, although win-stay behavior occurred for some subjects. The local reinforcement contingencies were discriminated poorly with longer intertrial intervals. In the absence of control by the local contingencies, choice proportion was determined by the molar contingencies, as indicated by high exponent values for the generalized matching law with long intertrial intervals, and lower values with short intertrial intervals. The results show that when molar contingencies of reinforcement and local contingencies are in opposition, both may have independent roles. Control by molar contingencies cannot generally be explained by local contingencies.     

Choice in the time-left procedure and in concurrent chains with a time-left terminal link.

In two experiments, rats chose between a standard fixed-duration food-associated stimulus and a stimulus whose duration was the time remaining to reinforcement in an elapsing comparison interval. In Experiment 1, 4 rats responded in a time-left procedure wherein a single initial-link variable-interval schedule set up two potential terminal links simultaneously. As time elapsed in the initial-link schedule, the choice was between a standard fixed-interval 30-s terminal link and a time-left terminal link whose programmed interval requirement equaled 90 s minus the elapsed time in the initial link. Rats generally responded more on the lever with the shortest programmed terminal-link duration, but the temporal parameters of the procedure were found to vary with response distributions. Contrary to previous reports, therefore, time-left data were well predicted by choice models that make no assumptions about animal timing. In Experiment 2, 8 rats responded on a concurrent-chains schedule with independent variable-interval initial links and a time-left terminal link in one of the choice schedules. On the time-left lever, the programmed terminal-link delay equaled 90 s minus the elapsed time in the time-left initial link. On the standard lever, terminal-link responses were reinforced according to a variable-interval schedule whose average value varied over four conditions. Relative time-left initial-link responses increased in the elapsing time-left initial-link schedule as the time-left terminal link became shorter relative to the standard terminal link. Scalar expectancy theory failed to predict the resultant data, but a modified version of the delay-reduction model made good predictions. An analysis of the elaboration of scalar expectancy theory for variable delays demonstrated that the model is poorly formulated for arithmetically distributed delays.     

Effects of schedule of reinforcement on a pentobarbital discrimination in rats.

The purpose of this study was to determine the effects of the schedule of reinforcement on a pentobarbital discrimination in rats. Five rats were trained to discriminate 10 mg/kg pentobarbital from saline under a multiple fixed-interval 180-s fixed-ratio 20 schedule of reinforcement. During both saline and pentobarbital training sessions, subjects emitted a higher percentage of correct responses under the fixed-ratio component as compared to the fixed-interval component of the multiple schedule. Determination of the pentobarbital dose-response curve under the fixed-ratio component resulted in a steep curve characterized by responding on the saline lever at low doses and on the drug lever at higher doses. Under the fixed-interval component, a graded dose-effect curve was produced, with considerable responding on both levers after intermediate doses of pentobarbital. The administration of phencyclidine and MK-801 resulted in an intermediate level of drug-lever responding for some subjects. Administration of d-amphetamine resulted in saline (nondrug) appropriate responding. The results of this study demonstrate that the schedule of reinforcement is a determinant of drug stimulus control, just as it is a determinant of other drug effects.     

Substitution and caloric regulation in a closed economy.

Three experiments were conducted to study the effect of an imperfect substitute for food on demand for food in a closed economy. In Experiments 1 and 2, rats pressed a lever for their entire daily food ration, and a fixed ratio of presses was required for each food pellet. In both experiments, the fixed ratio was held constant during a daily session but was increased between sessions. The fixed ratio was increased over a series of daily sessions once in the absence of concurrently available sucrose and again when sucrose pellets were freely available. For both series, increases in the fixed ratio reduced food intake, but body weight was reduced only in the no-sucrose condition. In the sucrose condition, body weight and total caloric intake (sucrose plus food) were relatively unaffected by increases in the fixed ratio. At all fixed ratios, food intake was proportionally reduced by the intake of sucrose. In Experiment 3, monkeys obtained food or saccharin by pressing keys; the fixed ratio of presses per food pellet was increased once when tap water was each monkey's only source of fluid, again when each monkey's water was sweetened with saccharin, and a third time when each monkey had concurrent access to the saccharin solution and plain water. Increases in the fixed ratio, but not the intake of the saccharin solution, reduced each monkey's food intake. Because neither rats' sucrose nor monkeys' saccharin intakes affected the slope of the respective demand curves for food, monkeys and rats increased their daily output of presses and thereby defended their daily intake of those complementary elements of food. However, sucrose reduced rats' food intake. The relative constancy of body weight and total caloric intake in the sucrose condition is consistent with the possibility that rats tended to regulate caloric intake.