Behavioral and pharmacological variables affecting risky choice in rats.

The effects of manipulations of response requirement, intertrial interval (ITI), and psychoactive drugs (ethanol, phencyclidine, and d-amphetamine) on lever choice under concurrent fixed-ratio schedules were investigated in rats. Responding on the "certain' lever produced three 45-mg pellets, whereas responding on the "risky" lever produced either 15 pellets (p = .33) or no pellets (p .67). Rats earned all food during the session, which ended after 12 forced trials and 93 choice trials or 90 min, whichever occurred first. When the response requirement was increased from 1 to 16 and the ITI was 20 s, percentage of risky choice was inversely related to fixed-ratio value. When only a single response was required but the ITI was manipulated between 20 and 120 s (with maximum session duration held constant), percentage of risky choice was directly related to length of the ITI. The effects of the drugs were investigated first at an ITI of 20 s, when risky choice was low for most rats, and then at an ITI of 80 s, when risky choice was higher for most rats. Ethanol usually decreased risky choice. Phencyclidine did not usually affect risky choice when the ITI was 20 s but decreased it in half the rats when the ITI was 80 s. For d-amphetamine, the effects appeared to he related to baseline probability of risky choice; that is, low probabilities were increased and high probabilities were decreased. Although increase in risky choice as a function of the ITI is at variance with previous ITI data, it is consistent with foraging data showing that risk aversion decreases as food availability decreases. The pharmacological manipulations showed that drug effects on risky choice may be influenced by the baseline probability of risky choice, just as drug effects can be a function of baseline response rate.     

Effects of reinforcer delays on choice as a function of income level

Three rats earned their daily food ration by responding during individual trials either on a lever that delivered one food pellet immediately or on a second lever that delivered three pellets after a delay that was continuously adjusted to ensure substantial responding to both alternatives. Choice of the delayed reinforcer increased when the number of trials per session was reduced. This result suggests that models seeking closure on choice effects must include a parameter reflecting how preference changes with sessionwide income. Moreover, models positing that reinforcer probability and immediacy (1/delay) function equivalently in choice are called into question by the finding that probability and immediacy produce opposing effects when income level is changed.     

Operant hoarding: a new paradigm for the study of self-control.

In the first of four experiments, rats were exposed to a modified multiple continuous reinforcement-extinction schedule during 15-min daily sessions. In one condition (saves condition) with the cuelight on, a single lever press produced a food pellet, briefly extinguished the cuelight, and started a clock. Saves (additional lever presses with interresponse times less than 1 s) produced an additional food pellet, briefly extinguished the cuelight, and restarted the interresponse time clock. The cuelight was extinguished 1 s after the last lever press and remained off during a 10-s period of extinction, during which no food pellets were delivered. In the other condition (savings account condition), the contingencies were the same except that the cuelight was extinguished and was not reilluminated after the initial lever press, and the delivery of all food pellets in the reinforcement component was delayed until the onset of extinction. In both conditions, rats made saves, but mean saves (total saves divided by the number of reinforcement components) were slightly reduced in the savings account condition. In Experiment 2, using six equally spaced 15-min sessions per day on alternate days, saves were either followed immediately with food and brief cuelight offset (saves condition) or were not reinforced at all. Mean saves were much greater when saves were reinforced. In Experiment 3, during 5-min daily sessions, saves earned a single pellet (savings account condition) or a number of pellets equal to the ordinal number of the lever press (interest condition). Rats made fewer mean saves, with little change in the food rate, when saves earned interest. In Experiment 4, the rats earned all their food in the operant situation during 24 daily 5-min sessions, these separated by 55-min intersession intervals during which no food was available; otherwise, the conditions were the same as in Experiment 3. In Experiment 4, the shift to interest for saves led to an increase in mean daily mean saves (total daily mean saves divided by the number of daily sessions) as well as to an increase in the number of food pellets delivered in each session. The results are discussed in terms of self-control and behavioral economics.     

Quinine pellets as an inferior good and a Giffen good in rats.

In Experiment 1, 4 rats earned their daily food ration by choosing between two levers. One lever delivered two regular and one quinine-adulterated food pellets, and the other delivered two regular and four quinine pellets. A 20-s intertrial interval separated successive choices. Sessions began with 10 forced trials during which only one lever, selected with p = .5 and cued by a light above it, could deliver its reinforcer. Forced trials were followed by 30 or 150 trials, depending on the condition, during which choices to either lever could be reinforced. Over this range, absolute choice of the four-quinine, two-regular-pellet lever was inversely related to the number of free-choice trials, establishing this reinforcer as an inferior good. In Condition 1 of Experiment 2, the prior design was altered in two ways: (a) one lever delivered four quinine pellets, and the other lever delivered one standard pellet; and (b) sessions ended after 140 free-choice trials. When the number of free-choice trials was reduced to 100 (Condition 2), all 3 rats increased their preference for quinine pellets, confirming their status as an inferior good. In the next several conditions, the number of quinine pellets provided for selecting its associated lever was varied between three and four. Preference for the quinine-pellet alternative was inversely related to the number of pellets it provided, a result defining it as a Giffen good. These findings are not accommodated readily by extant choice models and complicate the search for a unitary model of choice.     

Stock optimizing: maximizing reinforcers per session on a variable-interval schedule.

In Experiment 1, 2 monkeys earned their daily food ration by pressing a key that delivered food according to a variable-interval 3-min schedule. In Phases 1 and 4, sessions ended after 3 hr. In Phases 2 and 3, sessions ended after a fixed number of responses that reduced food intake and body weights from levels during Phases 1 and 4. Monkeys responded at higher rates and emitted more responses per food delivery when the food earned in a session was reduced. In Experiment 2, monkeys earned their daily food ration by depositing tokens into the response panel. Deposits delivered food according to a variable-interval 3-min schedule. When the token supply was unlimited (Phases 1, 3, and 5), sessions ended after 3 hr. In Phases 2 and 4, sessions ended after 150 tokens were deposited, resulting in a decrease in food intake and body weight. Both monkeys responded at lower rates and emitted fewer responses per food delivery when the food earned in a session was reduced. Experiment 1's results are consistent with a strength account, according to which the phases that reduced body weights increased food's value and therefore increased subjects' response rates. The results of Experiment 2 are consistent with an optimizing strategy, because lowering response rates when food is restricted defends body weight on variable-interval schedules. These contrasting results may be attributed to the discriminability of the contingency between response number and the end of a session being greater in Experiment 2 than in Experiment 1. In consequence, subjects lowered their response rates in order to increase the number of reinforcers per session (stock optimizing).     

Stock optimizing in choice when a token deposit is the operant.

Each of 2 monkeys typically earned their daily food ration by depositing tokens in one of two slots. Tokens deposited in one slot dropped into a bin where they were kept (token kept). Deposits to a second slot dropped into a bin where they could be obtained again (token returned). In Experiment 1, a fixed-ratio (FR) 5 schedule that provided two food pellets was associated with each slot. Both monkeys preferred the token-returned slot. In Experiment 2, both subjects chose between unequal FR schedules with the token-returned slot always associated with the leaner schedule. When the FRs were 2 versus 3 and 2 versus 6, preferences were maintained for the token-returned slot; however, when the ratios were 2 versus 12, preference shifted to the token-kept slot. In Experiment 3, both monkeys chose between equal-valued concurrent variable-interval variable-interval schedules. Both monkeys preferred the slot that returned tokens. In Experiment 4, both monkeys chose between FRs that typically differed in size by a factor of 10. Both monkeys preferred the FR schedule that provided more food per trial. These data show that monkeys will choose so as to increase the number of reinforcers earned (stock optimizing) even when this preference reduces the rate of reinforcement (all reinforcers divided by session time).     

Second-order schedules of token reinforcement with pigeons: implications for unit price

Four pigeons were exposed to second-order schedules of token reinforcement, with stimulus lights serving as token reinforcers. Tokens were earned according to a fixed-ratio (token-production) schedule, with the opportunity to exchange tokens for food (exchange period) occurring after a fixed number had been produced (exchange-production ratio). The token-production and exchange-production ratios were manipulated systematically across conditions. Response rates varied inversely with the token-production ratio at each exchange-production ratio. Response rates also varied inversely with the exchange-production ratio at each token-production ratio, particularly at the higher token-production ratios. At higher token-production and exchange-production ratios, response rates increased in token-production segments closer to exchange periods and food. Some conditions were conducted in a closed economy, in which the pigeons earned all their daily ration of food within the session. Relative to comparable open-economy conditions, response rates in the closed economy were less affected by changes in token-production ratio, resulting in higher levels of food intake and body weight. Some of the results are consistent with the economic concept of unit price, a cost-benefit ratio comprised of responses per unit of food delivery, but most are well accounted for by a consideration of the number of responses required to produce exchange periods, without regard to the amount of reinforcement available during those exchange periods.     

Choice between constant and variable alternatives by rats: effects of different reinforcer amounts and energy budgets

Two experiments, using rats as subjects, investigated the effect of different reinforcer amounts and energy budgets on choice between constant and variable alternatives under a closed economy. Rats were housed in the chamber and were exposed to a modified concurrent-chains schedule in which the choice phase was separated from a rest phase during which the rats could engage in other activities. In the choice phase, a single variable-interval schedule arranged entry into one of two equal terminal links (fixed-interval schedules). The constant terminal link ended with the delivery of a fixed number of food pellets (two or three, depending on the condition), whereas the variable terminal link ended with a variable number of food pellets (means of two or three, depending on the condition). Energy budget was defined as positive when body weights were over 90% of free-feeding weights, and as negative when they were under 80% of free-feeding weights. The different body weights were produced by varying the duration of the equal terminal-link schedules within daily 3-hr sessions. In Experiment 1, rats chose between a constant and a variable three pellets under both energy budgets. Rats preferred the constant three pellets more under the positive energy budget, whereas they were indifferent under the negative energy budget. In Experiment 2, rats chose between a constant three pellets and a variable two pellets, and chose between a constant two pellets and a variable three pellets under both energy budgets. The rats strongly preferred the constant three pellets over the variable two pellets under both energy budgets. In contrast, rats preferred the variable three pellets over the constant two pellets only under the negative energy budget, whereas they were indifferent under the positive energy budget. These results indicate that rats choices are sensitive to the difference in reinforcer amounts and to the energy budgets defined by the level of body weight. The present results are consistent with those obtained with small granivorous birds as well as with the predictions of a recent risk-sensitive foraging theory.     

Effects of food deprivation and reinforcement magnitude on conditioned suppression

In Experiment I, the responding of rats lever pressing on a variable-interval schedule for sucrose solution was partially suppressed by a variable duration conditioned stimulus followed by shock. When food deprivation was increased, response rates during and before the conditioned stimulus increased monotonically. Varying the concentration of sucrose across blocks of sessions or from session to session in a semi-random sequence had little effect on response rates either before or during the conditioned stimulus. With a fixed sequence of increasing concentrations across a five-session block, increased concentration produced much more rapid increases in response rates before than during the conditioned stimulus. In Experiment II, rats were presented with the same sequence of increasing concentrations across a five-session block. When tested at 80% body weight, response rates increased rapidly as concentration increased, but at 100%, body-weight rates increased only slightly. The effect of a change in body weight in Experiment II thus mimicked the effect of the conditioned stimulus in the latter part of Experiment I. These findings support the view that the effect of a pre-aversive conditioned stimulus is similar to that of a change in food deprivation, but unlike that of a change in reinforcement magnitude.     

Choice between food and heroin: effects of morphine, naloxone, and secobarbital.

Baboons responded on a choice task on which discrete trials involved choosing between an intravenous injection of heroin (.32 or 1.0 mg/kg) or the availability of food pellets. An intertrial interval of three hours followed the completion of each trial. Under baseline conditions baboons consistently completed the eight available trials each day. Typically, animals chose heroin on three or four trials a day and food on the remaining trials. Animals tended to space the selection of heroin rather than choosing heroin on consecutive trials. A series of single-day experimental manipulations was undertaken to characterize performance further. Manipulation of the heroin dose produced shifts in the relative frequency of choosing the drug option which were inversely related to dose. Manipulation of number of pellets per food trial produced little change in distribution of choices. Noncontingent administration of morphine produced dose-related decreases in relative frequency of heroin choices, and a higher dose decreased the number of trials completed. Noncontingent naloxone produced dose-related increases in the relative frequency of heroin choices. Noncontingent secobarbital had no effect on distribution of choices, and high doses reduced the number of trials completed per day. The results suggest that morphine and naloxone produce shifts in this choice behavior by selectively interacting with the reinforcing properties of the option involving heroin.     

Substitution and caloric regulation in a closed economy.

Three experiments were conducted to study the effect of an imperfect substitute for food on demand for food in a closed economy. In Experiments 1 and 2, rats pressed a lever for their entire daily food ration, and a fixed ratio of presses was required for each food pellet. In both experiments, the fixed ratio was held constant during a daily session but was increased between sessions. The fixed ratio was increased over a series of daily sessions once in the absence of concurrently available sucrose and again when sucrose pellets were freely available. For both series, increases in the fixed ratio reduced food intake, but body weight was reduced only in the no-sucrose condition. In the sucrose condition, body weight and total caloric intake (sucrose plus food) were relatively unaffected by increases in the fixed ratio. At all fixed ratios, food intake was proportionally reduced by the intake of sucrose. In Experiment 3, monkeys obtained food or saccharin by pressing keys; the fixed ratio of presses per food pellet was increased once when tap water was each monkey's only source of fluid, again when each monkey's water was sweetened with saccharin, and a third time when each monkey had concurrent access to the saccharin solution and plain water. Increases in the fixed ratio, but not the intake of the saccharin solution, reduced each monkey's food intake. Because neither rats' sucrose nor monkeys' saccharin intakes affected the slope of the respective demand curves for food, monkeys and rats increased their daily output of presses and thereby defended their daily intake of those complementary elements of food. However, sucrose reduced rats' food intake. The relative constancy of body weight and total caloric intake in the sucrose condition is consistent with the possibility that rats tended to regulate caloric intake.     

Determinants of reinforcer accumulation during an operant task.

Responses by rats on an earn lever made available food pellets that were delivered to a food cup by responses on a second, collect, lever. The rats could either collect and immediately consume or accumulate (defined as the percentage of multiple earn responses and as the number of pellets earned before a collect response) earned pellets. In Experiment 1, accumulation varied as a function of variations in the earn or collect response requirements and whether the earn and collect levers were proximal (31 cm) or distal (248 cm) to one another. Some accumulation occurred under all but one of the conditions, but generally was higher when the earn and collect levers were distal to one another, particularly when the earn response requirement was fixed-ratio (FR) 1. In Experiment 2, the contributions of responses and time to accumulation were assessed by comparing an FR 20 earn response requirement to a condition in which only a single earn response was required at the end of a time interval nominally yoked to the FR interval. When 248 cm separated the earn and collect levers, accumulation was always greater in the FR condition, and it was not systematically related to reinforcement rate. In Experiment 3, increasing the earn response requirement with a progressive-ratio schedule that reset only with a collect response increased the likelihood of accumulation when the collect and earn levers were 248 cm apart, even though such accumulation increased the next earn response requirement. Reinforcer accumulation is an understudied dimension of operant behavior that relates to the analysis of such phenomena as hoarding and self-control, in that they too involve accumulating versus immediately collecting or consuming reinforcers.     

Control of schedule-induced polydipsi: type, size, and spacing of meals

Rats were given daily 1-min variable-interval sessions for several types of food delivered in various amounts per reinforcement and the concurrent, schedule-induced polydipsia was measured. Dry, solid food was neither a necessary nor sufficient condition for the development of polydipsia. Small portions of liquid Standard Monkey Diet produced polydipsia, but 45-mg dextrose or sucrose pellets did not. Within the range studied, smaller portions of both solid and liquid foods produced more drinking than larger portions per reinforcement. Two-min variable-interval sessions produced a greater polydipsic response than 1-min variable-interval, even though the number of 45-mg Noyes pellets allowed per session was held constant. Polydipsia was greatly attenuated on these schedules when the number of pellets remained constant, but were delivered two at a time. Within the ranges studied, the concurrent polydipsic response was increased by decreasing the rate of food acquisition, either by using smaller portions of food per reinforcement or by increasing the interreinforcement time.     

Altering "motivational" variables alters induction produced by upcoming food-pellet reinforcement

Previous research has demonstrated that rats will increase their rates of lever pressing for sucrose rewards in the first half of an experimental session when food pellets, rather than the same sucrose, continually serve as the reward in the second half of the session. This effect has been coined induction, and the present study investigated whether it could be altered by altering "motivational" variables. Experiment 1 manipulated subjects' motivation by altering, across conditions, their level of food deprivation. Predictably, the size of induction varied directly with level of deprivation. Experiments 2 and 3 manipulated subjects' motivation by feeding them food pellets and sucrose, respectively, prior to their responding in the experimental session. These pre-session feedings decreased the size of the observed induction in both experiments. The results from the present study indicate that the size of induction is correlated with subjects' motivation to respond for the available reinforcers. They are also consistent with the idea that operant processes underlie the effect. The notion that induction might encompass the concept of "anticipation" is also discussed. Electronic Publication     

Motivation in concurrent variable-interval schedules with food and water reinforcers

The lever pressing of four food- and water-deprived rats was reinforced on concurrent variable-interval schedules. Food reinforced one response, and water reinforced the other. Response rates in baseline were higher in the food component than in the water component. After response patterns and body weights had stabilized, the animals were given access to either food only, water only, both food and water, or neither food nor water (baseline) before daily sessions. Giving food before a session decreased per cent time in the food component, decreased overall response rates for food, and increased overall response rates for water. Giving water before a session increased per cent time in the food component, increased overall response rates for food, and decreased overall response rates for water. Giving both food and water before a session resulted in a combination of prefeeding and prewatering effects. More food and more water were consumed when both were available than when only one was available before a session.     

The role of response-contingent incentives in lithium chloride-mediated suppression of an operant response

Three experiments investigated what role a novel incentive plays in the development of operant response suppression mediated by lithium chloride. In all experiments animals were trained to press two levers under concurrent schedules of reinforcement. In Experiment 1 responding on one lever delivered a familiar incentive (food pellets), whereas responding on an alternative lever delivered a novel incentive (sucrose solution) prior to lithium chloride injections. If lithium was administered immediately after the instrumental session, the action associated with the novel, but not with the familiar, incentive was suppressed. By comparison, in a control group for which responding on both levers led to the familiar incentive, both actions were suppressed. Experiment 2 examined whether the novelty, rather than the sensory properties, of the incentive is crucial for observing performance suppression. It was found that animals familiarized with the “target” incentive were insensitive to aversion conditioning by lithium, in that there was no difference in response rates between the action that delivered the familiar incentive from that which earned the “target”. In contrast, if animals were unfamiliar with the “target” incentive at the time of aversion conditioning, they suppressed responding on the lever that was associated with the novel incentive but did not suppress responding on the lever associated with the familiar incentive. Experiment 3 investigated the mechanism underlying instrumental performance suppression. After the completion of concurrent lever press training, novel sucrose was introduced in conjunction with the pellets for responding on one lever; responding on the other lever continued to deliver only familiar pellets. Lithium injections were then administered either immediately following the sessions or several hours after the sessions. It was found that the rate of responding on the lever associated with the contingent delivery of sucrose was suppressed below that of the pellet-alone action. By comparison, if lithium injections were administered several hours following the session, an elevation in responding on the sucrose-plus-pellet lever was observed. The outcomes of all three experiments demonstrate not only that the novelty of an incentive is important in obtaining performance suppression, but also that a novel incentive can punish instrumental responding if it has been associated with toxicosis.     

Transfer of persistence to the acquisition of a new behaviour

Two experiments tested whether the degree of effort required for the reinforcement of one behaviour would affect the acquisition of a second behaviour. In the first experiment, rats were placed in a conditioning chamber and: (a) were required to press a lever for food pellets on a fixed ratio schedule, (b) received free presentation of the pellets, or (c) did not receive pellets. Next, all rats were rewarded for a new behaviour, round trips across the length of a runway. As predicted, the fixed-ratio group had the greatest shuttle rate. In the second experiment, two groups were required to press a lever, and the number of presses per pellet was varied. For two other groups not required to press the lever, the amount of food presented per approach to the feeder was varied. The greater required number of lever presses and the lesser number of pellets per approach to the feeder produced the higher subsequent shuttle rates. Two alternative explanations were compared: the degree of accustomed effort per reinforcer becomes a learned component of behaviour, or high effort increases the habituation of frustration-produced disruptive responses.     

Second-order schedules of token reinforcement with pigeons: effects of fixed- and variable-ratio exchange schedules

Pigeons' key pecks produced food under second-order schedules of token reinforcement, with light-emitting diodes serving as token reinforcers. In Experiment 1, tokens were earned according to a fixed-ratio 50 schedule and were exchanged for food according to either fixed-ratio or variable-ratio exchange schedules, with schedule type varied across conditions. In Experiment 2, schedule type was varied within sessions using a multiple schedule. In one component, tokens were earned according to a fixed-ratio 50 schedule and exchanged according to a variable-ratio schedule. In the other component, tokens were earned according to a variable-ratio 50 schedule and exchanged according to a fixed-ratio schedule. In both experiments, the number of responses per exchange was varied parametrically across conditions, ranging from 50 to 400 responses. Response rates decreased systematically with increases in the fixed-ratio exchange schedules, but were much less affected by changes in the variable-ratio exchange schedules. Response rates were consistently higher under variable-ratio exchange schedules than tinder comparable fixed-ratio exchange schedules, especially at higher exchange ratios. These response-rate differences were due both to greater pre-ratio pausing and to lower local rates tinder the fixed-ratio exchange schedules. Local response rates increased with proximity to food under the higher fixed-ratio exchange schedules, indicative of discriminative control by the tokens.     

RISKY CHOICE IN PIGEONS: PREFERENCE FOR AMOUNT VARIABILITY USING A TOKEN-REINFORCEMENT SYSTEM

Pigeons were given repeated choices between variable and fixed numbers of token reinforcers (stimulus lamps arrayed above the response keys), with each earned token exchangeable for food. The number of tokens provided by the fixed‐amount option remained constant within blocks of sessions, but varied parametrically across phases, assuming values of 2, 4, 6, or 8 tokens per choice. The number of tokens provided by the variable‐amount option varied between 0 and 12 tokens per choice, arranged according to an exponential or rectangular distribution. In general, the pigeons strongly preferred the variable option when the fixed option provided equal or greater numbers of tokens than the variable amount. Preference for the variable amount decreased only when the alternatives provided widely disparate amounts favoring the fixed amount. When tokens were removed from the experimental context, preference for the variable option was reduced or eliminated, suggesting that the token presentation played a key role in maintaining risk‐prone choice patterns. Choice latencies varied inversely with preferences, suggesting that local analyses may provide useful ancillary measures of reinforcer value. Overall, the results indicate that systematic risk sensitivity can be attained with respect to reinforcer amount, and that tokens may be critical in the development of such preferences.     

Effects of reinforcer probability, delay, and response requirements on the choices of rats and pigeons: possible species differences

In Experiment 1 with rats, a left lever press led to a 5-s delay and then a possible reinforcer. A right lever press led to an adjusting delay and then a certain reinforcer. This delay was adjusted over trials to estimate an indifference point, or a delay at which the two alternatives were chosen about equally often. Indifference points increased as the probability of reinforcement for the left lever decreased. In some conditions with a 20% chance of food, a light above the left lever was lit during the 5-s delay on all trials, but in other conditions, the light was only lit on those trials that ended with food. Unlike previous results with pigeons, the presence or absence of the delay light on no-food trials had no effect on the rats' indifference points. In other conditions, the rats showed less preference for the 20% alternative when the time between trials was longer. In Experiment 2 with rats, fixed-interval schedules were used instead of simple delays, and the presence or absence of the fixed-interval requirement on no-food trials had no effect on the indifference points. In Experiment 3 with rats and Experiment 4 with pigeons, the animals chose between a fixed-ratio 8 schedule that led to food on 33% of the trials and an adjusting-ratio schedule with food on 100% of the trials. Surprisingly, the rats showed less preference for the 33% alternative in conditions in which the ratio requirement was omitted on no-food trials. For the pigeons, the presence or absence of the ratio requirement on no-food trials had little effect. The results suggest that there may be differences between rats and pigeons in how they respond in choice situations involving delayed and probabilistic reinforcers.