Choice in a variable environment: visit patterns in the dynamics of choice

Molar and molecular views of behavior imply different approaches to data analysis. The molecular view privileges moment-to-moment analyses, whereas the molar view supports analysis of more and less extended activities. In concurrent performance, the molar view supports study of both extended patterns of choice and more local patterns of visiting the choice alternatives. Analysis of the present data illustrated the usefulness of investigating order at various levels of extendedness. Seven different reinforcer ratios were presented within each session, without cues to identify them, and pigeons pecked at two response keys that delivered food on variable-interval schedules. Choice changed rapidly within components as reinforcers were delivered and, following each reinforcer, shifted toward the alternative that produced it. If several reinforcers were delivered consecutively by one alternative, choice favored that alternative, but shifted more slowly with each new reinforcer. A discontinuation of such a series of reinforcers by the delivery of a reinforcer by the other alternative resulted in a large shift of choice toward that alternative. These effects were illuminated by analysis of visits to the two alternatives. Changes in visit length occurred primarily in the first postreinforcer visit to the repeatedly reinforced alternative. All other visits tended to be brief and equal. Performance showed multiple signs of moving in the direction of a fix-and-sample pattern that characterized steady-state performance in earlier experiments with many sessions of maintaining each schedule pair. The analyses of extended and local patterns illustrate the flexibility of a molar view of behavior.     

Overmatching in rats: the barrier choice paradigm

The barrier choice paradigm was used to impose a cost on rats' behavior of traveling between two levers: Pressing on two levers was reinforced with food on concurrent random-interval schedules, but rats had to climb over a barrier to move from one lever to another. The height of the barrier separating the levers was increased from 30.5 to 45.7 cm across two phases that involved various pairs of random-interval schedules. With the 30.5-cm barrier, the generalized matching law showed slopes equal to or slightly above 1.0 for response and time allocation. With the 45.7-cm barrier, the generalized matching law showed slopes above 1.2 for responses, indicating that sensitivity to reinforcement increased with increasing barrier height. For time allocation the slopes remained close to 1.0; sensitivity to reinforcement did not seem to increase with increasing barrier height. The role of locomotion effort in choice situations is discussed.     

Signal modality and choice between signaled and unsignaled food

Choice between signaled and unsignaled response-independent food schedules was assessed in three experiments using a commitment procedure. In Experiment 1, subjects tested with a 5-s visual signal consistently changed from the signaled to the unsignaled schedule. Changing from the unsignaled to the signaled schedule was observed only occasionally and only at low levels. The same outcome was observed in Experiment 2 with different types of visual signals and with different stimulus combinations identifying the signal period, the signal-absent period, and the unsignaled schedule. In Experiment 3 the visual signal was replaced with an auditory signal for four of the subjects tested in Experiment 2. The subjects then changed from the unsignaled to the signaled schedule or showed a substantial reduction in choice for the unsignaled schedule. The data were assessed using a conditioned-reinforcement interpretation of choice.     

Conservation, choice, and the concurrent fixed-ratio schedule

Five rats got all of their water in daily 60-minute sessions. Two levers and a water spout were freely available throughout baseline sessions. Contingency sessions offered a choice between two alternative fixed-ratio components, in the form of a choice between the two levers. Each component required a specified number of lever presses for access to the spout, and then a specified number of licks for another choice between components. Given the observed relative frequency, the absolute frequency of selecting each component was predicted accurately by assuming that the subject conserved between baseline and contingency the total amount of a dimension attributable to lever pressing and licking. Several quantitative models for predicting relative frequency were examined. The best of these assumed that the subject would show a nonexclusive preference for the component requiring fewer lever presses.     

Behavioral and pharmacological variables affecting risky choice in rats.

The effects of manipulations of response requirement, intertrial interval (ITI), and psychoactive drugs (ethanol, phencyclidine, and d-amphetamine) on lever choice under concurrent fixed-ratio schedules were investigated in rats. Responding on the "certain' lever produced three 45-mg pellets, whereas responding on the "risky" lever produced either 15 pellets (p = .33) or no pellets (p .67). Rats earned all food during the session, which ended after 12 forced trials and 93 choice trials or 90 min, whichever occurred first. When the response requirement was increased from 1 to 16 and the ITI was 20 s, percentage of risky choice was inversely related to fixed-ratio value. When only a single response was required but the ITI was manipulated between 20 and 120 s (with maximum session duration held constant), percentage of risky choice was directly related to length of the ITI. The effects of the drugs were investigated first at an ITI of 20 s, when risky choice was low for most rats, and then at an ITI of 80 s, when risky choice was higher for most rats. Ethanol usually decreased risky choice. Phencyclidine did not usually affect risky choice when the ITI was 20 s but decreased it in half the rats when the ITI was 80 s. For d-amphetamine, the effects appeared to he related to baseline probability of risky choice; that is, low probabilities were increased and high probabilities were decreased. Although increase in risky choice as a function of the ITI is at variance with previous ITI data, it is consistent with foraging data showing that risk aversion decreases as food availability decreases. The pharmacological manipulations showed that drug effects on risky choice may be influenced by the baseline probability of risky choice, just as drug effects can be a function of baseline response rate.     

Effects of reinforcer delays on choice as a function of income level

Three rats earned their daily food ration by responding during individual trials either on a lever that delivered one food pellet immediately or on a second lever that delivered three pellets after a delay that was continuously adjusted to ensure substantial responding to both alternatives. Choice of the delayed reinforcer increased when the number of trials per session was reduced. This result suggests that models seeking closure on choice effects must include a parameter reflecting how preference changes with sessionwide income. Moreover, models positing that reinforcer probability and immediacy (1/delay) function equivalently in choice are called into question by the finding that probability and immediacy produce opposing effects when income level is changed.     

Acquisition and maintenance of postshock response pattern in nondiscriminated avoidance with rats

Five experimentally naive albino rats were placed under a nondiscriminated lever-press avoidance schedule in which the delay to the next shock for responses after a shock was longer than the delay for responses after a response. Four rats acquired the postshock response pattern and maintained it for a prolonged period. The results revealed that postshock responding was under operant control and was not purely shock-elicited. It was suggested that the two kinds of response-shock interval, i.e. the shock-response-shock interval and the response-response-shock interval, could and should be independently controlled in nondiscriminated avoidance schedules.     

An animal model of excessive eating: schedule-induced hyperphagia in food-satiated rats.

Nineteen rats were maintained throughout the experiment on ad libitum wet mash and water and were trained to press a lever on fixed-interval or fixed-ratio schedules of reinforcement with electrical brain stimulation. Fourteen rats ate at least 150% more mash during intermittent reinforcement sessions than during baseline, massed reinforcement control, and/or extinction sessions. In a 3-hr session, 11 of those 14 consumed more than 22 g of wet mash (13 g dry weight), the equivalent of nearly half an animal's daily food intake. In subsequent control sessions, the electrodes did not support stimulus-bound eating despite attempts to make stimulation parameters optimal. These results indicate that the eating was schedule induced or adjunctive, and suggest that the procedure may provide an animal model of excessive nonregulatory eating that contributes to obesity in humans.     

Delay discounting in Lewis and Fischer 344 rats: steady-state and rapid-determination adjusting-amount procedures

Lewis rats have been shown to make more impulsive choices than Fischer 344 rats in discrete-trial choice procedures that arrange fixed (i.e., nontitrating) reinforcement parameters. However, nontitrating procedures yield only gross estimates of preference, as choice measures in animal subjects are rarely graded at the level of the individual subject. The present study was designed to examine potential strain differences in delay discounting using an adjusting-amount procedure, in which distributed (rather than exclusive) choice is observed due to dynamic titration of reinforcer magnitude across trials. Using a steady-state version of the adjusting-amount procedure in which delay was manipulated between experimental conditions, steeper delay discounting was observed in Lewis rats compared to Fischer 344 rats; further, delay discounting in both strains was well described by the traditional hyperbolic discounting model. However, upon partial completion of the present study, a study published elsewhere (Wilhelm & Mitchell, 2009) demonstrated no difference in delay discounting between these strains with the use of a more rapid version of the adjusting-amount procedure (i.e., in which delay is manipulated daily). Thus, following completion of the steady-state assessment in the present study, all surviving Lewis and Fischer 344 rats completed an approximation of this rapid-determination procedure in which no strain difference in delay discounting was observed.     

Relative sensitivity to reinforcer amount and delay in a self-control choice situation.

Rats were exposed to concurrent-chains schedules in which a single variable-interval schedule arranged entry into one of two terminal-link delay periods (fixed-interval schedules). The shorter delay ended with the delivery of a single food pellet; the longer day ended with a larger number of food pellets (two under some conditions and six under others). In Experiment 1, the terminal-link delays were selected so that under all conditions the ratio of delays would exactly equal the ratio of the number of pellets. But the absolute duration of the delays differed across conditions. In one condition, for example, rats chose between one pellet delayed 5 s and six pellets delayed 30 s; in another condition rats chose between one pellet delayed 10 s and six pellets delayed 60 s. The generalized matching law predicts indifference between the two alternatives, assuming that the sensitivity parameters for amount and delay of reinforcement are equal. The rats' choices were, in fact, close to indifference except when the choice was between one pellet delayed 5 s and six pellets delayed 30 s. That deviation from indifference suggests that the sensitivities to amount and delay differ from each other depending on the durations of the delays. In Experiment 2, rats chose between one pellet following a 5-s delay and six pellets following a delay that was systematically increased over sessions to find a point of indifference. Indifference was achieved when the delay to the six pellets was approximately 55 s. These results are consistent with the possibility that the relative sensitivities to amount and delay differ as a function of the delays.     

Acquisition of lever-press responding in rats with delayed reinforcement: A comparison of three procedures

The present study examined the acquisition of lever pressing in rats under three procedures in which food delivery was delayed by 4, 8, and 16 seconds relative to the response. Under the nonresetting delay procedure, food followed the response selected for reinforcement after a specified interval elapsed; responses during this interval had no programmed effect. Under the resetting procedure, the response selected for reinforcement initiated an interval to food delivery that was reset by each subsequent response. Under the stacked delay procedure, every response programmed delivery of food t seconds after its occurrence. Two control groups were studied, one that received food immediately after each lever press and another that never received food. With the exception of the group that did not receive food, responding was established with every procedure at every delay value without autoshaping or shaping. Although responding was established under the resetting delay procedure, response rates were generally not as high as under the other two procedures. These findings support the results of other recent investigations in demonstrating that a response not previously reinforced can be brought to strength by delayed reinforcement in the absence of explicit training.     

Conditioned reinforcement dynamics in three-link chained schedules.

In two experiments rats were trained on three-link concurrent-chains schedules of reinforcement. In Experiment 1, additional entries to one terminal link were added during one of the middle links to a baseline schedule that was otherwise equal for the two chains, and, depending on the condition, these additional terminal-link presentations ended either in food or in no food. When food occurred, preference was always in favor of the chain with the additional terminal-link presentations (which also entailed a higher rate of reinforcement). When no food occurred at the end of the additional terminal links, the outcome depended on the nature of the stimuli associated with these additional terminal links. When stimuli different from the reinforced baseline terminal links were used for the no-food terminal links, preference was against the choice alternative that led to the extra periods of extinction. When the same stimulus was used for the two kinds of terminal links, preference was near indifference, that is, significantly greater than when different stimuli were used. In Experiment 2, rats learned repeated reversals of a simultaneous discrimination under a three-link concurrent-chains schedule, in which the food or no-food choice outcomes were delayed until the end of the chain. Different conditions were defined by the point in the chain at which differential stimuli occurred. When the middle and terminal links provided no differential stimuli, discrimination was acquired more slowly than when differential stimuli occurred in both links. When differential stimuli occurred in the middle but not the terminal links, acquisition rates were intermediate. Both experiments together show that the effects of stimuli in a chain schedule are due partly to the time to food correlated with the stimuli and partly to the time to the next conditioned reinforcer in the sequence.     

Quinine pellets as an inferior good and a Giffen good in rats.

In Experiment 1, 4 rats earned their daily food ration by choosing between two levers. One lever delivered two regular and one quinine-adulterated food pellets, and the other delivered two regular and four quinine pellets. A 20-s intertrial interval separated successive choices. Sessions began with 10 forced trials during which only one lever, selected with p = .5 and cued by a light above it, could deliver its reinforcer. Forced trials were followed by 30 or 150 trials, depending on the condition, during which choices to either lever could be reinforced. Over this range, absolute choice of the four-quinine, two-regular-pellet lever was inversely related to the number of free-choice trials, establishing this reinforcer as an inferior good. In Condition 1 of Experiment 2, the prior design was altered in two ways: (a) one lever delivered four quinine pellets, and the other lever delivered one standard pellet; and (b) sessions ended after 140 free-choice trials. When the number of free-choice trials was reduced to 100 (Condition 2), all 3 rats increased their preference for quinine pellets, confirming their status as an inferior good. In the next several conditions, the number of quinine pellets provided for selecting its associated lever was varied between three and four. Preference for the quinine-pellet alternative was inversely related to the number of pellets it provided, a result defining it as a Giffen good. These findings are not accommodated readily by extant choice models and complicate the search for a unitary model of choice.     

Choice and the rate of punishment in concurrent schedules

Rats' responses on two levers were reinforced according to independent random-interval 1.5-min food schedules. In addition, both lever presses were intermittently punished according to several concurrent random-interval random-interval shock schedules. For the left, the scheduled rate of punishment was kept constant according to a random-interval 6-min schedule. For the right, the rate of punishment varied. As the frequency of punishment for the right lever press increased, its rate decreased. The rate of the left punished lever press increased, however, even though its scheduled reinforcement rate and punishment rate remained unchanged.     

Choice behavior of rats in a concurrent-chains schedule: Amount and delay of reinforcement

Rats were exposed to concurrent-chains schedules in which the terminal links were equal fixed-interval schedules terminating in one or three food pellets. Choice proportions for large reward increased with increases in delay intervals programmed on fixed-interval schedules and supported the predictions derived from a general choice model originally formulated by Fantino and later developed by Navarick and Fantino. In addition, a functional equivalence of two alternatives was established by increasing delay intervals with large reward, whereas delay intervals for small reward were held constant. Functionally equivalent delay intervals with large reward, for each delay interval with small reward, can be described by a power function with exponent smaller than 1.0. A better prediction of choice proportions resulted when this function was used to derive predicted choice proportions.     

The relationship between feeding rate and patch choice.

Rats in a laboratory foraging simulation searched for sequential opportunities to feed in two patches that differed in the rate at which food pellets were delivered (controlled by fixed-interval schedules) and in the size of the pellets. The profitability of feeding in each patch was calculated in terms of time (grams per minute) and in terms of effort (grams per bar press). These values were the result of the imposed fixed interval, the size of the pellets, and the rate at which the rats pressed the bar in each condition. The rats ate more food and larger meals, but not more frequent meals, at the patch offering the higher rate of food consumption, calculated as grams per minute. The relative intake at any patch was a function of the relative rate of intake during meals at that patch compared to the other patch. Rats respond to explicit manipulations of feeding time in the same manner as they respond to manipulations of feeding effort.     

Choice between constant and variable alternatives by rats: effects of different reinforcer amounts and energy budgets

Two experiments, using rats as subjects, investigated the effect of different reinforcer amounts and energy budgets on choice between constant and variable alternatives under a closed economy. Rats were housed in the chamber and were exposed to a modified concurrent-chains schedule in which the choice phase was separated from a rest phase during which the rats could engage in other activities. In the choice phase, a single variable-interval schedule arranged entry into one of two equal terminal links (fixed-interval schedules). The constant terminal link ended with the delivery of a fixed number of food pellets (two or three, depending on the condition), whereas the variable terminal link ended with a variable number of food pellets (means of two or three, depending on the condition). Energy budget was defined as positive when body weights were over 90% of free-feeding weights, and as negative when they were under 80% of free-feeding weights. The different body weights were produced by varying the duration of the equal terminal-link schedules within daily 3-hr sessions. In Experiment 1, rats chose between a constant and a variable three pellets under both energy budgets. Rats preferred the constant three pellets more under the positive energy budget, whereas they were indifferent under the negative energy budget. In Experiment 2, rats chose between a constant three pellets and a variable two pellets, and chose between a constant two pellets and a variable three pellets under both energy budgets. The rats strongly preferred the constant three pellets over the variable two pellets under both energy budgets. In contrast, rats preferred the variable three pellets over the constant two pellets only under the negative energy budget, whereas they were indifferent under the positive energy budget. These results indicate that rats choices are sensitive to the difference in reinforcer amounts and to the energy budgets defined by the level of body weight. The present results are consistent with those obtained with small granivorous birds as well as with the predictions of a recent risk-sensitive foraging theory.