Drinking in a patchy environment: the effect of the price of water.

Rats in a laboratory foraging paradigm searched for sequential opportunities to drink in two water patches that differed in the bar-press price of each "sip" (20 licks) of water within a bout of drinking (Experiment 1) or the price and size (10, 20, or 40 licks) of each sip (Experiment 2). Total daily water intake was not affected by these variables. The rats responded faster at the patch where water was more costly. However, they accepted fewer opportunities to drink, and thus had fewer drinking bouts, and drinking bouts were smaller at the more costly patch than at the other patch. This resulted in the rats consuming a smaller proportion of their daily water from the more costly patch. The size of the differences in bout frequency and size between the patches appears to be based on the relative cost of water at the patches. The profitability of each patch was calculated in terms of the return (in milliliters) on either effort (bar presses) or time spent there. Although both measures were correlated with the relative total intake, bout size, and acceptance of opportunities at each patch, the time-based profitability was the better predictor of these intake measures. The rats did not minimize bar-press output; however, their choice between the patches and their bout sizes within patches varied in a way that reduced costs compared to what would have been expended drinking randomly. These data accord well with similar findings for choices among patches of food, suggesting that foraging for water and food occurs on the basis of comparable benefit-cost functions: In each case, the amount consumed is related to the time spent consuming.     

The relationship between feeding rate and patch choice.

Rats in a laboratory foraging simulation searched for sequential opportunities to feed in two patches that differed in the rate at which food pellets were delivered (controlled by fixed-interval schedules) and in the size of the pellets. The profitability of feeding in each patch was calculated in terms of time (grams per minute) and in terms of effort (grams per bar press). These values were the result of the imposed fixed interval, the size of the pellets, and the rate at which the rats pressed the bar in each condition. The rats ate more food and larger meals, but not more frequent meals, at the patch offering the higher rate of food consumption, calculated as grams per minute. The relative intake at any patch was a function of the relative rate of intake during meals at that patch compared to the other patch. Rats respond to explicit manipulations of feeding time in the same manner as they respond to manipulations of feeding effort.     

Positive and negative conditioned suppression in the pigeon: Effects of the locus and modality of the CS

In Experiment I, four pigeons were exposed to trials in which a 12-sec key light illumination was followed by free food. These trials were superimposed upon a baseline of key pecking for food reinforcement on a variable-interval schedule. When the signal for food was on the operant key, response rate was substantially higher during the signal than during the baseline procedure. When the signal was on a second, signal key, operant responding was suppressed during the signal and substantial pecking of the signal key occurred. The sum of signal key and operant key pecks far exceeded the operant baseline rate of responding. An explanation of opposite results obtained with rats and pigeons as subjects in experiments of this type was suggested in terms of the spatial relation between the signal for free food and the operant target which usually characterizes these experiments. Experiment II assessed the importance of signal location when shock rather than food was the US. Suppression of operant key pecking was unaffected by signal location. Experiment III assessed the relative effectiveness of visual and auditory stimuli (clicks) as signals for food and shock, and found that all combinations of signal and US were equally effective in suppressing operant key responding. The three experiments together suggested that the identification of important effects of species—typical behavior in one experimental situation does not imply that there will be like effects in similar situations.     

Comparing Locomotion With Lever-press Travel In An Operant Simulation Of Foraging

An operant model of foraging was studied. Rats searched for food by pressing on the left lever, the patch, which provided one, two, or eight reinforcers before extinction (i.e., zero reinforcers). Obtaining each reinforcer lowered the probability of receiving another reinforcer, simulating patch depletion. Rats traveled to another patch by pressing the right lever, which restored reinforcer availability to the left lever. Travel requirement changed by varying the probability of reset for presses on the right lever; in one condition, additional locomotion was required. That is, rats ran 260 cm from the left to the right lever, made one response on the right lever, and ran back to a fresh patch on the left lever. Another condition added three hurdles to the 260-cm path. The lever-pressing and simple locomotion conditions generated equivalent travel times. Adding the hurdles produced longer times in patches than did the lever-pressing and simple locomotion requirements. The results contradict some models of optimal foraging but are in keeping with McNair's (1982) optimal giving-up time model and add to the growing body of evidence that different environments may produce different foraging strategies.     

GROUP FORAGING SENSITIVITY TO PREDICTABLE AND UNPREDICTABLE CHANGES IN FOOD DISTRIBUTION: PAST EXPERIENCE OR PRESENT CIRCUMSTANCES?

The ideal free distribution theory (Fretwell & Lucas, 1970) predicts that the ratio of foragers at two patches will equal the ratio of food resources obtained at the two patches. The theory assumes that foragers have "perfect knowledge" of patch profitability and that patch choice maximizes fitness. How foragers assess patch profitability has been debated extensively. One assessment strategy may be the use of past experience with a patch. Under stable environmental conditions, this strategy enhances fitness. However, in a highly unpredictable environment, past experience may provide inaccurate information about current conditions. Thus, in a nonstable environment, a strategy that allows rapid adjustment to present circumstances may be more beneficial. Evidence for this type of strategy has been found in individual choice. In the present experiments, a flock of pigeons foraged at two patches for food items and demonstrated results similar to those found in individual choice. Experiment 1 utilized predictable and unpredictable sequences of resource ratios presented across days or within a single session. Current foraging decisions depended on past experience, but that dependence diminished when the current foraging environment became more unpredictable. Experiment 2 repeated Experiment I with a different flock of pigeons under more controlled circumstances in an indoor coop and produced similar results.     

Time horizons in rats foraging for food in temporally separated patches

An important tenet of optimal foraging theory is that foragers compare prey densities in alternative patches to determine an optimal distribution of foraging behavior over time. A critical question is over what time period (time horizon) this integration of information and behavior occurs. Recent research has indicated that rats do not compare food density in a depleting patch with that in a rich patch delayed by an hour or more (Timberlake, 1984). In the present research we attempted to specify over what time period a future rich patch would affect current foraging. The effect of future food was measured by early entry into the rich patch (anticipation) and by a decrease in food obtained in the depleting patch (suppression). The rats showed anticipation of a rich patch up to an hour distant, but suppressed current feeding only if the rich patch was 16 min distant or less. The suppression effect appeared mediated by competition for expression between anticipatory entries into the rich patch and continued foraging in the depleting patch. These results suggest that optimal foraging is based on a variety of specific mechanisms rather than a general optimizing algorithm with a single time horizon.     

Visual Reinforcement Signals Interfere with the Effects of Reinforcer Magnitude Manipulations

Three experiments examined the effect of reinforcement magnitude on free-operant response rates. In Experiment 1, rats that received four food pellets responded faster than rats that received one pellet on a variable ratio 30 schedule. However, when the food hopper was illuminated during reinforcer delivery, there was no difference between the rates of response produced by the two magnitudes of reward. In Experiment 2, there was no difference in response rates emitted by rats receiving either one or four pellets of food as reward on a random interval (RI) 60-s schedule. In Experiment 3, rats responding on an RI 30-s schedule did so at a lower rate with four pellets as reinforcement than with one pellet. This effect was abolished by the illumination of the food hopper during reinforcement delivery. These results indicate that the influence of magnitude is obscured by manipulations which signal the delivery of reinforcement.     

Testing a stochastic foraging model in an operant simulation: Agreement with qualitative but not quantitative predictions

An operant simulation of foraging through baited and empty patches was studied with 4 pigeons. On a three-key panel, side keys were designated as patches, and successive opportunities to complete 16 fixed-ratio 10 schedules on side keys were defined as encounters with feeders. In a random half of the patches in any session, some of the fixed-ratio 10 schedules yielded reinforcement (baited feeders) and the other schedules yielded nonreinforcement (empty feeders). In the other half of the patches, all feeders were empty. Pigeons could travel between patches at any time by completing a fixed-ratio schedule on the center key. An optimal foraging model was tested in Experiments 1 and 2 by varying center-key travel time and number of baited feeders in baited patches. The ordinal predictions that number of feeders visited in empty patches would increase with travel time and decrease as number of baited feeders increased were supported, but pigeons visited far more feeders in empty patches than the optimal number predicted by the model to maximize energy/time. In Experiment 3, evidence was found to suggest that the number of empty feeders encountered before the first baited feeder in baited patches is an important factor controlling leaving empty patches.     

Patch choice as a function of procurement cost and encounter rate.

The effects of patch encounter rate on patch choice and meal patterns were studied in rats foraging in a laboratory environment offering two patch types that were encountered sequentially and randomly. The cost of procuring access to one patch was greater than the other. Patches were either encountered equally often or the high-cost patch was encountered more frequently. As expected, rats exploited the low-cost patch on almost 100% of encounters and exploited the high-cost patch on a percentage of encounters that was inversely proportional to its cost. Meal size was the same at both patches. Surprisingly, when low-cost patches were rare, the rats did not increase their use of high-cost patches. This resulted in spending more time and energy searching for patches and a higher average cost per meal. The rats responded to this increased cost by reducing the frequency and increasing the size of meals at both patches and thereby limited total daily foraging cost and conserved total intake.     

Temporal control in a complex environment: An analysis of schedule-related behavior

Three experiments conducted in an automated ten-compartment chamber recorded collateral activities of rats reinforced for lever pressing on differential-reinforcement-of-low-rate schedules. In Experiment 1, the rate of lever pressing increased when stimulus support for collateral activities was removed, thus confirming earlier findings. However, there were no temporal or sequential patterns of collateral activities that predicted operant responding. In Experiment 2, the rate of lever pressing increased only if (a) access to all stimulus support for collateral activities was simultaneously prevented, and (b) the rat was forced to remain in the presence of the lever and food tray. The availability of any of the stimuli related to collateral activity was sufficient to keep lever-pressing rates from increasing. Experiment 3 examined collateral activities under a signaled differential-reinforcement-of-low-rate schedule. Preventing access to stimuli supporting collateral activities had little effect on stable lever pressing when the signal was maintained. When the signal was removed, collateral activities continued, but lever-pressing rates increased in three of the four rats and rates of food presentation declined in all rats. Hypotheses that collateral activities have (a) a timekeeping or discriminative function, or (b) directly inhibit operant responding were not supported. The results suggest that collateral activities may facilitate operant responding by simply removing the subject from the presence of reinforcement-related stimuli.     

Timing and Foraging: Gibbon's Scalar Expectancy Theory and Optimal Patch Exploitation

We present a study that links optimal foraging theory (OFT) to behavioral timing. OFT's distinguishing feature is the use of models that compute the most advantageous behavior for a particular foraging problem and compare the optimal solution to empirical data with little reference to psychological processes. The study of behavioral timing, in contrast, emphasizes performance in relation to time, most often without strategic or functional considerations. In three experiments, reinforcer-maximizing behavior and timing performance are identified and related to each other. In all three experiments starlings work in a setting that simulates food patches separated by a flying distance between the two perches. The patches contain a variable and unpredictable number of reinforcers and deplete suddenly without signal. Before depletion, patches deliver food at fixed intervals (FI). Our main dependent variables are the times of occurrence of three behaviors: the “peak” in pecking rate (Peak), the time of the last peck before “giving in” (GIT), and the time for “moving on” to a new patch (MOT). We manipulate travel requirement (Experiment 1), level of deprivation and FI (Experiment 2), and size of reinforcers (Experiment 3). For OFT, Peak should equal the FI in all conditions while GIT and MOT should just exceed it. Behavioral timing and Scalar Expectancy Theory (SET) in particular predict a Peak at around the FI and a longer (unspecified) GIT, and make no prediction for MOT. We found that Peak was close to the FI and GIT was approximately 1.5 times longer, neither being affected by travel, hunger, or reinforcer size manipulations. MOT varied between 1.5 and just over 3 times the FI, was responsive to both travel time and the FI, and did not change when the reinforcer rate was manipulated. These results support the practice of producing models that explicitly separate information available to the subject from strategic use of this information.     

Effect of signaled reinforcement on response variability

Three experiments examined the effect of signaling reinforcement on rats' lever pressing on contingencies that reinforced variable responding to extend the exploration of signaled reinforcement to a schedule that has previously not been examined in this respect. In Experiment 1, rats responding on a lag-8 variability schedule with signaled reinforcement displayed greater levels of variability (U values) than rats on the same schedule lacking a reinforcement signal. In Experiment 2, rats responding on a differential reinforcement of least frequent responses schedule also displayed greater operant variability with a signal for reinforcement compared with rats without a reinforcement signal. In Experiment 3, a reinforcement signal decreased the variability of a response sequence when there was no variability requirement. These results offer empirical corroboration that operant variability responds to manipulations in the same manner as do other forms of operant response and that a reinforcement signal facilitates the emission of the required operant.     

Resistance to the response-decrementing effects of response-independent reinforcement produced by delay and non-delay schedules of reinforcement

In two experiments, animals were initially exposed to response-dependent schedules of food before exposure to response-independent reinforcement matched for overall rate and temporal distribution of reinforcers to the preceding condition. In Experiment I, response decrements during the response-independent phase were smaller after delayed reinforcement training than after a comparable immediate reinforcement schedule, for both doves and rats. In Experiment II variable-interval and variable-ratio schedules, both with either immediate or delayed reinforcement, were used with rats. Both the delayed reinforcement schedules produced resistance to subsequent response-independent reinforcement, but response decrements were larger after either of the immediate reinforcement conditions. It was concluded that the critical factor in response maintenance under response-independent reinforcement was the type of response-reinforcer contiguities permitted under the response-dependent schedule rather than perception of response-reinforcer “contingencies”. If the response-dependent schedule was arranged so that behaviours other than a designated operant (key pecking or lever pressing) could be contiguous with food, responding was maintained well under response-independent schedules.     

Effects of pre-operant running and sucrose concentration on operant wheel running on a fixed interval schedule of reinforcement

Prior research proposed that temporal control over the pattern of operant wheel running on a fixed interval (FI) schedule of sucrose reinforcement is a function of automatic reinforcement generated by wheel running and the experimentally arranged sucrose reinforcement. Two experiments were conducted to assess this prediction. In the first experiment, rats ran for different durations (0, 30, 60, and 180 min) prior to a session of operant wheel running on a FI 120-s schedule. In the second experiment, the concentration of sucrose reinforcement on a FI 180-s schedule was varied across values of 0, 5, 15, and 25%. In Experiment 1, as the duration of pre-operant running increased, the postreinforcement pause before initiation of running lengthened while wheel revolutions in the latter part of the FI interval increased. In Experiment 2, wheel revolutions markedly increased then decreased to a plateau early in the FI interval. Neither manipulation increased temporal control of the pattern of wheel running. Instead, results indicate that operant wheel running is regulated by automatic reinforcement generated by wheel activity and an adjunctive pattern of running induced by the temporal presentation of sucrose. Furthermore, the findings question whether the sucrose contingency regulates wheel running as a reinforcing consequence.     

Instrumental performance on negative schedules

Three experiments examined the performance of rats pressing a lever for food reinforcement on a schedule in which high rates of response resulted in lowered rates of reinforcement (i.e. a schedule with a negative component). In Experiment 1, rats responded on a variable interval (VI) schedule with a conjoint component such that every 30 responses a reinforcement programmed by the VI schedule was cancelled. These subjects generally emitted a lower response rate than rats responding on a VI schedule yoked to the former subjects with respect to the delivery of reinforcement, although response rate differences were sometimes not large. Similar response-rate effects were obtained in Experiment 2 using a within-subject yoking procedure. In Experiment 3, reinforced interresponse times were matched on negative and VI schedules yoked in terms of reinforcement rate, and the response rate emitted in these conditions were similar. These results give support to theories of instrumental conditioning that stress the strengthening and shaping properties of reinforcement.     

Response-independent Events In The Behavior Stream

The metaphor of the behavior stream provides a framework for studying the effects of response-independent food presentations intruded into an environment in which operant responding of pigeons was maintained by variable-interval schedules. In the first two experiments, response rates were reduced when response-independent food was intruded during the variable-interval schedule according to a concomitantly present fixed-time schedule. These reductions were not always an orderly function of the percentage of response-dependent food. Negatively accelerated patterns of key pecking across the fixed-time period occurred in Experiment 1 under the concomitant fixed-time variable-interval schedules. In Experiment 2, positively and negatively accelerated and linear response patterns occurred even though the schedules were similar to those used in Experiment 1. The variable findings in the first two experiments led to three subsequent experiments that were designed to further illuminate the controlling variables of the effects of intruded response-independent events. When the fixed and variable schedules were correlated with distinct operanda by employing a concurrent fixed-interval variable-interval schedule (Experiment 3) or with distinct discriminative stimuli (Experiments 4 and 5), negatively accelerated response patterns were obtained. Even in these latter cases, however, the response patterns were a joint function of the physical separation of the two schedules and the ratio of fixed-time or fixed-interval to variable-interval schedule food presentations. The results of the five experiments are discussed in terms of the contributions of both reinforcement variables and discriminative stimuli in determining the effects of intruding response-independent food into a stream of operant behavior.     

Response persistence under ratio and interval reinforcement schedules

In Experiment 1, rats were exposed to progressive-ratio schedules of food reinforcement while other rats were exposed simultaneously to yoked-interval schedules that arranged equivalent interreinforcer intervals but required only a single response at the end of the interval for food delivery. In Experiment 2, a within-subject yoked-control procedure was employed in which pigeons were exposed to alternating sessions (one per day) of progressive-ratio schedules and yoked-interval schedules as described above. In both experiments, responding under the yoked-interval schedule persisted beyond the point at which responding under the progressive-ratio schedule had ceased. The progressive-ratio schedules controlled break-and-run distributions, and the yoked-interval schedules controlled more even distributions of responses in time. Response rates decreased and postreinforcement pauses increased over time within individual sessions under both schedules. The results suggest that responding maintained by interval schedules is more persistent than that maintained by ratio schedules. The limitations and implications of this conclusion are discussed in the context of other investigations of response strength and behavioral momentum.     

Signal duration and suppression of operant responding by free reinforcement

Pigeons were trained on a VI (variable interval) schedule of food presentation with a superimposed schedule of response-independent food. Substantial suppression of the operant response rate occurred when the free food was presented without a signal. When the free food was preceded by a short (4 sec) signal, the degree of suppression was similar to that with unsignaled free food. But when the signal was lengthened to 12 sec, the degree of suppression was substantially reduced. Experiment 3 assessed the effect of signal duration using a baseline schedule of delayed reinforcement, in which contingent reinforcers were themselves preceded by a signal. The signal preceding the free reinforcers was then either the same as or different from this contingent signal. Signal duration effects occurred only when the two types of signals were different. These differences as a function of signal duration have implications for both “context-blocking” and “comparator” interpretations of the effects of noncontingent reinforcement in both Pavlovian and operant procedures.     

DISCRIMINATION OF VARIABLE SCHEDULES IS CONTROLLED BY INTERRESPONSE TIMES PROXIMAL TO REINFORCEMENT

In Experiment 1, food‐deprived rats responded to one of two schedules that were, with equal probability, associated with a sample lever. One schedule was always variable ratio, while the other schedule, depending on the trial within a session, was: (a) a variable‐interval schedule; (b) a tandem variable‐interval, differential‐reinforcement‐of‐low‐rate schedule; or (c) a tandem variable‐interval, differential‐reinforcement‐of‐high‐rate schedule. Completion of a sample‐lever schedule, which took approximately the same time regardless of schedule, presented two comparison levers, one associated with each sample‐lever schedule. Pressing the comparison lever associated with the schedule just presented produced food, while pressing the other produced a blackout. Conditional‐discrimination accuracy was related to the size of the difference in reinforced interresponse times and those that preceded it (predecessor interresponse times) between the variable‐ratio and other comparison schedules. In Experiment 2, control by predecessor interresponse times was accentuated by requiring rats to discriminate between a variable‐ratio schedule and a tandem schedule that required emission of a sequence of a long, then a short interresponse time in the tandem's terminal schedule. These discrimination data are compatible with the copyist model from Tanno and Silberberg (2012) in which response rates are determined by the succession of interresponse times between reinforcers weighted so that each interresponse time's role in rate determination diminishes exponentially as a function of its distance from reinforcement.     

Punishment of schedule-induced drinking in rats by signaled and unsignaled delays in food presentation

Food-deprived rats were exposed to a fixed-time 60-s schedule of food-pellet presentation and developed schedule-induced drinking. Using an ABA reversal design, three experiments investigated the effects of events then made dependent on licks. In Experiment 1, lick-dependent signaled delays (10 s) in food presentation in general led to decreased drinking, which recovered when the signaled delays were discontinued. The drinking of yoked-control rats, which received food at the same times as those exposed to the signaled-delay contingency, showed much smaller changes. Experiment 2 showed that 10-s lick-dependent signals alone did not reduce drinking. In Experiment 3, when licks produced unsignaled 10-s delays in food there were less marked and more gradual changes in drinking than in Experiment 1, although these effects again were greater than with yoked-control animals. We concluded that both signaled and unsignaled delays functioned as punishers of drinking. These findings support the view that schedule-induced drinking, like operant behavior, is subject to control by its consequences.