Session duration and the VI response function: Within-session prospective and retrospective effects

Two experiments examined the effects of session duration on responding during simple variable-interval schedules. In Experiment 1, rats were exposed to a series of simple variable-interval schedules differing in both session duration (10 min or 30 min) and scheduled reinforcement rate (7.5 s, 15 s, 30 s, and 480 s). The functions relating response rate to reinforcement rate were predominantly monotonic for the short (10-min) sessions but were predominantly bitonic for the long (30-min) sessions, when data from the entire session were considered. Examination of responding within sessions suggested that differences in the whole-session data were produced by a combination of prospective processes (i.e., processes based on events scheduled to occur later in the session) and retrospective processes (i.e., processes based on events that had already occurred in the session). In Experiment 2, rats were exposed to a modified discrimination procedure in which pellet flavor (standard or banana) predicted session duration (10 min or 30 min). All rats came to respond faster during the short (10-min) sessions than during the first 10 min of the long sessions. As in Experiment 1, the results seemed to reflect the simultaneous operation of both prospective and retrospective processes. The results shed light on the recent controversy over the form of the variable-interval response function by identifying one variable (session duration) and two types of processes (prospective and retrospective) that influence responding on these schedules.     

Interresponse-time sensitivity during discrete-trial and free-operant concurrent variable-interval schedules

Two experiments investigated the sensitivity of pigeons' choice to elapsed time since the last response (i.e., to inter-response time [IRT]) during concurrent variable-interval variable-interval schedules. Experiment 1 used a two-key discrete-trial procedure with variable intertrial intervals. Experiment 2 employed a three-key free-operant procedure. In both experiments choice was found to be a function of the active-schedule IRT, defined as the time since the most recent response. Monte Carlo simulations show how this finding permits the joining of several seemingly incompatible data sets held to both support and contradict a kind of choice strategy, termed momentary maximizing, which attempts to maximize momentary reinforcement probabilities. The studies suggest that only two variables are needed to describe the static molecular structure of concurrent variable-interval choice: active-schedule IRTs and "response states" consisting of the last one or two schedule choices.     

Multiple determinants of the effects of reinforcement magnitude on free-operant response rates

Four experiments examined the effects of increasing the number of food pellets given to hungry rats for a lever-press response. On a simple variable-interval 60-s schedule, increased number of pellets depressed response rates (Experiment 1). In Experiment 2, the decrease in response rate as a function of increased reinforcement magnitude was demonstrated on a variable-interval 30-s schedule, but enhanced rates of response were obtained with the same increase in reinforcement magnitude on a variable-ratio 30 schedule. In Experiment 3, higher rates of responding were maintained by the component of a concurrent variable-interval 60-s variable-interval 60-s schedule associated with a higher reinforcement magnitude. In Experiment 4, higher rates of response were produced in the component of a multiple variable-interval 60-s variable-interval 60-s schedule associated with the higher reinforcement magnitude. It is suggested that on simple schedules greater reinforcer magnitudes shape the reinforced pattern of responding more effectively than do smaller reinforcement magnitudes. This effect is, however, overridden by another process, such a contrast, when two magnitudes are presented within a single session on two-component schedules.     

Income maximizing on concurrent ratio-interval schedules of reinforcement

Three experiments examined the effect of food availability on pigeons' choice behavior under concurrent schedules of reinforcement. In Experiment 1, 3 pigeons earned their daily food ration by choosing, in 30-min sessions, between concurrent variable-ratio 30 variable-interval 40-s schedules. Food presentations during both schedules lasted 2 or 12 s, depending upon the condition. Relative variable-ratio response rate was inversely related to hopper duration. In Experiment 2, 4 pigeons received their daily feeding by responding on the same schedule pair as in Experiment 1 (with 4-s food presentations) in sessions that varied in length from 10 to 30 min, depending on the condition. The length of a vertical slit projected on a response key increased with time so that “passage of time” might be more easily discriminable. As session duration decreased, relative variable-ratio response rate increased. In Experiment 3, 4 pigeons chose between two variable-interval 40-s schedules. One schedule operated without regard to the schedule selected, whereas the other operated only when the subject responded in its presence (dependent). Although these schedules had the same feedback function, preference for the dependent variable interval increased as session duration decreased from 30 to 10 min. The preference changes in these studies reveal the operation of an income-maximizing process in choice.     

Control rate of response or reinforcement and amphetamine's effect on behavior.

The roles of control response rate and reinforcement frequency in producing amphetamine's effect on operant behavior were evaluated independently in rats. Two multiple schedules were arranged in which one variable, either response rate or reinforcement frequency, was held constant and the other variable manipulated. A multiple differential-reinforcement-of-low-rate seven-second yoked variable-interval schedule was used to equate reinforcement frequencies at different control response rates between multiple-schedule components. Amphetamine increased responding under the variable-interval component. In contrast, amphetamine decreased responding equivalently between components of a multiple random-ratio schedule that produced similar control response rates at different reinforcement frequencies. The results provide experimental support to the rate-dependency principle that control rate of responding is an important determinant of amphetamine's effect on operant behavior.     

Control of schedule-induced polydipsi: type, size, and spacing of meals

Rats were given daily 1-min variable-interval sessions for several types of food delivered in various amounts per reinforcement and the concurrent, schedule-induced polydipsia was measured. Dry, solid food was neither a necessary nor sufficient condition for the development of polydipsia. Small portions of liquid Standard Monkey Diet produced polydipsia, but 45-mg dextrose or sucrose pellets did not. Within the range studied, smaller portions of both solid and liquid foods produced more drinking than larger portions per reinforcement. Two-min variable-interval sessions produced a greater polydipsic response than 1-min variable-interval, even though the number of 45-mg Noyes pellets allowed per session was held constant. Polydipsia was greatly attenuated on these schedules when the number of pellets remained constant, but were delivered two at a time. Within the ranges studied, the concurrent polydipsic response was increased by decreasing the rate of food acquisition, either by using smaller portions of food per reinforcement or by increasing the interreinforcement time.     

Income and choice between different goods

In Experiment 1, 3 rats chose between two simultaneously operating variable-interval schedules, one of which provided saccharin water and the other, food. In five conditions, the absolute (and equal) reinforcement rates provided by the pair of equal-valued schedules were manipulated in the range of 36 to 240 per hour. Experiment 2 was identical to Experiment 1 except that these schedules operated successively, arranged by requiring the rat to stand on the side of the chamber correlated with each schedule. Food/saccharin choice ratios were inversely related to reinforcement rate in both experiments, although this effect was stronger in Experiment 2. When delivery rates were high, preference for food over saccharin often reversed as the session progressed. The results were interpretable in terms of economic accounts of choice (e.g., the minimum-needs hypothesis), as well as in terms of traditional psychological accounts (e.g., matching theory).     

A comparison of ratio and interval reinforcement schedules with comparable interreinforcement times

Pigeons were trained to peck keys on fixed-ratio and fixed-interval schedules of food reinforcement. Both schedules produced a pattern of behavior characterized as pause and run, but the relation of pausing to time between reinforcers differed for the two schedules even when mean time between reinforcers was the same. Pausing in the fixed ratio occupied less of the time between reinforcers for shorter interreinforcer times. For two of three birds, the relation was reversed at longer interreinforcer times. As an interreinforcer time elapsed, there was an increasing tendency to return to responding for the fixed interval, but a roughly constant tendency to return to responding for the fixed-ratio schedule. In Experiment 1 these observations were made for both single-reinforcement schedules and multiple schedules of fixed-ratio and fixed-interval reinforcement. In Experiment 2 the observations were extended to a comparison of fixed-ratio versus variable-interval reinforcement schedules, where the distribution of interreinforcement times in the variable interval approximated that for the fixed ratio.     

Concurrent schedules: Spatial separation of response alternatives

Four pigeons were exposed to independent concurrent variable-interval 20-second variable-interval 60-second schedules of reinforcement. A transparent partition was inserted midway between the two response keys. The length of the partition was systematically manipulated. Increasing partition length produced a decrease in changeover rate in Experiment 1. Over-matching was observed with a partition length of 20 centimeters. In Experiment 2 a four-second limited hold was added to the schedules. Increasing partition length produced a decrease in changeover rate that exceeded the decrease observed in Experiment 1. This manipulation produced nearly exclusive choice of the variable-interval 20-second component. The present results, together with results obtained in related research, suggest that deviation from matching is a function of procedural variables that determine the consequences of a changeover response.     

Behavioral contrast in competitive and noncompetitive environments

Three experiments examined the effects of opportunities for an alternative response (drinking) on positive behavioral contrast of rats' food-reinforced bar pressing. In both Experiments 1 and 2 the baseline multiple variable-interval schedules were rich (variable interval 10-s), and contrast was examined both with and without a water bottle present. In Experiment 1, the rats were not water deprived. When one component of the multiple schedule was changed to extinction, the rate of bar pressing increased in the constant component (positive behavioral contrast). The magnitude of contrast was larger when the bottle was absent than when it was present, as predicted by the matching law. Drinking did not shift from the constant variable-interval component to the extinction component, as might have been expected from competition theory. In Experiment 2, the rats were water deprived. Contrast was larger when the bottle was present than when it was absent, and drinking did shift to the extinction component, as predicted by competition theory. In Experiment 3, water-deprived rats responded on leaner multiple variable-interval schedules (60-s) in the presence of a water bottle. When one component was changed to extinction, contrast did not occur, and drinking did not shift to the extinction component. The present results suggest that there are at least two different sources of behavioral contrast: “competitive” contrast, observed when an alternative response occurs with high probability, and “noncompetitive” contrast, observed when an alternative response occurs with low probability. The results, in conjunction with earlier studies, also suggest that the form of the alternative response and the rate of food reinforcement provided by the multiple schedule combine to determine the amount of contrast.     

Response-independent Events In The Behavior Stream

The metaphor of the behavior stream provides a framework for studying the effects of response-independent food presentations intruded into an environment in which operant responding of pigeons was maintained by variable-interval schedules. In the first two experiments, response rates were reduced when response-independent food was intruded during the variable-interval schedule according to a concomitantly present fixed-time schedule. These reductions were not always an orderly function of the percentage of response-dependent food. Negatively accelerated patterns of key pecking across the fixed-time period occurred in Experiment 1 under the concomitant fixed-time variable-interval schedules. In Experiment 2, positively and negatively accelerated and linear response patterns occurred even though the schedules were similar to those used in Experiment 1. The variable findings in the first two experiments led to three subsequent experiments that were designed to further illuminate the controlling variables of the effects of intruded response-independent events. When the fixed and variable schedules were correlated with distinct operanda by employing a concurrent fixed-interval variable-interval schedule (Experiment 3) or with distinct discriminative stimuli (Experiments 4 and 5), negatively accelerated response patterns were obtained. Even in these latter cases, however, the response patterns were a joint function of the physical separation of the two schedules and the ratio of fixed-time or fixed-interval to variable-interval schedule food presentations. The results of the five experiments are discussed in terms of the contributions of both reinforcement variables and discriminative stimuli in determining the effects of intruding response-independent food into a stream of operant behavior.     

Schedule-induced licking during multiple schedules

Schedule-induced polydipsia was studied in rats bar pressing under two-component multiple schedules of food reinforcement. The first component of the multiple schedule was a variable-interval 1-min schedule throughout the experiment. The schedule comprising the second component was varied over blocks of sessions in terms of rate and magnitude of reinforcement, and was either variable-interval 3-min (one pellet), variable-interval 3-min (three pellets), variable-interval 1-min (one pellet), or extinction. Water intake per session varied with the rate of reinforcement in the schedule comprising the second component and was highest when the schedule was variable-interval 1-min. Both bar-pressing behavior and licking behavior showed behavioral interactions between the two components of the multiple schedules. With magnitude of reinforcement held constant, a matching relationship was observed between lick rate and reinforcement rate; the relative frequency of licks in the constant component matched the relative frequency of reinforcement in that component. Bar pressing, however, showed only a moderate degree of relativity matching. During the schedule-induced licking, a burst of licking followed each delivery of a pellet (post-prandial drinking). The duration of these bursts of licking was observed to be a function of the inter-reinforcement interval.     

Effects of variations in the temporal distribution of reinforcements on interval schedule performance

Pigeons were exposed to variable-interval and fixed-interval schedules and schedules approximating variable-interval and fixed-interval schedules. The probabilities of the variable-interval and fixed-interval components in a mixed fixed-interval variable-interval schedule in Experiment I and the minimum and maximum interreinforcement intervals in Experiment II in a variable-interval schedule were manipulated to create intermediate schedule contingencies and contingencies approximating simple variable-interval or fixed-interval contingencies. Maximal control by time as defined by quantitative indices of the temporal pattern of response occurred as fixed-interval contingencies were approximated and minimal control occurred as variable-interval contingencies were approximated. Changes in the temporal pattern of response were systematically related to changes in the temporal distribution of reinforcements with both procedural definitions for manipulating the temporal distribution of reinforcements.     

Effects of signaling on temporal control of behavior in response‐initiated fixed intervals

Behavior and events distributed in time can serve as markers that signal delays to future events. The majority of timing research has focused on how behavior changes as the time to some event, usually food availability, decreases. The primary objective of the two experiments presented here was to assess how behavior changes as time passes between two time markers when the first time marker was manipulated but the second, food delivery, was held constant. Pigeons were exposed to fixed‐interval, response‐initiated fixed‐interval, and signaled response‐initiated fixed‐interval 15‐ and 30‐s schedules of reinforcement. In Experiment 1, first‐response latencies were systematically shorter in the signaled response‐initiated schedules than response‐initiated schedules, suggesting that the first response was a more effective time marker when it was signaled. In Experiment 2, responding in no‐food (i.e. “peak”) trials indicated that timing accuracy was equivalent in the three schedule types. Compared to fixed interval schedules, timing precision was reduced in the signaled response‐initiated schedules and was lowest in response‐initiated schedules. Results from Experiments 1 and 2 coupled with previous research suggest that the overall “informativeness” of a time marker relative to other events and behaviors in the environment may determine its efficacy.     

Stock optimizing: maximizing reinforcers per session on a variable-interval schedule.

In Experiment 1, 2 monkeys earned their daily food ration by pressing a key that delivered food according to a variable-interval 3-min schedule. In Phases 1 and 4, sessions ended after 3 hr. In Phases 2 and 3, sessions ended after a fixed number of responses that reduced food intake and body weights from levels during Phases 1 and 4. Monkeys responded at higher rates and emitted more responses per food delivery when the food earned in a session was reduced. In Experiment 2, monkeys earned their daily food ration by depositing tokens into the response panel. Deposits delivered food according to a variable-interval 3-min schedule. When the token supply was unlimited (Phases 1, 3, and 5), sessions ended after 3 hr. In Phases 2 and 4, sessions ended after 150 tokens were deposited, resulting in a decrease in food intake and body weight. Both monkeys responded at lower rates and emitted fewer responses per food delivery when the food earned in a session was reduced. Experiment 1's results are consistent with a strength account, according to which the phases that reduced body weights increased food's value and therefore increased subjects' response rates. The results of Experiment 2 are consistent with an optimizing strategy, because lowering response rates when food is restricted defends body weight on variable-interval schedules. These contrasting results may be attributed to the discriminability of the contingency between response number and the end of a session being greater in Experiment 2 than in Experiment 1. In consequence, subjects lowered their response rates in order to increase the number of reinforcers per session (stock optimizing).     

Momentary maximizing in concurrent schedules with a minimum interchangeover interval

Eight pigeons were trained on concurrent variable-interval variable-interval schedules with a minimum interchangeover time programmed as a consequence of changeovers. In Experiment 1 the reinforcement schedules remained constant while the minimum interchangeover time varied from 0 to 200 s. Relative response rates and relative time deviated from relative reinforcement rates toward indifference with long minimum interchangeover times. In Experiment 2 different reinforcement ratios were scheduled in successive experimental conditions with the minimum interchangeover time constant at 0, 2, 10, or 120 s. The exponent of the generalized matching equation was close to 1.0 when the minimum interchangeover time was 0 s (the typical procedure for concurrent schedules without a changeover delay) and decreased as that duration was increased. The data support the momentary maximizing theory and contradict molar maximizing theories and the melioration theory.     

Closed-economy multiple-schedule performance: Effects of deprivation and session duration

Three pigeons responded for food reinforcement on multiple variable-interval schedules in which the total consumption of food was entirely determined by the subjects' interaction with the schedules (a closed economy). The finding of overmatching, where response allocation between components is more extreme than the distribution of reinforcers, was reconfirmed. Generalized-matching sensitivity decreased from overmatching to undermatching values typical of conventional multiple schedules when food deprivation was increased by decreasing session duration, but not when deprivation was increased by decreasing overall reinforcer rate. Sensitivity also increased from undermatching to overmatching as session duration increased from 100 min to 24 hr, while deprivation was held constant by decreasing overall reinforcer rate. These results can be understood in terms of increases in the value of extraneous reinforcers relative to food reinforcers as deprivation decreases or as the economy for extraneous reinforcers becomes more closed. However, no published quantitative expression of the effects of extraneous reinforcers is entirely consistent with the results.     

The matching law in and within groups of rats

In each of the two experiments, a group of five rats lived in a complex maze containing four small single-lever operant chambers. In two of these chambers, food was available on variable-interval schedules of reinforcement. In Experiment I, nine combinations of variable intervals were used, and the aggregate lever-pressing rates (by the five rats together) were studied. The log ratio of the rates in the two chambers was linearly related to the log ratio of the reinforcement rates in them; this is an instance of Herrnstein's matching law, as generalized by Baum. Summing over the two food chambers, food consumption decreased, and response output increased, as the time required to earn each pellet increased. In Experiment II, the behavior of individual rats was observed by time-sampling on selected days, while different variable-interval schedules were arranged in the two chambers where food was available. Individual lever-pressing rates for the rats were obtained, and their median bore the same “matching” relationship to the reinforcement rates as the group aggregate in Experiment I. There were differences between the rats in their distribution of time and responses between the two food chambers; these differences were correlated with differences in the proportions of reinforcements the rats obtained from each chamber.     

Contrast and undermatching as a function of reinforcer duration and quality during multiple schedules

Eight pigeons pecked keys under multiple variable-interval two-minute variable-interval two-minute schedules. In Experiment 1, the reinforcers were 2, 4, or 8 seconds access to a food magazine. In Experiments 2 and 3, the reinforcers were grains that had been determined to be most-, moderately-, or non-preferred. Both positive and negative behavioral contrast occurred when the reinforcers in one component were held constant and the duration or type of reinforcer obtained in the other component varied. Undermatching occurred when the relative rate of responding during a component was plotted as a function of the relative duration of the reinforcers in that component.     

Choice in the time-left procedure and in concurrent chains with a time-left terminal link.

In two experiments, rats chose between a standard fixed-duration food-associated stimulus and a stimulus whose duration was the time remaining to reinforcement in an elapsing comparison interval. In Experiment 1, 4 rats responded in a time-left procedure wherein a single initial-link variable-interval schedule set up two potential terminal links simultaneously. As time elapsed in the initial-link schedule, the choice was between a standard fixed-interval 30-s terminal link and a time-left terminal link whose programmed interval requirement equaled 90 s minus the elapsed time in the initial link. Rats generally responded more on the lever with the shortest programmed terminal-link duration, but the temporal parameters of the procedure were found to vary with response distributions. Contrary to previous reports, therefore, time-left data were well predicted by choice models that make no assumptions about animal timing. In Experiment 2, 8 rats responded on a concurrent-chains schedule with independent variable-interval initial links and a time-left terminal link in one of the choice schedules. On the time-left lever, the programmed terminal-link delay equaled 90 s minus the elapsed time in the time-left initial link. On the standard lever, terminal-link responses were reinforced according to a variable-interval schedule whose average value varied over four conditions. Relative time-left initial-link responses increased in the elapsing time-left initial-link schedule as the time-left terminal link became shorter relative to the standard terminal link. Scalar expectancy theory failed to predict the resultant data, but a modified version of the delay-reduction model made good predictions. An analysis of the elaboration of scalar expectancy theory for variable delays demonstrated that the model is poorly formulated for arithmetically distributed delays.