Delay reduction: current status.

Delay-reduction theory states that the effectiveness of a stimulus as a conditioned reinforcer may be predicted most accurately by the reduction in time to primary reinforcement correlated with its onset. We review support for the theory and then discuss two new types of experiments that assess it. One compares models of choice in situations wherein the less preferred outcome is made more accessible; the other investigates whether frequency of conditioned reinforcement affects choice beyond the effect exerted by frequency of primary reinforcement.     

Momentary maximizing in concurrent schedules with a minimum interchangeover interval

Eight pigeons were trained on concurrent variable-interval variable-interval schedules with a minimum interchangeover time programmed as a consequence of changeovers. In Experiment 1 the reinforcement schedules remained constant while the minimum interchangeover time varied from 0 to 200 s. Relative response rates and relative time deviated from relative reinforcement rates toward indifference with long minimum interchangeover times. In Experiment 2 different reinforcement ratios were scheduled in successive experimental conditions with the minimum interchangeover time constant at 0, 2, 10, or 120 s. The exponent of the generalized matching equation was close to 1.0 when the minimum interchangeover time was 0 s (the typical procedure for concurrent schedules without a changeover delay) and decreased as that duration was increased. The data support the momentary maximizing theory and contradict molar maximizing theories and the melioration theory.     

TEMPORAL CONTROL ON INTERVAL SCHEDULES: WHAT DETERMINES THE POSTREINFORCEMENT PAUSE?

On fixed-interval or response-initiated delay schedules of reinforcement, the average pause following food presentation is proportional to the interfood interval. Moreover, when a number of intervals of different durations occur in a programmed cyclic series, postreinforcement pauses track the changes in interval value. What controls the duration of postreinforcement pauses under these conditions? Staddon, Wynne, and Higa (1991), in their linear waiting model, propose control by the preceding interfood interval. Another possibility is that delay to reinforcement, signaled by a key peck and/or stimulus change, determines the subsequent pause. The experiments reported here examined the role of these two possible time markers by studying the performance of pigeons under a chained cyclic fixed-interval procedure. The data support the linear waiting model, but suggest that more than the immediately preceding interfood interval plays a role in temporal control.     

Additional-delay schedules: A continuum of temporal contingencies by varying food delay

Pigeons performed on discrete-trial, temporally defined schedules in which the food delay (D) was adjusted according to the latency of the key peck (X) and two schedule parameters (t and A). The schedule function was D = A(t − X), where D is the experienced delay between a response and a reinforcer. The schedule parameter t is the maximum value below which the present contingencies occur. A is the additional delay to reinforcement for each second the response latency is shorter than the t value. When A = 0 s, the schedule is a continuous reinforcement schedule with immediate reinforcement. When A = 1 s, the schedule is a conjunctive fixed-ratio 1 fixed-time t-s schedule. When A approaches infinity, the schedule becomes a differential reinforcement of long latency schedule. The latencies for subjects with t = 10 s and t = 30 s were observed with the present schedules having seven values for A between 0 s and 11 s. In addition, the latencies for subjects for which t = 30 s were observed at an A value of 31 s to 41 s. As the A value increased, the latencies approached the t value for subjects for which t = 10 s. The latencies for 30-s-t subjects did not approach t, even when the A value was 41 s. The latencies for 10-s-t subjects at 11-s A value were longer than those under yoked conditions having exactly the same delays/interreinforcement intervals. These results demonstrated a continuum of latency related to the schedule continuum (value of A) at a small t value.     

Timeout from concurrent schedules.

Response-contingent timeouts of equal duration and frequency were added to both alternatives of unequal concurrent schedules of reinforcement. For each of 4 pigeons in Experiment 1, relative response rates generally became less extreme as the frequency of timeout increased. In Experiment 2, relative response rates consistently approached indifference as the duration of timeout was increased. Variation in time allocation was less consistent in both experiments. Absolute response rates did not vary with the timeout contingency in either experiment. In a third experiment, neither measure of choice varied systematically when the duration of a postreinforcement blackout was varied. In contrast to the present results, preference has been shown to vary directly with the parameters of shock delivery in related procedures. The pattern of results in the first two experiments follows that obtained with other manipulations of the overall rate of reinforcement in concurrent schedules. The results of the third experiment suggest that an intertrial interval following reinforcement is not a critical feature of the overall rate of reinforcement.     

Response-independent Events In The Behavior Stream

The metaphor of the behavior stream provides a framework for studying the effects of response-independent food presentations intruded into an environment in which operant responding of pigeons was maintained by variable-interval schedules. In the first two experiments, response rates were reduced when response-independent food was intruded during the variable-interval schedule according to a concomitantly present fixed-time schedule. These reductions were not always an orderly function of the percentage of response-dependent food. Negatively accelerated patterns of key pecking across the fixed-time period occurred in Experiment 1 under the concomitant fixed-time variable-interval schedules. In Experiment 2, positively and negatively accelerated and linear response patterns occurred even though the schedules were similar to those used in Experiment 1. The variable findings in the first two experiments led to three subsequent experiments that were designed to further illuminate the controlling variables of the effects of intruded response-independent events. When the fixed and variable schedules were correlated with distinct operanda by employing a concurrent fixed-interval variable-interval schedule (Experiment 3) or with distinct discriminative stimuli (Experiments 4 and 5), negatively accelerated response patterns were obtained. Even in these latter cases, however, the response patterns were a joint function of the physical separation of the two schedules and the ratio of fixed-time or fixed-interval to variable-interval schedule food presentations. The results of the five experiments are discussed in terms of the contributions of both reinforcement variables and discriminative stimuli in determining the effects of intruding response-independent food into a stream of operant behavior.     

Predicting and scaling hens' preferences for topographically different responses

Six hens were exposed to several concurrent (second-order) variable-interval schedules in which the response requirements on the alternatives were varied. The response requirements were one key peck versus five key pecks, one key peck versus one door push, and five key pecks versus one door push. Response- and time-allocation ratios undermatched the obtained reinforcement ratios but were well described by the generalized matching law. Time and response bias estimates from two pairs of response requirements were used to predict bias in the third pairing. The predicted values were close to those obtained; this result supports the notion that both numerically and topographically different responses act as constant sources of bias within the generalized matching law. The differences between the response and time biases could be accounted for by the different times needed to complete each response requirement. The results also suggest that the door push is a useful operant for research with domestic hens.     

Contrast and reallocation of extraneous reinforcers as a function of component duration and baseline rate of reinforcement

Four pigeons responded on multiple schedules arranged on a “main” key in a two-key experimental chamber. A constant schedule component was alternated with another component that was varied over conditions. On an extra response key, conjoint schedules of reinforcement that operated in both components were arranged concurrently with the multiple schedule on the main key. On the main key, changes in reinforcement rate in the varied component were inversely related to changes in response rates in the constant component (behavioral contrast). On the extra key, some reinforcers were reallocated between components, depending on the schedules in effect on the main key in the varied component. In the varied component, the obtained rates of reinforcement on the extra key were inversely related to main-key reinforcement rate. In the constant component, extra-key reinforcer rates were positively related to main-key reinforcer rates obtained in the varied component, and were not a function of response rates on the extra key. In two comparisons, the rate at which components alternated and the value of the main-key schedule in the constant component were varied. Consistent with earlier work, long components reduced the extent of contrast. Reductions in contrast as a function of component duration were accompanied by similar reductions in the extent of reinforcer reallocation on the extra key. In the second comparison, lowering the rate of reinforcement in the constant component increased the rate at which extra-key reinforcers were obtained, reduced the extent of reinforcer reallocation, and reduced contrast. Overall, the results are consistent with the suggestion that some contrast effects are due to the changes in extraneous reinforcement during the constant component, and that manipulations of component duration, and manipulations of the rate of reinforcement in the constant component, affect contrast because they influence the extent of extraneous reinforcer real-location.     

Resistance to reinforcement change in multiple and concurrent schedules assessed in transition and at steady state

Behavioral momentum theory relates resistance to change of responding in a multiple-schedule component to the total reinforcement obtained in that component, regardless of how the reinforcers are produced. Four pigeons responded in a series of multiple-schedule conditions in which a variable-interval 40-s schedule arranged reinforcers for pecking in one component and a variable-interval 360-s schedule arranged them in the other. In addition, responses on a second key were reinforced according to variable-interval schedules that were equal in the two components. In different parts of the experiment, responding was disrupted by changing the rate of reinforcement on the second key or by delivering response-independent food during a blackout separating the two components. Consistent with momentum theory, responding on the first key in Part 1 changed more in the component with the lower reinforcement total when it was disrupted by changes in the rate of reinforcement on the second key. However, responding on the second key changed more in the component with the higher reinforcement total. In Parts 2 and 3, responding was disrupted with free food presented during intercomponent blackouts, with extinction (Part 2) or variable-interval 80-s reinforcement (Part 3) arranged on the second key. Here, resistance to change was greater for the component with greater overall reinforcement. Failures of momentum theory to predict short-term differences in resistance to change occurred with disruptors that caused greater change between steady states for the richer component. Consistency of effects across disruptors may yet be found if short-term effects of disruptors are assessed relative to the extent of change observed after prolonged exposure.     

Food-deprivation effects on punished schedule-induced drinking in rats.

Food-deprived rats (at 80% of their free-feeding weights) were exposed to a fixed-time 60-s schedule of food-pellet presentation and developed schedule-induced drinking. Lick-dependent signaled delays (10 s) to food presentation led to decreased drinking, which recovered when the signaled delays were discontinued. A major effect of this punishment contingency was to increase the proportion of interpellet intervals without any licks. The drinking of yoked control rats, which received food at the same times as those exposed to the signaled delay contingency (masters), was not consistently reduced. When food-deprivation level was changed to 90%, all master and yoked control rats showed decreases in punished or unpunished schedule-induced drinking. When the body weights were reduced to 70%, most master rats increased punished behavior to levels similar to those of unpunished drinking. This effect was not observed for yoked controls. Therefore, body-weight loss increased the resistance of schedule-induced drinking to reductions by punishment. Food-deprivation effects on punished schedule-induced drinking are similar to their effects on food-maintained lever pressing. This dependency of punishment on food-deprivation level supports the view that schedule-induced drinking can be modified by the same variables that affect operant behavior in general.