Choice between sequences of fixed-ratio schedules: effects of ratio values and probability of food delivery.

Pigeons were exposed to schedules of food delivery that consisted of two sequential fixed ratios. When alternative sequences provided two food deliveries per 50 responses, the schedule with the shorter initial fixed-ratio value was consistently preferred. Progressively reducing from 1.0 to .25 the probability of food delivery following completion of the second fixed ratio of the sequence with the shorter initial fixed ratio did not reduce preference for this sequence. Moreover, the sequence with the shorter initial fixed ratio also was preferred when the probability of food delivery following completion of the initial ratio in that sequence was progressively reduced from 1.0 to .5, although preference shifted to the alternative when the probability was reduced to 0. These findings suggest that the length of the initial fixed ratio was a primary determinant of choice. Subsequent manipulations demonstrated, however, that when the initial fixed ratios of the two alternatives were equal, changes in the ratio value and probability of food delivery following completion of the second fixed ratio lawfully affected choice.     

Conjunctive schedules of reinforcement: III. A fixed-interval adjusting fixed-ratio schedule

Key pecking of three pigeons was studied under a conjunctive schedule that specified both a fixed-interval and an adjusting fixed-ratio requirement. The fixed-interval schedule was 6 min for one pigeon and 3 min for the other two. The size of the ratio requirement was determined within each cycle of the fixed interval by the duration of the pause before responding began. The fixed-ratio value was at maximum at the start of each fixed interval and decreased linearly until the first response occurred (adjusting fixed-ratio schedule). A peck produced food when the number of responses remaining on the fixed-ratio schedule was completed and when the fixed interval had elapsed. If no response occurred during the interval, the fixed-ratio requirement decreased to one and a single response after the interval elapsed produced food. The initial value of the adjusting fixed-ratio schedule was studied over a range of 0 to 900. Increases in the adjusting fixed-ratio schedule to about 300 responses increased both pause duration and running response rate and also modified the pattern of responding from that obtained under the fixed-interval schedule. Higher values of the adjusting fixed ratio generally decreased pause duration and running response rate and also disrupted responding. Interreinforcement time under the conjunctive schedule was increased substantially when the adjusting fixed-ratio size exceeded 300 responses.     

Studies of operant and reflexive key pecks in the pigeon

The duration of pigeons' key pecks was studied in three experiments. Experiment I revealed that key pecks early in exposure to continuous reinforcement were of short duration, as were key pecks observed on an omission procedure in which pecks prevented food delivery. Key pecks later in exposure to continuous reinforcement, and those that occurred on positive automaintenance procedures, were of long duration. In Experiment II, pigeons were exposed to fixed-interval and fixed-ratio reinforcement schedules, and durations were recorded separately for each quarter of each interval or ratio. On fixed interval, durations were shorter in the first quarter of each interval than in subsequent quarters; on fixed ratio, durations were longer in the first quarter of the ratio than in subsequent quarters. These data parallel observations of concurrent operant responding and salivation in dogs. In Experiment III, pigeons were exposed to a discrete trial, differential-reinforcement-of-low-rate 6-sec schedule. Durations of responses in the first 2 sec of the trial were substantially shorter than those of responses that occurred later. The data from all three experiments support the view that the pigeon's "key peck" actually consists of two subclasses of peck, one reflexive and one operant.     

Cocaine and food as reinforcers: effects of reinforcer magnitude and response requirement under second-order fixed-ratio and progressive-ratio schedules.

Reinforcer magnitude and fixed-ratio requirement were varied under two second-order schedules. Under one, the first sequence of a fixed number of responses completed after the lapse of a 10-min fixed interval produced reinforcement. Under the second, a second-order progressive-ratio schedule, the fixed number of responses increased after each reinforcement. Either cocaine (0 to 300 micrograms/kg/inj) or food (0 to 5,700 mg/delivery) reinforcers were delivered. Under some conditions, a 2-s illumination of stimulus lights occurred on completion of each ratio sequence. Under the second-order schedule, as cocaine dose or amount of food increased, rates of responding increased; at the highest values, rates of responding decreased. Increases in the ratio requirement from 10 to 170 responses minimally decreased overall response rates. Under the second-order progressive-ratio schedule, increases in dose of cocaine or amount of food increased rates of responding; at the highest amounts of food, rates of responding decreased but response rates at the highest dose of cocaine remained relatively high. The highest ratio requirement that was completed (breaking point) depended on the dose of cocaine but was less dependent on the amount of food. Removing brief-stimulus presentations had a greater effect on completion of ratio requirements with cocaine compared to food.     

The psychological distance to reward

Pigeons' responses in the presence of two concurrently available (initial-link) stimuli produced entry into one of two different and mutually exclusive terminal link stimuli according to identical but independent variable-interval schedules. In one experiment, a two-component chained fixed-interval schedule produced food in one terminal link while a simple fixed-interval schedule produced food in the other terminal link. When the interreinforcement intervals were equal in the two terminal links (i.e., the simple fixed-interval was twice the size of each of the components in the chained schedule) pigeons preferred the simple fixed-interval as measured by their relative rates of responding in the concurrently available initial links. This preference increased as the duration of the terminal links increased. The preference could be reversed by making the simple fixed-interval schedule sufficiently longer than the chained schedule. In the second experiment, the terminal links consisted of two- vs three-component chained fixed-intervals, again with equal interreinforcement intervals. Pigeons preferred the two-component chain to the three-component chain, although these results were less consistent and less dramatic than those in the first experiment. Again, preference increased as the duration of the terminal links increased. The results show that an organism's choice for a schedule will be substantially lowered by the chaining operation even when the interreinforcement interval remains constant.     

Timeout and concurrent fixed-ratio schedules with human subjects

Human subjects given choices among 10 different pairs of concurrent fixed-ratio schedules preferred the smaller ratio. After a preference had been determined, timeout of increasing duration followed the completion of the preferred schedule. The larger the fixed-ratio difference, the longer the timeout necessary to produce the shift to the previously nonpreferred ratio. Responses by two of three subjects were unaffected by changes from response-dependent to response-independent pay.     

Choice and transformed interreinforcement intervals

Pigeons chose between two aperiodic, time-based schedules of reinforcement. The arithmetic mean interreinforcement interval of the first schedule was short, but the harmonic mean was long, whereas the arithmetic mean interreinforcement interval of the second schedule was long, but the harmonic mean was short. The pigeons preferred the schedule with the shorter harmonic mean in a concurrent-chains procedure when a terminal link ended after the first scheduled reinforcer had been gained on a terminal-link entry, but reversed their preferences, such that they preferred the schedule with the shorter arithmetic mean, when the terminal links ended after a fixed duration of exposure to the schedule. Moreover, the pigeons preferred the schedule with the shorter arithmetic mean in a two-key concurrent variable-interval variable-interval procedure, as well as in a concurrent variable-time variable-time, changeover-key procedure. The data suggest that an aggregate property of a schedule may not yield valid information about the responding that schedule will maintain as a choice alternative.     

Tolerance to the effects of cocaine on performance under behavior-correlated reinforcement magnitude

Four pigeons responded under a fixed-interval 8-min schedule of food delivery in which the amount of food delivered at the end of each interval depended on performance during the interval (i.e., a correlated schedule). Specifically, duration of access to grain was contingent upon the number of responses made during the first 4 min of the interval. This differential outcome did not affect response rates or patterning relative to performance under a simple fixed-interval 8-min schedule. Behavior under the correlated schedule was then investigated under doses of cocaine ranging from 0.3 to 10.0 mg/kg. A bitonic dose-response function was obtained for response rates and the time with head in the food hopper, whereas dose-dependent decreases were observed in the mathematical index of curvature (Fry, Kelleher, & Cook, 1960). The dose that produced the greatest increase in the head-in-hopper time was then administered prior to each session. Following repeated administration of cocaine, disruptions in response patterning were attenuated for all 4 pigeons; tolerance was also observed to the rate-increasing effects and increased head-in-hopper time for 2 pigeons after chronic cocaine administration. Tolerance therefore developed despite the fact that the initial effect of cocaine was to increase the amount of food obtained.     

Preference for locus of punishment in a response sequence

Food-deprived pigeons pecked a key under a schedule in which grain was made available after the seventieth peck. In each sequence of 70 responses, either the first, middle, or final response was followed by electric shock. Before the first response of each sequence, each response on a second key changed the color of the food key and the schedule of shock that was correlated with the food key color. Each pigeon preferred a schedule of shock, in that each of the three shock schedules did not occur equally often. The preferred shock schedule and the strength of the preference varied among the pigeons. The overall rate of responding by a pigeon under a given shock schedule was directly related to the pigeon's relative preference for that schedule, except when shock after the first response in the sequence was the most preferred schedule.     

Comparing preference and resistance to change in constant- and variable-duration schedule components

Two experiments explored preference and resistance to change in concurrent chains in which the terminal links were variable-interval schedules that ended either after a single reinforcer had been delivered (variable duration) or after a fixed period of access to the schedule (constant duration). In Experiment 1, pigeons' preference between the same pair of terminal links overmatched relative reinforcement rate when the terminal links were of constant duration, but not when they were of variable duration. Responding during the richer terminal link decreased less, relative to baseline, when response-independent food was presented during the initial links according to a variable-time schedule. In Experiment 2, all subjects consistently preferred a terminal link that consisted of 20-s access to a variable-interval 20-s schedule over a terminal link that ended after one reinforcer had been delivered by the same schedule. Results of resistance-to-change tests corresponded to preference, as responding during the constant-duration terminal link decreased less, relative to baseline, when disrupted by both response-independent food during the initial links and prefeeding. Overall, these data extend the general covariation of preference and resistance to change seen in previous studies. However, they suggest that reinforcement numerosity, including variability in the number of reinforcers per terminal-link entry, may sometimes affect preference and resistance to change in ways that are difficult to explain in terms of current models.     

Concurrent second-order schedules of reinforcement

Responses on one key (the main key) of a two-key chamber produced food according to a second-order variable-interval schedule with fixed-interval schedule components. A response on a second key (the changeover key) alternated colors on the main key and provided a second independent second-order variable-interval schedule with fixed-interval components. The fixed-interval component on one variable-interval schedule was held constant at 8 sec, while the fixed interval on the other variable-interval schedule was varied from 0 to 32 sec. Under some conditions, a brief stimulus terminated each fixed interval and generated fixed-interval patterns; in other conditions, the brief stimulus was omitted. Relative response rate and relative time deviated substantially from scheduled relative reinforcement rate and, to a lesser extent, from obtained relative reinforcement rate under both brief-stimulus and no-stimulus conditions. Matching was observed with equal components on both schedules; with unequal components, increasingly greater proportions of time and responses than the matching relation would predict were spent on the variable-interval schedule containing the shorter component. Preference for the shorter fixed interval was typically more extreme under brief-stimulus than under no-stimulus schedules. The results limit the extension of the matching relation typically observed under simple concurrent variable-interval schedules to concurrent second-order variable-interval schedules.     

Towards an empirical calculus of reinforcement value

Only one of two keys reinforces the subject with food. This key can assume one of two colors, each associated with a different fixed-ratio schedule for obtaining reinforcement. The function of the second key is to permit the animal to switch from the long schedule to the short schedule. If the difference between the ratio schedules is large enough, a preference for the shorter schedule is demonstrable. A quantitative index of preference is obtained as follows: each time the animal switches to the shorter schedule, the number of pecks required to produce the next switch is increased. As the “ante” on the switching key increases, the effective difference between the two ratio schedules decreases. After each food reinforcement, when the bird is exposed to the choice-situation, it takes longer before the bird switches again. This is used to “titrate” the bird's preference. If it does not switch within x sec, the progressively increasing ratio schedule of the switching key is decreased. A specific value, in terms of a rather specific number of responses the bird settles at on the choice key, is obtained. This equilibrium is employed as a dependent variable. Several variables of which it is a function are explored.     

Second-order schedules of token reinforcement: effects of varying the schedule of food presentation

In the initial link of a complex schedule, one discriminative stimulus was presented and lever pressing produced tokens on fixed-ratio schedules. In the terminal link, signalled by a second discriminative stimulus, deposits of the tokens produced food. With two rats, the terminal link was presented after each sixth component schedule of token reinforcement was completed. With the other two rats, the terminal link was presented following the first component schedule completed after a fixed interval. During the terminal link, each token deposit initially produced food. The schedule of food presentation was subsequently increased such that an increasing number of token deposits in the terminal link was required for each food presentation. Rates of lever pressing in the initial link were inversely related to the schedule of food presentation in the terminal link. These results are similar to those of experiments that have varied schedules of food presentation in chained schedules. Rates and patterns of responding controlled throughout the initial link were more similar to those ordinarily controlled by second-order brief-stimulus schedules than to those controlled by comparable extended chained schedules.     

A TEST OF THE FORMAL AND MODERN THEORIES OF MATCHING

Pigeons' choosing between fixed-interval and random-interval schedules of reinforcement was investigated in three experiments using a discrete-trial procedure. In all three experiments, the random-interval schedule was generated by sampling a probability distribution at an interval (and in multiples of the interval) equal to that of the fixed-interval schedule. Thus the programmed delays to reinforcement on the random alternative were never shorter and were often longer than the fixed interval. Despite this feature, the fixed schedule was not strongly preferred. Increases in the probability used to generate the random interval resulted in decreased preferences for the fixed schedule. In addition, the number of consecutive choices on the preferred alternative varied directly with preference, whereas the consecutive number of choices on the nonpreferred alternative was fairly constant. The probability of choosing the random alternative was unaffected by the immediately prior interval encountered on that schedule, even when it was very long relative to the average value. The results loosely support conceptions of a "preference for variability" from foraging theory and the "utility of behavioral variability" from human decision-making literatures.     

Concurrent-chain performance: Effects of absolute and relative terminal-link entry frequency

Six pigeons were trained in a concurrent-chain procedure with constant variable-interval 6-s variable-interval 12-s terminal links. Five groups of conditions were arranged. Within a group of conditions, the duration of one initial-link schedule was held constant and the duration of the other initial link was varied. The duration of the varied initial link was always longer than, or equal to, the constant initial-link duration. The duration of the shorter initial link was varied across groups of conditions from 5 s to 70 s. The data from each group were well described by the generalized matching law. Sensitivity (a) to the terminal-link entry ratio increased as the shorter initial-link duration increased, but appeared to reach an asymptote at shorter initial-link durations greater than 32 s. Terminal-link bias did not change with changes in shorter initial-link duration for the response-allocation data, but showed a small increase with increasing shorter initial-link duration for the time-allocation data.     

Control of responding during stimuli that precede transitions in reinforcement frequency

Pigeons' responses were reinforced on a variant of a mixed variable-interval extinction schedule of reinforcement in which the transition to the higher reinforcement rate was signaled by a trace stimulus projected on the response key prior to the onset of the component correlated with food delivery. In the first of two experiments, the duration of the trace stimulus preceding the component correlated with food delivery was varied from 1.5 to 50.0 s and in the second experiment, the reinforcement frequency in the same component was varied from 10 to 60 reinforcers per hour. Pigeons pecked at the trace stimulus preceding the onset of the component correlated with food delivery even though responding was not reinforced in its presence and only one of the changes in reinforcement rate (i.e., from extinction to reinforcement) was signaled. The rate of pecking during the trace stimulus was a function of its duration but not of the reinforcement frequency in the following component. Higher rates generally occurred at the shorter trace-stimulus durations. Component responding following the offset of the trace stimulus was under discriminative control of the trace stimulus whether or not responding occurred in the presence of the trace stimulus.     

Food deliveries during the pause on fixed-interval schedules

Pigeons were trained on fixed-interval schedules of food delivery. In Experiments I and II, the fixed interval was initiated by the previous fixed-interval reinforcer; in Experiment III, the fixed interval was initiated by the first key peck following the preceding fixed-interval reinforcer (a chain fixed-ratio one, fixed-interval schedule). During the postreinforcement pause, variable-time schedules delivered food independent of any specific response. Rate of food delivery during the pause had only small effects on pause duration in Experiments I and II. In Experiment III, however, pause duration increased systematically with the rate of food delivery during the pause. These data suggest that the momentary proximity to reinforcement delivered via the fixed-interval schedule exerts potent control over pause termination. Additional analysis revealed that pause termination was unaffected by the intermittent delivery of food during the pause. Such data suggest that the temporal control by fixed-interval schedules is highly resistant to interference.     

Duration and rate of reinforcement as determinants of concurrent responding

The duration and frequency of food presentation were varied in concurrent variable-interval variable-interval schedules of reinforcement. In the first experiment, in which pigeons were exposed to a succession of eight different schedules, neither relative duration nor relative frequency of reinforcement had as great an effect on response distribution as they have when they are manipulated separately. These results supported those previously reported by Todorov (1973) and Schneider (1973). In a second experiment, each of seven pigeons was exposed to only one concurrent schedule in which the frequency and/or duration of reinforcement differed on the two keys. Under these conditions, each pigeon's relative rate of response closely matched the relative total access to food that each schedule provided. This result suggests that previous failures to obtain matching may be due to factors such as an insufficient length of exposure to each schedule or to the pigeons' repeated exposure to different concurrent schedules.     

Factors influencing responding under multiple schedules of conditioned and unconditioned reinforcement

Two experiments examined pigeons' responses under multiple schedules of conditioned and unconditioned reinforcement. In one component, responses produced food according to a fixed-interval schedule; in a second component, responses produced brief stimuli according to a fixed-ratio schedule. When brief-stimulus presentations were paired with food in the first component, rates in the second component were usually higher than 10 responses per minute. When pairing in the first component was eliminated, responding continued to be maintained in the second component. Elimination of food presentation from the first component substantially decreased responding in the second component, even though the brief stimulus had not been paired with food. Experiment II demonstrated that response rate was affected by the duration of both the second component and the brief stimulus. The results suggest that three conditions are important in maintaining responding with brief-stimulus presentations: (1) pairing the brief stimulus, at least initially, with food, (2) maintaining unconditioned reinforcement in one component, and (3) employing optimal brief-stimulus and component durations.     

Increasing variety of foods consumed by blending nonpreferred foods into preferred foods

A treatment with differential or noncontingent reinforcement and nonremoval of the spoon increased the acceptance of one or two of 16 foods for 2 participants with severe food refusal. These differential levels of acceptance were demonstrated empirically in an ABAB design in which A was the presentation of the accepted (preferred) foods and B was the presentation of foods the participants refused (nonpreferred foods). Subsequently, we implemented a blending treatment that consisted of mixing (blending) nonpreferred foods into preferred foods in various ratios (e.g., 10% nonpreferred/90% preferred, 20% nonpreferred/80% preferred). We then presented nonpreferred foods that had been exposed to blending to determine if consumption of nonpreferred foods would increase following the blending treatment. We also conducted periodic reversals in which we presented nonpreferred foods that had not been exposed to the blending treatment. Following initial implementation of the blending treatment, consumption was high for nonpreferred foods that had been blended and low for nonpreferred foods that had not been blended. Consumption increased for all foods (i.e., foods that had been exposed to blending and foods that had not been exposed to blending) after seven or eight foods had been exposed to the blending treatment. Thus, the variety of foods consumed by the participants increased from one or two to 16. These results are discussed in terms of stimulus fading, conditioned food preferences, and escape extinction. DESCRIPTORS: conditioned food preferences, food refusal, negative reinforcement, stimulus fading