Variable-time reinforcement in multiple and concurrent schedules

Experiment I examined the role of a reduced rate of responding in the occurrence of behavioral contrast. Four rats and a pigeon were exposed to a two-component multiple schedule in which one component was always a variable-interval schedule. The second component was, at different times, either a variable-time schedule in which food was delivered independently of responding, or extinction. Both extinction and the variable-time schedule reduced the rate of responding in the second component. Behavioral contrast was observed, however, only when extinction was scheduled in the second component. Experiment II examined preference, as measured by time allocation in concurrent schedules for a variable-interval schedule relative to a variable-time schedule. Two rats displayed a lack of preference between the two schedules. The results of these experiments support a preference interpretation of behavioral contrast, which holds that behavioral contrast is the result of the introduction of a less-preferred condition in one component of a multiple schedule.     

Multiple determinants of the effects of reinforcement magnitude on free-operant response rates

Four experiments examined the effects of increasing the number of food pellets given to hungry rats for a lever-press response. On a simple variable-interval 60-s schedule, increased number of pellets depressed response rates (Experiment 1). In Experiment 2, the decrease in response rate as a function of increased reinforcement magnitude was demonstrated on a variable-interval 30-s schedule, but enhanced rates of response were obtained with the same increase in reinforcement magnitude on a variable-ratio 30 schedule. In Experiment 3, higher rates of responding were maintained by the component of a concurrent variable-interval 60-s variable-interval 60-s schedule associated with a higher reinforcement magnitude. In Experiment 4, higher rates of response were produced in the component of a multiple variable-interval 60-s variable-interval 60-s schedule associated with the higher reinforcement magnitude. It is suggested that on simple schedules greater reinforcer magnitudes shape the reinforced pattern of responding more effectively than do smaller reinforcement magnitudes. This effect is, however, overridden by another process, such a contrast, when two magnitudes are presented within a single session on two-component schedules.     

Choice between constant and variable alternatives by rats: effects of different reinforcer amounts and energy budgets

Two experiments, using rats as subjects, investigated the effect of different reinforcer amounts and energy budgets on choice between constant and variable alternatives under a closed economy. Rats were housed in the chamber and were exposed to a modified concurrent-chains schedule in which the choice phase was separated from a rest phase during which the rats could engage in other activities. In the choice phase, a single variable-interval schedule arranged entry into one of two equal terminal links (fixed-interval schedules). The constant terminal link ended with the delivery of a fixed number of food pellets (two or three, depending on the condition), whereas the variable terminal link ended with a variable number of food pellets (means of two or three, depending on the condition). Energy budget was defined as positive when body weights were over 90% of free-feeding weights, and as negative when they were under 80% of free-feeding weights. The different body weights were produced by varying the duration of the equal terminal-link schedules within daily 3-hr sessions. In Experiment 1, rats chose between a constant and a variable three pellets under both energy budgets. Rats preferred the constant three pellets more under the positive energy budget, whereas they were indifferent under the negative energy budget. In Experiment 2, rats chose between a constant three pellets and a variable two pellets, and chose between a constant two pellets and a variable three pellets under both energy budgets. The rats strongly preferred the constant three pellets over the variable two pellets under both energy budgets. In contrast, rats preferred the variable three pellets over the constant two pellets only under the negative energy budget, whereas they were indifferent under the positive energy budget. These results indicate that rats choices are sensitive to the difference in reinforcer amounts and to the energy budgets defined by the level of body weight. The present results are consistent with those obtained with small granivorous birds as well as with the predictions of a recent risk-sensitive foraging theory.     

Positive behavioral contrast across food and alcohol reinforcers.

The present study examined behavioral contrast during concurrent and multiple schedules that provided food and alcohol reinforcers. Concurrent-schedule contrast occurred in the responding reinforced by food when alcohol reinforcers were removed. It also occurred in the responding reinforced by alcohol when food was removed. Multiple-schedule contrast appeared for food when alcohol reinforcers were removed, but not for alcohol when food was removed. These results show that behavioral contrast may, but does not always, occur across qualitatively different reinforcers. They also show that multiple-schedule contrast may be more difficult to produce than concurrent-schedule contrast. The results have implications for a model of alcohol consumption.     

The influence of ultraviolet radiation on the pigeon's color discrimination

Two experiments demonstrated the pigeon's sensitivity to ultraviolet light. In Experiment I, pigeons' responses were reinforced on a multiple schedule with a variable-interval reinforcement schedule in one component and extinction in the other component. Response rates were quite different in the two components where the 520-nm stimuli signalling each component differed only in that one of them contained a 366-nm ultraviolet component. In Experiment II, pigeons were trained to peck one side key when two halves of a split field were of different wavelength and to peck another side key when they were of the same wavelength. Initially, field halves contained both "visible" and ultraviolet components of energy. Discrimination performance improved when the ultraviolet component was removed from one field half. It was argued that the critical change in the stimulus was a color change, rather than a brightness one, or a fluorescence of structures in the pigeon's eye.     

Response-independent milk delivery enhances persistence of pellet-reinforced lever pressing by rats

If, during training, one stimulus is correlated with a higher rate of reinforcement than another, responding will be more resistant to extinction in the presence of that higher rate signal, even if many of the reinforcers have been presented independently of responding. For the present study we asked if the response-independent reinforcers must be the same as the response-dependent reinforcers to enhance the response's persistence. Twelve Long-Evans hooded rats obtained 45-mg food pellets by lever pressing (variable-interval 100-s schedules) in the presence of two discriminative stimuli (blinking vs. steady lights) that alternated every minute during daily sessions. Also, in the presence of one of the stimuli (counterbalanced across rats), the rats received additional response-independent deliveries of sweetened condensed milk (a variable-time schedule). Extinction sessions were exactly like training sessions except that neither pellets nor milk were presented. Lever pressing was more resistant to extinction in the presence of the milk-correlated stimulus when (a) the size of the milk deliveries during training (under a variable-time 30 s schedule) was 0.04 ml (vs. 0.01 ml) and (b) 120-s or 240-s blackouts separated components. Response-independent reinforcers do not have to be the same as the response-dependent reinforcers to enhance persistence.     

Resistance to extinction following variable-interval reinforcement: reinforcer rate and amount

Rats obtained food-pellet reinforcers by nose poking a lighted key. Experiment 1 examined resistance to extinction following single-schedule training with different variable-interval schedules, ranging from a mean interval of 16 min to 0.25 min. That is, for each schedule, the rats received 20 consecutive daily baseline sessions and then a session of extinction (i.e., no reinforcers). Resistance to extinction (decline in response rate relative to baseline) was negatively related to the rate of reinforcers obtained during baseline, a relation analogous to the partial-reinforcement-extinction effect. A positive relation between these variables emerged, however, when the unit of extinction was taken as the mean interreinforcer interval that had been in effect during training (i.e., as an omitted reinforcer during extinction). In a second experiment, rats received blocks of training sessions, all with the same variable-interval schedule but with a reinforcer of four pellets for some blocks and one pellet for others. Resistance to extinction was greater following training with the larger (four pellets) than with the smaller (one pellet) reinforcer. Taken together, these results support the principle that greater reinforcement during training (e.g., higher rate or larger amount) engenders greater resistance to extinction even when the different conditions of reinforcement are varied between blocks of sessions.     

Behavioral contrast in competitive and noncompetitive environments

Three experiments examined the effects of opportunities for an alternative response (drinking) on positive behavioral contrast of rats' food-reinforced bar pressing. In both Experiments 1 and 2 the baseline multiple variable-interval schedules were rich (variable interval 10-s), and contrast was examined both with and without a water bottle present. In Experiment 1, the rats were not water deprived. When one component of the multiple schedule was changed to extinction, the rate of bar pressing increased in the constant component (positive behavioral contrast). The magnitude of contrast was larger when the bottle was absent than when it was present, as predicted by the matching law. Drinking did not shift from the constant variable-interval component to the extinction component, as might have been expected from competition theory. In Experiment 2, the rats were water deprived. Contrast was larger when the bottle was present than when it was absent, and drinking did shift to the extinction component, as predicted by competition theory. In Experiment 3, water-deprived rats responded on leaner multiple variable-interval schedules (60-s) in the presence of a water bottle. When one component was changed to extinction, contrast did not occur, and drinking did not shift to the extinction component. The present results suggest that there are at least two different sources of behavioral contrast: “competitive” contrast, observed when an alternative response occurs with high probability, and “noncompetitive” contrast, observed when an alternative response occurs with low probability. The results, in conjunction with earlier studies, also suggest that the form of the alternative response and the rate of food reinforcement provided by the multiple schedule combine to determine the amount of contrast.     

Behavioral contrast as differential time allocation

In Experiment I, hooded rats were exposed to multiple variable-interval schedules of reinforcement in which manipulanda and reinforcement magazines at opposite ends of the experimental chamber were associated with the different components. Time allocated to each component was measured by recording the time spent by the subject in the appropriate half of the chamber. Positive behavioral contrast was observed for the comparison between multiple variable-interval 30-second variable-interval 30-second and multiple variable-interval 30-second variable-interval 90-second conditions for both response frequency and time allocation measures, but not for mean local response rate (response frequency per time allocated to a component). In Experiment II, rats were exposed to multiple variable-time schedules in which reinforcement was response independent. Time allocated to each component was measured for two conditions, multiple variable-time 30-second variable-time 30-second and multiple variable-time 30-second variable-time 90-second. Positive behavioral contrast of time allocation was exhibited. The results indicated that time allocation was differentially sensitive to changes in reinforcement probability, and that behavioral contrast may result from the differential allocation of time to the different components of the multiple schedule.     

The effects of a signal for free or for earned reward: Implications for the role of response-reinforcer associations in instrumental performance

Pearce and Hall (1978) investigated the effects of making a brief flash of light contingent upon response in rats lever-pressing for food on a variable-interval (VI) schedule. When this signal occurred in conjunction only with reinforced responses the response rate was lowered with respect to a condition in which an equal number of light flashes occurred uncorrelated with reinforcement. The experiments reported here compared these effects with those produced by signalling “free” food deliveries in a similar way. Experiments I and II compared the effects of presenting correlated and uncorrelated schedules of light and food to rats given no opportunity to lever-press. The different schedules did not produce differences in response rate when the levers were made available. In Experiment III, free food was delivered to rats responding on a VI schedule. Signalling the delivery of earned food pellets produced a low response rate in comparison with a condition in which the free pellets were signalled. It is concluded that signalling food delivery is effective only when the rat must respond to earn the food and it is argued that the signal has its effect by overshadowing a response-reinforcer association.     

Successive discrimination training with equated reinforcement frequencies: failure to obtain behavioral contrast

In two experiments, pigeons were trained on two-component multiple schedules in which responding in one component (S₁) was always maintained by a variable-interval schedule. In Experiment I, low response rates were reinforced in the second (S₂) component for six master subjects. This schedule was adjusted to equate reinforcement frequencies in the two components. These subjects were compared to yoked partners, for which reinforcement in the S₂ component was made available on a variable-interval schedule whose value was determined by the master subjects. A similar procedure was used in Experiment II, where the S₂ schedule for master subjects made reinforcers contingent on the absence of responding. No evidence was found in either experiment for a behavioral contrast effect in the S₁ component attributable to response reduction in the S₂ component. A reliable contrast effect was obtained from a group of pigeons given extinction conditions in the S₂ component, which was compared to a group maintained throughout on a multiple variable-interval schedule. The results suggest that previous indications of behavioral contrast in similar situations were probably caused by uneven reinforcement distributions or reflect uncontrolled fluctuations in response rates.     

Discrimination between outcomes in instrumental learning: Effects of preexposure to the reinforcers

In two experiments rats received instrumental training with two response levers, one response being reinforced by sucrose solution and the other by sucrose pellets. Prior to a test session, on which both levers were made available in the absence of reinforcement, the rats were given free access to one of the reinforcers, a procedure known to reduce its value. It was found that the rats responded at a lower rate on the lever that had produced the now-devalued reinforcer, but that this effect was substantial only in rats that had received preexposure to the two reinforcers before instrumental training was begun (Experiment 1). Experiment 2 demonstrated that this effect was obtained only when presentations of the two reinforcers were presented according to an inter-mixed schedule during preexposure. It is suggested that this result constitutes an instance of the perceptual learning effect in which intermixed preexposure to similar events enhances their discriminability.     

Visual Reinforcement Signals Interfere with the Effects of Reinforcer Magnitude Manipulations

Three experiments examined the effect of reinforcement magnitude on free-operant response rates. In Experiment 1, rats that received four food pellets responded faster than rats that received one pellet on a variable ratio 30 schedule. However, when the food hopper was illuminated during reinforcer delivery, there was no difference between the rates of response produced by the two magnitudes of reward. In Experiment 2, there was no difference in response rates emitted by rats receiving either one or four pellets of food as reward on a random interval (RI) 60-s schedule. In Experiment 3, rats responding on an RI 30-s schedule did so at a lower rate with four pellets as reinforcement than with one pellet. This effect was abolished by the illumination of the food hopper during reinforcement delivery. These results indicate that the influence of magnitude is obscured by manipulations which signal the delivery of reinforcement.     

Behavioral contrast: Pavlovian effects and anticipatory contrast.

Two sources of behavioral contrast have been identified previously: Pavlovian stimulus-reinforcer relations and component sequence effects (anticipatory contrast). This study sought to isolate these sources of control procedurally in a four-ply multiple schedule composed of two fixed two-component sequences. Different cues were associated with the first component of each sequence, and contrast effects were studied in these target components. In Experiment 1, differential cuing of Component 2 between sequences and availability of reinforcement during target components were varied across three groups of pigeons; the stimulus-reinforcer relation between target-component cues and schedule of reinforcement in Component 2 was varied within subjects. Control by the Pavlovian relation was demonstrated under all conditions, and anticipatory contrast was not observed. In Experiment 2, target-component duration was systematically varied in the three groups of Experiment 1. Control by the Pavlovian relation was reliably obtained only when target-component behavior was unreinforced, and diminished with increases in component duration. Anticipatory contrast emerged in the two groups for which target-component reinforcement was available. These and other data indicate that Pavlovian effects in multiple schedules may be obscured when the requisite conditions for anticipatory contrast are present.     

Effects of response-dependent and independent electric shock on schedule-induced polydipsia

In Experiment I, lever pressing by rats was maintained by the delivery of food pellets under a 45-sec fixed-interval schedule. Fixed-time 180-sec and fixed-interval 180-sec schedules of shock delivery were systematically superimposed on the baseline food schedule to study effects on schedule-induced water intake. Response-dependent shock had little, if any, effect on water intake, whereas shocks independent of lever pressing attenuated fluid intake. In Experiment 2, rats received food pellets under a fixed-time 60-sec schedule. Electric shock delivered concurrently under a variable-time 180-sec schedule, but never while the animal was licking or within 5 sec after licking terminated, led to similar attenuation of water intake. These findings suggest that schedule-induced polydipsia is sensitive to differences in the functional properties of response-independent and dependent electric shock.     

Positive behavioral contrast when pigeons press treadles during multiple schedules

Pigeons were placed on multiple variable-interval 15-second variable-interval 15-second and on multiple variable-interval 15-second extinction schedules in which treadle presses produced food reinforcers. Positive behavioral contrast occurred. That is, rates of responding were higher during the variable-interval 15-second component when the other component was extinction than when it was another variable-interval 15-second schedule. These results contradict the findings of other studies, which failed to find positive contrast when pigeons pressed treadles for food reinforcers. They may also question the additive theories of behavioral contrast that predict that contrast should not occur in this situation.     

Effects of intertrial reinforcers on self-control choice.

In three experiments, pigeons chose between a small amount of food delivered after a short delay and a larger amount delivered after a longer delay. A discrete-trial adjusting-delay procedure was used to estimate indifference points--pairs of delay-amount combinations that were chosen about equally often. In Experiment 1, when additional reinforcers were available during intertrial intervals on a variable-interval schedule, preference for the smaller, more immediate reinforcer increased. Experiment 2 found that this shift in preference occurred partly because the variable-interval schedule started sooner after the smaller, more immediate reinforcer, but there was still a small shift in preference when the durations and temporal locations of the variable-interval schedules were identical for both alternatives. Experiment 3 found greater increases in preference for the smaller, more immediate reinforcer with a variable-interval 15-s schedule than with a variable-interval 90-s schedule. The results were generally consistent with a model that states that the impact of any event that follows a choice response declines according to a hyperbolic function with increasing time since the moment of choice.     

Apparent Incentive Contrast Effects in Autoshaping with Rats

The effects of shifts in reward quality and quantity on Pavlovian acquisition were studied in rats. In Experiment 1, animals preexposed to unsignaled food pellets, 10% sucrose solution, or home cage controls subsequently received autoshaping training (response-independent lever-pellet or lever-solution pairings, in three groups each). Unsignaled preexposure to sucrose solution facilitated autoshaping for pellets (relative to unshifted controls), whereas unsignaled preexposure to pellets retarded autoshaping for sucrose solution. In Experiment 2, unsignaled preexposure to 30% sucrose solution impaired acquisition reinforced by food pellets, relative to 2% solution. Using a choice procedure, Experiment 3 demonstrated that rats prefer pellets to either 2 or 10% sucrose solutions, but they prefer the 30% solution to the pellets. Experiment 4 demonstrated the facilitatory effect after an upward shift in reward magnitude rather than quality (from 1 to 12 pellets), but provided weaker evidence for retardation following a downward magnitude shift. Experiment 5 was similar to Experiment 4, except that animals received autoshaping training from the outset. No evidence of successive positive contrast was obtained, but there was a significant successive negative contrast effect. Moreover, extinction was faster after acquisition with 12 pellets rather than 1. These results suggest the presence of incentive contrast effects under Pavlovian training conditions.     

Preference for reinforcers under varying schedule arrangements: A behavioral economic analysis

The field of behavioral economics combines concepts from economics and operant conditioning to examine the influence of schedules or price on preference for reinforcers. Three case studies are reported in which behavioral economic analyses were used to assess relative preference for reinforcers shown by people with intellectual disabilities when schedule requirements varied. The studies examined (a) preference for different reinforcers, (b) substitutability of reinforcers, and (c) changes in preference as a function of schedule requirements.     

Positive contrast, negative induction, and inhibitory stimulus control in the rat

In Experiment I, 24 rats were trained on a multiple variable-interval variable-interval schedule with a doorlight and white noise serving as component cues. Two groups were then shifted to a multiple extinction variable-interval schedule, and a third group was maintained on the multiple variable-interval variable-interval schedule. The multiple extinction variable-interval condition produced positive contrast when either the light or noise signalled extinction, and both of these cues acquired inhibitory stimulus control as measured by a combined cue test. In Experiment II, the multiple variable-interval variable-interval condition was shifted to multiple extinction variable-interval for one group, to multiple variable-time variable-interval for a second group, and was unchanged for the third group. The two experimental conditions produced identical patterns of response-rate reduction in the altered component, but the multiple extinction variable-interval condition produced positive contrast, whereas the multiple variable-time variable-interval condition did not. Subsequent combined cue and resistance to reinforcement tests revealed that the cue signalling extinction acquired stronger inhibitory stimulus control than the cue signalling variable time.