Delayed reinforcement in a multiple schedule

Three rats and a pigeon were first trained on a two-component multiple schedule in which reinforcement in the two components occurred immediately after a response. Later, reinforcement in one component was delayed by a few seconds. During both stages of the experiment, reinforcement was scheduled by equal variable- (pigeon) or random-interval (rats) schedules in the two components. The main effect of the delayed reinforcement was to increase the rate of responding in the unchanged (non-delay) component. This behavioral contrast effect did not appear in all cases to be dependent upon a reduction in the rate of responding or the frequency of reinforcement in the delay component. This finding suggests that a reduction in response rate and/or reinforcement frequency in one component of a multiple schedule may not be a necessary prerequisite for the occurrence of behavioral contrast. This finding is, however, consistent with an explanation that suggests that behavioral contrast results from the introduction of a less-preferred condition in one component of a multiple schedule, since it is known that animals “prefer” immediate to delayed reinforcement.     

The relation between response rates and reinforcement rates in a multiple schedule

In a multiple schedule, exteroceptive stimuli change when the reinforcement schedule is changed. Each performance in a multiple schedule may be considered concurrent with other behavior. Accordingly, two variable-interval schedules of reinforcement were arranged in a multiple schedule, and a third, common variable-interval schedule was programmed concurrently with each of the first two. A quantitative statement was derived that relates as a ratio the response rates for the first two (multiple) variable-interval schedules. The value of the ratio depends on the rates of reinforcement provided by those schedules and the reinforcement rate provided by the common variable-interval schedule. The following implications of the expression were evaluated in an experiment with pigeons: (a) if the reinforcement rates for the multiple variable-interval schedules are equal, then the ratio of response rates is unity at all reinforcement rates of the common schedule; (b) if the reinforcement rates for the multiple schedules are unequal, then the ratio of response rates increases as the reinforcement rate provided by the common schedule increases; (c) the limit of the ratio is equal to the ratio of the reinforcement rates. Satisfactory confirmation was obtained for the first two implications, but the third was left in doubt.     

Independence of response force and reinforcement rate on concurrent variable-interval schedule performance

Five pigeons were trained over 43 experimental conditions on a variety of concurrent variable-interval schedules on which the forces required on the response keys were varied. The results were well described by the generalized matching law with log reinforcement ratios and log force ratios exerting independent (noninteractive) effects on preference. A further analysis using the Akaike criterion, an information-theoretic measure of the efficiency of a model, showed that overall reinforcement rate and overall force requirement did not affect preference. Unlike reinforcement rate changes, force requirement increases did not change the response rate on the alternate key, and an extension of Herrnstein's absolute response rate function for force variation on a single variable-interval schedule is suggested.     

Time-dependent contrast effects in a multiple schedule of food reinforcement

Four rats were rewarded for running in a wheel under two alternating conditions of food reinforcement. These periods of frequent and infrequent reinforcement, each accompanied by a particular stimulus, were presented a number of times in each daily session. Following shifts from high to low frequency of reinforcement, responding decreased suddenly and markedly, and then recovered within the next few minutes. The magnitude of this temporary depression was an increasing function of the duration of the immediately preceding component of high-frequency reinforcement. A transient elevation in performance, which did not vary with the duration of the prior component, was noted in two subjects following shifts from low to high frequency of reinforcement. The elevation and depression effects did not appear simultaneously during the 48 experimental sessions. A possible relation between the difficulty of the discrimination and the extent of contrast effects is discussed.     

Relative rate of response and relative magnitude of reinforcement in multiple schedules

Pigeons were trained on a multiple schedule in which the duration of access to grain reinforcement was varied independently in the two components. The relative response rate in one component was an increasing function of the relative duration of reinforcement in that component. The similarity of this interaction to that found in multiple schedules of different reinforcement frequency is discussed. Extinction data were also similar to those obtained after training on multiple schedules of different reinforcement frequency.     

Fixed-time schedule effects as a function of baseline reinforcement rate

Using an arbitrary response, we evaluated fixed-time (FT) schedules that were either similar or dissimilar to a baseline (response-dependent) reinforcement schedule and extinction. Results suggested that both FT schedules and extinction resulted in decreased responding. However, FT schedules were more effective in reducing response rates if the FT reinforcer rate was dissimilar to baseline reinforcer rates. Possible reasons for this difference were evaluated with data analysis methods designed to identify adventitious response-reinforcer relations.     

Performance of children under a multiple random-ratio random-interval schedule of reinforcement

Three children, aged 1.5, 2.5, and 4.5 years, pressed telegraph keys under a two-component multiple random-ratio random-interval schedule of reinforcement. In the first condition, responses on the left key were reinforced under a random-interval schedule and responses on the right key were reinforced under a random-ratio schedule. In the second condition, the schedule components were reversed. In the third condition, the original arrangement was reinstated. For all subjects, rates of responding were higher in the random-ratio component despite higher rates of reinforcement in the random-interval component. The average interreinforcement interval of the random-interval component was increased in the fourth condition, resulting in more similar rates of reinforcement for both schedule components, and then returned to its original value in the fifth condition. In both conditions, all subjects continued to exhibit higher rates of responding in the ratio component than in the interval component. Although these observations are consistent with results from studies with pigeons, it is argued that the response-rate differences between the interval and ratio schedule components are sufficient to demonstrate schedule sensitivity.     

Observing behavior in squirrel monkeys under a multiple schedule of reinforcement availability

Observing behavior of two squirrel monkeys was examined under a multiple schedule of four components. Lever (observing) responses produced either a stimulus indicating the availability of food or another stimulus indicating food was not available. Key responses in the presence of the food-available stimulus produced food on a continuous reinforcement schedule. In the absence of food-available stimuli, responding on the key had no scheduled consequences. Observing responses produced food-available stimuli according to three different random-interval schedules with mean interstimulus availability times of 1, 2, and 4 min. In the fourth component of the multiple schedule (observing extinction) food-available stimuli never occurred. Each component of the schedule was correlated with a distinctive auditory stimulus. Observing rates decreased with decreasing frequency of the food-available stimulus. Observing rates during extinction continued decreasing when the brief stimulus indicating food unavailability was no longer produced by lever pressing. When the brief stimulus was reinstated response rates increased abruptly.     

Stimulus generalization and delay of reinforcement during one component of a multiple schedule

The key pecking of six pigeons was reinforced according to a variable-interval 1-min schedule during each of two successively presented stimuli. When the key was illuminated by a black line on a white background, reinforcement was delayed for 10 sec. When the key was illuminated by a plain white light, reinforcement was not delayed. All subjects responded at a lower rate during the presentation of the black line. A subsequent generalization test along the line-orientation dimension produced a U-shaped gradient, with the nadir located at or near the training stimulus, for each subject. These gradients suggested that the lower rate of response during the stimulus associated with delayed reinforcement may have been due to an inhibition of responding.     

Aggression during the fixed-ratio and extinction components of a multiple schedule of reinforcement

Pigeons were trained to key peck for food on multiple reinforcement schedules including components of continuous and fixed-ratio reinforcement and extinction. At the end of the chamber opposite the response key was a restrained target pigeon. The target restraining equipment was designed to record automatically blows struck against the target. When the experimental pigeons were paired with restrained target pigeons they attacked the target. Attack occurred during extinction after both continuous and fixed-ratio reinforcement. Attack also occurred occasionally during fixed-ratio 25 and fixed-ratio 40 and frequently during fixed-ratio 60 and fixed-ratio 120. No attack occurred during fixed-ratio 15 and continuous reinforcement. After a history of stable responding without a target bird present, the introduction of a target bird resulted in severely strained key-peck responding characterized by long periods of neither key pecking nor aggressing.     

Relationship between response rate and reinforcement frequency in variable-interval schedules: the effect of the concentration of sucrose reinforcement

Four rats were exposed to variable-interval schedules specifying a range of different reinforcement frequencies, using sucrose of two different concentrations and distilled water as the reinforcer. With sucrose, the rates of responding of all four rats were increasing negatively accelerated functions of reinforcement frequency, the data conforming closely to Herrnstein's equation; this was also true of the data from three of the four rats when distilled water was used as the reinforcer. The values of both constants in Herrnstein's equation were related to the sucrose concentration: the asymptotic response rate decreased, and the reinforcement frequency corresponding to the half-maximal response rate increased, with decreasing sucrose concentration.     

Shaping of three-man teams on a multiple DRL-DRH schedule using collective reinforcement

Five three-man teams were conditioned to a multiple DRL-DRH schedule during five daily sessions of 3 hr each. The teams were conditioned on the basis of their cumulative response rate regardless of the contributions of individual members. Two individually conditioned Ss were run under a similar schedule for comparison. Although all teams demonstrated some differentiation of response rate under the two legs of the schedule, this difference was not achieved as rapidly, nor was it as stable, as the differentiation in rates achieved in individually conditioned Ss. Team members displayed a variety of response rates as a result of collective team conditioning. Often, individual response rates interfered with obtainment of team reinforcement. The individual team members displayed little S^D control in comparison with the individually conditioned Ss. In general the results suggest that collective reinforcement is highly inefficient in comparison with procedures employing individual feedback.     

Effects of reinforcement history on response rate and response pattern in periodic reinforcement

Several researchers have suggested that conditioning history may have long-term effects on fixed-interval performances of rats. To test this idea and to identify possible factors involved in temporal control development, groups of rats initially were exposed to different reinforcement schedules: continuous, fixed-time, and random-interval. Afterwards, half of the rats in each group were studied on a fixed-interval 30-s schedule of reinforcement and the other half on a fixed-interval 90-s schedule of reinforcement. No evidence of long-term effects attributable to conditioning history on either response output or response patterning was found; history effects were transitory. Different tendencies in trajectory across sessions were observed for measures of early and late responding within the interreinforcer interval, suggesting that temporal control is the result of two separate processes: one involved in response output and the other in time allocation of responding and not responding.     

Effects of response rate, reinforcement frequency, and the duration of a stimulus preceding response-independent food

Food-reinforced key pecking in the pigeon was maintained under a four-component multiple schedule. In two components, responding was maintained at high rates under a random-ratio schedule. In the other two components, responding was maintained at low rates under a schedule that specified a minimum interresponse time. For both high and low response rates, one of the schedule components was associated with a high reinforcement frequency and the other components with a lower reinforcement frequency. During performance under these schedules, a stimulus terminated by access to response-independent food was periodically presented. The duration of this pre-food stimulus was 5, 30, 60, or 120 sec. Changes in rate of key pecking during the pre-food stimulus were systematically related to baseline response rate and the duration of the stimulus. Both high and low response rates were increased during the 5-sec stimulus. At longer stimulus durations, low response rates were unaffected and high response rates were decreased during the stimulus. For two of three pigeons, high response rates maintained under a lower frequency of reinforcement tended to be decreased more than high response rates maintained under a higher reinforcement frequency. In general, the magnitude of decrease in high response rates was inversely related to the duration of the pre-food stimulus.     

Relationship between response rate and reinforcement frequency in variable-interval schedules: II. Effect of the volume of sucrose reinforcement

Three rats were exposed to variable-interval schedules specifying a range of different reinforcement frequencies, using three different volumes of .32 molar sucrose (.10, .05, and .02 milliliters) as the reinforcer. With each of the three volumes, the rates of responding of all three rats were increasing, negatively accelerated functions of reinforcement frequency, the data conforming closely to Herrnstein's equation. In each rat the value of the constant K_H, which expresses the reinforcement frequency needed to obtain the half-maximal response rate, increased with decreasing reinforcer volume, the values obtained with .02 milliliters being significantly greater than the values obtained with .10 milliliters. The values of the constant R_max, which expresses the theoretical maximum response rate, were not systematically related to reinforcer volume. The effect of reinforcer volume upon the relationship between response rate and reinforcement frequency is thus different from the effect of the concentration of sucrose reinforcement: In a previous experiment (Bradshaw, Szabadi, & Bevan, 1978) it was found that sucrose concentration influenced the values of both constants, R_max increasing and K_H decreasing with increasing sucrose concentration.     

Probability of response and probability of reinforcement in a response-defined analogue of an interval schedule

Variable interval (VI) responding was hypothesized to be a function of differential reinforcement susceptibilities of various unspecified behavior chains that mediate interresponse times (IRTs). To test this hypothesis, probabilities of reinforcement were regulated for the lengths of chains of key pecking responses of pigeons, analogous to the way that VI regulates probabilities of reinforcement for IRTs. This procedure generated a number of VI-like effects, supporting the notion that VI behavior can be construed as a special case of an interaction between the organism's function relating reinforcement susceptibilities to chain length and the experimenter's function relating probabilities of reinforcement to chain length.