Response rate and sensitivity to the molar feedback function relating response and reinforcement rate on VI+ schedules of reinforcement

In 5 experiments, the author examined rats' sensitivity to the molar feedback function relating response rate to reinforcement rate on schedules of reinforcement. These studies demonstrated that, at lower rates of responding, rats' performance on variable ratio (VR), variable interval (VI), and variable interval with linear feedback loop (VI+) schedules was determined largely by reinforcement of interresponse times; response rates were faster on VR than on both VI and VI+ schedules. In contrast, when procedures were adopted to maintain high rates of response, rats showed sensitivity to the molar characteristics of the schedules; they responded as fast on a VI+ schedule as on a VR schedule and faster on both of these schedules than on a yoked VI schedule. When the variance of response rate was manipulated, this factor was noted as an important element in determining sensitivity to the molar characteristics of the schedule.     

On the primacy of molecular processes in determining response rates under variable-ratio and variable-interval schedules

This study focused on variables that may account for response-rate differences under variable-ratio (VR) and variable-interval (VI) schedules of reinforcement. Four rats were exposed to VR, VI, tandem VI differential-reinforcement-of-high-rate, regulated-probability-interval, and negative-feedback schedules of reinforcement that provided the same rate of reinforcement. Response rates were higher under the VR schedule than the VI schedule, and the rates on all other schedules approximated those under the VR schedule. The median reinforced interresponse time (IRT) under the VI schedule was longer than for the other schedules. Thus, differences in reinforced IRTs correlated with differences in response rate, an outcome suggestive of the molecular control of response rate. This conclusion was complemented by the additional finding that the differences in molar reinforcement-feedback functions had little discernible impact on responding.     

The schedule dependency of the signaled reinforcement effect

In Experiment 1, rats leverpressed for food reinforcement on either a variable ratio (VR) 30 schedule or a variable interval (VI) 15-s schedule. One group in each condition received a signal filling a 500-ms delay of reinforcement. This treatment enhanced rates on the VR schedule, and attenuated rates on the VI schedule, relative to the rate seen in an unsignaled control condition. In Experiment 2 there was no delay of reinforcement and the signal and food were presented simultaneously. Attenuated rates of responding were observed on VI schedules with a range of mean interval values (15 to 300 s). Experiment 3 used a range of VR schedules (10 to 150) with simultaneous presentations of signal and food. A signal-induced enhancement of response rate was found at all VR values. In Experiment 4, a signal elevated response rates on a tandem VI VR schedule, but depressed rates on a tandem VR VI schedule, compared to control conditions receiving unsignaled delayed reinforcement. These results are taken to show that the effect of a signal accompanying reinforcement depends upon the nature of the behavior that is reinforced during exposure to a given schedule.     

Behavioral momentum: the effects of the temporal separation of rates of reinforcement.

In Part 1 of the experiment, rats responded under a variable-interval (VI) 30-s schedule and a VI 120-s schedule, with each in effect for a block of consecutive sessions. That is, the two VI schedules were presented in successive conditions. In Part 2 the VI schedules alternated each day, and in Part 3 the schedules alternated within the session as a multiple schedule. For half of the rats in Parts 1 and 2, the VI schedule alternated every few minutes within the session with a stimulus that signaled extinction. For each part, once response rates had stabilized, resistance to change was measured by prefeeding and extinction. When the schedules were examined in successive conditions (Part 1), resistance to extinction was greater under the VI 120-s schedule of reinforcement than under the VI 30-s schedule, but no consistent differences in resistance to prefeeding were observed between the two VI schedules. When the VI schedules alternated each day (Part 2), resistance to extinction was greater under the VI 120-s schedule. However, no consistent differences in resistance to prefeeding were observed between the VI schedules without extinction in Group A, but resistance to prefeeding was greater under the VI 30-s schedule for rats with the added extinction component in Group B. When the VI schedules alternated within the session as a multiple schedule (Part 3), resistance to extinction and resistance to prefeeding were greater under the VI 30-s schedule. The data suggest that different rates of reinforcement, and their accompanying discriminative stimuli, must be compared within the same session (or at least on alternate days) to produce data consistent with the behavioral momentum model.     

Free-operant performance on variable interval schedules with a linear feedback loop: no evidence for molar sensitivities in rats

Four experiments examined rats' sensitivity to molar and molecular factors on instrumental schedules of reinforcement. Rats were exposed to a variable interval schedule with a positive feedback loop (VI+), such that faster responding led to a shorter interreinforcement interval. In Experiments 1 and 2, rats responded faster on a variable response (VR) schedule than on either a VI schedule matched for reinforcement rate or a VI+ schedule matched for the feedback function. In Experiment 3, rats responded no differently on a VI schedule than they did on a VI+ schedule with equated rates of reinforcement. In Experiment 4, rats responded faster on a VI+ schedule with an interresponse time requirement yoked to that experienced on a VR schedule, than on a VI+ schedule with the same feedback function as the VR schedule. Taken together these results suggest that rats are more sensitive to the molecular than the molar properties of the schedules.     

Interlocking schedules: the relationship between response and time requirements.

Rats were exposed to an interlocking fixed-ratio 150 fixed-interval 5-minute schedule of food reinforcement and then to yoked variable-ratio schedules in which individual ratios corresponded exactly to the ratios of responses to reinforcement obtained on the interlocking schedule. After additional training with the interlocking schedule, the rats were exposed to yoked variable-interval schedules in which intervals corresponded to the intervals between successive reinforcements obtained on the second interlocking schedule. Response rates were highest in the yoked VR condition and lowest in the yoked VI, while intermediate rates characterized the interlocking schedule. Break-run patterns of responding were generated by the interlocking schedule for all subjects, while both the yoked VR and VI schedules produced comparatively stable local rates of responding. These results indicate that responding is sensitive to the interlocking schedule's inverse relationship between reinforcement frequency and responses per reinforcement.     

Schedules of reinforcement as determinants of human causality judgments and response rates

Experiments examined the effect of relationships between a response and an outcome on human judgments of causal effectiveness. In Experiment 1, the time between outcomes obtained on a variable ratio (VR) schedule became the intervals for a yoked variable interval (VI) schedule. Response rates were higher on the VR than on the VI schedule. In Experiment 2, the number of responses required per outcome on a VR schedule were matched to that on a master VI 20-s schedule. Both ratings of causal effectiveness and response rates were higher in the VR schedule. In Experiment 3, tandem VI fixed-ratio (FR) schedules produced higher rates and judgments than equivalent conjunctive VI FR schedule. In Experiment 4, a VI schedule with a reinforcement requirement for a short interresponse time (IRT) produced higher rates and judgments than a simple VI schedule. These results corroborate the view that schedules are a determinant of both response rates and causal judgments. Few current theories of causal judgment predict this pattern of results.     

Human sensitivity to reinforcement feedback functions

Two experiments investigated human sensitivity to the temporally extended aspects of reinforcement schedules. Experiment 1 investigated human sensitivity to the extended and local aspects of three reinforcement schedules: variable ratio (VR), variable interval (VI), and variable-interval-plus-linear-feedback (VI+) schedules. Experiment 2 investigated this sensitivity on two reinforcement schedules: VI and VI+ schedules. In both experiments, there was evidence of sensitivity to the temporally extended aspects of the schedule: There were differences between the response rate on the VI+ and yoked-VI schedules, but no statistical difference in rates of response between the VR and VI+ schedules. The VI+ versus VI difference was much more pronounced when a lower response force needed to depress a lever was used. These results suggest that human subjects do show some sensitivity to temporally extended aspects of schedules of reinforcement.     

Rats' performance on variable-interval schedules with a linear feedback loop between response rate and reinforcement rate

Three experiments investigated whether rats are sensitive to the molar properties of a variable-interval (VI) schedule with a positive relation between response rate and reinforcement rate (i.e., a VI+ schedule). In Experiment 1, rats responded faster on a variable ratio (VR) schedule than on a VI+ schedule with an equivalent feedback function. Reinforced interresponse times (IRTs) were shorter on the VR as compared to the VI+ schedule. In Experiments 2 and 3, there was no systematic difference in response rates maintained by a VI+ schedule and a VI schedule yoked in terms of reinforcement rate. This was found both when the yoking procedure was between-subject (Experiment 2) and within-subject (Experiment 3). Mean reinforced IRTs were similar on both the VI+ and yoked VI schedules, but these values were more variable on the VI+ schedule. These results provided no evidence that rats are sensitive to the feedback function relating response rate to reinforcement rate on a VI+ schedule.     

Instrumental performance on negative schedules

Three experiments examined the performance of rats pressing a lever for food reinforcement on a schedule in which high rates of response resulted in lowered rates of reinforcement (i.e. a schedule with a negative component). In Experiment 1, rats responded on a variable interval (VI) schedule with a conjoint component such that every 30 responses a reinforcement programmed by the VI schedule was cancelled. These subjects generally emitted a lower response rate than rats responding on a VI schedule yoked to the former subjects with respect to the delivery of reinforcement, although response rate differences were sometimes not large. Similar response-rate effects were obtained in Experiment 2 using a within-subject yoking procedure. In Experiment 3, reinforced interresponse times were matched on negative and VI schedules yoked in terms of reinforcement rate, and the response rate emitted in these conditions were similar. These results give support to theories of instrumental conditioning that stress the strengthening and shaping properties of reinforcement.     

Homogeneous chains, heterogeneous chains, and delay of reinforcement

Three pigeons responded on two-component chain schedules in which the required response topography in the initial and terminal links was similar (a homogeneous chain) or dissimilar (a heterogeneous chain). Key-peck responding in the initial link under a variable-interval 60-second (VI 60) schedule produced a terminal link in which, in different conditions, either key pecking or foot treadling was reinforced according to a VI 60 schedule. Multiple VI 60 VI 60 schedules, in which the responses required in the chain schedules were maintained by primary reinforcement in the two components, preceded and followed each type of chain. These multiple schedules were used to ensure that both responses occurred reliably prior to introducing the chain schedule. Key-peck response rates in the initial link of the chain consistently were higher during the homogeneous chain than during the heterogeneous chain. These results illustrate that intervening events during a period separating an operant response from primary reinforcement influence that operant, independently of the delay between the response and reinforcement.     

Effects of reinforcer rate and reinforcer quality on time allocation: Extensions of matching theory to educational settings

We examined how 3 special education students allocated their responding across two concurrently available tasks associated with unequal rates and equal versus unequal qualities of reinforcement. The students completed math problems from two alternative sets on concurrent variable-interval (VI) 30-s VI 120-s schedules of reinforcement. During the equal-quality reinforcer condition, high-quality (nickels) and low-quality items ("program money" in the school's token economy) were alternated across sessions as the reinforcer for both sets of problems. During the unequal-quality reinforcer condition, the low-quality reinforcer was used for the set of problems on the VI 30-s schedule, and the high-quality reinforcer was used for the set of problems on the VI 120-s schedule. Equal- and unequal-quality reinforcer conditions were alternated using a reversal design. Results showed that sensitivity to the features of the VI reinforcement schedules developed only after the reinforcement intervals were signaled through countdown timers. Thereafter, when reinforcer quality was equal, the time allocated to concurrent response alternatives was approximately proportional to obtained reinforcement, as predicted by the matching law. However the matching relation was disrupted when, as occurs in most natural choice situations, the quality of the reinforcers differed across the response options.     

Response suppression on DRL by rats with septal damage

The effectiveness of the differential reinforcement for low rates of responding (DRL) contingency in suppressing response rates of septal rats was investigated by using a Multi-DRL-yoked-VI (variable interval) schedule of reinforcement. The yoking procedure equated the interreinforcement times on the two schedules. Each schedule was in effect for half of each session, and the change in schedule was signaled by the presence or absence of a cue light. Schedule order and DRL delay requirement were varied. For both normal and septal rats, the response rates were higher in the VI component than the DRL component; this effect demonstrates that the responding of septals as well as normals is suppressed by the differential reinforcement of a particular class of IRTs. A sharp difference in the level of responding occurred at the point of transition from one component of the multiple schedule to the other, which provides evidence of a discrimination between the two schedules for both normals and septals. The conclusion is that the responding of septals is suppressed by the DRL contingency and not controlled solely by the density and distribution of reinforcement.     

Arousal, changeover responses, and preference in concurrent schedules

Pigeons were trained on multiple schedules that provided concurrent reinforcement in each of two components. In Experiment 1, one component consisted of a variable-interval (VI) 40-s schedule presented with a VI 20-s schedule, and the other a VI 40-s schedule presented with a VI 80-s schedule. After extended training, probe tests measured preference between the stimuli associated with the two 40-s schedules. Probe tests replicated the results of Belke (1992) that showed preference for the 40-s schedule that had been paired with the 80-s schedule. In a second condition, the overall reinforcer rate provided by the two components was equated by adding a signaled VI schedule to the component with the lower reinforcer rate. Probe results were unchanged. In Experiment 2, pigeons were trained on alternating concurrent VI 30-s VI 60-s schedules. One schedule provided 2-s access to food and the other provided 6-s access. The larger reinforcer magnitude produced higher response rates and was preferred on probe trials. Rate of changeover responding, however, did not differ as a function of reinforcer magnitude. The present results demonstrate that preference on probe trials is not a simple reflection of the pattern of changeover behavior established during training.     

Stimulus generalization of behavioral history: Interspecies generality and persistence of generalization gradients

Four pigeons were exposed to a tandem variable-interval (VI) fixed-ratio (FR) schedule in the presence of a 50-pixel (about 15 mm) square or an 80-pixel (about 24 mm) square and to a tandem VI differential-reinforcement-of-low-rate (DRL) schedule when a second 80-pixel or 50-pixel square was present. The values of the VI and FR schedules were adjusted to equate reinforcement rates in the two tandem schedules. Following this, a square-size continuum generalization test was administered under a fixed-interval (FI) schedule or extinction. In the first testing session, response frequency was a graded function of the similarity of the test stimuli to the training stimuli for all pigeons. These systematic generalization gradients persisted longer under the FI schedule than under extinction.     

An auto-adjusting shaping procedure

A variable interval (VI) schedule is described that automatically adjusts the programmed rates of reinforcement in accordance with the rates of responding of subjects during the two immediately preceding 30-sec time intervals. The schedule prescribes that as rate of responding decreases, programmed reinforcement rate increases, and that when rate of responding increases, reinforcement rate decreases. Thus, programmed reinforcement rate is adjusted continuously until some target value is reached. Ten rats were exposed to this procedure five times a day at 1-h intervals. The target, set at VI 120 sec, was reached by most subjects within 4 days of training. Subsequently, all subjects responded consistently during five daily 1-h sessions with VI 120 sec. This procedure speeds up the training of subjects on long VI schedules and substantially reduces the time and effort spent observing the subjects and adjusting the schedule parameter value during the early development of responding.     

Independence of concurrent responding maintained by interval schedules of reinforcement

A pigeon's responses were reinforced on a variable-interval schedule on one key; and, concurrently, either a multiple or a fixed-interval schedule of reinforcement was in effect on a second key. These concurrent schedules, conc VI 3 (mult VI 3 EXT) or conc VI 3 FI 6, were programmed with or without a changeover delay (COD). Because the COD provided that responses on one key could not be followed by reinforced responses on the other key, responding on one key was not likely to accidentally come under the control of the reinforcement schedule on the other. When the COD was used, the performances on each key were comparable to the performances maintained when these interval schedules are programmed separately. The VI schedule maintained a relatively constant rate of responding, even though the rate of responding on the second key varied in a manner appropriate to the schedule on the second key. The mult VI 3 EXT schedule maintained two separate rates of responding: a relatively high rate during the VI 3 component, and almost no responding during the EXT component. The FI schedule maintained the gradually increasing rate of responding within each interval that is characteristic of the performance maintained by this schedule. The concurrent performances, however, did include certain interactions involving the local characteristics of responding and the over-all rates of responding maintained by the various schedules. The relevance of the present findings to an inter-response time analysis of VI responding, a chaining account of FI responding, and the concept of the reflex reserve was discussed.     

The relation between response rates and reinforcement rates in a multiple schedule

In a multiple schedule, exteroceptive stimuli change when the reinforcement schedule is changed. Each performance in a multiple schedule may be considered concurrent with other behavior. Accordingly, two variable-interval schedules of reinforcement were arranged in a multiple schedule, and a third, common variable-interval schedule was programmed concurrently with each of the first two. A quantitative statement was derived that relates as a ratio the response rates for the first two (multiple) variable-interval schedules. The value of the ratio depends on the rates of reinforcement provided by those schedules and the reinforcement rate provided by the common variable-interval schedule. The following implications of the expression were evaluated in an experiment with pigeons: (a) if the reinforcement rates for the multiple variable-interval schedules are equal, then the ratio of response rates is unity at all reinforcement rates of the common schedule; (b) if the reinforcement rates for the multiple schedules are unequal, then the ratio of response rates increases as the reinforcement rate provided by the common schedule increases; (c) the limit of the ratio is equal to the ratio of the reinforcement rates. Satisfactory confirmation was obtained for the first two implications, but the third was left in doubt.     

A simple BASIC program to generate values for variable-interval schedules of reinforcement

A BASIC program to generate values for variable-interval (VI) schedules of reinforcement is presented. A VI schedule should provide access to reinforcement with a constant probability over a time horizon. If the values in a VI schedule are calculated from an arithmetic progression, the probability of reinforcement is positively correlated with the time since the last reinforcer was delivered. Fleshler and Hoffman (1962) developed an iterative equation to calculate VI schedule values so that the probability of reinforcement remains constant. This easy-to-use program generates VI schedule values according to the Fleshler and Hoffman equation, randomizes the values, and saves the values in ASCII to a disk file.     

Sensitivity to relative reinforcer rate in concurrent schedules: independence from relative and absolute reinforcer duration

Twelve pigeons responded on two keys under concurrent variable-interval (VI) schedules. Over several series of conditions, relative and absolute magnitudes of reinforcement were varied. Within each series, relative rate of reinforcement was varied and sensitivity of behavior ratios to reinforcer-rate ratios was assessed. When responding at both alternatives was maintained by equal-sized small reinforcers, sensitivity to variation in reinforcer-rate ratios was the same as when large reinforcers were used. This result was observed when the overall rate of reinforcement was constant over conditions, and also in another series of concurrent schedules in which one schedule was kept constant at VI ached 120 s. Similarly, reinforcer magnitude did not affect the rate at which response allocation approached asymptote within a condition. When reinforcer magnitudes differred between the two responses and reinforcer-rate ratios were varied, sensitivity of behavior allocation was unaffected although response bias favored the schedule that arranged the larger reinforcers. Analysis of absolute response rates ratio sensitivity to reinforcement occurrred on the two keys showed that this invariance of response despite changes in reinforcement interaction that were observed in absolute response rates on the constant VI 120-s schedule. Response rate on the constant VI 120-s schedule was inversely related to reinforcer rate on the varied key and the strength of this relation depended on the relative magnitude of reinforcers arranged on varied key. Independence of sensitivity to reinforcer-rate ratios from relative and absolute reinforcer magnitude is consistent with the relativity and independence assumtions of the matching law.