Response rate and sensitivity to the molar feedback function relating response and reinforcement rate on VI+ schedules of reinforcement

In 5 experiments, the author examined rats' sensitivity to the molar feedback function relating response rate to reinforcement rate on schedules of reinforcement. These studies demonstrated that, at lower rates of responding, rats' performance on variable ratio (VR), variable interval (VI), and variable interval with linear feedback loop (VI+) schedules was determined largely by reinforcement of interresponse times; response rates were faster on VR than on both VI and VI+ schedules. In contrast, when procedures were adopted to maintain high rates of response, rats showed sensitivity to the molar characteristics of the schedules; they responded as fast on a VI+ schedule as on a VR schedule and faster on both of these schedules than on a yoked VI schedule. When the variance of response rate was manipulated, this factor was noted as an important element in determining sensitivity to the molar characteristics of the schedule.     

Rats' performance on variable-interval schedules with a linear feedback loop between response rate and reinforcement rate

Three experiments investigated whether rats are sensitive to the molar properties of a variable-interval (VI) schedule with a positive relation between response rate and reinforcement rate (i.e., a VI+ schedule). In Experiment 1, rats responded faster on a variable ratio (VR) schedule than on a VI+ schedule with an equivalent feedback function. Reinforced interresponse times (IRTs) were shorter on the VR as compared to the VI+ schedule. In Experiments 2 and 3, there was no systematic difference in response rates maintained by a VI+ schedule and a VI schedule yoked in terms of reinforcement rate. This was found both when the yoking procedure was between-subject (Experiment 2) and within-subject (Experiment 3). Mean reinforced IRTs were similar on both the VI+ and yoked VI schedules, but these values were more variable on the VI+ schedule. These results provided no evidence that rats are sensitive to the feedback function relating response rate to reinforcement rate on a VI+ schedule.     

Free-operant performance on variable interval schedules with a linear feedback loop: no evidence for molar sensitivities in rats

Four experiments examined rats' sensitivity to molar and molecular factors on instrumental schedules of reinforcement. Rats were exposed to a variable interval schedule with a positive feedback loop (VI+), such that faster responding led to a shorter interreinforcement interval. In Experiments 1 and 2, rats responded faster on a variable response (VR) schedule than on either a VI schedule matched for reinforcement rate or a VI+ schedule matched for the feedback function. In Experiment 3, rats responded no differently on a VI schedule than they did on a VI+ schedule with equated rates of reinforcement. In Experiment 4, rats responded faster on a VI+ schedule with an interresponse time requirement yoked to that experienced on a VR schedule, than on a VI+ schedule with the same feedback function as the VR schedule. Taken together these results suggest that rats are more sensitive to the molecular than the molar properties of the schedules.     

The schedule dependency of the signaled reinforcement effect

In Experiment 1, rats leverpressed for food reinforcement on either a variable ratio (VR) 30 schedule or a variable interval (VI) 15-s schedule. One group in each condition received a signal filling a 500-ms delay of reinforcement. This treatment enhanced rates on the VR schedule, and attenuated rates on the VI schedule, relative to the rate seen in an unsignaled control condition. In Experiment 2 there was no delay of reinforcement and the signal and food were presented simultaneously. Attenuated rates of responding were observed on VI schedules with a range of mean interval values (15 to 300 s). Experiment 3 used a range of VR schedules (10 to 150) with simultaneous presentations of signal and food. A signal-induced enhancement of response rate was found at all VR values. In Experiment 4, a signal elevated response rates on a tandem VI VR schedule, but depressed rates on a tandem VR VI schedule, compared to control conditions receiving unsignaled delayed reinforcement. These results are taken to show that the effect of a signal accompanying reinforcement depends upon the nature of the behavior that is reinforced during exposure to a given schedule.     

Schedules of reinforcement as determinants of human causality judgments and response rates

Experiments examined the effect of relationships between a response and an outcome on human judgments of causal effectiveness. In Experiment 1, the time between outcomes obtained on a variable ratio (VR) schedule became the intervals for a yoked variable interval (VI) schedule. Response rates were higher on the VR than on the VI schedule. In Experiment 2, the number of responses required per outcome on a VR schedule were matched to that on a master VI 20-s schedule. Both ratings of causal effectiveness and response rates were higher in the VR schedule. In Experiment 3, tandem VI fixed-ratio (FR) schedules produced higher rates and judgments than equivalent conjunctive VI FR schedule. In Experiment 4, a VI schedule with a reinforcement requirement for a short interresponse time (IRT) produced higher rates and judgments than a simple VI schedule. These results corroborate the view that schedules are a determinant of both response rates and causal judgments. Few current theories of causal judgment predict this pattern of results.     

Influence of the cost of responding

In three experiments, the effect of costs associated with responding on judgments of the causal effectiveness of the response was examined. In Experiment 1, the temporal interval between outcomes was matched on a variable interval (VI) and a variable ratio (VR) schedule. When each response was made at some “cost,” and outcomes represented some “gain” for the subject, the rating of causal effectiveness for responses was higher on the VR than on the VI schedule. This relationship was absent when the outcome was a triangle Hash. In Experiment 2, the number of responses required per outcome on a VI and a VR schedule were matched, and responses on the VR schedule were rated as more causally effective. In Experiment 3, a VI-to-VR yoking procedure was used. With minimal response costs, judgments were similar on the VI and VR schedules, but with greater response costs, responses performed on the VR schedule were rated as more causally effective than those emitted on the VI schedule.     

On the primacy of molecular processes in determining response rates under variable-ratio and variable-interval schedules

This study focused on variables that may account for response-rate differences under variable-ratio (VR) and variable-interval (VI) schedules of reinforcement. Four rats were exposed to VR, VI, tandem VI differential-reinforcement-of-high-rate, regulated-probability-interval, and negative-feedback schedules of reinforcement that provided the same rate of reinforcement. Response rates were higher under the VR schedule than the VI schedule, and the rates on all other schedules approximated those under the VR schedule. The median reinforced interresponse time (IRT) under the VI schedule was longer than for the other schedules. Thus, differences in reinforced IRTs correlated with differences in response rate, an outcome suggestive of the molecular control of response rate. This conclusion was complemented by the additional finding that the differences in molar reinforcement-feedback functions had little discernible impact on responding.     

Instrumental performance on negative schedules

Three experiments examined the performance of rats pressing a lever for food reinforcement on a schedule in which high rates of response resulted in lowered rates of reinforcement (i.e. a schedule with a negative component). In Experiment 1, rats responded on a variable interval (VI) schedule with a conjoint component such that every 30 responses a reinforcement programmed by the VI schedule was cancelled. These subjects generally emitted a lower response rate than rats responding on a VI schedule yoked to the former subjects with respect to the delivery of reinforcement, although response rate differences were sometimes not large. Similar response-rate effects were obtained in Experiment 2 using a within-subject yoking procedure. In Experiment 3, reinforced interresponse times were matched on negative and VI schedules yoked in terms of reinforcement rate, and the response rate emitted in these conditions were similar. These results give support to theories of instrumental conditioning that stress the strengthening and shaping properties of reinforcement.     

Cooperative responding in rats maintained by fixed‐ and variable‐ratio schedules

The present study investigated the effects of fixed‐ratio (FR) and variable‐ratio (VR) reinforcement schedules on patterns of cooperative responding in pairs of rats. Experiment 1 arranged FR 1, FR 10, and VR 10 schedules to establish cooperative responding (water delivery depended on the joint responding of two rats). Cooperative response rates and proportions were higher under intermittent schedules than under continuous reinforcement. The FR 10 schedule generated a break‐and‐run pattern, whereas the VR 10 schedule generated a relatively high and constant rate pattern. Experiment 2 evaluated the effects of parametric manipulations of FR and VR schedules on cooperative responding. Rates and proportions of cooperative responding generally increased between ratio sizes of 1 and 5 but showed no consistent trend as the ratio increased from 5 to 10. Experiment 3 contrasted cooperative responding between an FR6 schedule and a yoked control schedule. Coordinated behavior occurred at a higher rate under the former schedule. The present study showed that external consequences and the schedules under which the delivery of these consequences are based, select patterns of coordinated behavior.     

Some Effects Of Intertrial Interval Duration On Discrete-trial Choice

Pigeons were trained in Experiment 1 on a discrete-trial concurrent variable-interval (VI) 1-min VI 3-min schedule, and in Experiment 2 on a discrete-trial concurrent VI 1.5-min VI 1.5-min schedule. In each experiment, the intertrial-interval durations (ITIs) were 0 s, 6 s, 22 s, and 120 s, and the schedules were both independent and interdependent. The purpose of the research was to determine whether lengthening the ITI would disrupt any local control that existed, measured with respect to relative response rate and changeover probability. In Experiment 1, with the independent schedules, both preference and obtained relative reinforcement rate approximated .75 at short ITIs, but then decreased toward .50 with longer ITIs. With interdependent schedules, both preference and obtained relative reinforcement rate approximated .75 at all ITIs. In both experiments, with both independent and interdependent schedules, changeover probabilities for each response in a sequence of up to five successive responses to a given schedule were variable for individual birds. The average changeover probabilities for all birds suggested perseveration rather than a systematic increase within a given ITI or a systematic trend toward chance responding as ITI duration increased. Finally, the changeover functions did not differ when a sequence of responses was calculated to begin anew after reinforcement rather than with the first response on a schedule. Taken together, the data were inconsistent with a theory holding that only local processes underlie choice in discrete-trial procedures.     

The effect of signaled reinforcement on rats' fixed-interval responding

Four experiments examined the effect on rats' response rate of presenting a brief (500 ms) stimulus simultaneously with the delivery of food on fixed-interval (FI) schedules. In Experiment 1, reinforcement signals that were spatially diffuse (both tones and lights) elevated rates of responding, but responding was attenuated by localized visual stimuli. The remaining experiments examined the signal-induced potentiation of responding. In Experiment 2, a tone reinforcement signal potentiated response rates on an FI schedule, but attenuated response rates on a variable-interval (VI) schedule. This difference was obtained even though the overall rate of responding was equated on the two schedules before the introduction of the signal. Signal-induced potentiation of responding occurred over a range of FI values employed in Experiment 3. In Experiment 4, presenting a reinforcement signal when high local rates of response had occurred immediately before reinforcement resulted in potentiated rates of responding on an FI schedule. The opposite effect on response rate occurred when the reinforcement signal followed only low local rates of response. These results indicate that a variety of factors influence the effects of a reinforcement signal. They imply, however, that the local rate of response at the time of reinforcement is a key factor in establishing the nature of the signaling effect.     

Marking effects in instrumental performance on DRH schedules

Three experiments investigated the effect of presenting a brief stimulus after a response sequence on the rate of lever-pressing by rats on differential reinforcement of high rate (DRH) schedules. In Experiment 1 enhanced responding was produced by a visual stimulus presented during a 500-msec delay of reinforcement compared to a condition in which no stimulus was presented. In Experiment 2 rats responded on a multiple DRH DRH schedule in which the DRH contingency was reinforced on a 50% schedule in each component. Equivalent levels of responding occurred in the components when reinforcement was signalled in one component and when the signal was presented following the non-reinforced schedules in the other components. A further group of rats received the stimulus presented after non-reinforced schedules in one component but not at all in the other component; responding was enhanced in the former component relative to the latter component. In Experiment 3 brief stimuli presented after the completion of DRH components on a second-order VR (DRH) schedule elevated response rates irrespective of whether the signal was presented paired or unpaired with reinforcement. The present data support the view that a brief signal may serve to mark a response sequence in memory and facilitate instrumental performance.     

Choice behavior in a discrete-trial concurrent VI-VR: A test of maximizing theories of matching

Rats were trained on a discrete-trial procedure in which one alternative (VR) was correlated with a constant probability of reinforcement while the other was correlated with a VI schedule which ran during the intertrial intervals and held the scheduled reinforcer until they were obtained by the next VI response. Relative reinforcement rate was varied in series of conditions in which the VR schedule was varied and in series in which the VI was varied. Choice behavior was described well by the generalized matching law, although moderate undermatching occurred for all subjects. Contrary to the predictions of molar maximizing (optimality) theories, there was no consistent bias in favor of the ratio alternative, and the sensitivity to reinforcement allocation was not systematically affected by whether the ratio or interval schedule was varied. The results were also contrary to momentary maximizing accounts, as there was no correspondence between the probability of a changeover to the VI behavior and the time since the last response to the VI alternative. Neither variety of maximizing theory appears to provide a general explanation of matching in concurrent schedules.     

More on concurrent interval-ratio schedules: a replication and review.

It has been suggested that the failure to maximize reinforcement on concurrent variable-interval, variable-ratio schedules may be misleading. Inasmuch as response costs are not directly measured, it is possible that subjects are optimally balancing the benefits of reinforcement against the costs of responding. To evaluate this hypothesis, pigeons were tested in a procedure in which interval and ratio schedules had equal response costs. On a concurrent variable time (VT), variable ratio-time (VRT) schedule, the VT schedule runs throughout the session and the VRT schedule is controlled by responses to a changeover key that switches from one schedule to the other. Reinforcement is presented independent of response. This schedule retains the essential features of concurrent VI VR, but eliminates differential response costs for the two alternatives. It therefore also eliminates at least one significant ambiguity about the reinforcement maximizing performance. Pigeons did not maximize rate of reinforcement on this procedure. Instead, their times spent on the alternative schedules matched the relative rates of reinforcement, even when schedule parameters were such that matching earned the lowest possible overall rate of reinforcement. It was further shown that the observed matching was not a procedural artifact arising from the constraints built into the schedule.     

Effects of reinforcement rate and reinforcer magnitude on choice behavior of humans

Two experiments with human subjects investigated the effects of rate of reinforcement and reinforcer magnitude upon choice. In Experiment 1, each of five subjects responded on four concurrent variable-interval schedules. In contrast to previous studies using non-human organisms, relative response rate did not closely match relative rate of reinforcement. Discrepancies ranged from 0.03 to 0.43 (mean equal to 0.19). Similar discrepancies were found between relative amount of time spent responding on each schedule and the corresponding relative rates of reinforcement. In Experiment 2, in which reinforcer magnitude was varied for each of five subjects, similar discrepancies ranging from 0.05 to 0.50 (mean equal to 0.21), were found between relative response rate and relative proportion of reinforcers received. In both experiments, changeover rates were lower on the long-interval concurrent schedules than on the short-interval ones. The results suggest that simple application of previous generalizations regarding the effects of reinforcement rate and reinforcer magnitude on choice for variable-interval schedules does not accurately describe human behavior in a simple laboratory situation.     

On the effects of component durations and component reinforcement rates in multiple schedules

Four experiments, each using the same six pigeons, investigated the effects of varying component durations and component reinforcement rates in multiple variable-interval schedules. Experiment 1 used unequal component durations in which one component was five times the duration of the other, and the shorter component was varied over conditions from 120 seconds to 5 seconds. The schedules were varied over five values for each pair of component durations. Sensitivity to reinforcement rate changes was the same at all component durations. In Experiment 2, both component durations were 5 seconds, and the schedules were again varied using both one and two response keys. Sensitivity to reinforcement was not different from the values found in Experiment 1. In Experiment 3, various manipulations, including body-weight changes, reinforcer duration changes, blackouts, hopper lights correlated with keylights, and overall reinforcement rate changes were carried out. No reliable increase in reinforcement sensitivity resulted from any manipulation. Finally, in Experiment 4, reinforcement rates in the two components were kept constant and unequal, and the component durations were varied. Shorter components produced significantly increased response rates normally in the higher reinforcement rate component, but schedule reversals at short component durations eliminated the response rate increases. The effects of component duration on multiple schedule performance cannot be interpreted as changing sensitivity to reinforcement nor to changing bias.     

Interresponse-time sensitivity during discrete-trial and free-operant concurrent variable-interval schedules

Two experiments investigated the sensitivity of pigeons' choice to elapsed time since the last response (i.e., to inter-response time [IRT]) during concurrent variable-interval variable-interval schedules. Experiment 1 used a two-key discrete-trial procedure with variable intertrial intervals. Experiment 2 employed a three-key free-operant procedure. In both experiments choice was found to be a function of the active-schedule IRT, defined as the time since the most recent response. Monte Carlo simulations show how this finding permits the joining of several seemingly incompatible data sets held to both support and contradict a kind of choice strategy, termed momentary maximizing, which attempts to maximize momentary reinforcement probabilities. The studies suggest that only two variables are needed to describe the static molecular structure of concurrent variable-interval choice: active-schedule IRTs and "response states" consisting of the last one or two schedule choices.     

Further evaluation of prompt density,rate of reinforcement,and the persistence of manding

Previous research has evaluated the effects of prompt rates on the rate of communicative behavior. More recent research has suggested that dense prompting can result in communicative behavior that is more resistant to change. However, existing research has not considered the impact that higher response rates had on reinforcement rate, a variable known to impact response persistence. The current study systematically replicated previous research by evaluating communicative responding in contexts associated with dense- and lean-prompt schedules and extended existing research by (a) holding reinforcement rates similar across the two prompting schedules (lean and dense), and (b) evaluating the persistence of communicative responding in the contexts associated with each prompting schedule. The results for Experiment 1 clearly replicated and extended previous research. The results for Experiment 2 were equivocal and suggested that previous reinforcement history and response class size impacted outcomes.     

Some effects of response cost upon human operant behavior

Three experiments are reported which investigated the effects of cost (point loss per response) upon human-observer responses maintained by VI and FI schedules of reinforcement (acquisition of points via critical-signal detections). (I) Cost attenuated VI response rates without substantially disturbing the constancy of responding, regardless of the presentation sequence of the no-cost and cost conditions. (II) FI scalloping appeared only under cost conditions. Under no cost, a constant rate of responding (similar to VI performance) characterized inter-reinforcement intervals. Exposure to cost did not prevent the recovery of previously established no-cost baselines. (III) FI irregularities, analogous to those commonly observed under FI reinforcement schedules, may be produced by different temporal presentations of the no-cost and cost conditions.

The results of all three experiments emphasize the importance of cost as a factor in the maintenance of human behavior on schedules of positive reinforcement.

     

Variable-ratio schedules as variable-interval schedules with linear feedback loops.

The mathematical theory of linear systems has been used successfully to describe responding on variable-interval (VI) schedules. In the simplest extension of the theory to the variable-ratio (VR) case, VR schedules are treated as if they were VI schedules with linear feedback loops. The assumption entailed by this approach, namely, that VR and VI-plus-linear-feedback schedules are equivalent, was tested by comparing responding on the two types of schedule. Four human subjects' lever pressing produced monetary reinforcers on five VR schedules, and on five VI schedules with linear feedback loops that reproduced the feedback properties of the VR schedules. Pressing was initiated by instructions in 2 subjects, and was shaped by successive approximation in the other 2. The different methods of response initiation did not have differential effects on behavior. For each of the 4 subjects, the VR and the comparable VI-plus-linear-feedback schedules generated similar average response rates and similar response patterns. The subjects' behavior on both types of schedule was similar to that of avian and rodent species on VR schedules. These results indicate that the assumption entailed by the VI-plus-linear-feedback approach to the VR case is valid and, consequently, that the approach is worth pursuing. The results also confute interresponse-time theories of schedule performance, which require interval and ratio contingencies to produce different response rates.