Optimal behavior and concurrent variable interval schedules

Behavior maintained with 2-component concurrent variable interval schedules of reinforcement (CONC VIVI) is described well by the matching law. Deviations from matching behavior have been handled by adding free parameters to the matching law equation. With CONC VIVI schedules there are infinitely many solutions to the matching law equation at each value of the procedural parameters. However, at each value of the procedural parameters, only one combination of durations of intervals spent in each VI component (dwell times) yields the combined maximum reinforcement rate. The equations that yield the optimal dwell times solution for CONC VIVI schedules are mathematically incompatible with the matching law. Optimal performance and matching coincide only when the parameter values of the two VI components are equal. It seems reasonable to use optimal behavior to assess performance in these schedules. Researchers have not compared optimal and empirical performances in CONC VIVI possibly because the equations for optimal dwell times (ODT) can be solved only numerically. We present a table of ODT for a wide range of VIs and changeover delays. We also derive a function m that can be used to compare matching data and the matching behavior predictions of optimization. We prove that 0.5<m<1.003502, and we describe some of the more nteresting properties of the function.     

More on concurrent interval-ratio schedules: a replication and review.

It has been suggested that the failure to maximize reinforcement on concurrent variable-interval, variable-ratio schedules may be misleading. Inasmuch as response costs are not directly measured, it is possible that subjects are optimally balancing the benefits of reinforcement against the costs of responding. To evaluate this hypothesis, pigeons were tested in a procedure in which interval and ratio schedules had equal response costs. On a concurrent variable time (VT), variable ratio-time (VRT) schedule, the VT schedule runs throughout the session and the VRT schedule is controlled by responses to a changeover key that switches from one schedule to the other. Reinforcement is presented independent of response. This schedule retains the essential features of concurrent VI VR, but eliminates differential response costs for the two alternatives. It therefore also eliminates at least one significant ambiguity about the reinforcement maximizing performance. Pigeons did not maximize rate of reinforcement on this procedure. Instead, their times spent on the alternative schedules matched the relative rates of reinforcement, even when schedule parameters were such that matching earned the lowest possible overall rate of reinforcement. It was further shown that the observed matching was not a procedural artifact arising from the constraints built into the schedule.     

Response elimination in rats with schedules of omission training, including yoked and response-independent reinforcement comparisons

After rats were trained to lever press, response elimination began with factorial combinations of fixed vs variable and adjusting vs constant omission training schedules. A time interval scheduling reinforcement remained the same in a constant schedule, and its length was increased as response rate declined in the adjusting schedule. A variable time (VT) response-independent reinforcement schedule followed response elimination to test the durability of response cessation. Experiment I included groups whose reinforcement was yoked to that received by the omission schedule groups. Rate of response elimination was faster with an adjusting than a constant schedule, and slightly faster with a variable than a fixed schedule. There were no significant differences in rate of response elimination between omission schedule and yoked groups. Shorter delay of reinforcement tended to increase rate of response elimination. In the subsequent VT durability test all groups displayed near-zero response rates. In Experiment II adjusting fixed and variable omission schedules, including yoked groups, were compared with fixed time (FT) and VT reinforcement schedules. Response elimination was slower in the FT and VT groups, and they responded more in a subsequent VT durability test. It was concluded that differential reinforcement of other behavior fails to account for these omission training effects, and suggestions were made for an analysis based on the correlation between response and reinforcement rate.     

A COLLATERAL EFFECT OF REWARD PREDICTED BY MATCHING THEORY

Matching theory describes a process by which organisms distribute their behavior between two or more concurrent schedules of reinforcement (Herrnstein, 1961). In an attempt to determine the generality of matching theory to applied settings, 2 students receiving special education were provided with academic response alternatives. Using a combined simultaneous treatments design and reversal design, unequal ratio schedules of reinforcement were varied across two academic responses. Findings indicated that both subjects allocated higher rates of responses to the richer schedule of reinforcement, although only one responded exclusively to the richer schedule. The present results lend support to a postulation that positive reinforcement may have undesirable collateral effects that are predicted by matching theory (Balsam & Bondy, 1983).     

Variable-time reinforcement schedules in the treatment of socially maintained problem behavior

Noncontingent reinforcement (NCR) consists of delivering a reinforcer on a time-based schedule, independent of responding. Studies evaluating the effectiveness of NCR as treatment for problem behavior have used fixed-time (FT) schedules of reinforcement. In this study, the efficacy of NCR with variable-time (VT) schedules was evaluated by comparing the effects of VT and FT reinforcement schedules with 2 individuals who engaged in problem behavior maintained by positive reinforcement. Both FT and VT schedules were effective in reducing problem behavior. These findings suggest that VT schedules can be used to treat problem behavior maintained by social consequences.     

Human choice in concurrent ratio-interval schedules of reinforcement.

Seven undergraduates participated in a concurrent-choice experiment with monetary reinforcers. Response-independent analogues of variable-interval and variable-ratio schedules were used to assess whether subjects would maximize reinforcement rate. The optimal pattern of behavior, in terms of maximizing reinforcement rate, involved a large bias toward the ratio alternative, with only occasional sampling of the interval schedule. Most experiments with pigeons, however, demonstrate matching of response rates to reinforcement rates, with only slight biases for the ratio schedule. Although subjects in the present experiment allocated more time to the ratio alternative than required by matching, the magnitude of the bias did not approximate that predicted by a maximizing account. After exposure to clock stimuli correlated with the operation of each schedule, 1 subject's behavior did show a substantial level of bias, increasing the total number of reinforcers obtained, and lay at a point between the predictions of matching and maximizing. The other subjects, however, continued to respond less optimally. The present results can be accounted for by a view of matching that incorporates the effects of delayed reinforcement.     

Molar And Molecular Control In Variable-interval And Variable-ratio Schedules

Response rates are typically higher under variable-ratio than under variable-interval schedules of reinforcement, perhaps because of differences in the dependence of reinforcement rate on response rate or because of differences in the reinforcement of long interresponse times. A variable-interval-with-added-linear-feedback schedule is a variable-interval schedule that provides a response rate/reinforcement rate correlation by permitting the minimum interfood interval to decrease with rapid responding. Four rats were exposed to variable-ratio 15, 30, and 60 food reinforcement schedules, variable-interval 15-, 30-, and 60-s food reinforcement schedules, and two versions of variable-interval-with-added-linear-feedback 15-, 30-, and 60-s food reinforcement schedules. Response rates on the variable-interval-with-added-linear-feedback schedule were similar to those on the variable-interval schedule; all three schedules led to lower response rates than those on the variable-ratio schedules, especially when the schedule values were 30. Also, reinforced interresponse times on the variable-interval-with-added-linear-feedback schedule were similar to those on variable interval and much longer than those produced by variable ratio. The results were interpreted as supporting the hypothesis that response rates on variable-interval schedules in rats are lower than those on comparable variable-ratio schedules, primarily because the former schedules reinforce long interresponse times.     

Stimuli, reinforcers, and behavior: an integration

We propose that a fundamental unit of behavior is the concurrent discriminated operant, and we discuss in detail a quantitative model of the concurrent three-term contingency that is based on the notion that an animal's behavior is controlled to differing extents by both stimulus—behavior and behavior—reinforcer relations. We show how this model can describe performance in a variety of experimental procedures: conditional discrimination and matching to sample, both with and without reinforcement for responses that are traditionally identified as errors; conditional discrimination with more than two stimuli and choice alternatives; delayed matching to sample and delayed reinforcement in matching to sample; second-order and complex conditional discrimination; and multiple and concurrent schedules. Although the model is incomplete in its coverage, and may be incorrect, we believe that this conceptual approach will bear fruit in the development of behavior theory.     

Melloration and maximization of reinforcement minus costs of behavior

Eight pigeons were exposed to independent concurrent schedules. Concurrent variable-interval 60-second variable-interval 60-second schedules were presented to one group of four subjects. Following baseline training, a limited hold was added to one of the schedules and the duration of the hold was decreased in successive conditions. Concurrent variable-interval 120-second variable-interval 40-second schedules were presented to another group of four subjects. These subjects were first exposed to decreasing durations of a limited hold in the variable-interval 40-second component. After replication of the baseline, a limited hold in the variable-interval 120-second component was decreased in duration. The initial durations of the holds were determined from the subjects' responding in the baseline conditions. A duration was chosen such that approximately 25% of the scheduled reinforcers would be canceled if responding remained unchanged.

Approximate matching of time proportions and reinforcement proportions was observed when the limited hold was added to the variable-interval 60-second schedule and when the limited hold was added to the variable-interval 40-second schedule. Time proportions were less extreme than reinforcement proportions when the limited hold operated in a variable-interval 120-second schedule. Overall reinforcement rates tended to decrease with continued training in concurrent schedules with a limited hold. Absolute deviations from time matching also decreased. The results provide evidence against the principle of reinforcement maximization, and support Herrnstein and Vaughan's (1980) melioration hypothesis.

     

A test of the melioration theory of matching

Melioration theory entails that matching in concurrent schedules occurs because the subjects equalize the local reinforcement rates (reinforcers received for each alternative divided by the time allocated to each alternative). The role of local reinforcement rates was tested by using multiple schedules in which one component involved an alternative with a high absolute rate of reinforcement and a low local reinforcement rate while the second component involved an alternative with a low absolute rate and a high local rate. These alternatives were then presented simultaneously in probe trials to determine preference between them. Contrary to melioration, the absolute rate of reinforcement, not the local rate, was the controlling variable.     

The reductive effects of noncontingent reinforcement fixed-time versus variable-time schedules.

The effects of fixed‐time (FT) and variable‐time (VT) schedules on responding were evaluated with 2 adults with mental retardation. Multielement and reversal designs were used to compare the effects of FT and VT schedules in reducing responses previously maintained on variable‐ratio reinforcement schedules. The schedules were equally effective in reducing the target behavior.     

Performance in concurrent interval schedules: a systematic replication

Five pigeons were trained on a variety of concurrent interval schedules that arranged reinforcements at either fixed or variable times after the last reinforcement. Two measures were obtained: the number of responses on each schedule, and the time spent responding on each schedule. Ratios of response rates on the two schedules did not equal ratios of reinforcement rates when both schedules were variable nor when one was variable and the other fixed. Ratios of times spent responding approximately equalled ratios of reinforcement rates when both schedules were variable, but did not do so when one was fixed.     

Tests of transitivity in choices between fixed and variable reinforcer delays.

This experiment tested for transitivity in pigeons' choices between variable-time (VT) and fixed-time (FT) schedules. In a discrete-trials procedure, a subject chose between two alternatives by making a single key peck. Each choice was between a "standard alternative," which was the same schedule throughout a condition, and an "adjusting alternative," in which the delay to reinforcement was systematically increased or decreased many times a session. These adjustments enabled an approximate indifference point to be identified--the value of the adjusting delay at which the subject chose each alternative about equally often. Each test of transitivity involved four conditions. In one, the standard alternative was a variable-time schedule with a 2-s reinforcer, and the adjusting alternative also delivered a 2-s reinforcer. A second condition was similar except that the adjusting alternative delivered a 5-s reinforcer. The indifference point from each of these conditions was then converted to a fixed-time schedule for subsequent comparisons in the third and fourth conditions, respectively. Each of these last two conditions compared one of the fixed-time schedules (based upon the previous conditions and including their different reinforcer durations) with an adjusting schedule that delivered the alternative reinforcer duration, to determine whether the obtained indifference points would be those predicted from the prior alternative-duration comparisons with the VT schedule. There was little evidence for intransitivity of choice: Averaged across subjects and replications, the obtained indifference points deviated from perfect transitivity by less than 8%, and these deviations were not statistically significant. These results contrast with those of Navarick and Fantino (1972), who found frequent violations of transitivity between periodic and aperiodic schedules using a concurrent-chains procedure with variable-interval schedules in the initial links.     

Computer program to generate operant schedules

A computer program for programming schedules of reinforcement is described. Students can use the program to experience schedules of reinforcement that are typically used with nonhuman subjects. A cumulative recording of a student’s responding can be shown on the screen and/or printed with the computer’s printer. The program can also be used to program operant schedules for animal subjects. The program was tested with human subjects experiencing fixed ratio, variable ratio, fixed interval, and variable interval schedules. Performance for human subjects on a given schedule was similar to performance for nonhuman subjects on the same schedule.     

The effect of punishment on free-operant choice behavior in humans

During Phase I, three female human subjects pressed a button for monetary reinforcement in five variable-interval schedules specifying different frequencies of reinforcement. On alternate days, responding was also punished (by subtracting money) according to a variable-ratio 34 schedule. In the absence of punishment, response rates conformed to Herrnstein's equation for single variable-interval schedules. Punishment suppressed responding at all frequencies of reinforcement. This was reflected in a change in the values of both constants in Herrnstein's equation: the value of the theoretical maximum response-rate parameter was reduced, and the parameter describing the reinforcement frequency corresponding to the half-maximal response rate was elevated. During Phase II, the same five schedules (A) were in operation (without punishment), but in addition, a concurrent variable-interval schedule (B) of standard reinforcement frequency was introduced. On alternate days, responding in Component B was punished according to a variable-ratio 34 schedule. In the absence of punishment, absolute response rates conformed to equations proposed by Herrnstein to describe performance in concurrent schedules; the ratios of the response rates in the two components and the ratios of the times spent in the two components conformed to the Matching Law. When responding in Component B was punished, response rates in Component B were reduced and those in Component A were elevated, these changes being reflected in distortions of the matching relationship.     

An auto-adjusting shaping procedure

A variable interval (VI) schedule is described that automatically adjusts the programmed rates of reinforcement in accordance with the rates of responding of subjects during the two immediately preceding 30-sec time intervals. The schedule prescribes that as rate of responding decreases, programmed reinforcement rate increases, and that when rate of responding increases, reinforcement rate decreases. Thus, programmed reinforcement rate is adjusted continuously until some target value is reached. Ten rats were exposed to this procedure five times a day at 1-h intervals. The target, set at VI 120 sec, was reached by most subjects within 4 days of training. Subsequently, all subjects responded consistently during five daily 1-h sessions with VI 120 sec. This procedure speeds up the training of subjects on long VI schedules and substantially reduces the time and effort spent observing the subjects and adjusting the schedule parameter value during the early development of responding.     

The effect of signaled reinforcement availability on concurrent performances in humans

During Phase I, three female human subjects pressed a button for monetary reinforcement in two-component concurrent variable-interval schedules. Five different reinforcement frequencies were used in component A, whereas the reinforcement frequency in component B was held constant. Absolute rates of responding conformed to equations proposed by Herrnstein to describe concurent performances, and the ratios of the response rates and the times spent in the two components conformed to the matching law. During Phase II, the availability of reinforcement in component A was signaled by the illumination of a lamp. This resulted in suppression of response rates in component A and elevation of response rates in component B, these changes being reflected in a distortion of the matching relationship which took the form of a bias in favor of component B.     

Response rate and sensitivity to the molar feedback function relating response and reinforcement rate on VI+ schedules of reinforcement

In 5 experiments, the author examined rats' sensitivity to the molar feedback function relating response rate to reinforcement rate on schedules of reinforcement. These studies demonstrated that, at lower rates of responding, rats' performance on variable ratio (VR), variable interval (VI), and variable interval with linear feedback loop (VI+) schedules was determined largely by reinforcement of interresponse times; response rates were faster on VR than on both VI and VI+ schedules. In contrast, when procedures were adopted to maintain high rates of response, rats showed sensitivity to the molar characteristics of the schedules; they responded as fast on a VI+ schedule as on a VR schedule and faster on both of these schedules than on a yoked VI schedule. When the variance of response rate was manipulated, this factor was noted as an important element in determining sensitivity to the molar characteristics of the schedule.     

Competition between noncontingent and contingent reinforcement schedules during response acquisition

We examined the extent to which noncontingent reinforcement (NCR), when used as treatment to reduce problem behavior, might interfere with differential reinforcement contingencies designed to strengthen alternative behavior. After conducting a functional analysis to identify the reinforcers maintaining 2 participants' self-injurious behavior (SIB), we delivered those reinforcers under dense NCR schedules. We delivered the same reinforcers concurrently under differential-reinforcement-of-alternative-behavior (DRA) contingencies in an attempt to strengthen replacement behaviors (mands). Results showed that the NCR plus DRA intervention was associated with a decrease in SIB but little or no increase in appropriate mands. In a subsequent phase, when the NCR schedule was thinned while the DRA schedule remained unchanged, SIB remained low and mands increased. These results suggest that dense NCR schedules may alter establishing operations that result in not only suppression of problem behavior but also interference with the acquisition of appropriate behavior. Thus, the strengthening of socially appropriate behaviors as replacements for problem behavior during NCR interventions might best be achieved if the NCR schedule is first thinned.