Synthesizing concurrent interval performances

Concurrent schedules may be viewed as consisting of two pairs of stay and switch schedules, each pair associated with one of the alternatives. A stay schedule arranges reinforcers for staying and responding at one alternative, whereas the associated switch schedule arranges reinforcers for switching to the other alternative. In standard concurrent schedules, the stay schedule at each alternative is equivalent to the switch schedule at the other alternative. MacDonall (1999) exposed rats to one pair of stay and switch variable-ratio schedules and varied the response requirements across conditions. Combining results from symmetric pairs produced composite performances that were described by the generalized matching law. This outcome was noteworthy because the data were obtained from performances at two alternatives with contingencies that were functionally unrelated to each other. This result suggests that concurrent performances may consist of two unrelated performances that alternate as behavior moves between alternatives. The purpose of the present experiment was to extend those results to interval schedules. Rats were exposed to pairs of random-interval schedules, and across conditions their mean intervals were varied. When data from appropriately paired conditions were combined, the composite performances were consistent with the generalized matching law. In addition, the results supported two models of concurrent performances that were based on local variables at an alternative (behavior, and stay and switch reinforcers): a modified version of the contingency discrimination model (Davison & Jenkins, 1985) and the local model (MacDonall, 1999).     

Optimal behavior and concurrent variable interval schedules

Behavior maintained with 2-component concurrent variable interval schedules of reinforcement (CONC VIVI) is described well by the matching law. Deviations from matching behavior have been handled by adding free parameters to the matching law equation. With CONC VIVI schedules there are infinitely many solutions to the matching law equation at each value of the procedural parameters. However, at each value of the procedural parameters, only one combination of durations of intervals spent in each VI component (dwell times) yields the combined maximum reinforcement rate. The equations that yield the optimal dwell times solution for CONC VIVI schedules are mathematically incompatible with the matching law. Optimal performance and matching coincide only when the parameter values of the two VI components are equal. It seems reasonable to use optimal behavior to assess performance in these schedules. Researchers have not compared optimal and empirical performances in CONC VIVI possibly because the equations for optimal dwell times (ODT) can be solved only numerically. We present a table of ODT for a wide range of VIs and changeover delays. We also derive a function m that can be used to compare matching data and the matching behavior predictions of optimization. We prove that 0.5<m<1.003502, and we describe some of the more nteresting properties of the function.     

Probabilistically reinforced choice behavior in pigeons

A single principle, "momentary maximizing", may account for much of a pigeon's steady-state behavior in both probability learning and concurrent variable interval experiments. The principle states that a pigeon tends to choose the alternative that momentarily has the higher probability of reinforcement. A successive discrimination procedure, which produced matching in an earlier experiment, produced here a tendency to maximize if training were adequately extended. Maximizing was produced also by other procedures, in which no reinforcing event was presented on some trials: one procedure did and two did not provide a bird with information about the availability of reinforcement on a key after an unreinforced response on the other key. The latter two procedures were analogous to concurrent variable interval schedules in two respects: the reinforcement probability on one key increased while a bird responded on the other key; and they produced matching. But sequential statistics suggested that matching resulted from momentary maximizing. Depending on the procedure, the tendency to maximize produced different relative frequencies of pecking a key for a fixed relative frequency of reinforcement. Computer simulation of maximizing behavior in several concurrent variable interval schedules produced matching and sequential statistics similar to those produced by a real bird.     

Allocation of complex, sequential operants on multiple and concurrent schedules of reinforcement

Pigeons could produce food by pecking exactly four times on each of two keys, in any order. In the first experiment, these response sequences were reinforced on a series of multiple schedules of variable-interval reinforcement. In the second experiment, these response sequences were reinforced on a series of concurrent schedules of reinforcement. In both experiments, highly stereotyped response sequences developed. If these response sequences were treated as individual responses, the resulting data conformed to what is typically reported in studies of multiple and concurrent schedules involving individual responses. For example, behavioral contrast was observed with the multiple schedules, and matching was observed with the concurrent schedules. However, schedule manipulation had no effect on within-sequence characteristics of responses like accuracy, stereotypy, or rate. These data constitute further evidence that response sequences can become functional behavioral units.     

Choice dynamics in concurrent ratio schedules of reinforcement

The generalized matching law predicts performance on concurrent schedules when variable-interval schedules are programmed but is trivially applicable when independent ratio schedules are used. Responding usually is exclusive to the schedule with the lowest response requirement. Determining a method to program concurrent ratio schedules such that matching analyses can be usefully employed would extend the generality of matching research and lead to new avenues of research. In the present experiments, ratio schedules were programmed dependently such that responses to either of the two options progressed the requirement on both schedules. Responding is not exclusive because the probability of reinforcement increases on both schedules as responses are allocated to either schedule. In Experiment 1, performance on concurrent variable-ratio schedules was assessed, and reinforcer ratios were varied across conditions to investigate changes in sensitivity. Additionally, the length of a changeover delay was manipulated. In Experiment 2, performance was compared under concurrently available, dependently programmed variable-ratio and fixed-ratio schedules. Performance was well described by the generalized matching law. Increases in the changeover delay decreased sensitivity, whereas sensitivity was higher when variable-ratio schedules were employed, compared with fixed-ratio schedules. Concurrent ratio schedules can be a viable approach to studying functional differences between ratio and interval schedules.     

Dynamics in the fine structure of schedule-controlled behavior.

The variability in the behavioral equilibrium established by six basic schedules was characterized. The measures were the pause preceding the first response in each interreinforcement interval; the mean rate of responding in each interreinforcement interval; and the relative frequency of each interresponse time. The temporal windows ranged across the 780-session exposure, across a session, and across the interreinforcement interval. A display of individual interresponse times as a function of time in the interreinforcement interval indicated clear recurrent responding at somewhat less than 3 Hz in every bird, even after extended exposure to a schedule and regardless of the contingency. No strong sequential dependencies in the interresponse-time distributions were identified. A simulator, based on a simple recurrent pulser, was presented that produced output similar to the obtained data. An archival data base of the behavior chronically maintained by the simple schedules was also generated.     

Local contrast in behavior allocation during multiple-schedule components

Allocation of responses between two keys was studied during two alternating multiple-schedule components. Responses were recorded in successive quarters of each component. Variable-interval reinforcer schedules on the two keys were constant throughout the experiment for one (constant) component and were varied over conditions on one key for the other, producing changes in reinforcer ratios for the varied component. Behavior allocation for the first quarter of the constant component was inversely related to varied-component reinforcer ratios, a form of local contrast, but this relationship was not observed later in the component. During the first quarter of the varied component, slopes of matching lines were high and decreased later in the component. It is argued that this form of local contrast cannot be explained in terms of reallocation of extraneous reinforcers between components, and that the matching law for concurrent operants does not capture some sources of control over behavior allocation. A simple extension of the matching law is offered that adequately describes behavior changes during both components. A version of this formulation can predict contrast effects in absolute response rates.     

A model for residence time in concurrent variable interval performance

A component-functions model of choice behavior is proposed for performance on interdependent concurrent variable-interval (VI) variable-interval schedules based on the product of two component functions, one that enhances behavior and one that reduces behavior. The model is the solution to the symmetrical pair of differential equations describing behavioral changes with respect to two categories of reinforcers: enhancing and reducing, or excitatory and inhibitory. The model describes residence time in interdependent concurrent VI VI schedules constructed from arithmetic and exponential distributions. The model describes the data reported by Alsop and Elliffe (1988) and Elliffe and Alsop (1996) with a variance accounted for of 87% compared to 64% accounted for by the Davison and Hunter (1976) model and 42% by Herrnstein's (1970) hyperbola. The model can explain matching, undermatching, and overmatching in the same subject under different procedures and has the potential to be extended to performance on concurrent schedules with more than two alternatives, multiple schedules, and single schedules. Thus, it can be considered as an alternative to Herrnstein's quantitative law of effect.     

Human choice in concurrent ratio-interval schedules of reinforcement.

Seven undergraduates participated in a concurrent-choice experiment with monetary reinforcers. Response-independent analogues of variable-interval and variable-ratio schedules were used to assess whether subjects would maximize reinforcement rate. The optimal pattern of behavior, in terms of maximizing reinforcement rate, involved a large bias toward the ratio alternative, with only occasional sampling of the interval schedule. Most experiments with pigeons, however, demonstrate matching of response rates to reinforcement rates, with only slight biases for the ratio schedule. Although subjects in the present experiment allocated more time to the ratio alternative than required by matching, the magnitude of the bias did not approximate that predicted by a maximizing account. After exposure to clock stimuli correlated with the operation of each schedule, 1 subject's behavior did show a substantial level of bias, increasing the total number of reinforcers obtained, and lay at a point between the predictions of matching and maximizing. The other subjects, however, continued to respond less optimally. The present results can be accounted for by a view of matching that incorporates the effects of delayed reinforcement.     

Treating destructive behavior reinforced by increased caregiver compliance with the participant's mands

Functional analyses sometimes do not identify momentary fluctuations in the function of destructive behavior (Bowman, Fisher, Thompson, & Piazza, 1997). In such cases, individuals may mand for the reinforcer that is currently most preferred and display destructive behavior if that mand goes unreinforced. In this study, we conducted a mand analysis to test whether destructive behavior functioned as a precurrent response that increased reinforcement for the participant's mands. We then evaluated a treatment that matched this function of destructive behavior by providing differential or time-based reinforcement of participant mands in accordance with multiple or chained schedules with reinforcement-schedule thinning. Decreases in destructive behavior averaged 97.4% across cases. We discuss these results relative to the importance of matching treatments for destructive behavior to operant functions for both traditional and idiosyncratic functions of destructive behavior.     

A functional analysis of chained fixed-interval schedule performance

Three pigeons were trained on two-link chained fixed-interval fixed-interval schedules. Numbers of responses, time spent responding, and the total time spent in each component were measured. The data were analyzed according to the matching law for multiple and concurrent schedules. In most conditions, the ratio of response rates in the two links was a constant proportion of the ratio that would be predicted in a multiple schedule with the same components. Data on pauses during the interval schedules showed that, in most conditions, the pause duration was a linear function of the interval length, and greater in the initial link than in the terminal link. The experiment thus demonstrated a quantitative functional analysis of performance on a chained schedule.     

Optimization and the matching law as accounts of instrumental behavior

The interaction between instrumental behavior and environment can be conveniently described at a molar level as a feedback system. Two different possible theories, the matching law and optimization, differ primarily in the reference criterion they suggest for the system. Both offer accounts of most of the known phenomena of performance on concurrent and single variable-interval and variable-ratio schedules. The matching law appears stronger in describing concurrent performances, whereas optimization appears stronger in describing performance on single schedules.     

Concurrent variable-interval variable-ratio schedules can provide only weak evidence for matching

Herrnstein and Heyman (1979) showed that when pigeons' pecking is reinforced on concurrent variable-interval variable-ratio schedules, (1) their behavior ratios match the ratio of the schedules' reinforcer frequencies, and (2) there is more responding on the variable interval. Since maximizing the reinforcement rate would require responding more on the variable ratio, these results were presented as establishing the primacy of matching over maximizing. In the present report, different ratios of behavior were simulated on a computer to see how they would affect reinforcement rates on these concurrent schedules. Over a wide range of experimenter-specified choice ratios, matching obtained — a result suggesting that changes in choice allocation produced changes in reinforcer frequencies that correspond to the matching outcome. Matching also occurred at arbitrarily selected choice ratios when reinforcement rates were algebraically determined by each schedule's reinforcement-feedback function. Additionally, three birds were exposed to concurrent variable-interval variable-ratio schedules contingent on key pecking in which hopper durations were varied in some conditions to produce experimenter-specified choice ratios. Matching generally obtained between choice ratios and reinforcer-frequency ratios at these different choice ratios. By suggesting that reinforcer frequencies track choice on this procedure, instead of vice versa, this outcome questions whether matching-as-outcome was due to matching-as-process in the Herrnstein and Heyman study.     

A quantitative analysis of extreme choice

Six homing pigeons were trained on a variety of concurrent variable-interval schedules in a switching-key procedure. Unlike previous work, reinforcer ratios of up to 160 to 1 and concurrent extinction variable-interval schedules were arranged in order to investigate choice when reinforcer-frequency outcomes were extremely different. The data obtained over 11 conditions were initially analyzed according to the generalized matching law, which fitted the data well. The generalized matching law was then fitted only to conditions in which the reinforcer ratios were between 1 to 10 and 10 to 1. The deviations of choice measures from the other four more extreme reinforcer-ratio conditions were significantly more towards equal choice than predicted by this second generalized matching fit. A contingency-discriminability model, which predicts such deviations, described the data more effectively than did the generalized matching law, and also correctly predicted the maintenance of responding on both alternatives when one was associated with extinction.     

Effects of methylphenidate on sensitivity to reinforcement in children diagnosed with attention deficit hyperactivity disorder: an application of the matching law

The behavior of children diagnosed with attention deficit hyperactivity disorder (ADHD) has been hypothesized to be the result of decreased sensitivity to consequences compared to typical children. The present study examined sensitivity to reinforcement in 2 boys diagnosed with ADHD using the matching law to provide more precise and quantitative measurement of this construct. This experiment also evaluated the effects of methylphenidate (MPH) on sensitivity to reinforcement of children with ADHD. Subjects completed math problems to earn tokens under four different variable-interval (VI) schedules of reinforcement presented in random order under both medicated and nonmedicated conditions. Results showed that, in the medicated condition, the matching functions for both subjects resulted in higher asymptotic values, indicating an overall elevation of behavior rate under these conditions. The variance accounted for by the matching law was also higher under the medicated conditions, suggesting that their behavior more closely tracked the changing rates of reinforcement while taking MPH compared to placebo. Under medicated conditions, the reinforcing efficacy of response-contingent tokens decreased. Results are discussed with respect to quantifying behavioral changes and the extent to which the drug interacts with prevailing contingencies (i.e., schedule values) to influence behavioral variability.     

Dynamical concurrent schedules

Previous work on the matching law has predominantly focused on the molar effects of the contingency by examining only one reinforcer ratio for extended periods. Responses are distributed as a function of reinforcer ratios under these static conditions. But the outcome under a dynamic condition in which reinforcer ratios change continuously has not been determined. The present study implemented concurrent variable-interval schedules that changed continuously across a fixed 5-min trial. The schedules were reciprocally interlocked. The variable interval for one key changed continuously from a variable-interval 15-s to a variable-interval 480-s, while the schedule for the other key changed from a variable-interval 480-s to a variable-interval 15-s. This dynamical concurrent schedule shifted behavior in the direction of matching response ratios to reinforcer ratios. Sensitivities derived from the generalized matching law were approximately 0.62, the mean absolute bias was approximately 0.11, and r2s were approximately 0.86. It was concluded that choice behavior can come to adapt to reinforcer ratios that change continuously over a relatively short time and that this change does not require extensive experience with a fixed reinforcer ratio. The results were seen as supportive of the view that all behavior constitutes choice.