Nonstable concurrent choice in pigeons

Six pigeons were trained on concurrent variable-interval schedules in which the arranged reinforcer ratios changed from session to session according to a 31-step pseudorandom binary sequence. This procedure allows a quantitative analysis of the degree to which performance in an experimental session is affected by conditions in previous sessions. Two experiments were carried out. In each, the size of the reinforcer ratios arranged between the two concurrent schedules was varied between 31-step conditions. In Experiment 1, the concurrent schedules were arranged independently, and in Experiment 2 they were arranged nonindependently. An extended form of the generalized matching law described the relative contribution of past and present events to present-session behavior. Total performance in sessions was mostly determined by the reinforcer ratio in that session and partially by reinforcers that had been obtained in previous sessions. However, the initial exposure to the random sequence produced a lower sensitivity to current-session reinforcers but no difference in overall sensitivity to reinforcement. There was no evidence that the size of the reinforcer ratios available on the concurrent schedules affected either overall sensitivity to reinforcement or the sensitivity to reinforcement in the current session. There was also no evidence of any different performance between independent and nonindependent scheduling. Because of these invariances, this experiment validates the use of the pseudorandom sequence for the fast determination of sensitivity to reinforcement.     

Choice in transition: Replication and extension to preschool children in a naturalistic setting

This study replicated previous basic research into the dynamics of choice and extended this analysis to children's behavior in a naturalistic setting. Two preschoolers with disabilities were observed interacting with their teachers at baseline and during an experimental analysis involving four pairs of concurrent variable‐interval schedules of adult attention implemented by an experimenter. Each child was exposed to four experimental phases in which the relative reinforcer rates for on‐ and off‐task behavior were 10:1, 1:1, 1:10, and reversed back to 10:1. The 10:1 phase was designed to mimic the same schedules and types of adult attention observed at baseline. We used the generalized matching equation to model steady‐state behavior at the end of the transition phases and to evaluate changes in sensitivity at various points throughout the phases. Choice in transition was evaluated by plotting log behavior ratios by session, cumulated time on‐ and off‐task and cumulated attention for on‐ and off‐task behavior by session, and interreinforcer behavior ratios following different sequences of the first four reinforcer deliveries. The generalized matching equation accounted for a large proportion of variance in steady‐state responding, sensitivity values increased steadily throughout the phases, patterns of choice in transition were similar to those reported in basic research, and interreinforcer preference generally shifted toward the just‐reinforced alternative. These findings are consistent with previous basic research and support the generality of the dynamics of choice to children's on‐ and off‐task behavior reinforced by adult attention.     

QUANTIFICATION OF ETHANOL'S ANTIPUNISHMENT EFFECT IN HUMANS USING THE GENERALIZED MATCHING EQUATION

Increases in rates of punished behavior by the administration of anxiolytic drugs (called antipunishment effects) are well established in animals but not humans. The present study examined antipunishment effects of ethanol in humans using a choice procedure. The behavior of 5 participants was placed under six concurrent variable‐interval schedules of monetary reinforcement. In three of the six concurrent schedules, punishment, in the form of monetary loss, was superimposed on one alternative. Data were analyzed according to the generalized matching equation which distinguishes between bias (allocation of behavior beyond what matching to relative reinforcer densities would predict) and sensitivity to reinforcement (how well behavior tracks relative reinforcer densities). In addition, participants completed a pencil‐tapping test. Under placebo punishment conditions, all participants demonstrated low response rates and a bias against the alternative associated with punishment, despite a resultant loss of available reinforcers. Bias against the punished alternative was dose‐dependently reduced in participants shown to be most sensitive to ethanol (0.6, 1.2, and 1.8 g/kg) in measures of overall responding and on the pencil‐tapping test. No ethanol‐induced change in bias was noted when punishment was not imposed. Sensitivity to reinforcement also decreased for participants shown to be sensitive to ethanol. In addition to extending antipunishment effects to humans, these results also show that antipunishment effects can be quantified via the matching equation.     

Concurrent-schedule performance in dairy cows: persistent undermatching.

Performance of dairy cows responding under concurrent variable-interval variable-interval schedules of food delivery was examined, with results analyzed in terms of the generalized matching equation. In Experiment 1, bias measures indicated that crushed barley was preferred over meatmeal when these foods were available under the alternative schedules. For whole-session data, substantial undermatching of response and time-allocation ratios to obtained reinforcement ratios was evident. Postreinforcement pause time ratios approximately matched obtained reinforcement rates. Subtracting these times from total time-allocation values yielded net time-allocation ratios that undermatched obtained reinforcement ratios to a greater degree than did whole-session time-allocation ratios. In Experiment 2, substantial undermatching was evident when the same foods (hay for 2 cows, crushed barley for 2 others) were available under the alternative schedules. Food-related activities and other defined behavior not related to food were quantified by direct observation, and were found to occupy a substantial proportion (roughly 40% to 80%) of experimental sessions. Subtracting the time spent in these activities from the time allocated to each component schedule did not reduce the degree of undermatching obtained. Across all conditions in both experiments, slopes of regression lines relating behavioral outputs to environmental inputs characteristically were below 0.6, which agrees with prior findings and suggests that, contrary to suggestions in the literature, undermatching in dairy cows is not the result of using different foods under alternative schedules or differential pausing under those schedules.     

Successive independence and behavioral contrast in a closed economy.

Two pigeons had access to multiple concurrent schedules of reinforcement for 24 hours per day in their home cages. The variable-interval schedules comprising the multiple concurrent schedules were varied across 16 conditions. In three sets of conditions, one schedule was varied while its concurrent alternative and the concurrent schedules in the other component were held constant. Behavioral contrast was observed; that is, as the rate of reinforcement arranged by the varied schedule decreased, response rates on the constant schedules typically increased. These conditions formed part of two larger sets of conditions in which the concurrent schedules in one multiple-schedule component remained constant while the concurrent schedules in the other component were varied. Successive independence was found, in that behavior allocation during the constant component did not vary as a function of the reinforcer ratios in the varied component. Successive independence between components in multiple concurrent schedules is a robust result that occurs in closed economies and under conditions that promote behavioral contrast.     

Concurrent variable‐interval variable‐ratio schedules in a dynamic choice environment

Most studies of operant choice have focused on presenting subjects with a fixed pair of schedules across many experimental sessions. Using these methods, studies of concurrent variable‐ interval variable‐ratio schedules helped to evaluate theories of choice. More recently, a growing literature has focused on dynamic choice behavior. Those dynamic choice studies have analyzed behavior on a number of different time scales using concurrent variable‐interval schedules. Following the dynamic choice approach, the present experiment examined performance on concurrent variable‐interval variable‐ratio schedules in a rapidly changing environment. Our objectives were to compare performance on concurrent variable‐interval variable‐ratio schedules with extant data on concurrent variable‐interval variable‐interval schedules using a dynamic choice procedure and to extend earlier work on concurrent variable‐interval variable‐ratio schedules. We analyzed performances at different time scales, finding strong similarities between concurrent variable‐interval variable‐interval and concurrent variable‐interval variable‐ ratio performance within dynamic choice procedures. Time‐based measures revealed almost identical performance in the two procedures compared with response‐based measures, supporting the view that choice is best understood as time allocation. Performance at the smaller time scale of visits accorded with the tendency seen in earlier research toward developing a pattern of strong preference for and long visits to the richer alternative paired with brief “samples” at the leaner alternative (“fix and sample”).     

Concurrent reinforcement schedules for problem behavior and appropriate behavior: experimental applications of the matching law

This study evaluated how children who exhibited functionally equivalent problem and appropriate behavior allocate responding to experimentally arranged reinforcer rates. Relative reinforcer rates were arranged on concurrent variable-interval schedules and effects on relative response rates were interpreted using the generalized matching equation. Results showed that relative rates of responding approximated relative rates of reinforcement. Finally, interventions for problem behavior were evaluated and differential reinforcement of alternative behavior and extinction procedures were implemented to increase appropriate behavior and decrease problem behavior. Practical considerations for the application of the generalized matching equation specific to severe problem behavior are discussed, including difficulties associated with defining a reinforced response, and obtaining steady state responding in clinical settings.     

Concurrent fixed-interval variable-ratio schedules and the matching relation

Five rats responded under concurrent fixed-interval variable-ratio schedules of food reinforcement. Fixed-interval values ranged from 50-seconds to 300-seconds and variable-ratio values ranged from 30 to 360; a five-second changeover delay was in effect throughout the experiment. The relations between reinforcement ratios obtained from the two schedules and the ratios of responses and time spent on the schedules were described by Baum's (1974) generalized matching equation. All subjects undermatched both response and time ratios to reinforcement ratios, and all subjects displayed systematic bias in favor of the variable-ratio schedules. Response ratios undermatched reinforcement ratios less than did time ratios, but response ratios produced greater bias than did time ratios for every subject and for the group as a whole. Local rates of responding were generally higher on the variable-ratio than on the fixed-interval schedules. When responding was maintained by both schedules, a period of no responding on either schedule immediately after fixed-interval reinforcement typically was followed by high-rate responding on the variable-ratio schedule. At short fixed-interval values, when a changeover to the fixed-interval schedule was made, responding usually continued until fixed-interval reinforcement was obtained; at longer values, a changeover back to the variable-ratio schedule usually occurred when fixed-interval reinforcement was not forthcoming within a few seconds, and responding then alternated between the two schedules every few seconds until fixed-interval reinforcement finally was obtained.     

Choosing among multiple alternatives: Relative and overall reinforcer rates

Choice behavior among two alternatives has been widely researched, but fewer studies have examined the effect of multiple (more than two) alternatives on choice. Two experiments investigated whether changing the overall reinforcer rate affected preference among three and four concurrently scheduled alternatives. Experiment 1 trained six pigeons on concurrent schedules with three alternatives available simultaneously. These alternatives arranged reinforcers in a ratio of 9:3:1 with the configuration counterbalanced across pigeons. The overall rate of reinforcement was varied across conditions. Preference between the pair of keys arranging the 9:3 reinforcer ratio was less extreme than the pair arranging the 3:1 reinforcer ratio regardless of overall reinforcer rate. This difference was attributable to the richer alternative receiving fewer responses per reinforcer than the other alternatives. Experiment 2 trained pigeons on concurrent schedules with four alternatives available simultaneously. These alternatives arranged reinforcers in a ratio of 8:4:2:1, and the overall reinforcer rate was varied. Next, two of the alternatives were put into extinction and the random interval duration was changed from 60 s to 5 s. The ratio of absolute response rates was independent of interval length across all conditions. In both experiments, an analysis of sequences of visits following each reinforcer showed that the pigeons typically made their first response to the richer alternative irrespective of which alternative was just reinforced. Performance on these three‐ and four‐alternative concurrent schedules is not easily extrapolated from corresponding research using two‐alternative concurrent schedules.     

Choice, contingency discrimination, and foraging theory

Four pigeons were trained on eight or nine pairs of independent concurrent variable-interval schedules. The range of reinforcement ratios included extreme ratios (up to 532 to 1). Large samples of stable performance were gathered. Contrary to the findings of Davison and Jones (1995), the generalized matching law described choice more accurately than a contingency-discriminability model. Taking small samples (5 to 10 sessions) and applying a more liberal stability criterion used by Davison and Jones only increased the unsystematic variance in the data and in estimates of generalized-matching-law sensitivity. Because changing to dependent scheduling and inserting a changeover delay had no systematic effect, the deviations from generalized matching reported by Davison and Jones probably arose from imperfectly discriminated stimuli. Analysis of visits revealed that visits to the nonpreferred alternative were brief and approximately constant. When choice between the preferred (rich) and nonpreferred (lean) alternatives, regardless of position, was analyzed according to the generalized matching law, sensitivities approximated 1.0, with bias in favor of the lean alternative. This bias, which arose from an excessive frequency of visits to the lean alternative, explains undermatching as the result of fitting one line to a choice relation that consists of two displaced lines, both with a slope of 1.0. The pattern of deviation from the generalized matching line confirmed this account. The findings suggest an alternative analysis of choice that focuses on probability of visiting the lean alternative as the dependent variable. This probability was directly proportional to ratio of reinforcement. Matching, undermatching, and overmatching may all be explained by a view of concurrent performance based on foraging theory, in which responding occurs primarily at the rich alternative and is occasionally interrupted by brief visits to the lean alternative.     

Hill-climbing by pigeons

Pigeons were exposed to two types of concurrent operant-reinforcement schedules in order to determine what choice rules determine behavior on these schedules. In the first set of experiments, concurrent variable-interval, variable-interval schedules, key-peck responses to either of two alternative schedules produced food reinforcement after a random time interval. The frequency of food-reinforcement availability for the two schedules was varied over different ranges for different birds. In the second series of experiments, concurrent variable-ratio, variable-interval schedules, key-peck responses to one schedule produced food reinforcement after a random time interval, whereas food reinforcement occurred for an alternative schedule only after a random number of responses. Results from both experiments showed that pigeons consistently follow a behavioral strategy in which the alternative schedule chosen at any time is the one which offers the highest momentary reinforcement probability (momentary maximizing). The quality of momentary maximizing was somewhat higher and more consistent when both alternative reinforcement schedules were time-based than when one schedule was time-based and the alternative response-count based. Previous attempts to provide evidence for the existence of momentary maximizing were shown to be based upon faulty assumptions about the behavior implied by momentary maximizing and resultant inappropriate measures of behavior.     

Rat choice in rapidly changing concurrent schedules

In two experiments, experimentally naïve rats were trained in concurrent variable‐interval schedules in which the reinforcer ratios changed daily according to a pseudorandom binary sequence. In Experiment 1, relative response rates showed clear sensitivity to current‐session reinforcer ratios, but not to previous sessions' reinforcer ratios. Within sessions, sensitivity to the current session's reinforcement rates increased steadily, and by session end, response ratios approached matching to the current‐session reinforcer ratios. Across sessions, sensitivity to the current session's reinforcer ratio decreased with continued exposure to the pseudorandom binary sequence, contrary to expectations based on previous studies demonstrating learning sets. Using a second group of naïve rats, Experiment 2 replicated the main results from Experiment 1 and showed that although there were increases over sessions in both changeover rate and response rate during the changeover delay, neither could explain the accompanying reductions in sensitivity. We consider the role of reinforcement history, showing that our results can be simulated using two separate representations, one local and one nonlocal, but a more complex approach will be needed to bring together these results and other history effects such as learning sets and spontaneous recovery.     

The role of discriminative stimuli in concurrent performances

Key pecking in pigeons was examined under concurrent and parallel arrangements of two independent and simultaneously available variable-interval schedules. Pecks on the changeover key alternated the schedule of reinforcement for responses on the main key. Under concurrent schedules, discriminative stimuli were paired with the reinforcement schedule arranged in each component and changeover responses also alternated these stimuli. Under parallel schedules, changeover responses alternated the effective reinforcement schedule, but did not change the discriminative stimulus. On concurrent procedures, changeover response rate was inversely related to the difference in reinforcement rate between the two components, whereas on parallel schedules no consistent relationship was found. With both schedules, absolute response and reinforcement rates were positively related, although for a given set of reinforcement frequencies, rates were often higher on the concurrent schedules. On concurrent schedules, relative response rates and relative times were equal to relative reinforcement rates. On parallel schedules these ratios were positively related, but response and time ratios were much smaller than were obtained with comparable concurrent schedules. This inequality was most pronounced when absolute reinforcement frequencies were lowest.     

A local model of concurrent performance

Concurrent procedures may be conceptualized as consisting of two pairs of schedules with only one pair operating at a time. One schedule of each pair arranges reinforcers for staying in the current alternative, and the other schedule arranges reinforcers for switching to the other alternative. These pairs alternate operation as the animal switches between choices. This analysis of the contingencies suggests that variables operating within an alternative produce behavior that conforms to the generalized matching law. Rats were exposed to one pair of stay and switch schedules in each condition, and the probabilities of reinforcement varied across conditions. Both run length and visit duration were power functions of the ratio of the probabilities of reinforcement for staying and switching. The local model, a model of performance on concurrent procedures, was derived from this power function. Performance on concurrent schedules was synthesized from the performances on the separate pairs. Both the generalized matching law and the local model fitted the synthesized concurrent performances. These results are consistent with the view that the contingencies in the alternative, the probability of stay and switch reinforcement, are responsible for performance consistent with the generalized matching law. These results are compatible with momentary maximizing and molar maximizing accounts of concurrent performance. Models of concurrent performance that posit comparisons among the alternatives are not easily applied to these results.     

Momentary maximizing in concurrent schedules with a minimum interchangeover interval

Eight pigeons were trained on concurrent variable-interval variable-interval schedules with a minimum interchangeover time programmed as a consequence of changeovers. In Experiment 1 the reinforcement schedules remained constant while the minimum interchangeover time varied from 0 to 200 s. Relative response rates and relative time deviated from relative reinforcement rates toward indifference with long minimum interchangeover times. In Experiment 2 different reinforcement ratios were scheduled in successive experimental conditions with the minimum interchangeover time constant at 0, 2, 10, or 120 s. The exponent of the generalized matching equation was close to 1.0 when the minimum interchangeover time was 0 s (the typical procedure for concurrent schedules without a changeover delay) and decreased as that duration was increased. The data support the momentary maximizing theory and contradict molar maximizing theories and the melioration theory.     

Some effects of relative reinforcement rate and changeover delay in response-independent concurrent schedules of reinforcement

Reinforcements were arranged independently of the pigeon's behavior by concurrent variable-interval schedules. The reinforcements arranged by one of the schedules occurred when the chamber was illuminated with amber light, and the reinforcements arranged by the other schedule occurred when the chamber was illuminated with blue light. Both schedules functioned concurrently, but reinforcers were delivered by each only in the presence of the appropriate stimulus condition. A response on a white key, the only key in the chamber, alternated the stimulus condition and the effective schedule. The results of this procedure were similar to those obtained with concurrent response-dependent variable-interval schedules of reinforcement. The proportion of the total session time spent in the presence of a schedule component approximated the proportion of the total number of reinforcements in the component. Changeover rate was a decreasing function of the changeover delay and of the difference between the relative rates of reinforcement for each pair of concurrent schedules.     

Noncontingent reinforcement, alternative reinforcement, and the matching law: a laboratory demonstration

Basic researchers, but not most applied researchers, have assumed that the behavior-decelerating effects of noncontingent reinforcement result at least partly from adventitious reinforcement of competing behaviors. The literature contains only sketchy evidence of these effects because few noncontingent reinforcement studies measure alternative behaviors. A laboratory model is presented in which concurrent schedules of contingent reinforcement were used to establish a "target" and an "alternative" behavior. Imposing noncontingent reinforcement decreased target behavior rates and increased alternative behavior rates, outcomes that were well described by the standard quantitative account of alternative reinforcement, the generalized matching law. These results suggest that adventitious reinforcement of alternative behaviors can occur during noncontingent reinforcement interventions, although the range of conditions under which this occurs remains to be determined in future studies. As an adjunct to applied studies, laboratory models permit easy measurement of alternative behaviors and parametric manipulations needed to answer many research questions.     

Temporal constraint on choice: Sensitivity and bias in multiple schedules

In multiple schedules of reinforcement, ratios of responses in successive components are relatively insensitive to ratios of obtained reinforcers. An analysis is proposed that attributes changes in absolute response rates to concurrent interactions between programmed reinforcement and extraneous reinforcement in other components. The analysis predicts that ratios of responses in successive components vary with reinforcer ratios, qualified by a term describing the reinforcement context, that is, programmed and extraneous reinforcers. Two main predictions from the analysis were confirmed in an experiment in which pigeons' responses were reinforced in the components of a multiple schedule and analog extraneous reinforcement was scheduled for an alternative response in each component. Sensitivity of response and time ratios to reinforcer ratios in the multiple schedules varied as a function of the rate of extraneous reinforcers. Bias towards responding in one component of the multiple schedule varied as an inverse function of the ratios of extraneous reinforcer rate in the two components. The data from this and previous studies of multiple-concurrent performance were accurately predicted by our analysis and supported our contention that the allocation of behavior in multiple-schedule components depends on the relative values of concurrently-available reinforcers within each component.     

Short-term and long-term effects of reinforcers on choice

The relation between molar and molecular aspects of time allocation was studied in pigeons on concurrent variable-time variable-time schedules of reinforcement. Fifteen-minute reinforcer-free periods were inserted in the middle of every third session. Generalized molar matching of time ratios to reinforcer ratios was observed during concurrent reinforcement. Contrary to melioration theory, preference was unchanged during the reinforcer-free periods as well as in extinction. In addition to this long-term effect of reinforcement, short-term effects were observed: Reinforcers increased the duration of the stays during which they were delivered but had little consistent effect either on the immediately following stay in the same schedule or on the immediately following stay in the alternative schedule. Thus, an orderly effect of reinforcer delivery on molecular aspects of time allocation was observed, but because of its short-term nature, this effect cannot account for the matching observed at the molar level.     

Drug discrimination under concurrent variable-ratio variable-ratio schedules

Pigeons were trained to discriminate 5 mg/kg pentobarbital from saline under concurrent variable-ratio (VR) VR schedules, in which responses on the pentobarbital-biased lever were reinforced under the VR schedule with the smaller response requirements when pentobarbital was given before the session, and responses on the saline-biased key were reinforced under the VR schedule with the larger response requirements. When saline was administered before the session, the reinforcement contingencies associated with the two response keys were reversed. When responding stabilized under concurrent VR 20 VR 30, concurrent VR 10 VR 40, or concurrent VR 5 VR 50 schedules, pigeons responded almost exclusively on the key on which fewer responses were required to produce the reinforcer. When other doses of pentobarbital and other drugs were substituted for the training dose, low doses of all drugs produced responding on the saline-biased key. Higher doses of pentobarbital and chlordiazepoxide produced responding only on the pentobarbital-biased key, whereas higher doses of ethanol and phencyclidine produced responding only on this key less often. d-Amphetamine produced responding primarily on the saline-biased key. When drugs generalized to pentobarbital, the shape of the generalization curve under concurrent VR VR schedules was more often graded than quantal in shape. Thus, drug discrimination can be established under concurrent VR VR schedules, but the shapes of drug-discrimination dose-response curves under concurrent VR VR schedules more closely resemble those seen under interval schedules than those seen under fixed-ratio schedules. Graded dose-response curves under concurrent VR VR schedules may relate to probability matching and difficulty in discriminating differences in reinforcement frequency.