Relative reinforcer magnitude under a nonindependent concurrent schedule of cocaine reinforcement in rhesus monkeys.

Lever pressing by three rhesus monkeys was maintained under a two-lever concurrent schedule of cocaine reinforcement. Responding on one lever (constant-dose lever) produced a constant dose of 0.05 or 0.1 mg/kg/injection arranged according to a variable-interval 1-min schedule. Responding on the other lever (variable-dose lever) produced a comparison dose of cocaine (0.013 to 0.8 mg/kg/injection), also under a variable-interval 1-min schedule. The two variable-interval schedules were made nonindependent by arranging that the assignment of a reinforcer by one schedule inactivated the second schedule until the assigned reinforcer had been obtained. This modification ensured that the two cocaine doses were obtained with approximately equal frequency, regardless of the distribution of the subject's responding. Preference, indicated by relative response frequency on the variable-dose lever, was almost always for the larger of the doses and was a monotonic function of the comparison dose, except at the highest doses. Preferences at the highest comparison doses may have resulted from the low overall response rates exhibited at these doses. Relative response frequencies on the variable-dose lever roughly matched relative reinforcer magnitude (mg/kg/injection available on the variable-dose lever divided by the sum of mg/kg/injections available on each lever).     

Variable-ratio schedules of timeout from avoidance: effects of d-amphetamine and morphine.

Rats were trained on concurrent schedules in which pressing one lever postponed shock and pressing the other lever produced periods of signaled timeout from avoidance on variable-ratio schedules. These procedures generated high rates of timeout-reinforced responding and provided a baseline for studying the effects of drugs on behavior maintained by different types of negative reinforcement (shock postponement vs. timeout). Morphine (2.5 to 10.0 mg/kg) reduced behavior maintained by timeout at doses that increased or had no effect on avoidance responding. In contrast, d-amphetamine (0.125 to 2.0 mg/kg) produced large increases in timeout responding at doses that had minimal effect on avoidance in rats trained on variable-interval and variable-ratio schedules. Thus, the event-dependent effects of morphine, observed in previous studies in which timeout responding was maintained at low rates by interval schedules, were replicated with high timeout rates maintained by variable-ratio schedules. The effects of d-amphetamine could also be described as "event dependent" because timeout responding was stimulated more than avoidance regardless of the maintenance schedule or baseline rate.     

Variable-interval schedules of timeout from avoidance: effects of chlordiazepoxide, CGS 8216, morphine, and naltrexone.

Rats were trained on concurrent schedules in which pressing one lever postponed shock and pressing the other occasionally (variable-interval schedule) produced a 2-min timeout during which the shock-postponement schedule was suspended and its correlated stimuli were removed. These procedures provided a baseline for studying the effects of drugs on behavior maintained by different sources of negative reinforcement (shock avoidance and timeout from avoidance). Experiment 1 studied a benzodiazepine agonist, chlordiazepoxide, and antagonist, CGS 8216. Chlordiazepoxide (2.5-30 mg/kg) had little effect on avoidance responding except at higher doses, when it reduced responding. By comparison, responding on the timeout lever was increased in 5 of 6 rats. These effects were reversed by CGS 8216 (2.5-5 mg/kg) in the 2 rats tested, but CGS 8216 had no effect by itself. Experiment 2 studied an opiate agonist, morphine, and antagonist, naltrexone, with 3 rats. Morphine's (2.5-20 mg/kg) effects were opposite those of chlordiazepoxide: At doses that either increased or had no effect on avoidance responding, morphine depressed timeout responding. Naltrexone (5 mg/kg) reversed these actions but had no effect by itself.     

Drug discrimination under two concurrent fixed-interval fixed-interval schedules

Pigeons were trained to discriminate 5.0 mg/kg pentobarbital from saline under a two-key concurrent fixed-interval (FI) 100-s FI 200-s schedule of food presentation, and later tinder a concurrent FI 40-s FI 80-s schedule, in which the FI component with the shorter time requirement reinforced responding on one key after drug administration (pentobarbital-biased key) and on the other key after saline administration (saline-biased key). After responding stabilized under the concurrent FI 100-s FI 200-s schedule, pigeons earned an average of 66% (after pentobarbital) to 68% (after saline) of their reinforcers for responding under the FI 100-s component of the concurrent schedule. These birds made an average of 70% of their responses on both the pentobarbital-biased key after the training dose of pentobarbital and the saline-biased key after saline. After responding stabilized under the concurrent FI 40-s FI 80-s schedule, pigeons earned an average of 67% of their reinforcers for responding under the FI 40 component after both saline and the training dose of pentobarbital. These birds made an average of 75% of their responses on the pentobarbital-biased key after the training dose of pentobarbital, but only 55% of their responses on the saline-biased key after saline. In test sessions preceded by doses of pentobarbital, chlordiazepoxide, ethanol, phencyclidine, or methamphetamine, the dose-response curves were similar under these two concurrent schedules. Pentobarbital, chlordiazepoxide, and ethanol produced dose-dependent increases in responding on the pentobarbital-biased key as the doses increased. For some birds, at the highest doses of these drugs, the dose-response curve turned over. Increasing doses of phencyclidine produced increased responding on the pentobarbital-biased key in some, but not all, birds. After methamphetamine, responding was largely confined to the saline-biased key. These data show that pigeons can perform drug discriminations under concurrent schedules in which the reinforcement frequency under the schedule components differs only by a factor of two, and that when other drugs are substituted for the training drugs they produce dose-response curves similar to the curves produced by these drugs under other concurrent interval schedules.     

Drug discrimination under concurrent variable-ratio variable-ratio schedules

Pigeons were trained to discriminate 5 mg/kg pentobarbital from saline under concurrent variable-ratio (VR) VR schedules, in which responses on the pentobarbital-biased lever were reinforced under the VR schedule with the smaller response requirements when pentobarbital was given before the session, and responses on the saline-biased key were reinforced under the VR schedule with the larger response requirements. When saline was administered before the session, the reinforcement contingencies associated with the two response keys were reversed. When responding stabilized under concurrent VR 20 VR 30, concurrent VR 10 VR 40, or concurrent VR 5 VR 50 schedules, pigeons responded almost exclusively on the key on which fewer responses were required to produce the reinforcer. When other doses of pentobarbital and other drugs were substituted for the training dose, low doses of all drugs produced responding on the saline-biased key. Higher doses of pentobarbital and chlordiazepoxide produced responding only on the pentobarbital-biased key, whereas higher doses of ethanol and phencyclidine produced responding only on this key less often. d-Amphetamine produced responding primarily on the saline-biased key. When drugs generalized to pentobarbital, the shape of the generalization curve under concurrent VR VR schedules was more often graded than quantal in shape. Thus, drug discrimination can be established under concurrent VR VR schedules, but the shapes of drug-discrimination dose-response curves under concurrent VR VR schedules more closely resemble those seen under interval schedules than those seen under fixed-ratio schedules. Graded dose-response curves under concurrent VR VR schedules may relate to probability matching and difficulty in discriminating differences in reinforcement frequency.     

Matching under nonindependent variable-ratio schedules of drug reinforcement.

Response-contingent deliveries of oral pentobarbital maintained responding of 3 rhesus monkeys during daily 3-hr sessions. Deliveries of pentobarbital were arranged under nonindependent concurrent variable-ratio variable-ratio schedules. Responses to either schedule counted toward completion of both variable-ratio schedule requirements. This schedule is similar in some respects to conventional concurrent variable-interval variable-interval schedules, in which passage of time counts toward completion of the interval value on both schedules. Restricted nonindependent concurrent variable-ratio variable-ratio schedules were also studied. On that schedule, when a drug delivery was assigned to one spout, it had to be collected before responses on the opposite spout again counted toward completion of the schedule requirements. Relative reinforcer magnitude was varied by changing the drug concentration on one schedule while keeping the drug concentration constant on the other variable-ratio schedule. Under both types of concurrent variable-ratio schedules, the relative rate of responding corresponded to the relative drug intake. Unlike earlier studies of concurrent variable-interval variable-interval intravenous cocaine reinforcement, preference was proportionate to concentration, and exclusive preferences did not develop. The relationship between relative rate of responding and relative drug intake was well described by the generalized matching law.     

Preference for multiple versus mixed schedules of reinforcement

Five pigeons were trained in a concurrent-chain procedure. In the initial links, equal nonindependent variable-interval schedules were available concurrently on two keys. Completing the schedule on either key led to exclusive presentation of one of two further variable-interval schedules for a fixed period of time. During these terminal links, as many reinforcers as were scheduled could be obtained. If the response producing this terminal link occurred on one key, differential stimuli signaled which variable-interval schedule had been produced. If the response producing the terminal link occurred on the other key, no such differential stimuli were available. Once the fixed period of time elapsed, the initial links were reinstated. In Experiment 1, the period of time for which the terminal links were available was always 10 s and the absolute duration of the initial links was varied. Subjects preferred the alternative leading to the multiple schedule when the initial-link duration was short, but preferred the alternative leading to the mixed schedule when the initial-link durations were longer. In Experiment 2, both the initial-link duration and the duration of the terminal links were varied. The effect of initial-link duration was identical to that in Experiment 1 and there was no systematic effect of varying the terminal-link duration.     

Matching and contrast on several concurrent treadle-press schedules

Four White King pigeons pressed treadles for food reinforcement on several concurrent variable-interval variable-interval schedules. The rate of reinforcement available for responding in one of the two component schedules was held constant at 30 reinforcers per hour. The rate of reinforcement available for responding in the other was varied from 120 to 60 to 15, and then to 30 reinforcers per hour. The relative rate of responding in each component schedule equalled the relative rate of reinforcement that the component provided. And, behavioral contrast, defined as an inverse relationship between the rate of responding in the constant component and the rate of reinforcement obtained by responding in the other component, occurred for all schedules.     

Within-session Response Patterns On Conjoint Variable-interval Variable-time Schedules

Operant responding often changes within sessions, even when factors such as rate of reinforcement remain constant. The present study was designed to determine whether within-session response patterns are determined by the total number of reinforcers delivered during the session or only by the reinforcers earned by the operant response. Four rats pressed a lever and 3 pigeons pecked a key for food reinforcers delivered by a conjoint variable-interval variable-time schedule. The overall rate of reinforcement of the conjoint schedule varied across conditions from 15 to 480 reinforcers per hour. When fewer than 120 reinforcers were delivered per hour, the within-session patterns of responding on conjoint schedules were similar to those previously observed when subjects received the same total number of reinforcers by responding on simple variable-interval schedules. Response patterns were less similar to those observed on simple variable-interval schedules when the overall rate of reinforcement exceeded 120 reinforcers per hour. These results suggest that response-independent reinforcers can affect the within-session pattern of responding on a response-dependent schedule. The results are incompatible with a response-based explanation of within-session changes in responding (e.g., fatigue), but are consistent with both reinforcer-based (e.g., satiation) and stimulus-based (e.g., habituation) explanations.     

The effects of cocaine on behavior maintained by timeout from avoidance.

Rats were trained on concurrent schedules under which responses on one lever postponed shock (avoidance) and responses on the other lever produced brief (2-min) periods of signaled timeout from avoidance. For 6 rats, timeout from avoidance was programmed on a variable-interval 45-s schedule that generally resulted in rates that were lower than those on the avoidance lever. For another 6 rats, timeout was arranged on a variable-ratio 15 schedule that produced higher baseline rates. Cocaine (3 to 40 mg/kg) produced large, dose-dependent increases in behavior maintained by timeout in both groups of rats. Avoidance responding was also generally increased by cocaine, but the increases were of lesser magnitude. Increases in response rates were seen across a broad range of doses on behavior maintained by either interval or ratio schedules, an outcome that was unexpected on the basis of most studies of cocaine on food-maintained behavior. These results were similar to those of previous studies of the effects of amphetamine on behavior maintained by timeout from avoidance and suggest that stimulant drugs affect behavior maintained under a shock-postponement schedule differently than they affect behavior maintained by timeout from avoidance.     

Within-session Changes In Responding During Concurrent Variable-interval Schedules

Five rats and 4 pigeons responded for food delivered by several concurrent variable-interval schedules. The sum of the rates of reinforcement programmed for the two components varied from 15 to 480 reinforcers per hour in different conditions. Rates of responding usually changed within the experimental session in a similar manner for the two components of each concurrent schedule. The within-session changes were similar to previously reported changes during simple schedules that provided rates of reinforcement equal to the sum of all reinforcers obtained from the concurrent schedules. The number of changeovers also changed within sessions in a manner similar to the changes in instrumental responding. These results suggest that changeovers are governed by the same variables that govern instrumental responding. They also suggest that the within-session change in responding during each component of a concurrent schedule is determined by approximately the sum of the reinforcers obtained from both components when both components provide the same type of reinforcer.     

Run length, visit duration, and reinforcers per visit in concurrent performance

The contingencies in each alternative of concurrent procedures consist of reinforcement for staying and reinforcement for switching. For the stay contingency, behavior directed at one alternative earns and obtains reinforcers. For the switch contingency, behavior directed at one alternative earns reinforcers but behavior directed at the other alternative obtains them. In Experiment 1, responses on the main lever, in S1, incremented stay and switch schedules and obtained a stay reinforcer when it became available. Responses on the switch lever changed S1 to S2 and obtained switch reinforcers when available. In S2, neither responses on the main lever nor on the switch lever were reinforced, but a switch response changed S2 to S1. Run lengths and visit durations were a function of the ratio of the scheduled probabilities of reinforcement (staying/switching). From run lengths and visit durations, traditional concurrent performance was synthesized, and that synthesized performance was consistent with the generalized matching law. Experiment 2 replicated and extended this analysis to concurrent variable-interval schedules. The synthesized results challenge any theory of matching that requires a comparison among the alternatives.     

Drug discrimination under a concurrent schedule.

Three pigeons were trained to discriminate a 5.0 mg/kg dose of pentobarbital from saline under a two-key concurrent schedule with responding on the key associated with the presession injection, under both stimulus conditions, producing four times as many reinforcers as responding on the other key. This concurrent schedule resulted in approximately 70% responding to the higher reinforcement key under the pentobarbital stimulus and approximately 30% responding to that key under the saline stimulus. During testing, then, the pigeons were able to dose-dependently emit higher (>70%) or lower (<30%) values than were established under the control conditions. Dose-response curves were determined for pentobarbital (twice), methamphetamine, phencyclidine, chlordiazepoxide, and the combination of pentobarbital and the barbiturate antagonist bemegride. The results obtained with pentobarbital and chlordiazepoxide showed that, as the dose increased, pentobarbital-appropriate responding also increased. Methamphetamine produced relatively flat dose--response curves, whereas phencyclidine administration produced inconsistent effects on responding. The combination of the training dose of pentobarbital with increasing doses of bemegride produced a decrease in pentobarbital-appropriate responding. The results also showed that the dose-response curves for pentobarbital and chlordiazepoxide, instead of being all or none, were graded functions of the drug dose.     

Choice and segmented interreinforcement intervals

Pigeons were trained on a two-key concurrent schedule, where food reinforcers on one key were arranged by a simple variable-interval schedule and on the other key by a chain variable-interval variable-interval schedule. When the initial link of the chain was in effect, the pigeons tended to respond more on the simple variable-interval schedule, and hence less on the chain, than would be expected from a comparison of both the local and overall rates of reinforcement of the two schedules. When the terminal link of the chain was in effect, the pigeons responded more on the chain than would be expected from a comparison of the rates of reinforcement of the schedules then in effect. Overall responding on the chain was not proportional to overall reinforcement on the chain but rather was a by-product of responding during initial- and terminal-link phases.     

DOES SENSITIVITY TO MAGNITUDE DEPEND ON THE TEMPORAL DISTRIBUTION OF REINFORCEMENT?

Our research addressed the question of whether sensitivity to relative reinforcer magnitude in concurrent chains depends on the distribution of reinforcer delays when the terminal-link schedules are equal. In Experiment 1, 12 pigeons responded in a two-component procedure. In both components, the initial links were concurrent variable-interval 40-s variable-interval 40-s, and the terminal links were both 20-s interval schedules in which responses were reinforced by either 4-s of grain in one, or 2-s of grain in the other. The only difference between the components was whether the terminal-link schedules were fixed interval or variable intervals. For all subjects, the relative rate of responding in the initial links for the terminal link that produced the 4-s reinforcer was greater when the terminal links were fixed-interval schedules than when they were variable-interval schedules. This result is contrary to the prediction of Grace's (1994) contextual choice model, but is consistent with both Mazur's (2001) hyperbolic value-added model and Killeen's (1985) incentive theory. In Experiment 2, 4 pigeons responded in a concurrent-chains procedure in which 4-s or 2-s reinforcers were provided independently of responding according to equal fixed-time or mixed-time schedules. Preference for the 4-s reinforcer increased as the variability of the intervals comprising the mixed-time schedules was decreased. Generalized-matching sensitivity of initial-link response allocation to relative reinforcer magnitude was proportional to the geometric mean of the terminal-link delays.     

Variable-interval schedules of timeout from avoidance

Rats were trained on concurrent schedules in which pressing one lever postponed shock and pressing the other occasionally produced a 2-min timeout during which the shock-postponement schedule was suspended and its correlated stimuli were removed. Throughout, the shock-postponement schedule maintained proficient levels of avoidance. Nevertheless, in Experiment 1 responding on the timeout lever was established rapidly, was maintained at stable levels on variable-interval schedules, was extinguished by withholding timeout, was reestablished when timeout was reintroduced, and was brought under discriminative control with a multiple variable-interval extinction schedule of timeout. These results are in contrast with Verhave's (1962) conclusion that timeout is an ineffective reinforcer when presented to rats on intermittent schedules. In Experiment 2 the consequence of responding on the timeout lever was altered so that the shock-postponement schedule remained in effect even though the stimulus conditions associated with timeout were produced for 2 min. Responding extinguished, indicating that suspension of the shock-postponement schedule, not stimulus change, was the source of reinforcement. By establishing the reinforcing efficacy of timeout with standard variable-interval schedules, these experiments illustrate a procedure for studying negative reinforcement in the same way as positive reinforcement.     

Molecular analyses of the effects of d-amphetamine on fixed-interval schedule performances of rats.

A series of doses (0.5 to 2.0 mg/kg) of d-amphetamine was administered to rats whose lever pressing was maintained by fixed-interval 30-s, 60-s, or 120-s schedules of reinforcement by sucrose delivery. Under both saline and d-amphetamine conditions, molecular features of responding were reliably described in terms of the distribution of postreinforcement pauses and local response rate following the onset of responding. Postreinforcement pause always varied from interval to interval but, on average, shortened under the drug. Local response rate (response rate exclusive of pause time) tended to decrease under the drug, and where acceleration occurred within runs of responses, it was reduced by the drug. All of these effects were dose-related. These findings suggest that fixed-interval behavior can be analyzed effectively at a molecular level, and that the effects of d-amphetamine are best described as disruption of temporal discrimination.     

Preference for unsegmented interreinforcement intervals in concurrent chains

Five pigeons were trained under concurrent-chain schedules in which a pair of independent, concurrent variable-interval 60-s schedules were presented in the initial link and either both variable-interval or both fixed-interval schedules were presented in the terminal link. Except for the baseline, one of the terminal-link schedules was always a two-component chained schedule and the other was either a simple or a tandem schedule of equal mean interreinforcement interval. The values of the fixed-interval schedules were either 15 s or 60 s; that of the variable-interval schedules was always 60 s. A 1.5-s changeover delay operated during the initial link in some conditions. The pigeons preferred a simple or a tandem schedule to a chain. For the fixed-interval schedules, this preference was greater when the fixed interval was 60 s than when it was 15 s. For the variable-interval schedules, the preferences were less pronounced and occurred only when the changeover delay was in effect. For a given type of schedule and interreinforcement interval, similar preferences were obtained whether the nonchained schedule was a tandem or simple schedule. The changeover delay generally inflated preference and lowered the changeover rate, especially when the terminal-link schedules were either short (15 s) or aperiodic (variable-interval). The results were consistent with the notion that segmenting the interreinforcement interval of a schedule into a chain lowers the preference for it.     

A choice procedure to assess the aversive effects of drugs in rodents

The goal of this series of experiments was to develop an operant choice procedure to examine rapidly the punishing effects of intravenous drugs in rats. First, the cardiovascular effects of experimenter‐administered intravenous histamine, a known aversive drug, were assessed to determine a biologically active dose range. Next, rats responded on each of two levers with concurrently available fixed‐ratio 1 schedules of food reinforcement. Intravenous histamine was delivered along with food when responses were made on one of the options, and the lever on which both food and histamine were contingent was switched on a regular basis. A dose of 1.0 mg/kg/inj of histamine was effective in moving responding to the alternate lever, whereas saline, 0.1, or 0.3 mg/kg/inj of histamine were not. Histamine injections produced reliable selection of the alternate lever when they were presented on the same lever for three consecutive sessions, but not when they were switched between levers on each session. In addition, histamine produced greater selection of the alternate lever when it was presented with shorter intertrial interval durations. These findings indicate that, with appropriate parameters, the aversive effects of histamine and perhaps other drugs can be established rapidly using a concurrent choice procedure.     

On the measurement of reinforcement frequency in the study of preference

In a two-link, concurrent-chain schedule, pigeons' pecks on each key during the initial link occasionally produced a terminal link, during which only that key was operative. Responses in the terminal link were reinforced with food on either fixed-interval or variable-interval schedules. In one experiment, relative amount of responding in the initial link equaled the relative harmonic rate of reinforcement in the terminal links. In a second experiment, the selection of interreinforcement intervals in variable-interval schedules in the terminal links was such that rates of reinforcement based on the harmonic or on the arithmetic means of the interreinforcement intervals predicted opposite preferences in the initial links. The observed preference was consistent with that predicted by the harmonic rather than by the arithmetic rates of reinforcement.