Temporal control in a complex environment: An analysis of schedule-related behavior

Three experiments conducted in an automated ten-compartment chamber recorded collateral activities of rats reinforced for lever pressing on differential-reinforcement-of-low-rate schedules. In Experiment 1, the rate of lever pressing increased when stimulus support for collateral activities was removed, thus confirming earlier findings. However, there were no temporal or sequential patterns of collateral activities that predicted operant responding. In Experiment 2, the rate of lever pressing increased only if (a) access to all stimulus support for collateral activities was simultaneously prevented, and (b) the rat was forced to remain in the presence of the lever and food tray. The availability of any of the stimuli related to collateral activity was sufficient to keep lever-pressing rates from increasing. Experiment 3 examined collateral activities under a signaled differential-reinforcement-of-low-rate schedule. Preventing access to stimuli supporting collateral activities had little effect on stable lever pressing when the signal was maintained. When the signal was removed, collateral activities continued, but lever-pressing rates increased in three of the four rats and rates of food presentation declined in all rats. Hypotheses that collateral activities have (a) a timekeeping or discriminative function, or (b) directly inhibit operant responding were not supported. The results suggest that collateral activities may facilitate operant responding by simply removing the subject from the presence of reinforcement-related stimuli.     

Relative reinforcer magnitude under a nonindependent concurrent schedule of cocaine reinforcement in rhesus monkeys.

Lever pressing by three rhesus monkeys was maintained under a two-lever concurrent schedule of cocaine reinforcement. Responding on one lever (constant-dose lever) produced a constant dose of 0.05 or 0.1 mg/kg/injection arranged according to a variable-interval 1-min schedule. Responding on the other lever (variable-dose lever) produced a comparison dose of cocaine (0.013 to 0.8 mg/kg/injection), also under a variable-interval 1-min schedule. The two variable-interval schedules were made nonindependent by arranging that the assignment of a reinforcer by one schedule inactivated the second schedule until the assigned reinforcer had been obtained. This modification ensured that the two cocaine doses were obtained with approximately equal frequency, regardless of the distribution of the subject's responding. Preference, indicated by relative response frequency on the variable-dose lever, was almost always for the larger of the doses and was a monotonic function of the comparison dose, except at the highest doses. Preferences at the highest comparison doses may have resulted from the low overall response rates exhibited at these doses. Relative response frequencies on the variable-dose lever roughly matched relative reinforcer magnitude (mg/kg/injection available on the variable-dose lever divided by the sum of mg/kg/injections available on each lever).     

Concurrent performances: reinforcement by different doses of intravenous cocaine in rhesus monkeys

Different doses of intravenous cocaine reinforced the lever pressing of rhesus monkeys under two-lever concurrent or concurrent-chain schedules. Under the concurrent procedure, responding produced drug reinforcers arranged according to independent variable-interval 1-min schedules. Under the concurrent-chain procedure, responding in the variable-interval link led to one of two mutually exclusive, equal-valued, fixed-ratio links; completion of the ratio produced a drug reinforcer. Under both procedures, responding on one lever produced a constant dose of 0.05 or 0.1 mg/kg/injection, while on the other lever, dose was systematically varied within a range of 0.013 to 0.8 mg/kg/injection. Preference, indicated by relative response frequency on the variable-dose lever during the variable-interval link, was always for the larger of the doses. Relative response frequencies on the variable-dose lever roughly matched relative drug intake (mg/kg of drug obtained on variable lever divided by mg/kg of drug obtained on both levers). For many dose comparisons, responding occurred and reinforcers were obtained almost exclusively on the preferred lever. Overall variable-interval rates generally were lower than with other reinforcers, and these low rates, under the experimental conditions, may have occasioned the exclusive preferences.     

Variable-interval schedules of timeout from avoidance: effects of chlordiazepoxide, CGS 8216, morphine, and naltrexone.

Rats were trained on concurrent schedules in which pressing one lever postponed shock and pressing the other occasionally (variable-interval schedule) produced a 2-min timeout during which the shock-postponement schedule was suspended and its correlated stimuli were removed. These procedures provided a baseline for studying the effects of drugs on behavior maintained by different sources of negative reinforcement (shock avoidance and timeout from avoidance). Experiment 1 studied a benzodiazepine agonist, chlordiazepoxide, and antagonist, CGS 8216. Chlordiazepoxide (2.5-30 mg/kg) had little effect on avoidance responding except at higher doses, when it reduced responding. By comparison, responding on the timeout lever was increased in 5 of 6 rats. These effects were reversed by CGS 8216 (2.5-5 mg/kg) in the 2 rats tested, but CGS 8216 had no effect by itself. Experiment 2 studied an opiate agonist, morphine, and antagonist, naltrexone, with 3 rats. Morphine's (2.5-20 mg/kg) effects were opposite those of chlordiazepoxide: At doses that either increased or had no effect on avoidance responding, morphine depressed timeout responding. Naltrexone (5 mg/kg) reversed these actions but had no effect by itself.     

Discrimination of response-contingent and response-independent shock by rats: effects of chlordiazepoxide HCL and sodium amylobarbitone

Rats, trained to press a lever for sucrose reward on a random interval (RI) schedule, were presented while lever-pressing with two stimuli, each associated with a different schedule of shock delivery: in the presence of one stimulus (S_e), shock occurred on an RI schedule irrespective of the rat's behaviour; in the presence of the other (S_p) shocks were programmed by the same schedule but delivered only when the rat pressed the lever. Both stimuli suppressed lever-pressing. In addition, the rats developed significantly different response rates in the two stimuli, thus demonstrating a discrimination between response-contingent and response-independent shock. Group data showed faster responding in S_e than in S_p, supporting the view that response-contingent shock produces greater suppression than response-independent shock. Individual animal analyses, however, demonstrated that this was the case in the majority of animals, but not in all. Response suppression was alleviated by amylobarbitone sodium (15 mg/kg) or chlordiazepoxide HCI (5 mg/kg); the latter drug alleviated suppression significantly more in S_p than S_e and eliminated the difference between the response rates controlled by the two stimuli.     

Effects of d-amphetamine, chlorpromazine, and chlordiazepoxide on intercurrent behavior during spaced-responding schedules.

Effects of d-amphetamine, chlorpromazine, and chlordiazepoxide on lever pressing under direct control of spaced-responding schedules were compared with effects on intercurrent drinking and wheel running in the rat. Drug effects on lever pressing were systematically related to dose and were consistent for all animals; drug effects on intercurrent behavior were generally different for each animal. In the case of lever presses, increasing doses of d-amphetamine first increased and then decreased response rate, increasing doses of chlorpromazine produced graded decreases in response rate, and doses of chlordiazepoxide up to 40 mg/kg produced no effect on response rate. These data are discussed in context with the concept of schedule control, and it is suggested that the behavioral pharmacology of intercurrent behavior be explored as a useful procedure in the experimental analysis of intercurrent behavior.     

Effects of cocaine on fixed-interval responding reinforced by the opportunity to run

Rate-dependent drug effects have been observed for operant responding maintained by food, water, heat, light onset, electrical brain stimulation, shock-stimulus termination, and shock presentation. The present study sought to determine if the effects of cocaine on lever pressing maintained by the opportunity to run could also be described as rate dependent. Seven male Wistar rats were trained to respond on levers for the opportunity to run in a wheel. The schedule of reinforcement was fixed-interval 60 s, and the reinforcing consequence was the opportunity to run for 60 s. On this schedule, overall rates of responding were low, usually below six presses per minute, and pauses frequently exceeded the 60-s interval. Despite these differences, an overall scalloped pattern of lever pressing was evident for each rat. Doses of 1, 2, 4, 8, and 16 mg/kg cocaine were administered 10 min prior to a session. Only at the 16 mg/kg dose did the responding of the majority of rats change in a manner suggestive of a rate-dependent drug effect. Specifically, lower response rates at the beginning of the intervals increased and higher rates at the end of the intervals decreased, as indicated by the fact that slopes from the regression of drug rates on control rates decreased. These data provide tentative support for the generalization of rate-dependent effects to operant responding maintained by wheel running. Differences in the baseline performance maintained by wheel running compared to those for food and water point to the need for further experimentation before this effect can be firmly established.     

Running and responding reinforced by the opportunity to run: effect of reinforcer duration.

The present study investigated the effect of reinforcer duration on running and on responding reinforced by the opportunity to run. Eleven male Wistar rats responded on levers for the opportunity to run in a running wheel. Opportunities to run were programmed to occur on a tandem fixed-ratio 1 variable-interval 30-s reinforcement schedule. Reinforcer duration varied across conditions from 30 to 120 s. As reinforcer duration increased, the rates of running and lever pressing declined, and latency to lever press increased. The increase in latency to respond was consistent with findings that unconditioned inhibitory aftereffects of reinforcement increase with reinforcer magnitude. The decrease in local lever-pressing rates, however, was inconsistent with the view that response strength increases with the duration of the reinforcer. Response rate varied inversely, not directly, with reinforcer duration. Furthermore, within-session data challenge satiation, fatigue, and response deprivation as determinants of the observed changes in running and responding. In sum, the results point to the need for further research with nonappetitive forms of reinforcement.     

Time of supplemental feeding alters the effects of cocaine on lever pressing of rats

The present experiment assessed the effects of cocaine on the lever pressing of 4 rats maintained during 15-min sessions by a fixed-ratio 50 schedule of food reinforcement. Across phases, supplemental food was provided either immediately or 2 hr after sessions. Two rats began the experiment in the delayed-feeding condition, and 2 began the experiment in the immediate-feeding condition. Rates of lever pressing of 2 rats sometimes decreased to low levels near the ends of sessions when supplemental feeding was provided immediately, but were consistently high throughout sessions when supplemental feeding was delayed. Cocaine (1.0 to 17.0 or 30.0 mg/kg) was administered intraperitoneally 15 min prior to test sessions. In most cases, cocaine suppressed response rates at lower doses under immediate-feeding conditions. Decreases in overall response rates were correlated with dosedependent increases in the time rats spent not responding. It is suggested that delaying the time of postsession feeding increased response strength, as indicated by greater resistance to the rate-suppressive effects of cocaine.     

Within-Session Changes in Adjunctive and Instrumental Responding

Rats pressed levers for food delivered by several fixed interval schedules. A drinking spout or running wheel was also available during some conditions, but not during others. The rate of lever pressing, drinking and running often changed within experimental sessions. The within-session patterns of lever pressing did not differ when drinking or running was available and when it was not. The correlation between the amount of lever pressing and the amount of drinking or running at a particular time in the session was inconsistently positive or negative. Finding within-session changes in responding for adjunctive behaviors implies that the factors that produce these changes are present for both adjunctive and instrumental behavior. Finding inconsistent correlations between instrumental responding and adjunctive behaviors questions arousal and interference from adjunctive behaviors as explanations for within-session changes in instrumental responding.     

Contrast, induction, facilitation, suppression, and conservation

Ten rats received all of their water in daily 1-hr sessions. Following a baseline phase in which lever and water spout were freely available throughout each session, subjects were trained to press the lever for water on mixed schedules composed of two alternating components. Each component gave access to water for a fixed cumulation of drinking time every time the rat cumulated a fixed amount of lever-pressing time. Changes in one component produced contrast and induction effects, both positive and negative, with respect to both lever pressing and drinking in the unchanged component. All schedules facilitated lever pressing relative to baseline. All schedules suppressed drinking relative to baseline, even though contingency sessions allowed ample time to perform the baseline amount of drinking. The entire pattern of results was predicted in quantitative detail by assuming that the total amount of a dimension apportioned to lever pressing and drinking is conserved between baseline and contingency sessions. Conservation theory was shown to predict several effects produced by simple fixed-ratio schedules, and was compared favorably with probability-differential (Premack, 1971) and response-deprivation (Timberlake and Allison, 1974) theory.     

Comparing Locomotion With Lever-press Travel In An Operant Simulation Of Foraging

An operant model of foraging was studied. Rats searched for food by pressing on the left lever, the patch, which provided one, two, or eight reinforcers before extinction (i.e., zero reinforcers). Obtaining each reinforcer lowered the probability of receiving another reinforcer, simulating patch depletion. Rats traveled to another patch by pressing the right lever, which restored reinforcer availability to the left lever. Travel requirement changed by varying the probability of reset for presses on the right lever; in one condition, additional locomotion was required. That is, rats ran 260 cm from the left to the right lever, made one response on the right lever, and ran back to a fresh patch on the left lever. Another condition added three hurdles to the 260-cm path. The lever-pressing and simple locomotion conditions generated equivalent travel times. Adding the hurdles produced longer times in patches than did the lever-pressing and simple locomotion requirements. The results contradict some models of optimal foraging but are in keeping with McNair's (1982) optimal giving-up time model and add to the growing body of evidence that different environments may produce different foraging strategies.     

Techniques for establishing schedules with wheel running as reinforcement in rats.

In three experiments, access to wheel running was contingent on lever pressing. In each experiment, the duration of access to running was reduced gradually to 4, 5, or 6 s, and the schedule parameters were expanded gradually. The sessions lasted 2 hr. In Experiment 1, a fixed-ratio 20 schedule controlled a typical break-and-run pattern of lever pressing that was maintained throughout the session for 3 rats. In Experiment 2, a fixed-interval schedule of 6 min maintained lever pressing throughout the session for 3 rats, and for 1 rat, the rate of lever pressing was positively accelerated between reinforcements. In Experiment 3, a variable-ratio schedule of 20 or 35 was in effect and maintained lever pressing at a very stable pace throughout the session for 2 of 3 rats; for 1 rat, lever pressing was maintained at an irregular rate. When the session duration was extended to successive 24-hr periods, with food and water accessible in Experiment 3, lever pressing settled into a periodic pattern occurring at a high rate at approximately the same time each day. In each experiment, the rats that developed the highest local rates of running during wheel access also maintained the most stable and highest rates of lever pressing.     

Intercurrent and reinforced behavior under multiple spaced-responding schedules

Lever pressing in rats was reinforced with food under a multiple spaced-responding schedule. A lever, food cup, and drinking tube were mounted in a running wheel so that lever pressing, running, and licking could be recorded. Running and licking had no scheduled consequences. Lever pressing was reinforced under a multiple schedule with three spaced-responding components and an extinction component. Each component was associated with a different auditory stimulus. Spaced-responding components reinforced only lever presses terminating interresponse times equal to or greater than 10, 20, or 60 sec, respectively. Rates of lever pressing, reinforcement, and licking all decreased as schedule parameter increased. Efficiency of spaced responding, as measured by reinforcements per response, also decreased. Rate of wheel running either increased or increased and then decreased with increasing schedule parameter. Individual running rates differed substantially. Neither licking nor running rate correlated with individual differences in efficiency. Analysis of conditional probabilities among the several response classes showed that, as the schedule requirement increased, the probability of running after a lever press increased and the probability of licking after a lever press decreased. After reinforcement, one subject always pressed the lever next. In the other subjects, the conditional probability of lever pressing, given reinforcement, increased while the probability of licking, given reinforcement, decreased with increasing schedule requirement. Results are discussed in relation to the concepts of schedule-induced and mediating behavior.     

Differential acquisition of lever pressing in inbred and outbred mice: comparison of one-lever and two-lever procedures and correlation with differences in locomotor activity

Recent progress in mouse genetics has led to an increased interest in developing procedures for assessing mouse behavior, but relatively few of the behavioral procedures developed involve positively reinforced operant behavior. When operant methods are used, nose poking, not lever pressing, is the target response. In the current study differential acquisition of milk-reinforced lever pressing was observed in five inbred strains (C57BL/6J, DBA/2J, 129X1/SvJ, C3H/HeJ, and BALB/cJ) and one outbred stock (CD-1) of mice. Regardless of whether one or two levers (an "operative" and "inoperative" lever) were in the operant chamber, a concomitant variable-time fixed-ratio schedule of milk reinforcement established lever pressing in the majority of mice within two 120-min sessions. Substantial differences in lever pressing were observed across mice and between procedures. Adding an inoperative lever retarded acquisition in C57BL/6J, DBA/2J, 129X1/SvJ, and C3H/HeJ mice, but not in CD-1 and BALB/cJ mice. Locomotor activity was positively correlated with number of lever presses in both procedures. Analyses of durations of the subcomponents (e.g., time to move from hopper to lever) of operant behavior revealed further differences among the six types of mice. Together, the data suggest that appetitively reinforced lever pressing can be acquired rapidly in mice and that a combination of procedural, behavioral, and genetic variables contributes to this acquisition.     

Effects of D-amphetamine on response acquisition with immediate and delayed reinforcement.

The present study examined in 8-hour sessions the effects of d-amphetamine (1.0, 5.6, and 10 mg/kg) on the acquisition of lever-press responding in rats that were exposed to procedures in which water delivery was delayed by 0, 8, or 16 seconds relative to the response that produced it. Both nonresetting- and resetting-delay conditions were studied. Although neither shaping nor autoshaping occurred, substantial levels of operative-lever responding developed under all conditions in which responses produced water. The lowest dose (1.0 mg/kg) of d-amphetamine either had no effect on or increased operative-lever pressing, whereas higher doses typically produced an initial reduction in lever pressing. Nonetheless, overall rates of operative-lever pressing at these doses were as high as, or higher than, those observed with vehicle. Thus, response acquisition was observed under all reinforcement procedures at all drug doses. In the absence of the drug, most responding occurred on the operative lever when reinforcement was immediate. Such differential responding also developed under both nonresetting- and resetting-delay procedures when the delay was 8 seconds, but not when it was 16 seconds. d-Amphetamine did not affect the development of differential responding under any procedure. Thus, consistent with d-amphetamine's effects under repeated acquisition procedures, the drug had no detrimental effect on learning until doses that produced general behavioral disruption were administered.     

Hippocampal synaptic plasticity during instrumental learning

present research examined the role of hippocampal NMDA-dependent synaptic potentiation on appetitive instrumental conditioning under a continuous reinforcement schedule. In the first experiment, low (.025 mg.kg) or moderate (.05 mg/kg) dosages of the NMDA receptor antagonist, MK801, failed to increase the number of training days required to reach acquisition criterion; number of training days required to reach criterion for extinction were also unaffected. In the second experiment, a higher dosage (.10 mg/kg) of MK801 or induction of long-term potentiation failed to alter the number of responses occurring during acquisition. These data suggest that hippocampal synaptic potentiation does not play a prominent role in instrumental learning with simple contingency conditions. It is suggested that hippocampal LTP reflects a perceptual process that contributes differentially to spatial cognition, classical and instrumental conditioning.     

Hippocampal synaptic plasticity during instrumental learning

present research examined the role of hippocampal NMDA-dependent synaptic potentiation on appetitive instrumental conditioning under a continuous reinforcement schedule. In the first experiment, low (.025 mg.kg) or moderate (.05 mg/kg) dosages of the NMDA receptor antagonist, MK801, failed to increase the number of training days required to reach acquisition criterion; number of training days required to reach criterion for extinction were also unaffected. In the second experiment, a higher dosage (.10 mg/kg) of MK801 or induction of long-term potentiation failed to alter the number of responses occurring during acquisition. These data suggest that hippocampal synaptic potentiation does not play a prominent role in instrumental learning with simple contingency conditions. It is suggested that hippocampal LTP reflects a perceptual process that contributes differentially to spatial cognition, classical and instrumental conditioning.     

Repeated cocaine experience facilitates sucrose-reinforced operant responding in enriched and isolated rats

The purpose of the present experiment was to determine whether repeated cocaine exposure differentially affects sucrose-reinforced operant responding in rats raised in an enriched condition (EC) or an isolated condition (IC). Specifically, the performance of EC and IC rats pressing a lever for sucrose under a high fixed-ratio schedule (FR 30) prior to and after 10 days of exposure to cocaine (15 mg/kg, i.p.) or saline was compared. Regardless of rearing condition, rats repeatedly exposed to cocaine had shorter reacquisition latencies to complete a sucrose-reinforced FR 30 task than saline controls. The results suggest that cocaine exposure may have cross-sensitized both EC and IC rats to the reinforcing effects of sucrose or sucrose-associated cues, thus facilitating reacquisition of operant responding.     

Conditioned suppression and conditioned enhancement with the same positive UCS: an effect of CS duration

Previous experiments have shown that positively reinforced operant responding is suppressed during a conditioned stimulus terminated with an electric shock (conditioned suppression). In the present experiment, the conditioned stimulus was terminated with a positive unconditioned stimulus, and it was found that the duration of the conditioned stimulus was a key factor in determining whether response suppression or response enhancement was observed during the stimulus. The lever-pressing responses of rats were maintained by a variable-interval schedule of food reinforcement. While the rats were pressing the lever, a light was occasionally turned on, its offset coincident with a brief period of access to a sucrose solution. In consecutive blocks of sessions, the light duration was 40 sec, 12 sec, or 120 sec. Results showed that the rate of lever pressing was substantially suppressed during the 12-sec stimulus, slightly suppressed during the 40-sec stimulus, and enhanced during the 120-sec stimulus.