Concurrent responses during multiple schedules in rats

Abstract.— Rats' lever pressing was reinforced with a sucrose solution. In the presence of a tone pressing one lever was reinforced on a FR 40 schedule, in the absence of the tone pressing another lever was reinforced on a FR 20 schedule. The components alternated every fourth minute. In both components, the duration of pauses in lever pressing after reinforcement was positively correlated with the duration of licking the empty sucrose-delivery mechanism. In the FR 40 component, the duration of pauses was also positively correlated with the number of presses on the non-reforced lever. Licking and non-reinforced lever pressing were decreased when the rate of reinforced lever pressing was high. The response interactions were similar to those described for concurrent reinforcement schedules.     

Context, observing behavior, and conditioned reinforcement

Pigeons made observing responses for stimuli signalling either a fixed-interval 30-sec schedule or a fixed-ratio x schedule, where x was either 20, 30, 100, 140, or 200 and the schedules alternated at random after reinforcement. If observing responses did not occur, food-producing responses occurred to a stimulus common to both reinforcement schedules. When the fixed-interval schedule was paired with a low-value fixed ratio, i.e., 20 or 30, the presentation of the stimulus reliably signalling the fixed-ratio schedule reinforced observing behavior, but the presentation of the stimulus reliably signalling the fixed-interval schedule did not. The converse was the case when the fixed-interval schedule was paired with a large-valued fixed ratio, i.e., 100, 140, or 200. The results demonstrated that the occasional presentation of the stimulus signalling the shorter interreinforcement interval was necessary for the maintenance of observing behavior. The reinforcement relationship was a function of the schedule context and was reversed by changing the context. Taken together, the results show that the establishment and measurement of conditioned reinforcement is dependent upon the context or environment in which stimuli reliably correlated with differential events occur.     

Compound stimulus control by discriminative stimuli associated with high and moderate response rates

Four rats received water on a fixed-ratio schedule for lever pressing in the presence of a tone (or light) stimulus and on a variable-interval schedule in the presence of a light (or tone) stimulus. Following stabilization of a high response rate during the fixed-ratio component and a moderate response rate during the variable-interval component, brief periods with the light and tone presented simultaneously but with no responses reinforced were inserted into the regular training schedule. Response rates during the compound stimuli were intermediate between the response rates controlled by the individual fixed-ratio and variable-interval associated stimuli.     

Intercurrent and reinforced behavior under multiple spaced-responding schedules

Lever pressing in rats was reinforced with food under a multiple spaced-responding schedule. A lever, food cup, and drinking tube were mounted in a running wheel so that lever pressing, running, and licking could be recorded. Running and licking had no scheduled consequences. Lever pressing was reinforced under a multiple schedule with three spaced-responding components and an extinction component. Each component was associated with a different auditory stimulus. Spaced-responding components reinforced only lever presses terminating interresponse times equal to or greater than 10, 20, or 60 sec, respectively. Rates of lever pressing, reinforcement, and licking all decreased as schedule parameter increased. Efficiency of spaced responding, as measured by reinforcements per response, also decreased. Rate of wheel running either increased or increased and then decreased with increasing schedule parameter. Individual running rates differed substantially. Neither licking nor running rate correlated with individual differences in efficiency. Analysis of conditional probabilities among the several response classes showed that, as the schedule requirement increased, the probability of running after a lever press increased and the probability of licking after a lever press decreased. After reinforcement, one subject always pressed the lever next. In the other subjects, the conditional probability of lever pressing, given reinforcement, increased while the probability of licking, given reinforcement, decreased with increasing schedule requirement. Results are discussed in relation to the concepts of schedule-induced and mediating behavior.     

Compounding of discriminative stimuli that maintain responding on separate response levers

In Experiment 1, rats' responses were reinforced on a fixed-interval 30-sec schedule in the presence of either a light or a tone and were not reinforced in their absence. Each stimulus was correlated with its own response lever, with only one lever present during a session. When light and tone were compounded in the presence of the tone-correlated lever, no change in responding occurred. However, when tone was compounded with light in the presence of the light-correlated lever, level of responding was greater than to light alone (response summation). Summation was also found when each stimulus was correlated with the same lever. Next, light and tone were again correlated with separate levers, but both levers were always simultaneously present. Compounding produced both summation and emission of most responses on the light-correlated lever. This prepotency of light was reduced (1) by leaving a houselight on throughout the session; and (2) by correlating each stimulus with a different schedule (either fixed-interval 4.7-sec or fixed-interval 30-sec). With a medium- and high-intensity houselight and with the different reinforcement schedules, similar results were obtained during compounding, regardless of whether compounding occurred in the presence of the light- or tone-correlated lever.     

Marking effects in instrumental performance on DRH schedules

Three experiments investigated the effect of presenting a brief stimulus after a response sequence on the rate of lever-pressing by rats on differential reinforcement of high rate (DRH) schedules. In Experiment 1 enhanced responding was produced by a visual stimulus presented during a 500-msec delay of reinforcement compared to a condition in which no stimulus was presented. In Experiment 2 rats responded on a multiple DRH DRH schedule in which the DRH contingency was reinforced on a 50% schedule in each component. Equivalent levels of responding occurred in the components when reinforcement was signalled in one component and when the signal was presented following the non-reinforced schedules in the other components. A further group of rats received the stimulus presented after non-reinforced schedules in one component but not at all in the other component; responding was enhanced in the former component relative to the latter component. In Experiment 3 brief stimuli presented after the completion of DRH components on a second-order VR (DRH) schedule elevated response rates irrespective of whether the signal was presented paired or unpaired with reinforcement. The present data support the view that a brief signal may serve to mark a response sequence in memory and facilitate instrumental performance.     

SINGLE‐SAMPLE DISCRIMINATION OF DIFFERENT SCHEDULES' REINFORCED INTERRESPONSE TIMES

Food‐deprived rats in Experiment 1 responded to one of two tandem schedules that were, with equal probability, associated with a sample lever. The tandem schedules' initial links were different random‐interval schedules. Their values were adjusted to approximate equality in time to completing each tandem schedule's response requirements. The tandem schedules differed in their terminal links: One reinforced short interresponse times; the other reinforced long ones. Tandem‐schedule completion presented two comparison levers, one of which was associated with each tandem schedule. Pressing the lever associated with the sample‐lever tandem schedule produced a food pellet. Pressing the other produced a blackout. The difference between terminal‐link reinforced interresponse times varied across 10‐trial blocks within a session. Conditional‐discrimination accuracy increased with the size of the temporal difference between terminal‐link reinforced interresponse times. In Experiment 2, one tandem schedule was replaced by a random ratio, while the comparison schedule was either a tandem schedule that only reinforced long interresponse times or a random‐interval schedule. Again, conditional‐discrimination accuracy increased with the temporal difference between the two schedules' reinforced interresponse times. Most rats mastered the discrimination between random ratio and random interval, showing that the interresponse times reinforced by these schedules can serve to discriminate between these schedules.     

Behavioural contrast in discrete-trial discriminations: effects of non-reinforcement

Six pigeons performed an acquisition and three reversals of a discrete trial simultaneous discrimination in which responses to the positive key were reinforced on a fixed-interval 5-s schedule; trials were terminated by either a response to the negative key or the delivery of a reinforcement. In the initial stage of acquisition and each reversal, where errors were most frequent, response rates rose and latencies fell on positive trials; these effects declined as errors decreased. The birds were also run in two yoked control conditions, and results showed that the critical requirement for the positive trial rate and latency changes was neither the simultaneous presentation of two stimuli nor the formation of a discrimination, but the occurrence of non-reinforced responses. The similarity of these phenomena to conventional behavioural contrast is emphasized, and the results are discussed in terms of frustration theory.     

Omission Learning after Instrumental Pretraining

Hungry rats were pretrained to press two levers on a concurrent schedule for food pellets. Following either limited or extended pretraining, presentations of a sucrose solution were programmed on a random time schedule while the animals continued pressing on a concurrent schedule for food pellets. Presses on one of the levers-the omission lever-postponed deliveries of the sucrose solution scheduled to occur within a fixed period of time following the press, whereas presses on the other, control lever had no effect on the random time contingency. The animals pressed less frequently on the omission lever than on the control lever following limited pretraining but failed to discriminate between the two levers following extended pretraining. The insensitivity to the omission contingency produced by extended pretraining was due to either the number of presses performed during the initial training or the number of reinforced presses, rather than the number of reinforcers received. Finally, the insensitivity of performance on the omission lever to sucrose devaluation suggests that adaptation tothis negative contingency was mediated by an inhibitory stimulus-response association.     

Behaviour of humans in concurrent schedules programmed on spatially separated operanda

Six human subjects responded on either of two levers for monetary reinforcement in a continuous choice situation. Responses on one lever (lever A) were reinforced at different frequencies specified by five variable-interval schedules. Reinforcements for responding on the other lever (lever B) were delivered according to a variable-interval schedule of standard reinforcement frequency. Results indicated that absolute response rate on lever A increased, while absolute response rate on lever B decreased as a function of reinforcement frequency for lever A. In terms of preference, the data conformed closely to Baum's (1974) generalized matching equation. Performances departing from perfect matching were obtained in four cases, but these deviations were not systematic either in the direction of overmatching or undermatching.     

The generality of selective observing

Four rats obtained food pellets by poking a key and 5-s presentations of the discriminative stimuli by pressing a lever. Every 1 or 2 min, the prevailing schedule of reinforcement for key poking alternated between rich (either variable-interval [VI] 30 s or VI 60 s) and lean (either VI 240 s, VI 480 s, or extinction) components. While the key was dark (mixed-schedule stimulus), no exteroceptive stimulus indicated the prevailing schedule. A lever press (i.e., an observing response), however, illuminated the key for 5 s with either a steady light (S+), signaling the rich reinforcement schedule, or a blinking light (S-), signaling the lean reinforcement schedule. One goal was to determine whether rats would engage in selective observing (i.e., a pattern of responding that maintains contact with S+ and decreases contact with S-). Such a pattern was found, in that a 5-s presentation of S+ was followed relatively quickly by another observing response (which likely produced another 5-s period of S+), whereas exposure to S- resulted in extended breaks from observing. Additional conditions demonstrated that the rate of observing remained high when lever presses were effective only when the rich reinforcement schedule was in effect (S+ only condition), but decreased to a low level when lever presses were effective only during the lean reinforcement component (S- only condition) or when lever presses had no effect (in removing the mixed stimulus or presenting the multiple-schedule stimuli). These findings are consistent with relativistic conceptualizations of conditioned reinforcement and extend the generality of selective observing to procedures in which the experimenter controls the duration of stimulus presentations, the schedule components both offer intermittent food reinforcement, and rats serve as subjects.     

Discrimination of response-contingent and response-independent shock by rats: effects of chlordiazepoxide HCL and sodium amylobarbitone

Rats, trained to press a lever for sucrose reward on a random interval (RI) schedule, were presented while lever-pressing with two stimuli, each associated with a different schedule of shock delivery: in the presence of one stimulus (S_e), shock occurred on an RI schedule irrespective of the rat's behaviour; in the presence of the other (S_p) shocks were programmed by the same schedule but delivered only when the rat pressed the lever. Both stimuli suppressed lever-pressing. In addition, the rats developed significantly different response rates in the two stimuli, thus demonstrating a discrimination between response-contingent and response-independent shock. Group data showed faster responding in S_e than in S_p, supporting the view that response-contingent shock produces greater suppression than response-independent shock. Individual animal analyses, however, demonstrated that this was the case in the majority of animals, but not in all. Response suppression was alleviated by amylobarbitone sodium (15 mg/kg) or chlordiazepoxide HCI (5 mg/kg); the latter drug alleviated suppression significantly more in S_p than S_e and eliminated the difference between the response rates controlled by the two stimuli.     

Temporal control in a complex environment: An analysis of schedule-related behavior

Three experiments conducted in an automated ten-compartment chamber recorded collateral activities of rats reinforced for lever pressing on differential-reinforcement-of-low-rate schedules. In Experiment 1, the rate of lever pressing increased when stimulus support for collateral activities was removed, thus confirming earlier findings. However, there were no temporal or sequential patterns of collateral activities that predicted operant responding. In Experiment 2, the rate of lever pressing increased only if (a) access to all stimulus support for collateral activities was simultaneously prevented, and (b) the rat was forced to remain in the presence of the lever and food tray. The availability of any of the stimuli related to collateral activity was sufficient to keep lever-pressing rates from increasing. Experiment 3 examined collateral activities under a signaled differential-reinforcement-of-low-rate schedule. Preventing access to stimuli supporting collateral activities had little effect on stable lever pressing when the signal was maintained. When the signal was removed, collateral activities continued, but lever-pressing rates increased in three of the four rats and rates of food presentation declined in all rats. Hypotheses that collateral activities have (a) a timekeeping or discriminative function, or (b) directly inhibit operant responding were not supported. The results suggest that collateral activities may facilitate operant responding by simply removing the subject from the presence of reinforcement-related stimuli.     

Stimulus control and compounding with ambient odor as a discriminative stimulus on a free-operant baseline

Previous experiments have demonstrated that the simultaneous presentation of independently established discriminative stimuli can control rates of operant responding substantially higher than the rates occasioned by the individual stimuli. This "additive summation" phenomenon has been shown with a variety of different reinforcers (e.g., food, water, shock avoidance, cocaine, and heroin). Discriminative stimuli previously used in such studies have been limited to the visual and auditory sensory modalities. The present experiment sought to (1) establish stimulus control on a free-operant baseline with an ambient olfactory discriminative stimulus, (2) compare olfactory control to that produced with an auditory discriminative stimulus, and (3) determine whether compounding independently established olfactory and auditory discriminative stimuli produces additive summation. Rats lever pressed for food on a variable-interval schedule in the presence of either a tone or an odor, with comparable control developed to each stimulus. In the absence of these stimuli responding was not reinforced. During stimulus compounding tests, the tone-plus-odor compound occasioned more than double the responses occasioned by either the tone or odor presented individually. Thus, the current study (1) established stimulus control with an ambient olfactory discriminative stimulus in a traditional free-operant setting and (2) extended the generality of stimulus-compounding effects by demonstrating additive summation when olfactory and auditory discriminative stimuli were presented simultaneously.     

Behavior of rats under fixed consecutive number schedules: effects of drugs of abuse.

Four rats responded under a simple fixed consecutive number schedule in which eight or more consecutive responses on the run lever, followed by a single response on the reinforcement lever, produced the food reinforcer. Under this simple schedule, dose-response curves were determined for diazepam, morphine, pentobarbital, and phencyclidine. The rats were then trained to respond under a multiple fixed consecutive number schedule in which a discriminative stimulus signaled when the response requirement on the run lever had been completed in one of the two fixed consecutive number component schedules. Under control conditions, the percentage of reinforced runs under the multiple-schedule component with the discriminative stimulus added was much higher than the percentage of reinforced runs under the multiple-schedule component without the discriminative stimulus. All of the drugs decreased the percentage of reinforced runs under each of the fixed consecutive number schedules by increasing the conditional probability of short run lengths. This effect was most consistently produced by morphine. The drugs produced few differences in responding between the multiple fixed consecutive number components. Responding under the simple fixed consecutive number schedule, however, was affected at lower doses of the drugs than was responding under the same fixed consecutive number schedule when it was a component of the multiple schedule. This result may be due to the difference in schedule context or, perhaps, to the order of the experiments.     

Effects of reinforcement magnitude and ratio values on behaviour maintained by a cyclic ratio schedule of reinforcement

In Experiments 1 and 2, lever pressing by rats was reinforced on a cyclic ratio schedule of food reinforcement, comprising a repeated sequence of fixed-ratio component schedules. Reinforcement magnitude was varied, on occasional sessions in Experiment 1 and across blocks of sessions in Experiment 2, from one to two or three 45-mg food pellets. In the one-pellet condition, post-reinforcement pauses increased with component schedule value. At higher magnitudes, post-reinforcement pauses increased, and overall response rates declined. Response rate on component schedules was a decreasing linear function of the obtained rate of reinforcement in all conditions. Plotted against component schedule value, response rate increased exponentially to an asymptote that decreased when reinforcement magnitude increased. These findings are consistent with regulatory accounts of food reinforced behaviour. In Experiment 3, rats were trained under a cyclic ratio schedule comprising fixed-ratio components including higher values, and some inverted U-shaped response functions were obtained. Those rats that did not showthis relationship were trained on cyclic ratios with even higher values, and all showed inverted U-shaped response functions. This suggests that behaviour on cyclic ratio schedules can reflect activating of reinforcement as well as the satiating effects seen in Experiments 1 and 2.     

Sequential response effects in the white rat during conditioning and extinction on a DRL schedule,

Sequential IRT data were obtained for three rats on a DRL 60-sec reinforcement schedule. It was found that first-order sequential dependencies exist under this schedule, including the partial dependence of the length of any given IRT on the length of the preceding IRT. The sequential analysis also served to extend the finding in the literature, based on frequency distributions, that the likelihood of a reinforced IRT is greater after a reinforced IRT than a non-reinforced IRT. Rapid extinction and reconditioning were obtained.     

OBSERVING RESPONSES AND SERIAL STIMULI: SEARCHING FOR THE REINFORCING PROPERTIES OF THE S−

The control exerted by a stimulus associated with an extinction component (S−) on observing responses was determined as a function of its temporal relation with the onset of the reinforcement component. Lever pressing by rats was reinforced on a mixed random-interval extinction schedule. Each press on a second lever produced stimuli associated with the component of the schedule in effect. In Experiment 1 a response-dependent clock procedure that incorporated different stimuli associated with an extinction component of a variable duration was used. When a single S− was presented throughout the extinction component, the rate of observing remained relatively constant across this component. In the response-dependent clock procedure, observing responses increased from the beginning to the end of the extinction component. This result was replicated in Experiment 2, using a similar clock procedure but keeping the number of stimuli per extinction component constant. We conclude that the S− can function as a conditioned reinforcer, a neutral stimulus or as an aversive stimulus, depending on its temporal location within the extinction component.     

Conditioned behavioral and physiological changes associated with injections of a narcotic antagonist in morphine-dependent monkeys

Environmental stimuli which are repeatedly associated with the nalorphine-induced withdrawal syndrome in morphine-dependent monkeys acquire the ability to produce a variety of conditioned behavioral and physiological responses. Morphine-dependent rhesus monkeys were studied under a fixed-ratio schedule where every tenth lever press produced a food pellet. After several pairings of a stimulus (light or tone) with intravenous injection of a dose of nalorphine which produced an immediate and severe withdrawal syndrome, onset of the stimulus alone produced conditioned suppression of lever pressing, heartrate decrease, vomiting and salivation. Conditioned suppression of responding and conditioned heart-rate changes persisted in post-dependent monkeys for one to four months after termination of chronic morphine treatment. No conditioned electrocardiogram, respiration or temperature changes were ever seen. A second group of morphine-dependent rhesus monkeys was studied under a schedule where every lever press produced an intravenous injection of morphine. After 10 pairings of a light with the intravenous injection of a dose of nalorphine which produced marked withdrawal signs and increased responding for morphine, presentation of the light and injection of sahne produced conditioned increases in responding for morphine. A third group of morphine-dependent rhesus monkeys was studied under a schedule where every nth lever press (n = 1 to 10) terminated a stimulus light associated with periodic injections of nalorphine or naloxone; lever-press responding was engendered and subsequently maintained. Thus, stimuli associated with the nalorphineor naloxone-induced withdrawal syndrome can either suppress, enhance or maintain behavior depending on the schedule conditions.     

Rate changes after unscheduled omission and presentation of reinforcement

Changes in response rate similar to frustration effects were studied in a two-lever situation. Responding on one lever on a fixed-interval schedule produced access to water for 5 sec and an exteroceptive stimulus. In the presence of this stimulus, responding on another lever on a fixed-interval schedule produced access to water for 5 sec and terminated the stimulus. Occasional omission of a previously scheduled reinforcer after responding on the first lever resulted consistently in increases in rate on the second lever during the immediately succeeding interval. In another procedure, occasional presentation of a previously unscheduled reinforcer after responding on the first lever resulted consistently in decreases in rate on the second lever during the immediately succeeding interval. Changes occurred after the first omissions or presentations and were about the same in magnitude as the procedure continued over several sessions. Typically, an increase or decrease in rate was maintained throughout an entire 100-sec interval. Changes in rate on the second lever of approximately the same magnitude also occurred when rate on the first lever was near-zero under a schedule that differentially reinforced behavior other than lever pressing.