Effects of food deprivation and reinforcement magnitude on conditioned suppression

In Experiment I, the responding of rats lever pressing on a variable-interval schedule for sucrose solution was partially suppressed by a variable duration conditioned stimulus followed by shock. When food deprivation was increased, response rates during and before the conditioned stimulus increased monotonically. Varying the concentration of sucrose across blocks of sessions or from session to session in a semi-random sequence had little effect on response rates either before or during the conditioned stimulus. With a fixed sequence of increasing concentrations across a five-session block, increased concentration produced much more rapid increases in response rates before than during the conditioned stimulus. In Experiment II, rats were presented with the same sequence of increasing concentrations across a five-session block. When tested at 80% body weight, response rates increased rapidly as concentration increased, but at 100%, body-weight rates increased only slightly. The effect of a change in body weight in Experiment II thus mimicked the effect of the conditioned stimulus in the latter part of Experiment I. These findings support the view that the effect of a pre-aversive conditioned stimulus is similar to that of a change in food deprivation, but unlike that of a change in reinforcement magnitude.     

Extinction of responding maintained by timeout from avoidance

The resistance to extinction of lever pressing maintained by timeout from avoidance was examined. Rats were trained under a concurrent schedule in which responses on one lever postponed shock on a free-operant avoidance (Sidman) schedule (response-shock interval = 30 s) and responses on another lever produced 2 min of signaled timeout from avoidance on a variable-ratio 15 schedule. Following extended training (106 to 363 2-hr sessions), two experiments were conducted. In Experiment 1 two different methods of extinction were compared. In one session, all shocks were omitted, and there was some weakening of avoidance but little change in timeout responding. In another session, responding on the timeout lever was ineffective, and under these conditions timeout responding showed rapid extinction. The within-session patterns produced by extinction manipulations were different than the effects of drugs such as morphine, which also reduces timeout responding. In Experiment 2 shock was omitted for many consecutive sessions. Response rates on the avoidance lever declined relatively rapidly, with noticeable reductions within 5 to 10 sessions. Extinction of the timeout lever response was much slower than extinction of avoidance in all 4 rats, and 2 rats continued responding at baseline levels for more than 20 extinction sessions. These results show that lever pressing maintained by negative reinforcement can be highly resistant to extinction. The persistence of responding on the timeout lever after avoidance extinction is not readily explained by current theories.     

Compounding of discriminative stimuli that maintain responding on separate response levers

In Experiment 1, rats' responses were reinforced on a fixed-interval 30-sec schedule in the presence of either a light or a tone and were not reinforced in their absence. Each stimulus was correlated with its own response lever, with only one lever present during a session. When light and tone were compounded in the presence of the tone-correlated lever, no change in responding occurred. However, when tone was compounded with light in the presence of the light-correlated lever, level of responding was greater than to light alone (response summation). Summation was also found when each stimulus was correlated with the same lever. Next, light and tone were again correlated with separate levers, but both levers were always simultaneously present. Compounding produced both summation and emission of most responses on the light-correlated lever. This prepotency of light was reduced (1) by leaving a houselight on throughout the session; and (2) by correlating each stimulus with a different schedule (either fixed-interval 4.7-sec or fixed-interval 30-sec). With a medium- and high-intensity houselight and with the different reinforcement schedules, similar results were obtained during compounding, regardless of whether compounding occurred in the presence of the light- or tone-correlated lever.     

Autoshaping, random control, and omission training in the rat

The role of the stimulus-reinforcer contingency in the development and maintenance of lever contact responding was studied in hooded rats. In Experiment I, three groups of experimentally naive rats were trained either on autoshaping, omission training, or a random-control procedure. Subjects trained by the autoshaping procedure responded more consistently than did either random-control or omission-trained subjects. The probability of at least one lever contact per trial was slightly higher in subjects trained by the omission procedure than by the random-control procedure. However, these differences were not maintained during extended training, nor were they evident in total lever-contact frequencies. When omission and random-control subjects were switched to the autoshaping condition, lever contacts increased in all animals, but a pronounced retardation was observed in omission subjects relative to the random-control subjects. In addition, subjects originally exposed to the random-control procedure, and later switched to autoshaping, acquired more rapidly than naive subjects that were exposed only on the autoshaping procedure. In Experiment II, subjects originally trained by an autoshaping procedure were exposed either to an omission, a random-control, or an extinction procedure. No differences were observed among the groups either in the rate at which lever contacts decreased or in the frequency of lever contacts at the end of training. These data implicate prior experience in the interpretation of omission-training effects and suggest limitations in the influence of stimulus-reinforcer relations in autoshaping.     

Re-assessing causal accounts of learnt behavior in rats

Rats received either a common-cause (i.e., A→B, A→food) or a causal-chain training scenario (i.e., B→A, A→food) before their tendency to approach the food magazine during the presentation of B was assessed as a function of whether it was preceded by a potential alternative cause. Causal model theory predicts that the influence of an alternative cause should be restricted to the common-cause scenario. In Experiment 1, responding to B was reduced when it occurred after pressing a novel lever during the test phase. This effect was not influenced by the type of training scenario. In Experiment 2, rats were familiarized with the lever prior to test by training it as a potential cause of B. After this treatment, the lever now failed to influence test responding to B. In Experiment 3, rats given common-cause training responded more to B when it followed a cue that had previously been trained as a predictor of B, than when it followed another stimulus. This effect was not apparent in rats that received causal-chain training. This pattern of results is the opposite of that predicted by causal model theory. Thus, in three experiments, the presence of an alternative cause failed to influence test responding in manner consistent with causal model theory. These results undermine the application of causal model theory to rats, but are consistent with associative analyses.     

Duration-reduction of avoidance sessions as negative reinforcement

Five rats were exposed to a shock-postponement procedure in which responses on each of two levers initially had equivalent effects. After an initial training sequence that ensured at least some responding on each lever, an additional consequence was made conjointly operative on the previously less-preferred lever for each animal. Each response on this lever continued to postpone shock, but also reduced the session duration by one minute. The conjoint contingencies were operative until, through session-shortening responses and the passage of time, the session was scheduled to end in two minutes; during the final two minutes the session-shortening contingency was disabled while the shock-postponement contingency continued to be operative on both levers. When responding shifted to a predominance on the session-shortening lever, the conjoint contingency was shifted to the other lever; for four of the five rats this reversal was followed by two additional reversals. Two of the rats' responding showed clear, strong, and unambiguous sensitivity to the session-shortening contingency. The responding of two others was also systematically controlled by that contingency, but the effects were less clearcut. The fifth animal showed an initial shift when session-shortening was introduced, but its subsequent behavior proved insensitive to reversals of procedure. The results clearly indicate a sensitivity of behavior to events on a time scale quite distinct from that of immediate consequences. They also support an interpretation of avoidance sessions, considered in their entirety, as events whose contingent relationship to behavior can affect that behavior—even in the absence of stimuli that delineate those relationships. Finally, these results support an interpretation of aversively based conditioning within a broader context, analogous to the “open versus closed economy” interpretation of appetitively controlled behavior.     

Discrimination between outcomes in instrumental learning: Effects of preexposure to the reinforcers

In two experiments rats received instrumental training with two response levers, one response being reinforced by sucrose solution and the other by sucrose pellets. Prior to a test session, on which both levers were made available in the absence of reinforcement, the rats were given free access to one of the reinforcers, a procedure known to reduce its value. It was found that the rats responded at a lower rate on the lever that had produced the now-devalued reinforcer, but that this effect was substantial only in rats that had received preexposure to the two reinforcers before instrumental training was begun (Experiment 1). Experiment 2 demonstrated that this effect was obtained only when presentations of the two reinforcers were presented according to an inter-mixed schedule during preexposure. It is suggested that this result constitutes an instance of the perceptual learning effect in which intermixed preexposure to similar events enhances their discriminability.     

Response specificity in animal timing.

The stimuli that control responding in the peak procedure were investigated by training rats, in separate sessions, to make two different responses for food reinforcement. During one type of session, lever pressing was normally reinforced 32 s after the onset of a light. During the other type of session, chain pulling was normally reinforced either 8 s after the onset of one auditory cue or 128 s after the onset of a different auditory cue. For both types of sessions, only the appropriate manipulandum was available, and 20% of the trials lasted 240 s and involved no response-contingent consequences. Rats were then tested with the auditory cues in the presence of the lever and the light in the presence of the chain. If the time of reinforcement associated with each stimulus was learned, response rates should peak at these times during transfer testing. However, if a specific response pattern was learned for each stimulus, little transfer should occur. The results did not clearly support either prediction, leading to the conclusion that both a representation of the time of reinforcement and the rat's own behavior may control responding in this situation.     

Variable-interval schedules of timeout from avoidance

Rats were trained on concurrent schedules in which pressing one lever postponed shock and pressing the other occasionally produced a 2-min timeout during which the shock-postponement schedule was suspended and its correlated stimuli were removed. Throughout, the shock-postponement schedule maintained proficient levels of avoidance. Nevertheless, in Experiment 1 responding on the timeout lever was established rapidly, was maintained at stable levels on variable-interval schedules, was extinguished by withholding timeout, was reestablished when timeout was reintroduced, and was brought under discriminative control with a multiple variable-interval extinction schedule of timeout. These results are in contrast with Verhave's (1962) conclusion that timeout is an ineffective reinforcer when presented to rats on intermittent schedules. In Experiment 2 the consequence of responding on the timeout lever was altered so that the shock-postponement schedule remained in effect even though the stimulus conditions associated with timeout were produced for 2 min. Responding extinguished, indicating that suspension of the shock-postponement schedule, not stimulus change, was the source of reinforcement. By establishing the reinforcing efficacy of timeout with standard variable-interval schedules, these experiments illustrate a procedure for studying negative reinforcement in the same way as positive reinforcement.     

Concurrent performances: a baseline for the study of conditioned anxiety

Three rats were trained to lever press on concurrent random interval 2-min random interval 2-min schedules of milk reinforcement. With a 5-sec changeover delay, relative response rate matched the relative reinforcement duration associated with each lever. A stimulus, during which unavoidable shocks occurred at random intervals, was superimposed on this concurrent baseline, and shifts in preference were found. However, data from this procedure were ambiguous, apparently confounded by shock-elicited response bursts. Termination of the shocks during the stimulus resulted in a rapid recovery of matching, which was preceded by a brief facilitation of responding on the less-preferred lever. The procedure was then changed to a conventional conditioned anxiety paradigm with a variable duration pre-shock stimulus. A marked shift in relative response rate towards the preferred lever was found in all three rats; that is, responding on the preferred lever was far less suppressed during the pre-shock stimulus than responding on the less-preferred lever.     

Control of responding by sounds of different quality: an evolutionary analysis.

Two experiments investigated the acquisition of discriminations between two acoustic stimuli of different quality (noise bursts vs. a 2-kHz pulsed signal) when features of the everyday environment were incorporated into the experiments. In Experiment 1, rats were trained, using food, to press a lever. Throughout all sessions, 5-s trials of noise bursts (the random stimulus) were presented, after variable intertrial intervals, through a remote speaker mounted outside the experimental enclosure. The noise burst occurred randomly with respect to reinforcement of lever pressing and had no programmed relationship to the animal's behavior. When lever pressing was established, the 2-kHz signal was presented through a speaker adjacent to the response lever according to a different set of variable intertrial intervals. A response in the presence of the 2-kHz signal terminated the trial and was reinforced. The 2-kHz signal acquired control of responding within the first few trials, whereas the random stimulus exerted no control of responding. In Experiment 2, rats were trained to press the lever in the presence of the 2-kHz signal presented through the adjacent speaker on a variable intertrial interval. After 14 sessions, 5-s trials of noise bursts (random stimulus) were presented through the remote speaker on the second variable intertrial interval. The random stimulus initially elicited exploratory behavior, which then rapidly declined. Subsequently, the random stimulus exerted no or weak control of responding. The introduction of the random stimulus had no effect on responding in the presence of the adjacent stimulus. In Experiments 3 and 4 the random stimulus was presented through the adjacent speaker, and the stimulus correlated with reinforcement was presented through the remote speaker. In both experiments, there was persistent control of responding by the random stimulus and slow development of control by the stimulus correlated with reinforcement. In Experiment 5, both stimuli were presented through the adjacent speaker. There was persistent control of responding by the random stimulus.     

Conditioned suppression of drl responding: Effect of ucs intensity, schedule parameter, and schedule context

In Experiment I, groups of rats were trained to press a lever for food reinforcement on differential reinforcement of low rate (DRL) schedules which differed in parameter value. A stimulus which terminated with either a 0.5-mA or 2.0-mA electric shock was then superimposed upon each DRL baseline. In general, the magnitude of conditioned suppression was an inverse function of DRL schedule parameter and a direct function of shock intensity. Experiment II demonstrated that the rate of responding maintained by the DRL component of a multiple DRL-extinction schedule decreased during a stimulus preceding a 0.5-mA shock, whereas the rate of responding maintained by the DRL component of a multiple DRL-variable interval schedule showed little change or increased slightly during a stimulus preceding a 0.5-mA shock.     

Drug discrimination in rats under concurrent variable-interval variable-interval schedules

Eight rats were trained to discriminate pentobarbital from saline under a concurrent variable-interval (VI) VI schedule, on which responses on the pentobarbital-biased lever after pentobarbital were reinforced under VI 20 s and responses on the saline-biased lever were reinforced under VI 80 s. After saline, the reinforcement contingencies programmed on the two levers were reversed. The rats made 62.3% of their responses on the pentobarbital-biased lever after pentobarbital and 72.2% on the saline-biased lever after saline, both of which are lower than predicted by the matching law. When the schedule was changed to concurrent VI 50 s VI 50 s for test sessions with saline and the training dose of pentobarbital, responding on the pentobarbital-biased lever after the training dose of pentobarbital and on the saline-biased lever after saline became nearly equal, even during the first 2 min of the session, suggesting that the presence or absence of the training drug was exerting minimal control over responding and making the determination of dose-effect relations of drugs difficult to interpret. When the pentobarbital dose-response curve was determined under the concurrent VI 50-s VI 50-s schedule, responding was fairly evenly distributed on both levers for most rats. Therefore, 6 additional rats were trained to respond under a concurrent VI 60-s VI 240-s schedule. Under this schedule, the rats made 62.6% of their responses on the pentobarbital-biased lever after pentobarbital and 73.5% of their responses on the saline-biased lever after saline, which also is lower than the percentages predicted by perfect matching. When the schedule was changed to a concurrent VI 150-s VI 150-s schedule for 5-min test sessions with additional drugs, the presence or absence of pentobarbital continued to control responding in most rats, and it was possible to generate graded dose-response curves for pentobarbital and other drugs using the data from these 5-min sessions. The dose-response curves generated under these conditions were similar to the dose-response curves generated using other reinforcement schedules and other species.     

Response acquisition and fixed-ratio escalation based on interresponse times in rats

The effectiveness of a fixed‐ratio (FR) escalation procedure, developed by Pinkston and Branch (2004) and based on interresponse times (IRTs), was assessed during lever‐press acquisition. Forty‐nine experimentally naïve adult male Long Evans rats were deprived of food for 24 hr prior to an extended acquisition session. Before the start of the session, three food pellets were placed in the magazine. Otherwise, no magazine training, shaping, nor autoshaping procedure was employed. The first 20 presses each resulted in the delivery of a 45‐mg food pellet. Then, the FR increased (2, 4, 8, 11, 16, 20, 25, 30) when each IRT in the ratio was less than 2 s during three consecutive ratios. Sessions lasted 13 hr or until 500 pellets were earned. On average, rats reached a terminal ratio of 11 (mean) or 16 (median) during the first session. Seven rats reached the maximum value of FR 30 and only one rat did not acquire the response. In most rats, a break‐and‐run pattern of responding characteristic of FR schedules began to develop in this acquisition session. Subsequently, the ratio‐escalation procedure continued during daily 2‐hr sessions. In these sessions, the starting ratio requirement was set at the terminal ratio reached in the previous session. Using this procedure, over half (26) of the rats reached the FR 30 requirement by the fourth session. These data demonstrate that a ratio‐escalation procedure based on IRTs provides a time‐efficient way of establishing ratio responding.     

Behavioral economics of concurrent ethanol-sucrose and sucrose reinforcement in the rat: effects of altering variable-ratio requirements.

These experiments examined the own-price and cross-price elasticities of a drug (ethanol mixed with 10% sucrose) and a nondrug (10% sucrose) reinforcer. Rats were presented with ethanol-sucrose and sucrose, both available on concurrent independent variable-ratio (VR) 8 schedules of reinforcement. In Experiment 1, the variable ratio for the ethanol mix was systematically raised to 10, 12, 14, 16, 20, and 30, while the variable ratio for sucrose remained at 8. Five of the 6 rats increased ethanol-reinforced responding at some of the increments and defended baseline levels of ethanol intake. However, the rats eventually ceased ethanol-reinforced responding at the highest variable ratios. Sucrose-reinforced responding was not systematically affected by the changes in variable ratio for ethanol mix. In Experiment 2, the variable ratio for sucrose was systematically increased while the ethanol-sucrose response requirement remained constant. The rats decreased sucrose-reinforced responding and increased ethanol-sucrose-reinforced responding, resulting in a two- to 10-fold increase in ethanol intake. Experiment 3 examined the substitutability of qualitatively identical reinforcers: 10% sucrose versus 10% sucrose. Increases in variable-ratio requirements at the preferred lever resulted in a switch in lever preference. Experiment 4 examined whether 10% ethanol mix substituted for 5% ethanol mix, with increasing variable-ratio requirements of the 5% ethanol. All rats eventually responded predominantly for the 10% ethanol mix, but total amount of ethanol consumed per session did not systematically change. In Experiment 5, the variable-ratio requirements for both ethanol and sucrose were simultaneously raised to VR 120; 7 of 8 rats increased ethanol-reinforced responding while decreasing sucrose-reinforced responding. These data suggest that, within this ethanol-induction procedure and within certain parameters, demand for ethanol-sucrose was relatively inelastic, and sucrose consumption was independent of ethanol-sucrose consumption. Demand for sucrose, on the other hand, was relatively elastic, and ethanol-sucrose readily substituted for it. The results are discussed in terms of applying a behavioral economic approach to relationships between drug and nondrug reinforcers.     

Food-paired stimuli as conditioned reinforcers: effects of d-amphetamine.

Seven pigeons were studied in two experiments in which key pecks were reinforced under a second-order schedule wherein satisfaction of variable-interval schedule requirements produced food or a brief stimulus. In the second part of each session, responses produced only the brief stimulus according to a variable-interval schedule (food extinction). For the 4 pigeons in Experiment 1, the response key was red throughout the session. In separate phases, the brief stimulus was either paired with food, not paired with food, or not presented during extinction. d-Amphetamine (0.3 to 10.0 mg/kg) dose-dependently reduced food-maintained responding during the first part of the session and, at intermediate dosages, increased responding during the extinction portion of the session. The magnitude of these increases, however, did not consistently depend on whether the brief stimulus was paired, not paired, or not presented. It was also true that under nondrug conditions, response rates during extinction did not differ reliably depending on pairing operations for the brief stimulus. In Experiment 2, 3 different pigeons responded under a procedure wherein the key was red in the component with food presentations and blue in the extinction component (i.e., multiple schedule). Again, d-amphetamine produced dose-related decreases in responding during the first part of a session and increases in responding in the second part of the session. These increases, however, were related to the pairing operations; larger increases were observed when the brief stimulus was paired with food than when it was not or when it was not presented at all. Under nondrug conditions, the paired brief stimulus controlled higher response rates during extinction than did a nonpaired stimulus or no stimulus. These findings suggest that d-amphetamine can enhance the efficacy of conditioned reinforcers, and that this effect may be more robust if conditioned reinforcers occur in the context of a signaled period of extinction.     

Irrelevant incentive learning during training on ratio and interval schedules

Two experiments investigated the effect of a motivationally-induced change in the value of the training reinforcer on instrumental performance. Initially, thirsty rats were trained to lever press for either a sodium or a potassium solution. Responding in an extinction test was then measured following the induction of sodium appetite. In Experiment I sodium-trained rats responded faster in a test given one day following the end of instrumental training. Furthermore, the relative size of this irrelevant incentive effect did not depend upon whether a ratio or interval schedule was employed during training. Delaying the test for eight days following the end of training abolished the difference between the test performance of sodium- and potassium-trained animals. Experiment II provided a further study of the effect of the training schedule when the introduction of the sodium reinforcer was delayed until responding was well established. Again the relative size of the difference between the performance of sodium- and potassium-trained animals was comparable following training on ratio and interval schedules. The insensitivity of this irrelevant incentive effect to the training contingency is in marked contrast to previous failures to detect an effect of reinforcer revaluation brought about by aversion conditioning following training on an interval schedule (Dickinson, Nicholas and Adams, 1983).     

Reevaluation of things past

The ability of rats to reevaluate previously presented information in light of subsequently provided information was evaluated using a CER (conditioned emotional response) procedure. In Experiment 1, rats suppressed responding to a compound light + tone stimulus that was repeatedly paired with shock. Groups of rats were then presented with only one element of the compound (the tone), either presented alone or paired with shock, for 15 days. During this 15-day period, two control groups received trials with the shock alone or neither stimulus nor shock. All of the rats were then tested with the other element of the compound (the light). Rats that had received the tone paired with shock during the intervening training continued to suppress to the light, whereas rats that had received the tone alone showed rapid extinction of the CER to light. The control groups showed that this effect was not due to the number of shock presentations received. A subsequent experiment also demonstrated that these results were not due to nonspecific stimulus effects. Apparently, a subsequent change in the associative strength of one element results in a similar change to the other element of a previously established compound stimulus.     

Mechanisms of resurgence II: Response-contingent reinforcers can reinstate a second extinguished behavior

Three experiments with rat subjects examined resurgence of an extinguished instrumental response using the procedure introduced by Epstein (1983) with pigeons. There were three phases: (1) initial acquisition of pressing on a lever (L1) for pellet reward, (2) extinction of L1, and (3) a test session in which a second lever (L2) was inserted, briefly reinforced, and then extinguished. Experiment 1 confirmed that if pressing L2 delivered 20 pellets followed by extinction, rats would resume L1 responding in the final test. Experiment 2 compared the effects of response-contingent and non-contingent rewards delivered upon insertion of L2. Although insertion of L2 alone did not increase L1 responding, response-contingent and non-contingent rewards led to comparable increases in L1 responding. Experiment 3 found that the delivery of non-contingent pellets during extinction of L1, which would be expected to reduce the ability of pellets to set the occasion for the L1 response, also reduced the effects of both response-contingent and non-contingent rewards during the final test. The results indicate that in this method, the resurgence treatment leads to an increase in L1 pressing due to simple presentation of the pellet; delivering the reinforcer after extinction of L1 reinstates L1 responding by setting the occasion for the L1 response.     

Renewal of operant performance formerly eliminated by omission or noncontingency training upon return to the acquisition context

Renewal of operant performance formerly eliminated by omission or noncontingency training was explored in two experiments with rats. When pressing a lever was trained with food reinforcement in one context (A) and then eliminated in a second context (B), responding was renewed by returning the rats to the original context (A). This ABA renewal effect was demonstrated in Experiment 1 when the elimination training was an omission procedure (delivery of food for withholding responding) and in Experiment 2 when it was a noncontingency procedure (delivery of food irrespective of responding). Because omission training (differential reinforcement of other behavior) and noncontingency training have been used in applied settings as effective procedures to reduce undesired human behaviors, the clinical implications of our findings for the relapse of undesirable behavior were discussed.