The effect of intertrial food presentations on anticipatory goal-tracking in the rat

Four experiments examined the sensitivity of anticipatory goal-tracking in the rat to stimulus-food contingency. Contingency was manipulated by varying the probability of food delivery in the absence of a food-tray-light or clicker conditional stimulus (CS), while holding constant the probability of food coincident with the CS. CS control of anticipatory food tray investigation was examined after a period of context extinction in all experiments. Acquisition of stimulus control was undermined by the scheduling of intertrial food deliveries (Experiment 1). The rate of intertrial food deliveries influenced subsequent acquisition of CS control when all intertrial food deliveries were omitted (Experiment 2). When intertrial food deliveries were added to the training regimen subsequent to acquisition of CS control, that control was impaired (Experiments 3 and 4).     

Contingency effects with maintained instrumental reinforcement

In three experiments we investigated the effect on the performance of thirsty rats of varying the instrumental contingency between lever pressing and the delivery of a saccharin reinforcer. In Experiment 1, the subjects performed more slowly in a non-contingent condition, in which the momentary probability of reinforcement was unaffected by whether or not the animals pressed, than in a contingent condition in which the reinforcer was never presented except following a lever press. This was true of performance under both random ratio and interval schedules in which the function determining the probability of reinforcement following a lever press remained the same across the contingent and non-contingent conditions. Experiment 2 demonstrated that instrumental performance was less affected when the contingency was degraded by the introduction of free reinforcers if these reinforcers were signalled. In Experiment 3, lever pressing was reinstated to some degree after non-contingent training by giving non-reinforced exposure to the operant chamber in the absence of the lever. These results suggest that free reinforcers depress instrumental behaviour through a performance mechanism engaged by their ability to support conditioning of the contextual cues.     

The effects of satiation and reinforcer develuation on signal-centered behavior in the rat

Two experiments are described which investigate the effects of satiation and reinforcer devaluation on signal-centered behavior in rats. In Experiment 1 lever contacts were established in hungry rats by pairing retractable lever presentations (CS) with response-independent food (UCS). Subsequently, food satiating these subjects significantly reduced the level of CS contact during an extinction test and, in particular, suppressed CS-directed licking and pawing. In Experiment 2 lever contacts which were established by lever-food pairings were suppressed when the food reinforcer was paired with lithium chloride (LiCl) induced illness. In particular, CS-directed licking, sniffing, and orienting were significantly suppressed by these food-LiCl pairings. These results suggest that signal-centered behavior (i) is not simply a manifestation of “conditioned hunger,” (ii) is determined to some extent by the animal's current need state, and (iii) is influenced by the status of specific reinforcer representations.     

Automaintenance in guinea pigs: effects of feeding regimen and omission training

Behavior maintained by stimulus-reinforcer pairings was examined. Guinea pigs maintained at 85 per cent of free-feeding weights reliably contacted a retractable lever presented before delivery of a single piece of guinea-pig chow or a 45-milligram guinea-pig pellet. When animals were given free access to one food and received the second food preceded by the lever, contact responses persisted. Such responses seldom occurred when a single food was freely available and was also delivered after lever presentation. Introduction of an omission training (negative automaintenance) procedure, in which lever contacts resulted in lever retraction and prevented food delivery, strongly reduced lever contacts. Observation indicated that mouthing the food cup, instead of the lever, became the prominent behavior during the prefood stimulus under the omission training procedure.     

Autoshaping in the rat: Effects of omission on the form of the response

Two experiments were conducted to investigate the effect of an omission contingency on behavior related to and characterizing autoshaped lever contacts in the rat. In Experiment I an omission contingency imposed on autoshaped lever contacts forceful enough to produce a press (.078N) resulted in a significant decrease in lever presses, but had no effect on frequency of lever touches (contacts of insufficient force to produce a press) or rate of food tray entry during lever presentation. In contrast, rats which received a similar number of lever-food pairings, but whose behavior had no programmed consequences (yoked control subjects), showed an increase in lever press rate, a significant decrease in rate of food tray entry, and no change in rate of lever-touches. In Experiment II, the effect of a similar omission contingency on the topography of lever contact responses was investigated. Prior to omission training subjects contacted the lever primarily by pawing it. Following omission training this behavior was suppressed, with a subsequent increase in lever contacts characterized as nosing. Yoked control subjects showed no significant changes in lever contact topography. The results indicate that (1) an omission contingency does not simply eliminate wholesale those topographies which incur the contingency but produces subtle adaptive changes in lever contact topography; and (2) the nature of the autoshaped response in the rat does not appear to be rigid enough to depend solely upon the nature of the unconditioned stimulus or the conditioned stimulus, but can also be determined by the relationships existing between the animal's behavior and these stimuli.     

Stimulus- and response-reinforcer contingencies in autoshaping, operant, classical, and omission training procedures in rats

Separate groups of rats received 500 trials of lever-press training under autoshaping (food delivery followed 10-second lever presentations, or occurred immediately following a response); operant conditioning (responding was necessary for food delivery); and classical conditioning (food followed lever presentations regardless of responding). Each group then received 500 trials on an omission procedure in which food was omitted on trials with a response. Another group received 1000 trials on the omission procedure, and a fifth group, random control, received 1000 uncorrelated presentations of lever and food. The autoshaping, operant, and classical groups reached high response levels by the end of initial training. Acquisition was fastest in the autoshaping group. Responding remained consistently low in the control group. The omission group responded at a level between the control group and the other three groups. During omission training, responding in these three groups declined to the omission-group level. During omission training, the rats continued contacting the lever frequently after lever pressing had declined. Response maintenance under omission training seems not to require topographic similarity between the response and reinforcer-elicited consummatory behaviors.     

Effects of response-dependent and independent electric shock on schedule-induced polydipsia

In Experiment I, lever pressing by rats was maintained by the delivery of food pellets under a 45-sec fixed-interval schedule. Fixed-time 180-sec and fixed-interval 180-sec schedules of shock delivery were systematically superimposed on the baseline food schedule to study effects on schedule-induced water intake. Response-dependent shock had little, if any, effect on water intake, whereas shocks independent of lever pressing attenuated fluid intake. In Experiment 2, rats received food pellets under a fixed-time 60-sec schedule. Electric shock delivered concurrently under a variable-time 180-sec schedule, but never while the animal was licking or within 5 sec after licking terminated, led to similar attenuation of water intake. These findings suggest that schedule-induced polydipsia is sensitive to differences in the functional properties of response-independent and dependent electric shock.     

Counterconditioning and exposure-only in the treatment of specific (conditioned suppression) and generalized fear in rats

In Experiment 1 the conditioned suppression technique was used to condition specific fear, suppression of operent lever pressing for food to a discrete CS. The efficacy of four treatment conditions on fear reduction was evaluated. Counterconditioning in which exposure to the CS was contiguously paired with food was significantly less effective than noncontiguous CS exposure and food. An exposure-only effect was indicated by the superiority of all three treatments involving CS exposure (the above two plus a typical conditioned suppression extinction procedure) to treatment consisting of food only. The reverse counterconditioning effect and the exposure effect are consistent with current views that emphasize the centrality of aversive stimulus exposure in fear reduction. Experiment 2 investigated elimination of generalized fear produced by unsignalled, inescapable shocks in the lever-pressing apparatus. Two treatments (counterconditioning and exposure-only) were equally effective and they were superior to no exposure control treatment. The results of the two experiments reinforce recent attempts toward a reevaluation of the role of anxiety-competing responses in elimination of fear.     

Topography of signal-centered behavior in the rat: Effects of deprivation state and reinforcer type

In a series of three experiments, groups of food-deprived and water-deprived rats were given pairings of a retractable lever (CS⁺) with response-independent deliveries of either solid or liquid reinforcers. In Experiment 1 food-deprived rats given a solid-pellet reinforcer differentially tended to sniff, paw, mouth, and bite the CS⁺ lever more often than a lever that was not paired with food (CS⁻), whereas food-deprived rats given a liquid reinforcer tended to differentially sniff, paw, and lick the CS⁺ lever. 23½-hour water-deprived rats given liquid reinforcers showed very little CS⁺ contact. In Experiment 2 increasing the severity of water deprivation from 23½ to 47½ hours significantly increased CS⁺ contact. In Experiment 3, subjects that were simultaneously food and water deprived and given a water reinforcer failed to exhibit differential CS⁺ contact, but subjects that were simultaneously food and water deprived and given a food reinforcer did acquire differential CS⁺-contact behavior. These results suggest that (a) even under a single motivational state the nature of signal-centered behavior can be determined by type of reinforcer, (b) although water reinforcement produces less signal contact than food reinforcement, this can be facilitated with more severe water-deprivation levels, and (c) high CS-contact rates using food reinforcement are not simply a product of reductions in body weight with food deprivation.     

Temporal control in a complex environment: An analysis of schedule-related behavior

Three experiments conducted in an automated ten-compartment chamber recorded collateral activities of rats reinforced for lever pressing on differential-reinforcement-of-low-rate schedules. In Experiment 1, the rate of lever pressing increased when stimulus support for collateral activities was removed, thus confirming earlier findings. However, there were no temporal or sequential patterns of collateral activities that predicted operant responding. In Experiment 2, the rate of lever pressing increased only if (a) access to all stimulus support for collateral activities was simultaneously prevented, and (b) the rat was forced to remain in the presence of the lever and food tray. The availability of any of the stimuli related to collateral activity was sufficient to keep lever-pressing rates from increasing. Experiment 3 examined collateral activities under a signaled differential-reinforcement-of-low-rate schedule. Preventing access to stimuli supporting collateral activities had little effect on stable lever pressing when the signal was maintained. When the signal was removed, collateral activities continued, but lever-pressing rates increased in three of the four rats and rates of food presentation declined in all rats. Hypotheses that collateral activities have (a) a timekeeping or discriminative function, or (b) directly inhibit operant responding were not supported. The results suggest that collateral activities may facilitate operant responding by simply removing the subject from the presence of reinforcement-related stimuli.     

Deprivation and satiation: The interrelations between food and wheel running

Two experiments were designed to assess whether depriving rats of food would increase the reinforcement effectiveness of wheel running (Experiment 1) and whether satiation for wheel running would decrease the reinforcement effectiveness of food (Experiment 2). In Experiment 1, a progressive-ratio schedule was used to measure the reinforcement effectiveness of wheel running when rats were deprived or not deprived of food. Completion of a fixed number of lever presses released a brake on a running wheel for 60 s, and the response requirement was systematically increased until the rat stopped pressing or until 8 hr had elapsed. The ratio value reached (and the total number of lever presses) was an inverted-U function of food deprivation (percentage body weight). In Experiment 2, when wheel running preceded test sessions, fewer food-reinforced lever presses were maintained by the progressive-ratio schedule, and responding occurred at a lower rate on a variable-interval schedule. An interpretation of these results is that deprivation or satiation with respect to one event (such as food) alters the reinforcement effectiveness of a different event (such as access to wheel running).     

Effects of reinforcer probability, delay, and response requirements on the choices of rats and pigeons: possible species differences

In Experiment 1 with rats, a left lever press led to a 5-s delay and then a possible reinforcer. A right lever press led to an adjusting delay and then a certain reinforcer. This delay was adjusted over trials to estimate an indifference point, or a delay at which the two alternatives were chosen about equally often. Indifference points increased as the probability of reinforcement for the left lever decreased. In some conditions with a 20% chance of food, a light above the left lever was lit during the 5-s delay on all trials, but in other conditions, the light was only lit on those trials that ended with food. Unlike previous results with pigeons, the presence or absence of the delay light on no-food trials had no effect on the rats' indifference points. In other conditions, the rats showed less preference for the 20% alternative when the time between trials was longer. In Experiment 2 with rats, fixed-interval schedules were used instead of simple delays, and the presence or absence of the fixed-interval requirement on no-food trials had no effect on the indifference points. In Experiment 3 with rats and Experiment 4 with pigeons, the animals chose between a fixed-ratio 8 schedule that led to food on 33% of the trials and an adjusting-ratio schedule with food on 100% of the trials. Surprisingly, the rats showed less preference for the 33% alternative in conditions in which the ratio requirement was omitted on no-food trials. For the pigeons, the presence or absence of the ratio requirement on no-food trials had little effect. The results suggest that there may be differences between rats and pigeons in how they respond in choice situations involving delayed and probabilistic reinforcers.     

Operant hoarding: a new paradigm for the study of self-control.

In the first of four experiments, rats were exposed to a modified multiple continuous reinforcement-extinction schedule during 15-min daily sessions. In one condition (saves condition) with the cuelight on, a single lever press produced a food pellet, briefly extinguished the cuelight, and started a clock. Saves (additional lever presses with interresponse times less than 1 s) produced an additional food pellet, briefly extinguished the cuelight, and restarted the interresponse time clock. The cuelight was extinguished 1 s after the last lever press and remained off during a 10-s period of extinction, during which no food pellets were delivered. In the other condition (savings account condition), the contingencies were the same except that the cuelight was extinguished and was not reilluminated after the initial lever press, and the delivery of all food pellets in the reinforcement component was delayed until the onset of extinction. In both conditions, rats made saves, but mean saves (total saves divided by the number of reinforcement components) were slightly reduced in the savings account condition. In Experiment 2, using six equally spaced 15-min sessions per day on alternate days, saves were either followed immediately with food and brief cuelight offset (saves condition) or were not reinforced at all. Mean saves were much greater when saves were reinforced. In Experiment 3, during 5-min daily sessions, saves earned a single pellet (savings account condition) or a number of pellets equal to the ordinal number of the lever press (interest condition). Rats made fewer mean saves, with little change in the food rate, when saves earned interest. In Experiment 4, the rats earned all their food in the operant situation during 24 daily 5-min sessions, these separated by 55-min intersession intervals during which no food was available; otherwise, the conditions were the same as in Experiment 3. In Experiment 4, the shift to interest for saves led to an increase in mean daily mean saves (total daily mean saves divided by the number of daily sessions) as well as to an increase in the number of food pellets delivered in each session. The results are discussed in terms of self-control and behavioral economics.     

DISCRIMINATION OF VARIABLE SCHEDULES IS CONTROLLED BY INTERRESPONSE TIMES PROXIMAL TO REINFORCEMENT

In Experiment 1, food‐deprived rats responded to one of two schedules that were, with equal probability, associated with a sample lever. One schedule was always variable ratio, while the other schedule, depending on the trial within a session, was: (a) a variable‐interval schedule; (b) a tandem variable‐interval, differential‐reinforcement‐of‐low‐rate schedule; or (c) a tandem variable‐interval, differential‐reinforcement‐of‐high‐rate schedule. Completion of a sample‐lever schedule, which took approximately the same time regardless of schedule, presented two comparison levers, one associated with each sample‐lever schedule. Pressing the comparison lever associated with the schedule just presented produced food, while pressing the other produced a blackout. Conditional‐discrimination accuracy was related to the size of the difference in reinforced interresponse times and those that preceded it (predecessor interresponse times) between the variable‐ratio and other comparison schedules. In Experiment 2, control by predecessor interresponse times was accentuated by requiring rats to discriminate between a variable‐ratio schedule and a tandem schedule that required emission of a sequence of a long, then a short interresponse time in the tandem's terminal schedule. These discrimination data are compatible with the copyist model from Tanno and Silberberg (2012) in which response rates are determined by the succession of interresponse times between reinforcers weighted so that each interresponse time's role in rate determination diminishes exponentially as a function of its distance from reinforcement.     

Effects of a brief stimulus accompanying reinforcement on instrumental responding in pigeons

The responding of pigeons on a variable interval schedule of reinforcement was investigated in four experiments. In some conditions in each experiment reinforced keypecks were accompanied by a brief (0.5-sec) flash of the houselight. This procedure resulted in a low rate of response in comparison with that found in conditions when response-contingent light flashes occurred uncorrelated with reinforcement (Experiments 1 and 2) or when no light flash was presented (Experiment 3). Experiment 4 allowed a comparison between the effects of a signal accompanying the reinforced response and one accompanying the delivery of “free” food. Signaling the delivery of earned food produced a lower rate of response than did signaling the delivery of free food. The role of stimulus-reinforcer and response-reinforcer associations in producing these effects is discussed.     

Techniques for establishing schedules with wheel running as reinforcement in rats.

In three experiments, access to wheel running was contingent on lever pressing. In each experiment, the duration of access to running was reduced gradually to 4, 5, or 6 s, and the schedule parameters were expanded gradually. The sessions lasted 2 hr. In Experiment 1, a fixed-ratio 20 schedule controlled a typical break-and-run pattern of lever pressing that was maintained throughout the session for 3 rats. In Experiment 2, a fixed-interval schedule of 6 min maintained lever pressing throughout the session for 3 rats, and for 1 rat, the rate of lever pressing was positively accelerated between reinforcements. In Experiment 3, a variable-ratio schedule of 20 or 35 was in effect and maintained lever pressing at a very stable pace throughout the session for 2 of 3 rats; for 1 rat, lever pressing was maintained at an irregular rate. When the session duration was extended to successive 24-hr periods, with food and water accessible in Experiment 3, lever pressing settled into a periodic pattern occurring at a high rate at approximately the same time each day. In each experiment, the rats that developed the highest local rates of running during wheel access also maintained the most stable and highest rates of lever pressing.     

Lever-contact responses in rats: automaintenance with and without a negative response-reinforcer dependency

Pairing the presentations of one lever (cueing lever) with food led to the acquisition and persistence of lever contacts by rats. This behavior did not occur with a second lever, which was presented randomly with regard to food delivery. This finding obtained whether food delivery was independent of cueing-lever contacts (positive automaintenance) or dependent on the absence of cueing-lever contacts (negative automaintenance). The general findings were: (1) cueing-lever contacts on the positive automaintenance procedure occurred on a higher proportion of trials and at higher rates when contacts occurred than on the negative automaintenance procedure; (2) instances of the cueing lever's failure to support responding were more frequent on the negative than the positive automaintenance procedure; and (3) the topography and median contact duration of positively automaintained responding differed from negatively automaintained responding. These findings agree substantially with the automaintenance literature on pigeons, suggesting that similar processes may characterize automaintained responding in both pigeons and rats.     

SCHEDULE-INDUCED DRINKING: ELICITATION,ANTICIPATION, OR BEHAVIORAL INTERACTION?

We carried out five experiments with rats on fixed-time schedules in order to define the relation between drinking and individual food-pellet presentations. In Experiment 1, unsignaled extra food occurred at the end of occasional fixed intervals, and we compared subsequent drinking patterns with drinking before the extra food presentation. In Experiment 2 we presented signaled and unsignaled extra food and measured elicited and anticipatory drinking patterns. In Experiment 3, we observed the persistence of modified drinking patterns when several consecutive intervals ended with extra pellets. In Experiments 4 and 5, we varied the magnitude of food delivery across (rather than within) sessions to replicate published findings. Results show that schedule-induced drinking is neither elicited by food presentations nor induced by stimuli associated with a high food rate. All subjects seemed to follow a simple rule: during any stimulus signaling an increase in the local probability of food delivery within a session, engage in food-related behavior to the exclusion of drinking. Schedule-induced drinking appears to be the result of dynamic interactions among food-related behavior, drinking, and other motivated behavior, rather than a direct effect of the contingencies of food reinforcement.     

Re-assessing causal accounts of learnt behavior in rats

Rats received either a common-cause (i.e., A→B, A→food) or a causal-chain training scenario (i.e., B→A, A→food) before their tendency to approach the food magazine during the presentation of B was assessed as a function of whether it was preceded by a potential alternative cause. Causal model theory predicts that the influence of an alternative cause should be restricted to the common-cause scenario. In Experiment 1, responding to B was reduced when it occurred after pressing a novel lever during the test phase. This effect was not influenced by the type of training scenario. In Experiment 2, rats were familiarized with the lever prior to test by training it as a potential cause of B. After this treatment, the lever now failed to influence test responding to B. In Experiment 3, rats given common-cause training responded more to B when it followed a cue that had previously been trained as a predictor of B, than when it followed another stimulus. This effect was not apparent in rats that received causal-chain training. This pattern of results is the opposite of that predicted by causal model theory. Thus, in three experiments, the presence of an alternative cause failed to influence test responding in manner consistent with causal model theory. These results undermine the application of causal model theory to rats, but are consistent with associative analyses.     

Reinforcement of mediating behavior on a spaced-responding schedule

This paper describes a procedure for gaining experimental control over mediating behavior on a spaced-responding schedule of food reinforcement. Three rats, food-deprived, were trained on a DRL 16 sec schedule of food reinforcement. Then, a concurrent schedule of food reinforcement was introduced on a second (mediating) lever, such that the first response to occur on the mediating lever, after the DRL interval had timed out, was reinforced with food, as was the next response to occur on the DRL lever. Reinforcement via the mediating lever became a discriminative stimulus for a food-reinforcement opportunity on the DRL lever. Next, food reinforcement for the mediating behavior was replaced by a conditioned reinforcer consisting of onset of a buzzer signaling timing-out of the DRL interval. Under these conditions, chaining of behavior on the two levers was strong, and timing on the DRL lever was more accurate than under ordinary DRL conditions. As the DRL requirement was lengthened from 16 sec to 24 sec to 60 sec, mediating behavior weakened slightly. When the inter-response requirement for food reinforcement on the DRL lever was made shorter than the inter-response requirement for conditioned reinforcement on the mediating lever, the mediating behavior extinguished. Performance in the experiment was analyzed into a four-component chain, and the factors contributing to the maintenance, and later extinction, of mediating behavior are discussed.