The role of response-contingent incentives in lithium chloride-mediated suppression of an operant response

Three experiments investigated what role a novel incentive plays in the development of operant response suppression mediated by lithium chloride. In all experiments animals were trained to press two levers under concurrent schedules of reinforcement. In Experiment 1 responding on one lever delivered a familiar incentive (food pellets), whereas responding on an alternative lever delivered a novel incentive (sucrose solution) prior to lithium chloride injections. If lithium was administered immediately after the instrumental session, the action associated with the novel, but not with the familiar, incentive was suppressed. By comparison, in a control group for which responding on both levers led to the familiar incentive, both actions were suppressed. Experiment 2 examined whether the novelty, rather than the sensory properties, of the incentive is crucial for observing performance suppression. It was found that animals familiarized with the “target” incentive were insensitive to aversion conditioning by lithium, in that there was no difference in response rates between the action that delivered the familiar incentive from that which earned the “target”. In contrast, if animals were unfamiliar with the “target” incentive at the time of aversion conditioning, they suppressed responding on the lever that was associated with the novel incentive but did not suppress responding on the lever associated with the familiar incentive. Experiment 3 investigated the mechanism underlying instrumental performance suppression. After the completion of concurrent lever press training, novel sucrose was introduced in conjunction with the pellets for responding on one lever; responding on the other lever continued to deliver only familiar pellets. Lithium injections were then administered either immediately following the sessions or several hours after the sessions. It was found that the rate of responding on the lever associated with the contingent delivery of sucrose was suppressed below that of the pellet-alone action. By comparison, if lithium injections were administered several hours following the session, an elevation in responding on the sucrose-plus-pellet lever was observed. The outcomes of all three experiments demonstrate not only that the novelty of an incentive is important in obtaining performance suppression, but also that a novel incentive can punish instrumental responding if it has been associated with toxicosis.     

Contextual conditioning and reinstatement of extinguished instrumental responding

In three experiments we studied the relationship between contextual conditioning and the reinstatement of extinguished lever pressing that occurs when noncontingent food is introduced following extinction. In all three experiments the non-contingent food was presented off-baseline (with the response levers not present). On subsequent tests, with the response levers present, animals that had been exposed to food showed more reinstatement of lever pressing than control animals. This finding rules out alternative mechanisms for the reinstated responding that rely on the interaction of non-contingent food and responding, such as superstitious reinforcement or the discriminative after-effects of food. In addition, in each experiment we demonstrated that manipulations known to affect contextual conditioning (signalling the food in Experiment 1, context extinction in Experiment 2, and switching contexts in Experiment 3) reduced the reinstatement. These results are consistent with the claim that contextual conditioning is important in controlling instrumental conditioning and closely parallel findings concerning the reinstatement of Pavlovian responsing following extinction.     

The influence of context and UCS intensity on the UCS preexposure effect in a flavor aversion paradigm

Three studies were conducted to evaluate the influence of environmental context and UCS preexposure intensity on the acquisition of a flavor aversion. In the first study, rats were exposed to two UCS-alone presentations at one of three dose levels of LiCl. The rats were given the two UCS preexposures in one of two contexts (familiar or novel). Following the UCS-alone treatment, all of the subjects received two pairings of sucrose and LiCl (1.25 meq) in the familiar context. Animals receiving UCS preexposures in the same context as used during conditioning consumed significantly more sucrose than did saline control animals. In contrast, the influence of prior UCS preexposure was not evident when UCS preexposure was experienced in a context different from that experienced during conditioning. In Experiment 2, all animal received UCS preexposure and conditioning in the novel room. Animals received either two or four preexposures at 0, 1.25, or 3.75 meq dose of LiCl. Two UCS preexposures at 3.75 meq dose produced a stronger UCS preexposure effect than did the 1.25 meq dose. Animals in Experiment 3 received two exposures to either 0-, 1.25-, or 3.75-meq LiCl in one of two contexts, followed by conditioning with the 1.25-meq LiCl dose in the same context as preexposure. Greater UCS interference occurred in animals preexposed to LiCl in the novel rather than familiar environment, with no specific dose effects in either context. These observations are discussed in terms of context blocking and generalization decrement models of the UCS preexposure effect.     

Alcohol seeking by rats: Action or habit?

In two experiments, we examined the relative susceptibility to outcome devaluation of lever pressing by rats for either a 10% ethanol solution or food pellets. The rats were trained to press different levers for these two reinforcers using a sucrose-substitution procedure. An aversion was then conditioned from either the ethanol solution or the food pellets by pairing consumption with illness induced by lithium chloride. When instrumental performance was subsequently tested in extinction, the rats pressed less on the pellet lever if the pellets, rather than the ethanol, had been devalued by aversion conditioning. By contrast, performance on the ethanol lever was unaffected by whether the ethanol or pellets were devalued. Moreover, noncontingent presentations of the devalued reinforcer had no impact on test performance. The differential resistance to outcome devaluation suggests that, in contrast to food seeking, alcohol seeking is a stimulus-response habit rather than a goal-directed action mediated by a representation of the action-outcome contingency.     

The effect of rearing environments on the contrafreeloading phenomenon in rats

Eight naive rats were reared in enriched or impoverished environments for 39 days after weaning and then lived in operant chambers, in which they could obtain food pellets freely or by lever pressing, for 25 or 30 days. The animals raised in an impoverished environment acquired the bar-press response quickly when placed in the operant chambers and maintained a preference for obtaining food via bar pressing. Animals raised in an enriched environment did not learn to lever press, as demonstrated by low levels of responding and the lack of bar pressing when free food was subsequently removed. It was concluded that restricting animals' postweaning environments facilitated learning in a choice situation, probably because of increased activity levels. The results are interpreted in relation to previous studies on rearing environments and on contrafreeloading.     

Incentive learning and the motivational control of instrumental performance

Hungry rats were trained to press a lever and pull a chain concurrently, with one action being reinforced with a sucrose solution and the other with food pellets. In addition, in the first two experiments all animals experienced non-contingent presentations of the two incentives in the absence of the operant manipulanda while either thirsty or hungry and either before (Experiment 1A) or after (Experiment 1B) the instrumental training. When lever pressing was assessed subsequently in extinction under thirst, the animals pressed at a relatively high rate only if (1) this action had been reinforced with the sucrose solution rather than the food pellets during training and (2) they had received the non-contingent presentations of the sucrose solution and food pellets on days on which they were thirsty rather than hungry. A third experiment demonstrated that non-contingent exposure to the sucrose solution alone, but not to water under thirst was sufficient to bring about this type of motivational control of instrumental performance.     

Is induction produced by upcoming food-pellet reinforcement the outcome of an increase in overall activity or in operant responding?

Researchers have demonstrated that rats reliably increase their rates of pressing a lever for 1% liquid-sucrose reinforcement if they will soon have the opportunity to press a lever for food-pellet reinforcement. In the present experiments, the authors investigated if this increase in response rates occurred because the upcoming food pellets produced an increase in all behaviors (i.e., general arousal) or an increase in only the specific operant response (i.e., lever pressing). The results of Experiments 1 and 2 showed that the appearance of induction in rats' lever pressing for 1% sucrose reinforcement when food-pellet reinforcement was upcoming did not coincide with increases in the frequency of running in a wheel or making a nonreinforced nose-poke response. On the other hand, in Experiment 3, the authors found the appearance of induction coincided with increase nonreinforced lever presses on an adjacent lever. These results shed doubt on the idea that induction is a result of a general increase in all activity, and suggest instead that the increase in responding that occurs during induction is limited to the operant response.     

Toxicosis affects instrumental behaviour in rats

Three experiments examined the effect of toxicosis on instrumental responding. These studies were prompted by Morrison and Collyer's (1974, Experiment 1) finding that the induction of toxicosis after an instrumental conditioning session produces greater response suppression if the response is reinforced by a novel saccharin solution rather than familiar water during conditioning. Experiments 1 and 2 investigated whether this suppression was mediated by the Pavlovian contingency between the contextual cues and the saccharin solution or the instrumental relationship between the response and the reward. A role for the instrumental contingency was indicated by the greater suppression of the response producing novel saccharin rather than water when the context of both responses was equally associated with the saccharin and illness. Experiment 3 found that extinction of the aversion to a novel reinforcer following aversive conditioning would re-establish an action previously associated with that reinforcer, in contrast to an action whose reinforcer remained aversive. This result was a further indication that the instrumental contingency between the response and reward contributes to response suppression.     

Post-Conditioning Devaluation of an Instrumental Reinforcer has no Effect on Extinction Performance

The extent to which a representation of the reinforcer controls an instrumental response can be assessed by studying the effect of post-conditioning changes in the reinforcer value. In the first experiment rats were trained to press a lever for sucrose pellets on a variable-interval (VI) schedule. The sucrose was subsequently devalued by pairing with Lithium Chloride (LiCl). This had no effect on lever pressing in extinction, although it profoundly reduced reacquisition responding and consumption. In Experiment II rats were trained to shuttle between the two distinctive chambers of a choice-box, in which lever pressing was reinforced in one chamber by sucrose and in the other chamber by food pellets programmed on independent VI schedules. A LiCl-induced taste-aversion was conditioned to the sucrose, and although this markedly affected reacquisition, extinction responding in the sucrose chamber and chamber preference were unaffected. These results indicate that instrumental performance can be independent of the current value of the reinforcer, and are discussed with reference to stimulus-response theory and second-order Pavlovian conditioning.     

Behavior analysis and revaluation

Revaluation refers to phenomena in which the strength of an operant is altered by reinforcer-related manipulations that take place outside the conditioning situation in which the operant was selected. As an example, if lever pressing is acquired using food as a reinforcer and food is later paired with an aversive stimulus, the frequency of lever pressing decreases when subsequently tested. Associationist psychology infers from such findings that conditioning produces a response-outcome (i.e., reinforcer) association and that the operant decreased in strength because pairing the reinforcer with the aversive stimulus changed the value of the outcome. Here, we present an approach to the interpretation of these and related findings that employs neural network simulations grounded in the experimental analysis of behavior and neuroscience. In so doing, we address some general issues regarding the relations among behavior analysis, neuroscience, and associationism.     

Context Conditioning and Free operant Acquisition under Delayed Reinforcement

Three experiments examined the effect of context conditioning on the acquisition of freeoperant lever pressing by hungry rats when the presentation of the food reinforcer was delayed for 32 sec. The first study replicated the preexposure effect reported by Dickinson, Watt, and Griffiths (1992): Exposure to the contextual cues with the lever withdrawn prior to each instrumental training session enhanced acquisition, an effect that was attenuated by the presentation of non-contingent reinforcement during the preexposure periods. Signalling the non-contingent reinforcers during the preexposure periods with a brief auditory stimulus enhanced acquisition in a second study, suggesting that the non-contingent reinforcement interferes with acquisition through context conditioning. The final study confirmed this conclusion using a within-subject procedure in which pressing different levers was reinforced in two contexts, one of which was also associated with non-contingent reinforcers.     

Conditional acceleration and external disinhibition of operant lever pressing by prereward, neutral, and reinforcing stimuli

Rats responding under a differential-reinforcement-of-low-rate schedule increased their rates of lever pressing during a 20-second click/flash stimulus that preceded the delivery of a response-independent food pellet. The increase could not be attributed to suppression of collateral behavior that has been said to mediate temporally-spaced responding. We propose that the prereward stimulus functioned as an external disinhibitor of lever pressing that had been inhibited by the constraints of the operant schedule. Support is derived from the observed disinhibitory effects of a 10-second unpaired click/flash stimulus and of unsignaled, response-independent pellets that were presented while the animals were responding under the same schedule.     

The role of the instrumental contingency in the motivational control of performance

In four experiments we investigated an irrelevant incentive effect based upon a transition from hunger to thirst. Hungry rats were trained to lever press either for sucrose solution or for food pellets before performance was tested in extinction while they were thirsty. Reinforcer-specific motivational control was found in the first experiment in that the animals pressed the lever more on tests following training with the sucrose solution rather than with food pellets. Moreover, this effect was seen only when testing was conducted following water, but not following food deprivation. The outcome of the remaining experiments suggests that this motivational control is not mediated by the instrumental contingency between lever pressing and the sucrose reinforcer during training. In these studies lever pressing and chain pulling were reinforced concurrently, one with sucrose and the other with food pellets, in order to equate the noninstrumental functions of the incentives. Following this training, lever pressing in extinction under thirst was unaffected by the type of incentive used as its reinforcer during training.     

Altering "motivational" variables alters induction produced by upcoming food-pellet reinforcement

Previous research has demonstrated that rats will increase their rates of lever pressing for sucrose rewards in the first half of an experimental session when food pellets, rather than the same sucrose, continually serve as the reward in the second half of the session. This effect has been coined induction, and the present study investigated whether it could be altered by altering "motivational" variables. Experiment 1 manipulated subjects' motivation by altering, across conditions, their level of food deprivation. Predictably, the size of induction varied directly with level of deprivation. Experiments 2 and 3 manipulated subjects' motivation by feeding them food pellets and sucrose, respectively, prior to their responding in the experimental session. These pre-session feedings decreased the size of the observed induction in both experiments. The results from the present study indicate that the size of induction is correlated with subjects' motivation to respond for the available reinforcers. They are also consistent with the idea that operant processes underlie the effect. The notion that induction might encompass the concept of "anticipation" is also discussed. Electronic Publication     

Stimulus- and response-reinforcer contingencies in autoshaping, operant, classical, and omission training procedures in rats

Separate groups of rats received 500 trials of lever-press training under autoshaping (food delivery followed 10-second lever presentations, or occurred immediately following a response); operant conditioning (responding was necessary for food delivery); and classical conditioning (food followed lever presentations regardless of responding). Each group then received 500 trials on an omission procedure in which food was omitted on trials with a response. Another group received 1000 trials on the omission procedure, and a fifth group, random control, received 1000 uncorrelated presentations of lever and food. The autoshaping, operant, and classical groups reached high response levels by the end of initial training. Acquisition was fastest in the autoshaping group. Responding remained consistently low in the control group. The omission group responded at a level between the control group and the other three groups. During omission training, responding in these three groups declined to the omission-group level. During omission training, the rats continued contacting the lever frequently after lever pressing had declined. Response maintenance under omission training seems not to require topographic similarity between the response and reinforcer-elicited consummatory behaviors.     

A comparison of operant licking and lever pressing in the rat

Two groups of rats were trained on a signaled, free-operant, avoidance procedure to lick or to lever press in order to avoid shock while water-deprived or satiated and, in the case of licking, while ingesting deionized water, isotonic saline, or 10% sucrose. The most effective avoidance licking occurred while the rats were water-deprived and ingesting 10% sucrose. Water deprivation level had no effect on lever pressing for shock avoidance. Two other groups of rats were operantly conditioned to lick or to lever press for food pellets while waterdeprived or satiated and, in the case of licking, while ingesting deionized water or 10% sucrose. The most effective licking for food reinforcement occurred while the rats were water-deprived and ingesting 10% sucrose. Water deprivation level had no effect on lever pressing for food reinforcement. The data indicated that effective operant licking must be supported by factors related to water regulation and taste palatability.     

Temporal control in a complex environment: An analysis of schedule-related behavior

Three experiments conducted in an automated ten-compartment chamber recorded collateral activities of rats reinforced for lever pressing on differential-reinforcement-of-low-rate schedules. In Experiment 1, the rate of lever pressing increased when stimulus support for collateral activities was removed, thus confirming earlier findings. However, there were no temporal or sequential patterns of collateral activities that predicted operant responding. In Experiment 2, the rate of lever pressing increased only if (a) access to all stimulus support for collateral activities was simultaneously prevented, and (b) the rat was forced to remain in the presence of the lever and food tray. The availability of any of the stimuli related to collateral activity was sufficient to keep lever-pressing rates from increasing. Experiment 3 examined collateral activities under a signaled differential-reinforcement-of-low-rate schedule. Preventing access to stimuli supporting collateral activities had little effect on stable lever pressing when the signal was maintained. When the signal was removed, collateral activities continued, but lever-pressing rates increased in three of the four rats and rates of food presentation declined in all rats. Hypotheses that collateral activities have (a) a timekeeping or discriminative function, or (b) directly inhibit operant responding were not supported. The results suggest that collateral activities may facilitate operant responding by simply removing the subject from the presence of reinforcement-related stimuli.     

Incentive learning following reinforcer devaluation is not conditional upon the motivational state during re-exposure

Three experiments analysed the effect of re-exposure to the reinforcer following aversion conditioning on instrumental performance. In the first experiment, groups of hungry and thirsty rats were trained to press a lever for sucrose, which was then followed by a single injection of lithium chloride (LiCl). On the following day, half the animals in each motivational condition received re-exposure to the sucrose solution; the remaining animals were not re-exposed. In a subsequent extinction test animals that had received re-exposure to the sucrose pressed less than animals that were not re-exposed. Moreover, the effect of re-exposure to the sucrose solution was similar following training under hunger and thirst. In the remaining studies, animals were trained to lever-press for sucrose while either hungry or thirsty. They were then injected with LiCl and re-exposed to the sucrose while either hungry or thirsty, i.e. in the same or different motivational state employed during training, or they were not re-exposed. Lever pressing was then tested in extinction in the training motivational state. As in the first experiment, re-exposure to the reinforcer after aversion conditioning enhanced the magnitude of the reinforcer devaluation effect. More importantly, re-exposure to the sucrose produced a comparable effect on instrumental performance, whether re-exposure was given under the same or different motivational state to that employed during training. These results suggest that the instrumental reinforcer devaluation effect depends upon a process of incentive learning, but that this process is not conditional upon the current motivational state of the animal.     

Context-dependent latent inhibition in taste aversion learning

The effects of taste stimulus preexposure, either in the presence or in the absence of a specific contextual cue consisting of a specific noise-producing bottle, upon the conditioning and testing of conditioned taste aversions to the taste (saccharin) plus context (noisy-bottle) compound stimulus were investigated. Four groups of rats were given preexposure trials (latent inhibition) to either: (1) novel saccharin in novel noisy bottles, (2) novel saccharin in familiar silent bottles, (3) familiar water in novel noisy bottles, (4) familiar water in familiar silent bottles, in six trials. During conditioning, saccharin was presented in the noisy bottles followed by lithium chloride for all the groups. At testing, saccharin was presented in the noisy bottles for both one-bottle and two-bottle tests of aversion. It was indicated that the conditioning decrement produced after both saccharin and noisy bottle preexposure was overwhelmingly greater than any produced after preexposure to the elements. These results, discussed in relation to current theories of latent inhibition and perceptual learning, further underline the overwhelming significance of exteroceptive contextual elements in conditioned taste aversions.     

Motivational control of instrumental performance: The role of prior experience of the reinforcer

In four experiments we investigated the conditions that are necessary for instrumental performance to adjust appropriately to a change in drive state. Rats were trained to press a lever and pull a chain concurrently, with one action being reinforced by sucrose solution and the other by food pellets. In Experiment 1 for animals that were hungry throughout training the rate of lever pressing in an extinction test under thirst was unaffected by the type of reinforcer produced by this action during training, even though the sucrose solution would maintain a higher rate than the food pellets during training under thirst. In contrast, animals that experienced the instrumental contingencies arranged by the concurrent schedule while thirsty at some point during training pressed the lever more during the extinction test under thirst when this action had been reinforced with the sucrose solution rather than the food pellets. The remaining three experiments demonstrated that for this incentive effect to occur it is sufficient that the sucrose solution be delivered non-contingently under thirst at some stage of training. Thus, it would appear that performance mediated by an instrumental contingency adjusts appropriately to reinforcer revaluation brought about by a drive shift only if the animals have had prior experience with the incentive under the test drive state. This observation was interpreted in terms of Tolman's “cathexis” theory of motivation.