The effect of the instrumental training contingency on susceptibility to reinforcer devaluation

Two experiments investigated performance of instrumental lever pressing by rats following post-conditioning devaluation of the sucrose reinforcer produced by establishing an aversion to it. In Experiment I rats responded less in an extinction test after being averted from the sucrose following training on a ratio schedule, but not following an equivalent amount of training on an interval schedule. This was true even though the devalued sucrose would not act as an effective reinforcer on either the ratio or interval schedule. Experiment II provided a further investigation of the insensitivity of interval responding to reinforcer devaluation by comparing test performance under simple extinction with responding when the devalued reinforcer was presented on either a response-contingent or non-contingent schedule during the test. Once again simple extinction performance was unaffected by prior reinforcer devaluation. Furthermore, neither non-contingent nor contingent presentations of the devalued reinforcer significantly depressed responding below the level seen in the extinction condition. Ratio, but not interval performance appears to be controlled by knowledge about the instrumental contingency that encodes specific properties of the training reinforcer.     

Discrimination between outcomes in instrumental learning: Effects of preexposure to the reinforcers

In two experiments rats received instrumental training with two response levers, one response being reinforced by sucrose solution and the other by sucrose pellets. Prior to a test session, on which both levers were made available in the absence of reinforcement, the rats were given free access to one of the reinforcers, a procedure known to reduce its value. It was found that the rats responded at a lower rate on the lever that had produced the now-devalued reinforcer, but that this effect was substantial only in rats that had received preexposure to the two reinforcers before instrumental training was begun (Experiment 1). Experiment 2 demonstrated that this effect was obtained only when presentations of the two reinforcers were presented according to an inter-mixed schedule during preexposure. It is suggested that this result constitutes an instance of the perceptual learning effect in which intermixed preexposure to similar events enhances their discriminability.     

The role of the instrumental contingency in the motivational control of performance

In four experiments we investigated an irrelevant incentive effect based upon a transition from hunger to thirst. Hungry rats were trained to lever press either for sucrose solution or for food pellets before performance was tested in extinction while they were thirsty. Reinforcer-specific motivational control was found in the first experiment in that the animals pressed the lever more on tests following training with the sucrose solution rather than with food pellets. Moreover, this effect was seen only when testing was conducted following water, but not following food deprivation. The outcome of the remaining experiments suggests that this motivational control is not mediated by the instrumental contingency between lever pressing and the sucrose reinforcer during training. In these studies lever pressing and chain pulling were reinforced concurrently, one with sucrose and the other with food pellets, in order to equate the noninstrumental functions of the incentives. Following this training, lever pressing in extinction under thirst was unaffected by the type of incentive used as its reinforcer during training.     

Signalling and incentive processes in instrumental reinforcer devaluation

We have previously reported that conditioning an aversion to the reinforcer using an isotonic lithium chloride (LiCl) solution following instrumental training reduces performance in a subsequent extinction test only if animals are re-exposed to the reinforcer prior to the test. Rescorla (1992), in contrast, reported an immediate devaluation effect using a hypertonic LiCl solution that did not depend upon re-exposure. In two experiments we examined the effect of using a hypertonic LiCl solution to condition the aversion to the reinforcer on subsequent instrumental performance in extinction, with and without re-exposure. In Experiment 1 thirsty rats were trained to press a lever for a sucrose solution before being injected with 0.6 M LiCl either immediately or after a delay. Half of the immediate and delay groups were then re-exposed to the sucrose in the absence of the lever, with the remainder being exposed to water. Contrary to the previously reported effects of isotonic LiCl, a hypertonic solution induced a reinforcer devaluation effect in all the immediately poisoned animals, which did not depend upon re-exposure to the reinforcer. In Experiment 2 the possibility that this devaluation effect was induced by the discomfort associated with the hypertonicity of the solution was assessed by replicating Experiment 1 but, in addition, using two immediately poisoned groups given the LiCl injection under anaesthesia. In the absence of anaesthesia, the devaluation effect observed without re-exposure to the reinforcer in Experiment 1 was replicated. When the injection was given under anaesthesia, however, a reinforcer devaluation effect was observed only in animals that were re-exposed to the reinforcer prior to the extinction test. These results were interpreted as evidence that a reinforcer devaluation effect induced by pairing the reinforcer with illness depends upon a process of incentive learning, whereas a devaluation effect mediated by learning a signalling relationship between the reinforcer and somatic discomfort does not.     

Irrelevant incentive learning during instrumental conditioning: The role of the drive-reinforcer and response-reinforcer relationships

Four experiments investigated the processes by which a motivationally-induced change in the value of the training reinforcer affects instrumental performance. Initially, thirsty rats were trained to lever press for either a sodium or non-sodium solution. In Experiment I sodium-trained rats responded faster in extinction following the induction of a sodium appetite, but not following either food or water deprivation. Thus, enhanced extinction performance depends upon the relevance of the training reinforcer to the test drive state. The remaining experiments examined the role of the instrumental contingency. Animals received response-contingent presentations of one solution alternated either within (Experiments II and III) or between sessions (Experiment IV) with non-contingent presentations of another solution. Neither procedure yielded convincing evidence that contingent sodium presentations generated more responding in extinction under a sodium appetite than did non-contingent sodium presentations. On the basis of these results, we argue that the instrumental contingency itself does not play a major role in this irrelevant incentive effect.     

Effect of reinforcer devaluation on discriminative control of instrumental behavior

Two experiments examined the effect of reinforcer devaluation on the ability of a discriminative stimulus (Sd) to control instrumental behavior in Sprague-Dawley rats. In Experiment 1 reinforcer devaluation reduced, but did not eliminate, the ability of the Sd to control performance of the original response and to transfer its control to a new response trained with the same reinforcer. The effect of devaluation was more complete in Experiment 2, in which the reinforcer was delivered directly into the oral cavity. However, retraining the response with a different reinforcer partially restored the ability of the Sd to control performance of that response. These results suggest that an Sd may not augment its trained responses when the reinforcer has been completely devalued but may promote responses with which it shares a reinforcer, as long as those responses are associated with some reinforcer that retains its value. The implications of these results for the way that discriminative stimuli control instrumental behavior are discussed.     

Incentive learning following reinforcer devaluation is not conditional upon the motivational state during re-exposure

Three experiments analysed the effect of re-exposure to the reinforcer following aversion conditioning on instrumental performance. In the first experiment, groups of hungry and thirsty rats were trained to press a lever for sucrose, which was then followed by a single injection of lithium chloride (LiCl). On the following day, half the animals in each motivational condition received re-exposure to the sucrose solution; the remaining animals were not re-exposed. In a subsequent extinction test animals that had received re-exposure to the sucrose pressed less than animals that were not re-exposed. Moreover, the effect of re-exposure to the sucrose solution was similar following training under hunger and thirst. In the remaining studies, animals were trained to lever-press for sucrose while either hungry or thirsty. They were then injected with LiCl and re-exposed to the sucrose while either hungry or thirsty, i.e. in the same or different motivational state employed during training, or they were not re-exposed. Lever pressing was then tested in extinction in the training motivational state. As in the first experiment, re-exposure to the reinforcer after aversion conditioning enhanced the magnitude of the reinforcer devaluation effect. More importantly, re-exposure to the sucrose produced a comparable effect on instrumental performance, whether re-exposure was given under the same or different motivational state to that employed during training. These results suggest that the instrumental reinforcer devaluation effect depends upon a process of incentive learning, but that this process is not conditional upon the current motivational state of the animal.     

Induction when rats lick for 1% liquid-sucrose reinforcement

Previous research reported that rats responding for 1% liquid-sucrose reinforcement when 32% sucrose reinforcement is upcoming will decrease their response rate (contrast) if licking is the dependent measure and increase their response rate (induction) if pressing a lever is the dependent measure. The present study investigated whether induction could be observed when licking served as the dependent measure and whether induction in lever pressing and contrast in licking behaviour could be concurrently observed. Experiment 1 found induction when rats licked to earn the rewards but consumed them at a location separate from the spout licked to earn them. Experiment 2 also found induction when rats earned (and consumed) rewards by licking the same spout throughout the session. Experiment 3 separately measured instrumental lever pressing for sucrose rewards and licking the sucrose during the reward period. We found that both measures increased for 1% sucrose when 32% sucrose reinforcement was upcoming. The present results indicate that the type of response is not the sole determinant of whether contrast or induction is observed. Rather, they suggest that other procedural details, such as the location of reinforcer delivery, influence which effect is observed. The results also indicate that associative processes underlie the appearance of induction in responding for 1% sucrose.     

Induction when rats lick for 1% liquid-sucrose reinforcement

Previous research reported that rats responding for 1% liquid-sucrose reinforcement when 32% sucrose reinforcement is upcoming will decrease their response rate (contrast) if licking is the dependent measure and increase their response rate (induction) if pressing a lever is the dependent measure. The present study investigated whether induction could be observed when licking served as the dependent measure and whether induction in lever pressing and contrast in licking behaviour could be concurrently observed. Experiment 1 found induction when rats licked to earn the rewards but consumed them at a location separate from the spout licked to earn them. Experiment 2 also found induction when rats earned (and consumed) rewards by licking the same spout throughout the session. Experiment 3 separately measured instrumental lever pressing for sucrose rewards and licking the sucrose during the reward period. We found that both measures increased for 1% sucrose when 32% sucrose reinforcement was upcoming. The present results indicate that the type of response is not the sole determinant of whether contrast or induction is observed. Rather, they suggest that other procedural details, such as the location of reinforcer delivery, influence which effect is observed. The results also indicate that associative processes underlie the appearance of induction in responding for 1% sucrose.     

Motivational control of instrumental performance: The role of prior experience of the reinforcer

In four experiments we investigated the conditions that are necessary for instrumental performance to adjust appropriately to a change in drive state. Rats were trained to press a lever and pull a chain concurrently, with one action being reinforced by sucrose solution and the other by food pellets. In Experiment 1 for animals that were hungry throughout training the rate of lever pressing in an extinction test under thirst was unaffected by the type of reinforcer produced by this action during training, even though the sucrose solution would maintain a higher rate than the food pellets during training under thirst. In contrast, animals that experienced the instrumental contingencies arranged by the concurrent schedule while thirsty at some point during training pressed the lever more during the extinction test under thirst when this action had been reinforced with the sucrose solution rather than the food pellets. The remaining three experiments demonstrated that for this incentive effect to occur it is sufficient that the sucrose solution be delivered non-contingently under thirst at some stage of training. Thus, it would appear that performance mediated by an instrumental contingency adjusts appropriately to reinforcer revaluation brought about by a drive shift only if the animals have had prior experience with the incentive under the test drive state. This observation was interpreted in terms of Tolman's “cathexis” theory of motivation.     

Irrelevant incentive learning during training on ratio and interval schedules

Two experiments investigated the effect of a motivationally-induced change in the value of the training reinforcer on instrumental performance. Initially, thirsty rats were trained to lever press for either a sodium or a potassium solution. Responding in an extinction test was then measured following the induction of sodium appetite. In Experiment I sodium-trained rats responded faster in a test given one day following the end of instrumental training. Furthermore, the relative size of this irrelevant incentive effect did not depend upon whether a ratio or interval schedule was employed during training. Delaying the test for eight days following the end of training abolished the difference between the test performance of sodium- and potassium-trained animals. Experiment II provided a further study of the effect of the training schedule when the introduction of the sodium reinforcer was delayed until responding was well established. Again the relative size of the difference between the performance of sodium- and potassium-trained animals was comparable following training on ratio and interval schedules. The insensitivity of this irrelevant incentive effect to the training contingency is in marked contrast to previous failures to detect an effect of reinforcer revaluation brought about by aversion conditioning following training on an interval schedule (Dickinson, Nicholas and Adams, 1983).     

Induction produced by upcoming food-pellet reinforcement: Effects on subsequent operant responding

Research has demonstrated that rats' rates of operant behavior maintained by 1% sucrose reinforcement in the first half of an experimental session are heightened when food-pellet reinforcers, rather than 1% sucrose, will be available in the second half. Experiment 1 showed that rats that had been displaying this positive induction effect acquired a new response more quickly when 1% sucrose was used to reinforce the novel response than did rats that had not been displaying induction. Experiment 2 showed that this enhanced acquisition remained even when the new response was learned in a new environment. Experiment 3 showed that unconditioned rates of making a new response did not differ between subjects that had or had not been displaying induction. Experiment 4 further showed that significantly different rates of responding on a novel task were not observed when that response was reinforced with a new, non-sucrose reinforcer. Together, the present results suggest that induction results in and/or from an increase in the reinforcing value of the 1% sucrose. As such, the present results have both theoretical and practical implications.     

Facilitation of instrumental behavior by a Pavlovian appetitive conditioned stimulus

Three experiments examined appetitive Pavlovian-instrumental interactions by presenting separately trained conditioned stimuli (CSs) during reinforced instrumental responding in rabbits. Intra-oral reinforcement was used to minimize interference from peripheral responses such as magazine approach. In experiment 1, the rabbits were first trained to perform an instrumental head-raising response for sucrose reward. A conditioned jaw movement response was then established to a 2-sec CS by pairing it with sucrose; a control stimulus was unpaired with sucrose. Instrumental responding maintained by a variable-interval 40-sec schedule was enhanced during 10-sec presentations of the paired, but not the unpaired, CS. Responding on a variable-ratio 15 schedule was unaffected except on trials on which the pre-CS baseline response rate was low; in such cases the paired CS caused a long-lasting acceleration of responding. Noncontingent presentation of the sucrose reinforcer itself briefly suppressed responding but had no long-term effect. In Experiment 2, a CS that had been conditioned at a 10-sec duration produced the same pattern of effects as in the first study, indicating that facilitation resulted from CS presentation rather than from the frustrative effects of non-reinforcement of the CS. In Experiment 3 an inhibitory CS blocked facilitation by the excitatory CS but did not itself affect instrumental responding. These results support the view that Pavlovian processes play a positive role in instrumental performance and suggest that previous findings of suppression by a short-duration CS reflect peripheral interference. The dependence of facilitation on the baseline level of responding is discussed in terms of associative and motivational theories of Pavlovian mediation.     

Toxicosis affects instrumental behaviour in rats

Three experiments examined the effect of toxicosis on instrumental responding. These studies were prompted by Morrison and Collyer's (1974, Experiment 1) finding that the induction of toxicosis after an instrumental conditioning session produces greater response suppression if the response is reinforced by a novel saccharin solution rather than familiar water during conditioning. Experiments 1 and 2 investigated whether this suppression was mediated by the Pavlovian contingency between the contextual cues and the saccharin solution or the instrumental relationship between the response and the reward. A role for the instrumental contingency was indicated by the greater suppression of the response producing novel saccharin rather than water when the context of both responses was equally associated with the saccharin and illness. Experiment 3 found that extinction of the aversion to a novel reinforcer following aversive conditioning would re-establish an action previously associated with that reinforcer, in contrast to an action whose reinforcer remained aversive. This result was a further indication that the instrumental contingency between the response and reward contributes to response suppression.     

Analysis of treatments to alleviate forgetting in rats

Two experiments with rats investigated the effectiveness of prior-cuing treatments for alleviating forgetting of aversive conditioning. The aim was to see which retrieval cues would be most effective within different contexts. Experiment 1 examined the contexts of classical fear conditioning and instrumental avoidance training. The results indicated that the response components were sufficient to reinstate avoidance training, whereas the unconditioned stimulus (US) was most effective for classical fear conditioning. In Experiment 1, the reinforcer per se was ineffective in reinstating instrumental avoidance training. Experiment 2 manipulated the training context and found that the US could be made an effective prior-cuing treatment for instrumental training if classical conditioning components were more prevalent during training. These results are interpreted to mean that a "critical context" must be reinstated by the cuing treatment if this treatment is to promote retrieval of the memory.     

Instrumental performance following reinforcer devaluation depends upon incentive learning

Conditioning an aversion to the reinforcer following instrumental training reduces performance in a subsequent extinction test. Three experiments examined whether this reinforcer-devaluation effect depends upon experience with the devalued reinforcer prior to the extinction test. In Experiments 1 and 2 thirsty rats were trained to press a lever for sucrose solution in a single session. All animals then received an injection of lithium chloride (LiCl) either immediately following the session or after a delay of 6 hr. On the next day either the sucrose solution or water was presented non-contingently either in the operant chamber without the lever present or in a separate drinking cage. In a subsequent extinction test only the animals that had received immediate LiCl and re-exposure to non-contingent sucrose pressed less than those in the delayed-LiCl control groups. Experiment 3 demonstrated that this difference depended, at least in part, on post-conditioning exposure to a contingent reinforcer. Lever pressing and chain pulling were reinforced concurrently with either a sucrose or a sodium chloride solution in a single session immediately before the administration of LiCl. All animals then received non-contingent presentations of one of the reinforcers in the absence of both manipulanda. Finally, performance of both actions was assessed in an extinction test. Re-exposure to a reinforcer produced a relative reduction in the performance of its associated action on test. These results are interpreted as evidence that the instrumental reinforcer devaluation effect depends upon a process of incentive learning.     

Relations between Pavlovian-instrumental transfer and reinforcer devaluation

Relations between posttraining reinforcer devaluation and Pavlovian-instrumental transfer were examined in 2 experiments. When a single reinforcer was used, extended training of the instrumental response increased transfer but reduced devaluation effects. When multiple instrumental reinforcers were used, both reinforcer-specific transfer and devaluation effects were less influenced by the amount of instrumental training. Finally, although reinforcer devaluation decreased both Pavlovian conditioned responses and baseline instrumental responding, it had no effect on either single-reinforcer or reinforcer-specific transfer. These results indicate that transfer and reinforcer devaluation can reflect different aspects of associative learning and that the nature of associative learning can be influenced by parameters such as the amount of training and the use of multiple reinforcers.     

Incentive learning and the motivational control of instrumental performance

Hungry rats were trained to press a lever and pull a chain concurrently, with one action being reinforced with a sucrose solution and the other with food pellets. In addition, in the first two experiments all animals experienced non-contingent presentations of the two incentives in the absence of the operant manipulanda while either thirsty or hungry and either before (Experiment 1A) or after (Experiment 1B) the instrumental training. When lever pressing was assessed subsequently in extinction under thirst, the animals pressed at a relatively high rate only if (1) this action had been reinforced with the sucrose solution rather than the food pellets during training and (2) they had received the non-contingent presentations of the sucrose solution and food pellets on days on which they were thirsty rather than hungry. A third experiment demonstrated that non-contingent exposure to the sucrose solution alone, but not to water under thirst was sufficient to bring about this type of motivational control of instrumental performance.     

The control of appetitive instrumental responding does not depend on classical conditioning to the discriminative stimulus

In Experiment I, rats were exposed to a classical relationship between a clicker-light compound and response-independent food. Conditioning to the light was blocked if the clicker had previously served as a classical signal for food, but not if it had been established as a discriminative stimulus for food-reinforced lever pressing. In Experiment II, a tone-light compound served as a discriminative stimulus for lever pressing. Control by the light was blocked if the tone was independently trained as a discriminative stimulus, but not if it was trained as a classical signal for response-independent food. These results suggest that discriminative stimuli do not come to control appetitive instrumental responding by virtue of their implicit classical relationship to the instrumental reinforcer.     

Aggression as a reinforcer: operant behavior in the mouse-killing rat

Two experiments examined mouse killing as a reinforcer of key pressing by rats that killed mice. In Experiment I, mouse-killing rats performed the key-pressing response when each press was reinforced with presentation of a mouse. Offered a choice between a key that yielded presentation of mice and one that did not, the rats preferred the key that yielded mice. When the contingency was reversed, the rats preferred the other key and continued to kill mice. In Experiment II, mouse-killing rats that did not kill rat pups performed a key-pressing response reinforced with presentation of mice on a variable-interval schedule. In tests for responding reinforced on that schedule with presentation of normal mice, anesthetized mice, dead mice, or rat pups, these rats that killed mice but not rat pups exhibited a decline in response rate when rat pups were the reinforcer. Altering the condition of the mice did not significantly affect performance.