Instrumental responding following reinforcer devaluation

In two experiments, hungry rats were given instrumental lever-press training for an appetitive reinforcer and, in addition, were exposed to another type of food which was not contingent on lever pressing. In the first experiment, exposure to each type of food was on separate days, whereas in the second experiment rats were exposed to each type of food in strict alternation within each session. Subsequently, a food aversion was conditioned to the reinforcer for the experimental group and to the non-contingent food for the control group. In both experiments, animals with an aversion to the reinforcer responded less in an extinction test than animals with an aversion to the non-contingent food. Subsequent reacquisition tests confirmed that the aversion to the non-contingent food in the control group was of comparable strength with that to the reinforcer in the experimental group. The results were discussed in terms of whether the reinforcer is encoded in the associative structure set up by exposure to an instrumental contingency.     

Instrumental performance following a shift in primary motivation depends on incentive learning.

In 5 experiments the role of incentive learning in instrumental performance following a shift in primary motivation was examined. In Experiments 1 and 2 rats trained to perform an instrumental action reinforced by either pellets or maltodextrin when in a low-deprivation state were shifted to a high-deprivation state and tested in extinction. This shift in deprivation increased performance only if the animals had been exposed to the reinforcer in the high-deprivation state prior to instrumental training. Experiments 3, 4, and 5 examined the reverse shift training in a high-deprivation state and testing in a low-deprivation state, and found, similarly, that performance was only sensitive to this shift if animals were previously exposed to the reinforcer while in the low-deprivation state. These experiments support the conclusion that instrumental performance following revaluation of the reinforcer depends on a process of incentive learning.     

Incentive learning following reinforcer devaluation is not conditional upon the motivational state during re-exposure

Three experiments analysed the effect of re-exposure to the reinforcer following aversion conditioning on instrumental performance. In the first experiment, groups of hungry and thirsty rats were trained to press a lever for sucrose, which was then followed by a single injection of lithium chloride (LiCl). On the following day, half the animals in each motivational condition received re-exposure to the sucrose solution; the remaining animals were not re-exposed. In a subsequent extinction test animals that had received re-exposure to the sucrose pressed less than animals that were not re-exposed. Moreover, the effect of re-exposure to the sucrose solution was similar following training under hunger and thirst. In the remaining studies, animals were trained to lever-press for sucrose while either hungry or thirsty. They were then injected with LiCl and re-exposed to the sucrose while either hungry or thirsty, i.e. in the same or different motivational state employed during training, or they were not re-exposed. Lever pressing was then tested in extinction in the training motivational state. As in the first experiment, re-exposure to the reinforcer after aversion conditioning enhanced the magnitude of the reinforcer devaluation effect. More importantly, re-exposure to the sucrose produced a comparable effect on instrumental performance, whether re-exposure was given under the same or different motivational state to that employed during training. These results suggest that the instrumental reinforcer devaluation effect depends upon a process of incentive learning, but that this process is not conditional upon the current motivational state of the animal.     

Signalling and incentive processes in instrumental reinforcer devaluation

We have previously reported that conditioning an aversion to the reinforcer using an isotonic lithium chloride (LiCl) solution following instrumental training reduces performance in a subsequent extinction test only if animals are re-exposed to the reinforcer prior to the test. Rescorla (1992), in contrast, reported an immediate devaluation effect using a hypertonic LiCl solution that did not depend upon re-exposure. In two experiments we examined the effect of using a hypertonic LiCl solution to condition the aversion to the reinforcer on subsequent instrumental performance in extinction, with and without re-exposure. In Experiment 1 thirsty rats were trained to press a lever for a sucrose solution before being injected with 0.6 M LiCl either immediately or after a delay. Half of the immediate and delay groups were then re-exposed to the sucrose in the absence of the lever, with the remainder being exposed to water. Contrary to the previously reported effects of isotonic LiCl, a hypertonic solution induced a reinforcer devaluation effect in all the immediately poisoned animals, which did not depend upon re-exposure to the reinforcer. In Experiment 2 the possibility that this devaluation effect was induced by the discomfort associated with the hypertonicity of the solution was assessed by replicating Experiment 1 but, in addition, using two immediately poisoned groups given the LiCl injection under anaesthesia. In the absence of anaesthesia, the devaluation effect observed without re-exposure to the reinforcer in Experiment 1 was replicated. When the injection was given under anaesthesia, however, a reinforcer devaluation effect was observed only in animals that were re-exposed to the reinforcer prior to the extinction test. These results were interpreted as evidence that a reinforcer devaluation effect induced by pairing the reinforcer with illness depends upon a process of incentive learning, whereas a devaluation effect mediated by learning a signalling relationship between the reinforcer and somatic discomfort does not.     

Localized disruption of Narp in medial prefrontal cortex blocks reinforcer devaluation performance

Neuronal activity regulated pentraxin (Narp) is a secreted protein that regulates α-amino-3-hydroxyl-5-methyl-4-isoxazole-propionate receptors (AMPAR) aggregation and synaptogenesis. Mapping of Narp-positive neurons in brain has revealed it is prominently expressed in several limbic system projection pathways. Consistent with this localization pattern, Narp knockout mice show deficits in using the current value of a reinforcer to guide behavior, a critical function of the limbic system. To help assess whether this behavioral deficit is due to impairment of synaptogenesis during development or in modulating synaptic signaling in the mature brain, we have used a dominant negative Narp viral construct which blocks trafficking of endogenous Narp to axons. Focal injection of this viral construct into the medial prefrontal cortex (mPFC) of adult mice, a region containing Narp-positive projection neurons, blocked reinforcer devaluation. Thus, these results indicate that Narp released from mPFC neurons plays a key role in mediating synaptic changes underlying instrumental reinforcer devaluation.     

Relations between Pavlovian-instrumental transfer and reinforcer devaluation

Relations between posttraining reinforcer devaluation and Pavlovian-instrumental transfer were examined in 2 experiments. When a single reinforcer was used, extended training of the instrumental response increased transfer but reduced devaluation effects. When multiple instrumental reinforcers were used, both reinforcer-specific transfer and devaluation effects were less influenced by the amount of instrumental training. Finally, although reinforcer devaluation decreased both Pavlovian conditioned responses and baseline instrumental responding, it had no effect on either single-reinforcer or reinforcer-specific transfer. These results indicate that transfer and reinforcer devaluation can reflect different aspects of associative learning and that the nature of associative learning can be influenced by parameters such as the amount of training and the use of multiple reinforcers.     

Amount of training and cue-evoked taste-reactivity responding in reinforcer devaluation

In two experiments, rats received minimal (16) pairings of one auditory conditioned stimulus (CS) cue with a sucrose reinforcer, and extensive (112) pairings of another auditory CS with that reinforcer. After sucrose was devalued by pairing it with lithium chloride in some rats (Devalue groups) but not others (Maintain groups), taste reactivity (TR) and other responses to unflavored water were assessed in the presence of the auditory CSs alone. The minimally trained CS controlled substantially more evaluative TR responses than the extensively trained CS. Those TR responses were hedonic (positive) in the Maintain groups, but aversive (negative) in the Devalue groups. By contrast, food cup entry and other responses thought not to reflect evaluative taste processing were controlled more by the extensively trained cue. These responses were reduced by sucrose devaluation comparably, regardless of the amount of training. The results suggest rapid changes in the content of learning as conditioning proceeds. Early in training, CSs may be capable of activating preevaluative processing of an absent food reinforcer that includes information about its palatability, but that capability is lost as training proceeds.     

Effect of reinforcer devaluation on discriminative control of instrumental behavior

Two experiments examined the effect of reinforcer devaluation on the ability of a discriminative stimulus (Sd) to control instrumental behavior in Sprague-Dawley rats. In Experiment 1 reinforcer devaluation reduced, but did not eliminate, the ability of the Sd to control performance of the original response and to transfer its control to a new response trained with the same reinforcer. The effect of devaluation was more complete in Experiment 2, in which the reinforcer was delivered directly into the oral cavity. However, retraining the response with a different reinforcer partially restored the ability of the Sd to control performance of that response. These results suggest that an Sd may not augment its trained responses when the reinforcer has been completely devalued but may promote responses with which it shares a reinforcer, as long as those responses are associated with some reinforcer that retains its value. The implications of these results for the way that discriminative stimuli control instrumental behavior are discussed.     

Variations in the sensitivity of instrumental responding to reinforcer devaluation

In five experiments hungry rats were trained to make a lever press response for a sucrose reinforcer. That sucrose was subsequently devalued by conditioning a food-aversion to it, and the ability of the rats to integrate knowledge about the instrumental contingency with that gained from aversion training was assessed in an extinction test. Experiment I showed successful integration following limited but not extended instrumental training. Experiment II suggested that the crucial factor was the spacing of training; successful integration was seen after massed but not distributed training. The third experiment implicated distributed experience with the reinforcer, rather than distributed response practice, in failures of integration. Experiment IV showed that if the distribution of food-aversion learning was dissimilar to that of instrumental training then a failure of integration could result; this finding was able to account for the distribution of training effects seen in previous studies, but not the effect of extended training. Experiment V replicated the extended training effect seen in Experiment I, and provided evidence that this may reflect the degree of exposure to the reinforcer rather than the extent of response practice.     

The effect of the instrumental training contingency on susceptibility to reinforcer devaluation

Two experiments investigated performance of instrumental lever pressing by rats following post-conditioning devaluation of the sucrose reinforcer produced by establishing an aversion to it. In Experiment I rats responded less in an extinction test after being averted from the sucrose following training on a ratio schedule, but not following an equivalent amount of training on an interval schedule. This was true even though the devalued sucrose would not act as an effective reinforcer on either the ratio or interval schedule. Experiment II provided a further investigation of the insensitivity of interval responding to reinforcer devaluation by comparing test performance under simple extinction with responding when the devalued reinforcer was presented on either a response-contingent or non-contingent schedule during the test. Once again simple extinction performance was unaffected by prior reinforcer devaluation. Furthermore, neither non-contingent nor contingent presentations of the devalued reinforcer significantly depressed responding below the level seen in the extinction condition. Ratio, but not interval performance appears to be controlled by knowledge about the instrumental contingency that encodes specific properties of the training reinforcer.     

Instrumental performance on negative schedules

Three experiments examined the performance of rats pressing a lever for food reinforcement on a schedule in which high rates of response resulted in lowered rates of reinforcement (i.e. a schedule with a negative component). In Experiment 1, rats responded on a variable interval (VI) schedule with a conjoint component such that every 30 responses a reinforcement programmed by the VI schedule was cancelled. These subjects generally emitted a lower response rate than rats responding on a VI schedule yoked to the former subjects with respect to the delivery of reinforcement, although response rate differences were sometimes not large. Similar response-rate effects were obtained in Experiment 2 using a within-subject yoking procedure. In Experiment 3, reinforced interresponse times were matched on negative and VI schedules yoked in terms of reinforcement rate, and the response rate emitted in these conditions were similar. These results give support to theories of instrumental conditioning that stress the strengthening and shaping properties of reinforcement.     

Novelty as a reinforcer for position learning in children

The stimulus-familiarization-effect (SFE) paradigm, a reaction-time (RT) task based on a response to novelty procedure, was modified to assess response for novelty, i.e., a response-reinforcement sequence. Ninetysix kindergarten children were familiarized on one of two colored lights. Then Ss were confronted with a simple position discrimination task in which they could choose between two response buttons which produced either the previously familiarized light or a new (i.e., novel) light. Results showed that the choice response for the novel stimulus (NS) was facilitated by three variables: number of exposures to and duration per exposure of the familiar stimulus (FS), and degree of change provided by the NS relative to the FS. The parallel between the effective variables of this study and those which produce the SFE was noted, implicating an attention interpretation of the function of novelty as a reinforcer. The potential implications of attention for reinforcement theory and learning in general are discussed.     

Motor sequence learning performance in Parkinson's disease patients depends on the stage of disease

It is still unclear, whether patients with Parkinson's disease (PD) are impaired in the incidental learning of different motor sequences in short succession, although such a deficit might greatly impact their daily life. The aim of this study was thus to clarify the relation between disease parameters of PD and incidental motor learning of two different sequences in short succession. Results revealed that the PD patients were able to acquire two sequences in short succession but needed more time than healthy subjects. However, both the severity of axial manifestations, as assessed on a subsection of the Unified Parkinson's Disease Rating Scale III (UPDRS III) and the Hoehn and Yahr score, and the levodopa-equivalent dose (LED) were negatively correlated with the sequence learning performance. These findings indicate that, although PD patients are able to learn two sequences in short succession, they need more time and their overall sequence learning performance is strongly correlated with the stage of disease.     

Delayed matching-to-sample performance: Effects of relative reinforcer frequency and of signaled versus unsignaled reinforcer magnitudes

Six pigeons were trained on a delayed red-green matching-to-sample task that arranged four delays within sessions. Matching responses intermittently produced either 1.5-s access to food or 4.5-s access to food, and nonmatching responses produced either 1.5-s or 4.5-s blackout. Two phases were conducted: a signaled phase in which the reinforcer magnitudes (small and large) were signaled by houselights (positioned either on the left or right of the chamber), and an unsignaled phase in which there was no correlation between reinforcer magnitude and houselight position. In both phases, the relative frequency with which red and green matching responses produced food was varied across five values. Both matching accuracy and the sensitivity of performance to the distribution of reinforcers for matching responses decreased with increasing delays in both phases. In addition, accuracy and reinforcer sensitivity were significantly lower on signaled small-reinforcer trials compared with accuracy and sensitivity values on signaled large-reinforcer trials and on both types of unsignaled trials. These results are discussed in the context of research on both nonhuman animal and human memory.     

Pigeons' performance under differential reinforcement of low rate schedules depends upon the operant

Pigeons were reinforced with grain for pecking a key or depressing a foot treadle according to differential reinforcement of low rate (DRL) schedules. Birds which depressed a treadle performed efficiently on DRL schedules as high as DRL 35-sec; while birds reinforced for keypecking showed low efficiency under DRL 14-sec. While treadle pressing and keypecking differ along a number of dimensions (including force requirement of the operant and differences in temporal distributions of responses), the present results are consistent with an interpretation based on differences in the degree to which these two responses are elicited by periodic presentations of food.