A note on depression of instrumental responding after one trial of outcome devaluation

After a response has been rewarded with an outcome, subsequent devaluation of that outcome by the toxin LiCl reduces responding. Balleine and Dickinson (1991) argued that this effect depends on incentive learning, which requires reexposure to the outcome after its pairing with the toxin. However, Rescorla (1992) reported that a single pairing of the outcome with LiCl was sufficient to depress responding even without reexposure. This result has been attributed by Balleine and Dickinson (1992) to the use of a hypertonic solution. Two experiments are reported here which replicate the prior single trial effect using an equivalent level of toxin in an isotonic solution.     

Overshadowing of a stimulus-reinforcer association by an instrumental response

In two experiments, rats were first exposed to pairings of a clicker and food; they were subsequently, in order to measure the effectiveness of the clicker as a conditioned reinforcer, given the opportunity to press a lever which turned the clicker on. For one group of animals the food originally delivered in the presence of the clicker had been contingent on their performance of an instrumental response (running in a running wheel); for a second the contingency between clicker and food had been purely classical. Although the actual correlation between clicker and food was identical for the two groups, the clicker was a less effective conditioned reinforcer for the first group than for the second. In a third experiment, all animals were initially required to run to obtain food in the presence of the clicker, but one group received additional trials on which food was delivered contingent on running in the absence of the clicker. This group showed less tendency to lever press for the clicker than a second group that had received free food on trials when the clicker was not presented. The results of all three experiments suggest that conditioning to the clicker could be overshadowed if the occurrence of food was more reliably predicted by the execution of an instrumental running response; they thus support the view that instrumental conditioning depends on the establishment of an association between response and reinforcer similar to the association between stimulus and reinforcer underlying classical conditioning.     

The control of instrumental action following outcome devaluation in young children aged between 1 and 4 years

To determine the role of action-outcome learning in the control of young children's instrumental behavior, the authors trained 18- to 48-month-olds to manipulate visual icons on a touch-sensitive display to obtain different types of video clips as outcomes. Subsequently, one of the outcomes was devalued by repeated exposure, and children's propensity to perform the trained actions was tested in extinction. On test, children with a mean age greater than 2.5 years performed the action trained with the devalued outcome less than those trained with the still-valued outcome, thereby demonstrating that their actions were mediated by action-outcome learning. By contrast, the instrumental responses of younger children (mean age<2 years) were resistant to outcome devaluation and may have been elicited directly by the icons associated with each response, rather than mediated by a specific action-outcome expectation.     

Effect of reinforcer devaluation on discriminative control of instrumental behavior

Two experiments examined the effect of reinforcer devaluation on the ability of a discriminative stimulus (Sd) to control instrumental behavior in Sprague-Dawley rats. In Experiment 1 reinforcer devaluation reduced, but did not eliminate, the ability of the Sd to control performance of the original response and to transfer its control to a new response trained with the same reinforcer. The effect of devaluation was more complete in Experiment 2, in which the reinforcer was delivered directly into the oral cavity. However, retraining the response with a different reinforcer partially restored the ability of the Sd to control performance of that response. These results suggest that an Sd may not augment its trained responses when the reinforcer has been completely devalued but may promote responses with which it shares a reinforcer, as long as those responses are associated with some reinforcer that retains its value. The implications of these results for the way that discriminative stimuli control instrumental behavior are discussed.     

Variations in the sensitivity of instrumental responding to reinforcer devaluation

In five experiments hungry rats were trained to make a lever press response for a sucrose reinforcer. That sucrose was subsequently devalued by conditioning a food-aversion to it, and the ability of the rats to integrate knowledge about the instrumental contingency with that gained from aversion training was assessed in an extinction test. Experiment I showed successful integration following limited but not extended instrumental training. Experiment II suggested that the crucial factor was the spacing of training; successful integration was seen after massed but not distributed training. The third experiment implicated distributed experience with the reinforcer, rather than distributed response practice, in failures of integration. Experiment IV showed that if the distribution of food-aversion learning was dissimilar to that of instrumental training then a failure of integration could result; this finding was able to account for the distribution of training effects seen in previous studies, but not the effect of extended training. Experiment V replicated the extended training effect seen in Experiment I, and provided evidence that this may reflect the degree of exposure to the reinforcer rather than the extent of response practice.     

Fear as a discriminative stimulus for an appetitive instrumental response

Two experiments tested the hypothesis that anticipation of shock could be established as a discriminative stimulus for an appetitive instrumental response. In multiphase experiments, bar pressing for food was brought under the discriminative control of intermittent and gradually increasing electric shock. In a second phase, tones were estalished as either a CS+, or CS? for shock. Subsequently, the CSs were introduced on to the operant baseline. Animals trained with shock as the S^D showed an increase in responding to the CS+ and a slight decrease to the CS?. Conversely, animals trained with shock as the S^Δ showed decreased responding to the CS+ and slight increase to the CS?. These findings are seen as supportive of the Discriminative Stimulus hypothesis of learned resistance to punishment.     

The effect of the instrumental training contingency on susceptibility to reinforcer devaluation

Two experiments investigated performance of instrumental lever pressing by rats following post-conditioning devaluation of the sucrose reinforcer produced by establishing an aversion to it. In Experiment I rats responded less in an extinction test after being averted from the sucrose following training on a ratio schedule, but not following an equivalent amount of training on an interval schedule. This was true even though the devalued sucrose would not act as an effective reinforcer on either the ratio or interval schedule. Experiment II provided a further investigation of the insensitivity of interval responding to reinforcer devaluation by comparing test performance under simple extinction with responding when the devalued reinforcer was presented on either a response-contingent or non-contingent schedule during the test. Once again simple extinction performance was unaffected by prior reinforcer devaluation. Furthermore, neither non-contingent nor contingent presentations of the devalued reinforcer significantly depressed responding below the level seen in the extinction condition. Ratio, but not interval performance appears to be controlled by knowledge about the instrumental contingency that encodes specific properties of the training reinforcer.     

Acquisition of instrumental conditioned reinforcement is resistant to the devaluation of the unconditioned stimulus

The associative mechanisms responsible for the efficacy of Pavlovian stimuli during first- and second-order conditioning have been extensively studied, but little is known about the representations underlying instrumental conditioned reinforcement. The present study investigated the associative structure underlying conditioned reinforcement, by employing an unconditioned stimulus (US) devaluation procedure on a commonly used instrumental task: the acquisition of a new response with conditioned reinforcement. Whilst US-directed behaviour was abolished following devaluation, the conditioned stimulus acting as a conditioned reinforcer supported the acquisition of instrumental responding. In this preparation then, the conditioned reinforcer appears to be impervious to devaluation of its associated US, suggesting that the underlying representation maintaining behaviour is independent of the current value of the US and may reflect the activation of a central appetitive motivational state.     

The effect of exercise on the selective performance of an instrumental response

Dogs were used to condition two instrumental reactions—bending of the left or of the right forepaw respectively. In the experimental situation a dish containing food was presented to the dog at a distance but could be drawn close and retained by lifting the forepaw to which (left or right) a lever had been attached. If both paws were simultaneously attached to levers, the dog had to choose between two reactions. Preliminary trials showed a right paw preference in spite of the equal effectiveness (food reinforcement) of the left paw. The aim of this experiment was to alter this right paw preference by a paradigm of differential practice with left paw dominance. The plan was to have ten training sessions followed by one test with choice. According to the probabilities, P_left = 0.8, P_right = 0.2, a random table was used to determine the order of the right-left paw attachments to the levers during training. In all, 1,000 training sessions and 100 tests with choice were carried out in each of three dogs. According to the training program, in every 100 training sessions the left paw had to perform the reaction a total of 80 times and 20 times by the right paw. An absolute alteration in paw preference occurred P_lelt = 1.0) even though both forepaws were equally effective in obtaining food reinforcement on the test trials. The present results indicate that not only the Law of Effect but Exercise is also important in instrumental conditioning.