The role of the instrumental contingency in the motivational control of performance

In four experiments we investigated an irrelevant incentive effect based upon a transition from hunger to thirst. Hungry rats were trained to lever press either for sucrose solution or for food pellets before performance was tested in extinction while they were thirsty. Reinforcer-specific motivational control was found in the first experiment in that the animals pressed the lever more on tests following training with the sucrose solution rather than with food pellets. Moreover, this effect was seen only when testing was conducted following water, but not following food deprivation. The outcome of the remaining experiments suggests that this motivational control is not mediated by the instrumental contingency between lever pressing and the sucrose reinforcer during training. In these studies lever pressing and chain pulling were reinforced concurrently, one with sucrose and the other with food pellets, in order to equate the noninstrumental functions of the incentives. Following this training, lever pressing in extinction under thirst was unaffected by the type of incentive used as its reinforcer during training.     

Orbital prefrontal cortex and guidance of instrumental behavior of rats by visuospatial stimuli predicting reward magnitude

The orbital prefrontal cortex (OPFC) is part of a circuitry mediating the perception of reward and the initiation of adaptive behavioral responses. We investigated whether the OPFC is involved in guidance of the speed of instrumental behavior by visuospatial stimuli predictive of different reward magnitudes. Unoperated rats, sham-lesioned rats, and rats with bilateral lesions of the OPFC by N-methyl-D-aspartate (NMDA) were trained in a visuospatial discrimination task. The task required a lever press on the illuminated lever of two available to obtain a food reward. Different reward magnitudes were permanently assigned to lever presses to respective sides of the operant chamber; that is, responses to one lever (e.g., the left one) were always rewarded with one pellet and responses to the other lever with five pellets. On each trial, the position of the illuminated lever was pseudorandomly determined in advance. Results revealed that OPFC lesions did not impair acquisition of the task, as the speed of conditioned responses was significantly shorter with expectancy of a high reward magnitude. In addition, during reversal, shift and reshift of lever position–reward magnitude contingencies and under extinction conditions, performance of the OPFC-lesioned and control groups did not differ. It is concluded that the OPFC in rats might not be critical for adapting behavioral responses to changes of stimulus–reward magnitude contingencies signaled by visuospatial cues.     

Motivational control of instrumental performance following a shift from thirst to hunger

Thirsty rats were trained to press a lever for either a sucrose solution or saline before performance was tested in extinction while the animals were either hungry alone or experiencing both hunger and a sodium appetite. Reinforcer-specific motivational control was observed in that the animals trained with the sucrose solution pressed more than those trained with the saline when they were tested hungry, but not when they were tested under combined hunger and sodium appetite. In order to assess the role of a Pavlovian incentive process in this effect, thirsty animals received non-contingent pairings of one stimulus with the sucrose solution and another with saline in the second experiment. In an extinction test the sucrose stimulus augmented lever pressing relative to the saline stimulus when the animals were hungry, but not when they were thirsty. In the subsequent experiments the contribution of the Pavlovian process was equated by giving concurrent training with both incentives. Lever pressing and chain pulling were reinforced concurrently, one with the sucrose solution and the other with saline, while the animals were thirsty. Once again, the animals pressed more in extinction if this action had been trained with the sucrose solution rather than the saline, but only if they were hungry rather than thirsty. Thus, instrumental performance across a thirst-to-hunger shift can also be controlled by an instrumental incentive process. The direct engagement of the instrumental process by this motivational shift contrasts to the absence of such control following a hunger-to-thirst transition (Dickinson & Dawson, 1987a), a fact attributed to the asymmetrical motivational interactions produced by water and food deprivation.     

动物信号追踪和目标追踪行为及其神经机制

信号追踪和目标追踪的实质是条件刺激和非条件刺激结合诱导的条件性接近反应.条件刺激的预测作用和诱因作用是信号追踪和目标追踪的重要机制;伏隔核、中央杏仁核和前扣带回在信号追踪和目标追踪过程中具有重要作用;然而,大脑皮层和海马损伤对两者影响较小.DA功能降低可损害信号追踪获得和表达;而脑内DA含量增加可提高目标追踪.未来的研究中,应该规范条件刺激和非条件刺激的类型和设置方法,针对不同的研究目的个别地或同时地测量信号追踪和目标追踪.     

Incentive learning and the motivational control of instrumental performance

Hungry rats were trained to press a lever and pull a chain concurrently, with one action being reinforced with a sucrose solution and the other with food pellets. In addition, in the first two experiments all animals experienced non-contingent presentations of the two incentives in the absence of the operant manipulanda while either thirsty or hungry and either before (Experiment 1A) or after (Experiment 1B) the instrumental training. When lever pressing was assessed subsequently in extinction under thirst, the animals pressed at a relatively high rate only if (1) this action had been reinforced with the sucrose solution rather than the food pellets during training and (2) they had received the non-contingent presentations of the sucrose solution and food pellets on days on which they were thirsty rather than hungry. A third experiment demonstrated that non-contingent exposure to the sucrose solution alone, but not to water under thirst was sufficient to bring about this type of motivational control of instrumental performance.     

Context blocking in rat autoshaping: Sign-tracking versus goal-tracking

Prior experience of unsignaled food can interfere with subsequent acquisition by birds of autoshaped key-pecking at a signal light. This has been understood to indicate that unsignaled food results in context conditioning, which blocks subsequent learning about the keylight-food relationship. In the present experiment with rats lever insertion as the conditioned stimulus (CS) preceded sucrose delivery as the unconditioned stimulus (US). Half the rats initially received unsignaled USs in the conditioning context, while the remainder did not. Both lever-presses (sign-tracking) and magazine-entries (goal-tracking) were recorded. Under immediate reinforcement conditions, prior unsignaled US interfered with sign-tracking, but had no effect on goal-tracking. In two further groups, a trace condition prevented development of sign-tracking. In this case, prior context conditioning interfered with goal-tracking. These results suggest that interference with sign-tracking may reflect response competition, while interference with goal-tracking under trace conditions may reflect failure to acquire a CS-US association.     

Omission Learning after Instrumental Pretraining

Hungry rats were pretrained to press two levers on a concurrent schedule for food pellets. Following either limited or extended pretraining, presentations of a sucrose solution were programmed on a random time schedule while the animals continued pressing on a concurrent schedule for food pellets. Presses on one of the levers-the omission lever-postponed deliveries of the sucrose solution scheduled to occur within a fixed period of time following the press, whereas presses on the other, control lever had no effect on the random time contingency. The animals pressed less frequently on the omission lever than on the control lever following limited pretraining but failed to discriminate between the two levers following extended pretraining. The insensitivity to the omission contingency produced by extended pretraining was due to either the number of presses performed during the initial training or the number of reinforced presses, rather than the number of reinforcers received. Finally, the insensitivity of performance on the omission lever to sucrose devaluation suggests that adaptation tothis negative contingency was mediated by an inhibitory stimulus-response association.     

Apparent Incentive Contrast Effects in Autoshaping with Rats

The effects of shifts in reward quality and quantity on Pavlovian acquisition were studied in rats. In Experiment 1, animals preexposed to unsignaled food pellets, 10% sucrose solution, or home cage controls subsequently received autoshaping training (response-independent lever-pellet or lever-solution pairings, in three groups each). Unsignaled preexposure to sucrose solution facilitated autoshaping for pellets (relative to unshifted controls), whereas unsignaled preexposure to pellets retarded autoshaping for sucrose solution. In Experiment 2, unsignaled preexposure to 30% sucrose solution impaired acquisition reinforced by food pellets, relative to 2% solution. Using a choice procedure, Experiment 3 demonstrated that rats prefer pellets to either 2 or 10% sucrose solutions, but they prefer the 30% solution to the pellets. Experiment 4 demonstrated the facilitatory effect after an upward shift in reward magnitude rather than quality (from 1 to 12 pellets), but provided weaker evidence for retardation following a downward magnitude shift. Experiment 5 was similar to Experiment 4, except that animals received autoshaping training from the outset. No evidence of successive positive contrast was obtained, but there was a significant successive negative contrast effect. Moreover, extinction was faster after acquisition with 12 pellets rather than 1. These results suggest the presence of incentive contrast effects under Pavlovian training conditions.     

Incentive learning following reinforcer devaluation is not conditional upon the motivational state during re-exposure

Three experiments analysed the effect of re-exposure to the reinforcer following aversion conditioning on instrumental performance. In the first experiment, groups of hungry and thirsty rats were trained to press a lever for sucrose, which was then followed by a single injection of lithium chloride (LiCl). On the following day, half the animals in each motivational condition received re-exposure to the sucrose solution; the remaining animals were not re-exposed. In a subsequent extinction test animals that had received re-exposure to the sucrose pressed less than animals that were not re-exposed. Moreover, the effect of re-exposure to the sucrose solution was similar following training under hunger and thirst. In the remaining studies, animals were trained to lever-press for sucrose while either hungry or thirsty. They were then injected with LiCl and re-exposed to the sucrose while either hungry or thirsty, i.e. in the same or different motivational state employed during training, or they were not re-exposed. Lever pressing was then tested in extinction in the training motivational state. As in the first experiment, re-exposure to the reinforcer after aversion conditioning enhanced the magnitude of the reinforcer devaluation effect. More importantly, re-exposure to the sucrose produced a comparable effect on instrumental performance, whether re-exposure was given under the same or different motivational state to that employed during training. These results suggest that the instrumental reinforcer devaluation effect depends upon a process of incentive learning, but that this process is not conditional upon the current motivational state of the animal.     

Rapid acquisition of an auditory localization discrimination by rats

Acquisition of a sound localization discrimination by rats was investigated. Two loudspeakers were located outside an experimental enclosure containing two levers and a dipper feeder. In the same-side condition, responses on the lever nearest the sound-producing speaker were reinforced. Animals in this condition acquired the discrimination rapidly, generally within the first session. In the opposite-side condition, responses on the lever furthest from the sound-producing speaker were reinforced. Acquisition for animals in this condition began below the chance level (50% correct responses) and took on the order of 10 sessions to approach the final, high level. The course of acquisition in both cases appeared to depend upon an initial tendency of rats to respond on the lever nearest the source of sound in this situation. The rise-decay time of the 4-kHz tone burst signal clearly affected the performance level reached. It did not, however, affect the rate at which the discrimination was acquired.     

Post-Conditioning Devaluation of an Instrumental Reinforcer has no Effect on Extinction Performance

The extent to which a representation of the reinforcer controls an instrumental response can be assessed by studying the effect of post-conditioning changes in the reinforcer value. In the first experiment rats were trained to press a lever for sucrose pellets on a variable-interval (VI) schedule. The sucrose was subsequently devalued by pairing with Lithium Chloride (LiCl). This had no effect on lever pressing in extinction, although it profoundly reduced reacquisition responding and consumption. In Experiment II rats were trained to shuttle between the two distinctive chambers of a choice-box, in which lever pressing was reinforced in one chamber by sucrose and in the other chamber by food pellets programmed on independent VI schedules. A LiCl-induced taste-aversion was conditioned to the sucrose, and although this markedly affected reacquisition, extinction responding in the sucrose chamber and chamber preference were unaffected. These results indicate that instrumental performance can be independent of the current value of the reinforcer, and are discussed with reference to stimulus-response theory and second-order Pavlovian conditioning.     

Motivational control of instrumental performance: The role of prior experience of the reinforcer

In four experiments we investigated the conditions that are necessary for instrumental performance to adjust appropriately to a change in drive state. Rats were trained to press a lever and pull a chain concurrently, with one action being reinforced by sucrose solution and the other by food pellets. In Experiment 1 for animals that were hungry throughout training the rate of lever pressing in an extinction test under thirst was unaffected by the type of reinforcer produced by this action during training, even though the sucrose solution would maintain a higher rate than the food pellets during training under thirst. In contrast, animals that experienced the instrumental contingencies arranged by the concurrent schedule while thirsty at some point during training pressed the lever more during the extinction test under thirst when this action had been reinforced with the sucrose solution rather than the food pellets. The remaining three experiments demonstrated that for this incentive effect to occur it is sufficient that the sucrose solution be delivered non-contingently under thirst at some stage of training. Thus, it would appear that performance mediated by an instrumental contingency adjusts appropriately to reinforcer revaluation brought about by a drive shift only if the animals have had prior experience with the incentive under the test drive state. This observation was interpreted in terms of Tolman's “cathexis” theory of motivation.     

The Isolation of Motivational, Motoric, and Schedule Effects on Operant Performance: A Modeling Approach

Dissociating motoric and motivational effects of pharmacological manipulations on operant behavior is a substantial challenge. To address this problem, we applied a response‐bout analysis to data from rats trained to lever press for sucrose on variable‐interval (VI) schedules of reinforcement. Motoric, motivational, and schedule factors (effort requirement, deprivation level, and schedule requirements, respectively) were manipulated. Bout analysis found that interresponse times (IRTs) were described by a mixture of two exponential distributions, one characterizing IRTs within response bouts, another characterizing intervals between bouts. Increasing effort requirement lengthened the shortest IRT (the refractory period between responses). Adding a ratio requirement increased the length and density of response bouts. Both manipulations also decreased the bout‐initiation rate. In contrast, food deprivation only increased the bout‐initiation rate. Changes in the distribution of IRTs over time showed that responses during extinction were also emitted in bouts, and that the decrease in response rate was primarily due to progressively longer intervals between bouts. Taken together, these results suggest that changes in the refractory period indicate motoric effects, whereas selective alterations in bout initiation rate indicate incentive‐motivational effects. These findings support the use of response‐bout analyses to identify the influence of pharmacological manipulations on processes underlying operant performance.     

The role of response-contingent incentives in lithium chloride-mediated suppression of an operant response

Three experiments investigated what role a novel incentive plays in the development of operant response suppression mediated by lithium chloride. In all experiments animals were trained to press two levers under concurrent schedules of reinforcement. In Experiment 1 responding on one lever delivered a familiar incentive (food pellets), whereas responding on an alternative lever delivered a novel incentive (sucrose solution) prior to lithium chloride injections. If lithium was administered immediately after the instrumental session, the action associated with the novel, but not with the familiar, incentive was suppressed. By comparison, in a control group for which responding on both levers led to the familiar incentive, both actions were suppressed. Experiment 2 examined whether the novelty, rather than the sensory properties, of the incentive is crucial for observing performance suppression. It was found that animals familiarized with the “target” incentive were insensitive to aversion conditioning by lithium, in that there was no difference in response rates between the action that delivered the familiar incentive from that which earned the “target”. In contrast, if animals were unfamiliar with the “target” incentive at the time of aversion conditioning, they suppressed responding on the lever that was associated with the novel incentive but did not suppress responding on the lever associated with the familiar incentive. Experiment 3 investigated the mechanism underlying instrumental performance suppression. After the completion of concurrent lever press training, novel sucrose was introduced in conjunction with the pellets for responding on one lever; responding on the other lever continued to deliver only familiar pellets. Lithium injections were then administered either immediately following the sessions or several hours after the sessions. It was found that the rate of responding on the lever associated with the contingent delivery of sucrose was suppressed below that of the pellet-alone action. By comparison, if lithium injections were administered several hours following the session, an elevation in responding on the sucrose-plus-pellet lever was observed. The outcomes of all three experiments demonstrate not only that the novelty of an incentive is important in obtaining performance suppression, but also that a novel incentive can punish instrumental responding if it has been associated with toxicosis.     

Discrimination between reinforced action patterns in the rat

Laboratory rats were rewarded for face-washing, rearing, or scratching by being given the opportunity to press retractable levers for food reward. Yoked control animals received the same number of lever presentations and food rewards, but did not have to face-wash, rear, or scratch to obtain the levers. The experimental animals showed increases in number of bouts of reinforced target behavior above control levels, and the total amount of time spent face-washing increased when a 1.5-sec criterion for reinforced bout length was introduced. The activities in this experiment were made to serve a discriminative as well as an instrumental function, since the cue to tell the rat which lever to press for reward when the levers were presented was the activity that the rat had engaged in to obtain lever presentation. In two separate experiments high levels of discrimination between behaviors were obtained. Discrimination was worse following scratching than after other actions, and scratching also showed relatively poor instrumental conditioning. The relationship between Pavlovian and instrumental conditioning processes in this situation is discussed.     

吗啡对不同距离条件下大鼠信号追踪和目标追踪的影响

条件刺激短暂呈现并消失后, 奖赏立即呈现, 多次匹配后诱导出动物对条件刺激(信号追踪)或奖赏呈现装置如食盒(目标追踪)的接近。条件刺激与食盒间的距离是影响信号/目标追踪反应和损害联结学习的重要变量, 成瘾药物能够增加奖赏的诱因动机, 进而增加个体的奖赏寻求行为。距离能否通过损害联结学习而减弱成瘾药物的动机放大作用尚未见到报道。本实验采用autoshaping模型, 考察8、30和60 cm距离条件下吗啡处理对大鼠信号追踪和目标追踪的影响。结果发现：(1)信号追踪随距离增加而减少, 目标追踪对距离不敏感。(2)急性吗啡处理减少8、30和60 cm条件下信号追踪而增加8和60 cm条件下目标追踪, 慢性吗啡处理在8和30 cm条件下减少信号追踪增加目标追踪; 消退检测中, 吗啡前暴露减少8和60 cm条件下信号追踪而增加60 cm条件下目标追踪。(3)辨别反转学习中, 吗啡前暴露使30和60 cm条件下的大鼠偏爱旧信号、辨别力受损, 减少8、30和60 cm条件下大鼠对新信号的接触。这些结果提示, 距离较少影响吗啡的信号追踪抑制作用和目标追踪增强效应, 而易化吗啡前暴露对反转学习的损害。说明距离是易化成瘾药物对联结学习不利影响而非反转其动机放大作用的重要因素。     

Effects of negative incentive shifts in food reward on rats' consumption of concurrent ethanol solutions

Frustration stress, typically operationalized as the unexpected loss of reinforcement, has been shown to engender substance use. Abrupt reductions in reinforcer magnitude likely also function as frustration stressors. These negative incentive shifts were previously shown to produce tap‐ and sweetened‐water drinking in rats. The purpose of this study was to investigate whether these shifts in food reward would occasion oral ethanol self‐administration. Nine male Long‐Evans rats operated on a two‐component multiple fixed‐ratio schedule with signaled components producing either a large (4 pellets) or small (1pellet) reinforcer. Components were pseudorandomly arranged to present 4 transitions between past and upcoming reinforcer magnitudes: small‐to‐large, small‐to‐small, large‐to‐large, and large‐to‐small (negative incentive shift). Experiment 1 investigated the effects of negative incentive shifts on consumption of concurrent, freely available 10% sucrose, 10% sucrose plus 10% ethanol, and following sucrose fading, 10% ethanol. Experiment 2 entailed continuation of schedule contingencies with a dose manipulation of 4 ethanol concentrations (0, 5, 10, and 20%) to assess dose‐dependent differences in transition‐type control and consumption. A lever‐press extinction condition was then conducted with 10% ethanol availability. In this novel model of frustration stress, negative incentive shifts prompted ethanol self‐administration at each dose investigated, whereas the other transitions did not.     

Complex effects of NMDA receptor antagonist APV in the basolateral amygdala on acquisition of two-way avoidance reaction and long-term fear memory

Although much has been learned about the role of the amygdala in Pavlovian fear conditioning, relatively little is known about an involvement of this structure in more complex aversive learning, such as acquisition of an active avoidance reaction. In the present study, rats with a pretraining injection of the N-methyl-D-aspartate (NMDA) receptor antagonist, 2-amino-5-phosphonopentanoic acid (APV), into the basolateral amygdala (BLA) were found to be impaired in two-way active avoidance learning. During multitrial training in a shuttle box, the APV-injected rats were not different from the controls in sensitivity to shock or in acquisition of freezing to contextual cues. However, APV injection led to impaired retention of contextual fear when tested 48 h later, along with an attenuation of c-Fos expression in the amygdala. These results are consistent with the role of NMDA receptors of the BLA in long-term memory of fear, previously documented in Pavlovian conditioning paradigms. The APV-induced impairment in the active avoidance learning coincided with deficits in directionality of the escape reaction and in attention to conditioned stimuli. These data indicate that normal functioning of NMDA receptors in the basolateral amygdala is required during acquisition of adaptive instrumental responses in a shuttle box but is not necessary for acquisition of short-term contextual fear in this situation.     

Pavlovian processes in the motivational control of instrumental performance

Hungry rats were rewarded for pressing a lever on a multiple schedule. During one component the reward was a sucrose solution, whereas food pellets acted as the reward during another component. Lever pressing was never rewarded during the third component. When the drive state was switched from hunger to thirst and the animals tested in extinction, they pressed more in the presence of the component stimulus that had been associated with the sucrose reward during training. A similar effect was observed during the extinction test of a second study in which the component stimuli had signalled non-contingent presentations of either the sucrose or pellet rewards in the absence of the lever. This suggests that the instrumental irrelevant incentive effect observed in the first experiment was due, at least in part, to the Pavlovian relationship between the component stimuli and the reinforcers during training. In fact, when the size of the effects controlled by purely Pavlovian and supposedly instrumental contingencies was compared directly in the final study, no difference could be detected.     

Choice in the time-left procedure and in concurrent chains with a time-left terminal link.

In two experiments, rats chose between a standard fixed-duration food-associated stimulus and a stimulus whose duration was the time remaining to reinforcement in an elapsing comparison interval. In Experiment 1, 4 rats responded in a time-left procedure wherein a single initial-link variable-interval schedule set up two potential terminal links simultaneously. As time elapsed in the initial-link schedule, the choice was between a standard fixed-interval 30-s terminal link and a time-left terminal link whose programmed interval requirement equaled 90 s minus the elapsed time in the initial link. Rats generally responded more on the lever with the shortest programmed terminal-link duration, but the temporal parameters of the procedure were found to vary with response distributions. Contrary to previous reports, therefore, time-left data were well predicted by choice models that make no assumptions about animal timing. In Experiment 2, 8 rats responded on a concurrent-chains schedule with independent variable-interval initial links and a time-left terminal link in one of the choice schedules. On the time-left lever, the programmed terminal-link delay equaled 90 s minus the elapsed time in the time-left initial link. On the standard lever, terminal-link responses were reinforced according to a variable-interval schedule whose average value varied over four conditions. Relative time-left initial-link responses increased in the elapsing time-left initial-link schedule as the time-left terminal link became shorter relative to the standard terminal link. Scalar expectancy theory failed to predict the resultant data, but a modified version of the delay-reduction model made good predictions. An analysis of the elaboration of scalar expectancy theory for variable delays demonstrated that the model is poorly formulated for arithmetically distributed delays.