Relations between Pavlovian-instrumental transfer and reinforcer devaluation

Relations between posttraining reinforcer devaluation and Pavlovian-instrumental transfer were examined in 2 experiments. When a single reinforcer was used, extended training of the instrumental response increased transfer but reduced devaluation effects. When multiple instrumental reinforcers were used, both reinforcer-specific transfer and devaluation effects were less influenced by the amount of instrumental training. Finally, although reinforcer devaluation decreased both Pavlovian conditioned responses and baseline instrumental responding, it had no effect on either single-reinforcer or reinforcer-specific transfer. These results indicate that transfer and reinforcer devaluation can reflect different aspects of associative learning and that the nature of associative learning can be influenced by parameters such as the amount of training and the use of multiple reinforcers.     

The role of contextual associations in producing the partial reinforcement acquisition deficit

Three conditioned suppression experiments with rats as subjects assessed the contributions of the conditioned stimulus (CS)-context and context-unconditioned stimulus (US) associations to the degraded stimulus control by the CS that is observed following partial reinforcement relative to continuous reinforcement training. In Experiment 1, posttraining associative deflation (i.e., extinction) of the training context after partial reinforcement restored responding to a level comparable to the one produced by continuous reinforcement. In Experiment 2, posttraining associative inflation of the context (achieved by administering unsignaled outcome presentations in the context) enhanced the detrimental effect of partial reinforcement. Experiment 3 found that the training context must be an effective competitor to produce the partial reinforcement acquisition deficit. When the context was down-modulated, the target regained behavioral control thereby demonstrating higher-order retrospective revaluation. The results are discussed in terms of retrospective revaluation, and are used to contrast the predictions of a performance-focused model with those of an acquisition-focused model.     

Effect of reinforcer devaluation on discriminative control of instrumental behavior

Two experiments examined the effect of reinforcer devaluation on the ability of a discriminative stimulus (Sd) to control instrumental behavior in Sprague-Dawley rats. In Experiment 1 reinforcer devaluation reduced, but did not eliminate, the ability of the Sd to control performance of the original response and to transfer its control to a new response trained with the same reinforcer. The effect of devaluation was more complete in Experiment 2, in which the reinforcer was delivered directly into the oral cavity. However, retraining the response with a different reinforcer partially restored the ability of the Sd to control performance of that response. These results suggest that an Sd may not augment its trained responses when the reinforcer has been completely devalued but may promote responses with which it shares a reinforcer, as long as those responses are associated with some reinforcer that retains its value. The implications of these results for the way that discriminative stimuli control instrumental behavior are discussed.     

Substitutability between conditioned and primary reinforcers in discrimination acquisition.

Rats and pigeons were trained on a series of reversals of a conditional simultaneous discrimination. The percentage of reinforcement for correct trials was varied across reversals. When nonreinforced correct trials produced the same feedback as incorrect trials, the number of errors to reach an acquisition criterion was greater for smaller percentages of reinforcement, but the number of reinforcers required was either approximately constant or smaller for the smaller percentages. When a stimulus paired with food (the conditioned reinforcer) was added on nonreinforced correct trials, both measures were substantially decreased. When the same stimulus was presented, but without a history of food pairing, learning rate was similar to when no stimulus was presented on nonreinforced trials. The results provide direct evidence that conditioned reinforcers may substitute, although imperfectly, for a primary reinforcer, and that pairing with the primary reinforcer is a necessary condition for such substitutability to occur.     

Blocking of the response-reinforcer association: Additional evidence

Response rates maintained by free-operant schedules are sometimes reduced when a brief signal is interposed between the response and reinforcer. Early interpretations of this rate reduction were in terms of associative competition between the response and signal, analogous to blocking and overshadowing effects obtained in Pavlovian conditioning. But subsequent evidence has disputed this interpretation by showing that the signal enhances response rate in several circumstances. The present study attempted to define the conditions determining the different effects by investigating the role of signal location in the delay-of-reinforcement interval. When it occurred immediately after a response, response rate was enhanced. When it occurred at the end of the delay interval, immediately preceding the reinforcer, the signal decreased response rate. A similar decremental effect occurred when the signal occurred in the middle of the delay interval, contiguous with neither the response nor the reinforcer. Similar blocking effects occurred with a response acquisition procedure. The results demonstrate that blocking effects do indeed occur in instrumental conditioning but that the usual procedure of using signals immediately contingent on responding confounds such blocking effects with the competing effects of conditioned reinforcement/marking.     

Evaluation of the relation between Pavlovian occasion-setting and instrumental discriminative stimuli: a blocking analysis

Experiments with rat subjects used a blocking design (A+, then AB+) to assess the relation between Pavlovian occasion-setting and instrumental discriminative stimuli. Prior conditioning of both associative and occasion-setting functions of A in a serial feature-positive procedure blocked acquisition of an instrumental discriminative function by a novel stimulus (B) trained in compound with A. However, neither prior conditioning of only an A-US (unconditioned stimulus) association nor prior conditioning of Stimulus A using a Pavlovian simultaneous feature-positive procedure, which does not endow A with an occasion-setting function, blocked acquisition of an instrumental discriminative function by B. Prior acquisition of an instrumental discriminative function by A blocked acquisition of a Pavlovian occasion-setting function by a novel stimulus (B) trained in compound with A, but did not block acquisition of an association between B and the US. These outcomes indicate that Pavlovian occasion-setting and instrumental discriminative properties of a stimulus are functionally equivalent and that both Pavlovian occasion-setting and instrumental discriminative properties of a stimulus are functionally independent of simple associative relations between that stimulus and other events.     

Differential effects of two ways of devaluing the unconditioned stimulus after Pavlovian appetitive conditioning

Three experiments with rat subjects investigated the effects of two methods of devaluing a food unconditioned stimulus (US) after pairings of an auditory conditioned stimulus (CS) with that US. Experiment 1 found no effect of postconditioning pairings of the food US with lithium chloride (LiCl) on general activity to a tone CS, even though those pairings substantially reduced food consumption. Experiments 2 and 3 compared the effects on conditioned responding of postconditioning pairings of food with LiCl and with high-speed rotation. In these experiments the general activity measure was supplemented by a detailed visual analysis of the rats' behavior. Experiment 2 found that food-rotation pairings had larger effects than food-LiCl pairings on general activity responding and on two detailed behavioral measures but that food-LiCl pairings had larger effects on food consumption and on one behavioral measure. Experiment 3 replicated the findings of Experiment 2 and found that the ability of the CS to serve as a reinforcer for second-order conditioning after US devaluation was reduced more by food-LiCl pairings.     

非条件刺激降低再评估对条件性恐惧消退的影响

诸多情绪障碍治疗的关键在于促进条件性恐惧反应的消退, 研究证明非条件刺激再评估能够引起恐惧反应的变化。本研究将非条件刺激降低再评估范式应用于消退训练, 以主观预期值、皮电反应作为恐惧反应指标, 考察恐惧习得后非条件刺激强度降低对条件性恐惧消退的影响, 并运用评价性条件作用探讨非条件刺激再评估的作用机制。结果表明：在消退阶段, 降低组对条件刺激的皮电反应显著低于控制组。同时评价性条件作用结果显示：经过非条件刺激再评估, 降低组较控制组对条件刺激的评价更为正性。说明非条件刺激降低再评估有效改变条件刺激的负性效价, 降低个体的恐惧反应、促进恐惧消退。     

Conditioning of the rabbit's nictitating membrane response to a CSA-CSB-US serial compound: manipulations of CSB's associative character.

Response acquisition to a trace conditioned stimulus (CSA) can be facilitated by insertion of a second stimulus (CSB) at the end of the trace interval just before the unconditioned stimulus (US). This effect may arise from serial mediation of trace conditioning, second-order conditioning, or both. Whereas serial mediation relies only on the presence of CSB, associative transfer relies on CSB's associative strength. In the present experiments, the presence of CSB was fixed, whereas CSB's associative strength was manipulated by (a) extinction of CSB, (b) latent inhibition of CSB, and (c) prior CSB-US pairings. In the first 2 cases, the level of responding to CSA was reduced in a fashion parallel to that of CSB. However, in the third case, partial blocking of conditioned response (CR) acquisition to CSA was observed. The results are discussed with reference to the role of associative transfer to both facilitating and blocking CR acquisition to CSA.     

Effect of telencephalon ablation on the reinforcing and eliciting properties of species-specific events in Betta splendens

In male Betta splendens, aggressive behavior is drastically attenuated following telencephalon ablation. Because instrumental training and Pavlovian conditioning experiments with intact fish have suggested that associative factors may play an important role in the performance of agonistic behaviors, the effect of ablation on instrumental learning and Pavlovian conditioning was studied. In Experiment 1, ablation had no effect on the learning of the instrumental tunnel-swimming response reinforced by mirror presentation (i.e., viewing a conspecific), although the mirror presentations in yoked-control groups elicited fewer responses in ablates than in normal and sham-operated control fish. Yoked controls further established that instrumental responding was maintained by the reinforcement contingency and was not merely the result of increased motor activity. Experiment 2 studied Pavlovian conditioning of the components of the agonistic display. Unconditioned fin erection, gill erection, and tail beating (i.e., unconditioned responses, URs) to the mirror US all were less frequent in ablates than in normals or shams. Of these, only gill cover erection showed evidence of true conditioning (i.e., conditioned responses; CRs) in which responses to the conditioned stimulus (CS) are due to the pairings of CS and US (unconditioned stimulus). However, ablates suffered no impairment of conditioned gill erections. Ablates performed fewer fin erections to the CS; however, fin erection responses were not due to CS-US pairings but were attributable to pseudoconditioning. These results are related to hypotheses postulating the involvement of learning mechanisms in ablation-produced deficits and normal aggressive behavior.     

Trace classical conditioning in rainbow trout (Oncorhynchus mykiss): what do they learn?

There are two main memory systems: declarative and procedural memory. Knowledge of these two systems in fish is scarce, and controlled laboratory studies are needed. Trace classical conditioning is an experimentally tractable model of declarative memory. We tested whether rainbow trout (Oncorhynchus mykiss) can learn by trace conditioning and form stimulus–stimulus, as opposed to stimulus–response, associations. We predicted that rainbow trout trained by trace conditioning would show appetitive behaviour (conditioned response; CR) towards the conditioned stimulus (CS; light), and that the CR would be sensitive to devaluation of the unconditioned stimulus (US; food). The learning group (L, N = 14) was trained on a CS + US contingency schedule with a trace interval of 3.4 s. The control group (CtrL, N = 4) was kept on a completely random schedule. The fish that learnt were further trained as either an experimental (L, N = 6) or a memory control (CtrM, N = 3) group. The L group had the US devalued. The CtrM group received only food. No fish in the CtrL group, but nine fish from the L group conditioned to the light. When tested, five L fish changed their CRs after US devaluation, indicating learning by stimulus–stimulus association of the light with the food. CtrM fish retained their original CRs. To the best of our knowledge, this experiment is the first to show that rainbow trout can learn by trace classical conditioning. The results indicate that the fish learnt by ‘facts-learning’ rather than by reflex acquisition in this study.     

Rapid reaquisition in conditioning of the rabbit's nictitating membrane response.

Reacquisition after extinction often appears faster than original acquisition. However, data from conditioned suppression studies indicate that this effect may arise from spontaneous recovery and reinstatement of unextinguished contextual stimuli related to the unconditioned stimulus (US). In the present experiments using the rabbit nictitating membrane preparation, spontaneous recovery was eradicated before reaquisition training. US contextual stimuli were controlled by retaining the US during extinction through explicit unpairings of the conditioned stimulus (CS) and US. Attempts were also made to drive the associative strength of the CS into the inhibitory region by differential conditioning and conditioned inhibition procedures. In all cases, reacquisition was very rapid in comparison with a rest control. The results are discussed with respect to their implications for CS and US processing models of conditioning.     

Facilitation of instrumental behavior by a Pavlovian appetitive conditioned stimulus

Three experiments examined appetitive Pavlovian-instrumental interactions by presenting separately trained conditioned stimuli (CSs) during reinforced instrumental responding in rabbits. Intra-oral reinforcement was used to minimize interference from peripheral responses such as magazine approach. In experiment 1, the rabbits were first trained to perform an instrumental head-raising response for sucrose reward. A conditioned jaw movement response was then established to a 2-sec CS by pairing it with sucrose; a control stimulus was unpaired with sucrose. Instrumental responding maintained by a variable-interval 40-sec schedule was enhanced during 10-sec presentations of the paired, but not the unpaired, CS. Responding on a variable-ratio 15 schedule was unaffected except on trials on which the pre-CS baseline response rate was low; in such cases the paired CS caused a long-lasting acceleration of responding. Noncontingent presentation of the sucrose reinforcer itself briefly suppressed responding but had no long-term effect. In Experiment 2, a CS that had been conditioned at a 10-sec duration produced the same pattern of effects as in the first study, indicating that facilitation resulted from CS presentation rather than from the frustrative effects of non-reinforcement of the CS. In Experiment 3 an inhibitory CS blocked facilitation by the excitatory CS but did not itself affect instrumental responding. These results support the view that Pavlovian processes play a positive role in instrumental performance and suggest that previous findings of suppression by a short-duration CS reflect peripheral interference. The dependence of facilitation on the baseline level of responding is discussed in terms of associative and motivational theories of Pavlovian mediation.     

Timing at the Start of Associative Learning

Some theories of associative learning imply that time plays a fundamental role in the acquisition process. Consistent with these theories, this paper presents evidence that the time from the onset of a conditioned stimulus (CS) until presentation of the unconditioned stimulus (US) is learned very rapidly at the start of training. We report two autoshaping studies and a study on aversive conditioning in goldfish in which we examine timing at the start of conditioning. We also review data from a number of other conditioning preparations, including fear-potentiated startle, appetitive conditioning in rats, and eyeblink conditioning in rabbits, that report conditioned response (CR) timing early in training. Acquisition speed and the very first expressions of conditioned responding often show sensitivity to the time of US presentation. In instances where temporal control is slowly expressed, it is likely due to performance factors, not to slow learning about time. In fact, the learning about time may be a necessary condition for associative learning.     

Response rates track the history of reinforcement times

When conditioning involves a consistent temporal relationship between the conditioned stimulus (CS) and unconditioned stimulus (US), the expression of conditioned responses within a trial peaks at the usual time of the US relative to the CS. Here we examine the temporal profile of responses during conditioning with variable CS-US intervals. We conditioned stimuli with either uniformly distributed or exponentially distributed random CS-US intervals. In the former case, the frequency of each CS-US interval within a specified range is uniform but the momentary probability of the US (the hazard function) increases as time elapses during the trial; with the latter distribution, short CS-US intervals are more frequent than longer intervals, but the momentary probability of the US is constant across time within the trial. We report that, in a magazine approach paradigm, rats' response rates remained stable as time elapses during the CS when the CS-US intervals were uniformly distributed, whereas their response rates declined when the CS-US intervals were exponentially distributed. In other words, the profile of responding during the CS matched the frequency distribution of the US times, not the momentary probability of the US during the CS. These results are inconsistent with real-time associative models, which predict that associative strength tracks the momentary probability of the US, but may provide support for timing models of conditioning in which conditioned responding is tied to remembered times of reinforcement.     

How transitions from nonrewarded to rewarded trials regulate responding in Pavlovian and instrumental learning following extensive acquisition training

In both discrimination learning and partial reinforcement, transitions may occur from nonrewarded to rewarded trials (NR transition). In discrimination learning, NR transitions may occur in two different stimulus alternatives (NR different transitions). In partial reward, NR transitions may occur in a single stimulus alternative (NR same transitions). Available instrumental learning data indicate that resistance to extinction is increased by both types of NR transitions following limited acquisition training. Following more extensive acquisition training, resistance to extinction appears to be increased by NR same transitions but not by NR different transitions. In Experiment 1, it was shown for the first time that following extensive acquisition training the effects of the two types of transitions are the same in the Pavlovian situation as in the instrumental situation. This finding indicates that on the current trial the rat remembered the conditioned stimulus (CS) and the unconditioned stimulus (US) from the prior trial and those memories along with the CS on the current trial become the signal for the current US. Experiments 2 and 3, which tested hypotheses about instrumental learning, identified why NR different transitions lose their capacity to promote vigorous responding following extensive acquisition training. This is because cues occurring on the rewarded trials of NR different transitions more validly signal reward than other situational cues and thus overshadow them. Finally, some implications of the present findings for understanding the role of NR different transitions in discrimination learning situations were discussed.     

The comparator hypothesis of conditioned response generation: manifest conditioned excitation and inhibition as a function of relative excitatory strengths of CS and conditioning context at the time of testing

In the present research water-deprived rats were used in a conditioned lick suppression paradigm to test and further develop Rescorla's (1968) contingency theory, which posits that excitatory associations are formed when a conditioned stimulus (CS) signals an increase in unconditioned stimulus (US) likelihood and that inhibitory associations develop when the CS signals a decrease in US likelihood. In Experiment 1 we found that responding to a CS varied inversely with the associative status of the context in which the CS was trained and that this response was unaltered when testing occurred in a distinctively dissimilar context with a different conditioning history, provided associative summation with the test context was minimized. These results suggest that manifest excitatory and inhibitory conditioned responding is modulated by the associative value of the training context rather than that of the test context. In Experiment 2 it was demonstrated that postconditioning decreases in the associative value of the CS training context reduced the effective inhibitory value of the CS even when testing occurred outside of the training context. Moreover, this contextual deflation effect was specific to the CS training context as opposed to any other excitatory context. Collectively, these studies support the comparator hypothesis, which states that conditioned responding is determined by a comparison of the associative strengths of the CS and its training context that occurs at the time of testing rather than at the time of conditioning. This implies that all associations are excitatory and that responding indicative of conditioned inhibition reflects a CS-US association that is below (or near) the associative strength of its comparator stimulus. It is suggested that response rules which go beyond a monotonic relation between associative value and response strength can partially relieve learning theories of their explanatory burdens, thereby allowing for simpler models of acquisition.     

Bidirectional Instrumental Conditioning

Three experiments examined bidirectional instrumental conditioning by training hungry rats to push a pole in one direction for food pellets and in the other for either a sugar or a starch solution. In the first study we examined whether the animals learned about the actionreinforcer relations using a specific satiety procedure. Prefeeding one type of reinforcer before an extinction test selectively depressed the performance of the action that had been paired with this reinforcer during training. The second experiment investigated the sensitivity of the bidirectional actions to variations in the action-reinforcer contingencies. When the instrumental contingency was degraded by presenting unpaired reinforcers, the animals pushed less in the direction that was paired with the reinforcer type that was the same as the non-contiguous one. A third study revealed that increasing the rate of reinforcement for one action enhanced its rate of performance without significantly affecting the performance of the other action. We conclude that the effects of reinforcer devaluation, the action-outcome contingency, and the rate of reinforcement are not mediated by Pavlovian associations between the manipulandum and the reinforcer.     

Eye movements during recall of aversive memory decreases conditioned fear

Cognitive-behavioral therapy for anxiety disorders typically involves exposure to the conditioned stimulus (CS). Despite its status as an effective and primary treatment, many patients do not show clinical improvement or relapse. Contemporary learning theory suggests that treatment may be optimized by adding techniques that aim at revaluating the aversive consequence (US) of the feared stimulus. This study tested whether US devaluation via a dual task – imagining the US while making eye movements – decreases conditioned fear. Following fear acquisition one group recalled the US while making eye movements (EM) and one group merely recalled the US (RO). Next, during a test phase, all participants were re-presented the CSs. Dual tasking, relative to the control condition, decreased memory vividness and emotionality. Moreover, only in the dual task condition reductions were observed in self-reported fear, US expectancy, and CS unpleasantness, but not in skin conductance responses. Findings provide the first evidence that the dual task decreases conditioned fear and suggest it may be a valuable addition to exposure therapy.     

Backward inhibitory learning in honeybees: a behavioral analysis of reinforcement processing

One class of theoretical accounts of associative learning suggests that reinforcers are processed according to learning rules that minimize the predictive error between the expected strength of future reinforcement and its actual strength. The omission of reinforcement in a situation where it is expected leads to inhibitory learning of stimuli indicative for such a violation of the prediction. There are, however, results indicating that inhibitory learning can also be induced by other mechanisms. Here, we present data from olfactory reward conditioning in honeybees that show that (1) one- and multiple-trial backward conditioning results in conditioned inhibition (CI); (2) the inhibition is maximal for a 15-sec interval between US and CS; (3) there is a nonmonotonic dependency on the degree of CI from the US-CS interval during backward pairing; and (4) the prior association of context stimuli with reinforcement is not necessary for the development of CI. These results cannot be explained by models that only minimize a prediction error. Rather, they are consistent with models of associative learning that, in addition, assume that learning depends on the temporal overlap of a CS with two processes, a fast excitatory and a slow inhibitory one, both evoked by a reinforcer. The fmdings from this behavioral analysis of reinforcement processing are compared with the known properties of an individual, identified neuron involved in reinforcement processing in the bee brain, to further understand the mechanisms underlying predictive reward learning.