Stimulus generalization of suppression in rats following aversively motivated instrumental or Pavlovian training.

Following 100 or 300 avoidance training trials, instrumental subjects and their yoked Pavlovian counterparts were tested for generalization of lick suppression along the frequency dimension of the avoidance conditioned stimulus. Gradients of stimulus control were evident after 300 instrumental avoidance training trials, and additional intradimensional Pavlovian discrimination training further sharpened the gradients. After 100 trials, the yoked Pavlovian subjects suppressed more than their instrumental counterparts. However, with increased Pavlovian training, flatter gradients with decreased suppression were obtained. Results from a second experiment revealed that, whereas Pavlovian experience decreased suppression to the tone, subjects suppressed drinking in the presence of static, environmental cues. Data from both experiments supported interpretations that stress the role of response control over environmental events.     

Stimulus control of avoidance behavior

The introduction of a warning signal preceding shocks greatly increased the effectiveness of avoidance responding. Periods of “warm-up” at the beginning of the session were eliminated, and the number of shocks received by the subjects was greatly reduced. With response-shock interval constant, response rate increased as the interval between the response and the onset of the warning signal was shortened. The response tended to occur shortly after the onset of the warning signal regardless of the duration of these “safe” periods. A greatly elevated response rate was maintained even when the duration of the safe period was reduced to 0.3 sec. Thus, the pre-shock signal obtained nearly exclusive control of the responding and overrode the usual “temporal discrimination” of the response-shock interval.     

Pavlovian conditioned inhibition of fear during shuttlebox avoidance behavior

Three experiments explored the development and function of conditioned inhibition of fear during the acquisition and maintenance of shuttlebox avoidance behavior. The development of inhibition to an exteroceptive feedback stimulus was found to be a function of the number of successive avoidance responses to which the animal had been trained and of the duration of the intertrial interval, a parameter shown also to affect the rate of acquisition of avoidance learning. Master animals who learned the instrumental avoidance response, and yoked animals who did not, showed equivalent inhibitory fear conditioning in each experiment. The results of one experiment suggest that conditioned inhibition plays no important role in “protecting” fear conditioned to the discrete warning signal during avoidance maintenance. These data indicate that feedback stimuli develop their inhibitory properties by a Pavlovian process and that certain aspects of their function may, therefore, be readily understood within the framework of mediational two-process learning theory.     

Stimulus aspects of aversive controls: stimulus generalization of conditioned suppression following discrimination training

A tone ending with unavoidable electrical shock was periodically presented to pigeons while they pecked a key for food. A second group of birds was exposed to these tone-shock contingencies and also to a second tone which never ended with shock. The gradient of stimulus generalization following training to a single stimulus was symmetrical, whereas the gradient following discrimination training was asymmetrical, but there was no evidence of peak shift. During testing, suppression to all tones gradually extinguished and in both groups the slope of the gradient increased markedly. A second experiment with the discrimination birds revealed that free shock caused a recovery of the gradient, but the asymmetry persisted.     

Facilitation of instrumental behavior by a Pavlovian appetitive conditioned stimulus

Three experiments examined appetitive Pavlovian-instrumental interactions by presenting separately trained conditioned stimuli (CSs) during reinforced instrumental responding in rabbits. Intra-oral reinforcement was used to minimize interference from peripheral responses such as magazine approach. In experiment 1, the rabbits were first trained to perform an instrumental head-raising response for sucrose reward. A conditioned jaw movement response was then established to a 2-sec CS by pairing it with sucrose; a control stimulus was unpaired with sucrose. Instrumental responding maintained by a variable-interval 40-sec schedule was enhanced during 10-sec presentations of the paired, but not the unpaired, CS. Responding on a variable-ratio 15 schedule was unaffected except on trials on which the pre-CS baseline response rate was low; in such cases the paired CS caused a long-lasting acceleration of responding. Noncontingent presentation of the sucrose reinforcer itself briefly suppressed responding but had no long-term effect. In Experiment 2, a CS that had been conditioned at a 10-sec duration produced the same pattern of effects as in the first study, indicating that facilitation resulted from CS presentation rather than from the frustrative effects of non-reinforcement of the CS. In Experiment 3 an inhibitory CS blocked facilitation by the excitatory CS but did not itself affect instrumental responding. These results support the view that Pavlovian processes play a positive role in instrumental performance and suggest that previous findings of suppression by a short-duration CS reflect peripheral interference. The dependence of facilitation on the baseline level of responding is discussed in terms of associative and motivational theories of Pavlovian mediation.     

Methylphenidate and stimulus control of avoidance behavior

The introduction of a warning signal that preceded a scheduled shock modified the temporal distribution of free-operant avoidance responses. With response-shock and shock-shock intervals held constant, response rates increased only slightly when the response-signal interval was reduced. The result is consistent with Sidman's (1955) findings under different conditions, but at variance with Ulrich, Holz, and Azrin's (1964) findings under similar conditions. Methylphenidate in graded doses increased response rates, modifying frequency distributions of interresponse times. Drug treatment may have disrupted a "temporal discrimination" formed within the signal-shock interval. More simply, methylphenidate influenced response rates by increasing short response latencies after signal onset; this effect was more prominent than the drug's tendency to increase the frequency of pre-signal responses. When signal-onset preceded shock by 2 sec, individual differences in performance were marked; methylphenidate suppressed responding in one rat as a function of increasing dose levels to a greater degree than in a second animal, but both subjects received more shocks than under control conditions.     

Factors influencing inhibitory stimulus control: differential reinforcement of other behavior during discrimination training

Pigeons were trained to respond on a multiple variable-interval 1-min variable-interval 1-min schedule, then switched to a multiple variable-interval 1-min differential-reinforcement-of-other-behavior discrimination training. The rate of reinforcement was held constant during the shift from non-differential reinforcement to discrimination training. Behavioral contrast and post-discrimination inhibitory stimulus control followed the observation of a reduction in the rate of responding to the stimulus correlated with reinforcement for the non-occurrence of pecking.     

Reduction of contextual control of conditioned responses by extended Pavlovian training in an insect

The effect of repetitive training on learned behavior has been an important subject in neuroscience. In instrumental conditioning in mammals, learned action early in training is often goal-driven and controlled by outcome expectancy, but as training progresses, it becomes more habitual and insensitive to outcome devaluation. Similarly, we recently showed in Pavlovian conditioning in crickets (Gryllus bimaculatus) that a conditioned response (CR) is initially sensitive to devaluation of the unconditioned stimulus but becomes insensitive to it after extended training. It is known that habitual responses after extended instrumental training are characterized by a higher context specificity than are initial goal-directed actions in mammals. In this study, we investigated whether this is applicable to Pavlovian conditioning in crickets. In crickets that received a standard amount of training to associate an odor with water reward under illumination, CR under illumination was stronger than that in the dark. In crickets that received extended training under illumination, on the other hand, the level of CR did not differ in different light conditions. Further experiments confirmed that context specificity decreases with the development of behavioral automaticity by extended training, as opposed to findings in instrumental training in mammals. We conclude that the nature of habitual behaviors after extended training differs in different learning systems of animals.

In humans, habitual behavior often underlies dysfunctional behaviors such as drug addiction (Everitt et al. 2001) and obsessive-compulsive disorder (Gillan et al. 2011). It is for this reason that much research conducted on behavioral neuroscience and psychology has aimed to understand the effects of repetitive training on the performance of learned actions or, in other words, the development of habits (Gardner 2015; Wood and Rünger 2016). One of the most common findings in instrumental conditioning in mammals is that actions in an early stage of training are more goal-driven; i.e., they are controlled by the specific outcome (reinforcer) of their actions. However, as training proceeds, behaviors often become more automatic and independent of the actual value of the outcome, although both the goal-driven and habitual behavioral components often coexist in both early and later stages of training (Dickinson 1985; Yin and Knowlton 2006; Kosaki and Dickenson 2010; Smith and Graybiel 2014). Goal-directed actions and habitual behavior can be distinguished by using an outcome devaluation procedure in which the value of the outcome (reward) is reduced by aversive conditioning (Adams 1982; Dickinson 1985) or by outcome-specific satiation (Balleine and Dickinson 1998).We have investigated associative processes underlying Pavlovian conditioning and its possible neural substrates in the cricket Gryllus bimaculatus (Unoki et al. 2005; Mizunami et al. 2009; Mizunami and Matsumoto 2017; Mizunami et al. 2018), and we recently found that the conditioned response (CR) to an odor (conditioned stimulus, CS) diminishes when the value of water reward (unconditioned stimulus [US]) is reduced after satiation early in training but that it becomes insensitive to US devaluation after extended training in crickets (Mizunami et al. 2019). To our knowledge, these results provide the first evidence of increased automaticity of learned behavior by extended Pavlovian training in any animals. Crickets that received four-trial training to associate an odor CS with water US, which we refer to as standard training, exhibited no CR after water had been given until satiation before the test. On the other hand, crickets that received extended training (i.e., four conditioning trials per day on three consecutive days) exhibited the same level of CR regardless of whether they had been provided with water until satiation or not before the test.Notably, in instrumental conditioning in mammals, it has been documented that habitual behavior that is insensitive to outcome devaluation has features that are distinct from those of goal-directed behavior typically seen after limited training. The most notable one is its higher context specificity: Habitual responses are more likely to occur in the context in which training occurred than outside that context (in rats) (Thrailkill and Bouton 2015). Similarly, in humans, such context specificity is considered a hallmark of the habitual behavior formed by repetition of the same learned responses and it is considered a critical feature to which special attention must be paid in the therapy of mal-adaptive habits (Tricomi et al. 2009; Gardner 2015; Wood and Rünger 2016).The aim of this study was to determine whether the same context specificity that is observed in the habitual response after extended training in mammals is applicable to Pavlovian conditioning in insects. To do so, we performed standard or extended training for crickets under illumination, and their CRs were tested under illumination or in the dark. We used light and darkness as contexts since we observed that a change of background light level between training and testing influences the level of CRs in one paradigm of Pavlovian conditioning in crickets (Matsumoto and Mizunami 2004).     

Transfer across drives of the discriminative effect of a Pavlovian conditioned stimulus

Previous results suggest that a stimulus paired in Pavlovian fashion with reward should exert some discriminative control over an unrelated operant response acquired under a different drive-reward system. In the following experiment, a stimulus was first paired with food reinforcement for a hungry rat. Subsequently, the animal learned to lever-press for water reinforcements when thirsty but not hungry. Finally, the control over lever-pressing of the food-paired stimulus was tested by presenting it at various times during extinction of the lever-pressing response. All animals in the experiment showed the expected effect; each emitted more lever-presses during periods of the food-paired stimulus than during alternate control periods.     

Stimulus control of spatial behavior on the eight-arm maze in rats

Rats were trained on an eight-arm, elevated radial maze in a large cylindrical chamber where extramaze stimuli could be manipulated. The first experiment indicated that rats could use extramaze stimuli to locate the arms if such stimuli were available, whereas they performed less effectively and tended to employ response chaining if these stimuli were not available. The second experiment demonstrated that maze performance was disrupted by transposition of the stimuli but was relatively unaffected by rotation of the same stimuli. The third experiment suggested that the disruptive effect of stimulus transposition might be due to a “resetting” process elicited by the alteration in the configuration of the stimuli after stimulus transposition. These results suggest that when extramaze stimuli are available, rats tend to use such stimuli in a configurational manner to locate the arms rather than as a list of items processed independently of their spatial relationships to each other.     

Stimulus generalization of conditioned taste aversion in rats

Relatively little information is available regarding the intradimensional stimulus generalization of conditioned taste aversion (CTA). Experiment 1 employed a between-groups generalization test to examine the extent to which conditioned flavor aversion to one sucrose solution generalized to other concentrations of sucrose in adult rats. Evidence of a gradient of aversion was obtained. Because generalization gradients in other tasks have been found to flatten over a retention interval, Experiment 2 investigated the effects of delayed testing (2, 7, or 21 days) upon the slope of the generalization gradient. The generalization gradient flattened at the intervals, suggesting that subjects forgot the specific attributes of the conditioning concentration and avoided generalized stimuli as if they were the original CS. Experiment 3 used a long delay between taste and toxicosis to degrade the associative contingency and found no evidence that the generalization gradients found in the first two experiments could be explained in terms of enhanced neophobia due to poisoning. These findings provide further evidence (cf. A. W. Logue, 1979, Psychological Bulletin, 86, 276-296; M. Domjan, 1980, in J. S. Rosenblatt, R. A. Hinde, C. Beer, & M. Busnel (Eds.), Advances in the study of behavior, Vol. 11, New York: Academic Press) that CTA shares a number of similarities with other learning processes. Further, they illustrate that stimulus forgetting can be detected in a paradigm considered relatively immune to retention loss.     

The development and the extinction of hyperthermia induced by conditioned avoidance behavior in rats

Hyperthermia was observed at the end of each conditioning session in rats trained to avoid electric shocks by jumping on a platform. The temperature rise was also observed after the conditioned behavioral response was well established and was elicited even in the absence of shocks. There was no tendency for the hyperthermia to diminish over the course of nine extinction sessions.     

Some effects of methylphenidate on stimulus control of ratio avoidance behavior in the rat

After exposure to an avoidance schedule which included a warning signal, a rat was placed on a multiple schedule in which the first component was the same as before, i.e., a single response reset the response-shock interval, delaying shock, and the second component differed only in that four bar-presses were required to postpone shock. A fixed ratio requirement of four responses (FR 4) generated behavior resembling a fixed ratio requirement of one response (FR 1) since responding was controlled by the warning signal but more shocks were received. At a dosage of 2 mg/kg, methylphenidate given intraperitoneally decreased shock frequency during FR 4 periods while FR 1 behavior was not affected; at 4 mg/kg, stimulus control of avoidance responding was impaired during both components. Results at 4 mg/kg were partially confirmed by two animals exposed to an FR 4 avoidance schedule which included a warning signal but with different parameters. Response distributions showed that methylphenidate increased response rates in the absence of the warning signal, i.e., stimulus control of ratio-avoidance behavior was impaired although the increased response rates reduced shock frequency. One hour later responses again occurred more frequently during the signal than in its absence but shocks were less frequent than during control (non-drug) periods.     

On the failure of transfer of control from separately conducted Pavlovian conditioning to free-operant avoidance conditioning in rats

In Experiment 1, presentation of a tone previously correlated with foot shock produced an increase in the rate of rats' foot-shock avoidance responding, while presentation of a tone previously correlated with tail shock had no reliable effect on the rate of foot-shock avoidance responding. In Experiment 2, a tone correlated with tail shock elicited heart-rate conditioned responses, but had no reliable effect on the rate of subsequent foot-shock avoidance responding. In Experiment 3, presentation of a tone previously correlated with foot shock, and a tone previously correlated with tail shock in separate groups produced increases in the rate of tail-shock avoidance responding. In Experiment 4, a tone correlated with tail shock in one setting accelerated tail-shock avoidance responding in a second setting. Interaction between US and negative reinforcer modifiability by skeletal responding may contribute to the pattern of results obtained.     

Enhancement of Pavlovian conditioned inhibition achieved by posttraining inflation of the training excitor

In two conditioned lick suppression experiments using water-deprived rats, we examined the effects of following Pavlovian conditioned inhibition training (i.e., A–US/AX-NoUS) with pairings of the training excitor (A) and the unconditioned stimulus (US). Experiments 1 and 2 assessed the effects of this posttraining inflation treatment on Pavlovian conditioned inhibition using a summation test and a retardation test, respectively. Both experiments revealed that subjects exposed to inflation treatment demonstrated behavior indicative of enhanced conditioned inhibition compared to subjects that did not receive inflation treatment. The results in conjunction with other recent findings suggest that posttraining associative deflation (i.e., extinction of A) and inflation effects are more symmetrical than has previously been realized.