Duration of extinction trials as a determinant of instrumental extinction in terrestrial toads (<Emphasis Type="Italic">Rhinella arenarum</Emphasis>)

Instrumental learning guides behavior toward resources. When such resources are no longer available, approach to previously reinforced locations is reduced, a process called extinction. The present experiments are concerned with factors affecting the extinction of acquired behaviors in toads. In previous experiments, total reward magnitude in acquisition and duration of extinction trials were confounded. The present experiments were designed to test the effects of these factors in factorial designs. Experiment 1 varied reward magnitude (900, 300, or 100 s of water access per trial) and amount of acquisition training (5 or 15 daily trials). With total amount of water access equated in acquisition, extinction with large rewards was faster (longer latencies in 900/5 than 300/15), but with total amount of training equated, extinction with small rewards was faster (longer latencies in 100/15 than 300/15). Experiment 2 varied reward magnitude (1200 or 120 s of water access per trial) while holding constant the number of acquisition trials (5 daily trials) and the duration of extinction trials (300 s). Extinction performance was lower with small, rather than large reward magnitude (longer latencies in 120/300 than in 1200/300). Thus, instrumental extinction depends upon the amount of time toads are exposed to the empty goal compartment during extinction trials.     

Response-duration of lever pressing in the rat

Two aspects of the lever-holding behaviour of rats in a Skinner-box have been analysed: firstly, the changes in the duration of the responses during the acquisition and experimental extinction phase, and, secondly, the bunching of responses during the experimental extinction phase. The response-durations on the first few acquisition trials were found to be bimodally distributed, but to become stabilized at 0.42 seconds as practice increased. During experimental extinction response-durations increased. The rate of increase depended on the conditions of secondary reward. Rats who had the source of secondary reward removed, or restricted, yielded a steeper slope than those who responded under the same conditions of secondary reward as during training.

The analysis of the extinction scores revealed that unrewarded responses were emitted in groups. Response-group latencies were shown to progress as a positive function of the number of response-groups, and the average response-duration for consecutive response-groups to increase progressively. The slope of the duration curve plotted for successive response-groups increased, and this was related to an increase in the value of the time-intercept for successive response-groups.     

Effect of schedule and magnitude of reinforcement on instrumental learning in the toad, Bufo arenarum

Extinction after training with continuous (CR) or 50% partial (PR) reinforcement, and with different magnitudes of reward, was studied in the amphibian Bufo arenarum, in a runway situation. In Experiment 1, a group of toads received massed-trial, CR training with access to water as the reward. Performance improved during acquisition, including an improvement on the first trial of each session. Extinction was rapid and there was evidence for spontaneous recovery of the running response. In Experiment 2, groups of toads received PR or CR training at a rate of one trial per day. PR impaired acquisition and resulted in poor responding during extinction, compared to CR. Experiment 3 factorially studied the effects of schedule (PR vs CR) and distribution of practice (15 s vs 300 s intertrial interval). Acquisition was impaired by PR training but had little effect on extinction performance. Different magnitudes of water reinforcement were used in Experiment 4 in a one-trial-per-day situation. Terminal acquisition performance was a monotonic function of reward magnitude, but there were no differences in extinction performance across groups. The results are discussed in relation to comparative and developmental data on the paradoxical effects of reward.     

Autoshaping: further study of "negative automaintenance"

The key pecking of pigeons was autoshaped to three key colors paired with food in discrete trials. Then, the effects of three different color-correlated contingencies were compared: reward (presentation of food contingent on pecking), omission (presentation of food prevented by pecking), and extinction (no food). Two measures of performance were used: initial response (the number of trials with each color on which at least one peck was made) and multiple response (the total number of pecks per trial). In general, the reward color produced more pecking than the omission color, the omission color more than the extinction color, and the extinction color more than on blank trials with an unlighted key, although (relative to reward) omission produced a higher level of initial than of multiple responding. These results point clearly to the importance of stimulus-reinforcer continguity in the control of pecking.     

Deepened extinction following compound stimulus presentation: noradrenergic modulation

Behavioral extinction is an active form of new learning involving the prediction of nonreward where reward has previously been present. The expression of extinction learning can be disrupted by the presentation of reward itself or reward-predictive stimuli (reinstatement) as well as the passage of time (spontaneous recovery) or contextual changes (renewal). The following experiments replicated the demonstration that presenting multiple previously rewarded stimuli in compound during extinction enhances extinction learning. To explore the pharmacological basis for this we next examined the effects of pharmacological treatments that either facilitated or blocked noradrenergic activity to test the hypothesis that increased noradrenergic activity at the time of extinction training would improve, whereas blockade of noradrenergic activity would impair the extinction of appetitive stimulus-reward memories. Different groups of rats were trained in a discriminative stimulus paradigm to lever-press for food reward. Once stable responding was achieved, responding was extinguished for 2 d. Prior to a third extinction session, rats received systemic administration of either saline, yohimbine (α2 antagonist), atomoxetine (norepinephrine reuptake inhibitor), or propranolol (β-receptor antagonist). Spontaneous recovery of responding to the stimuli was tested 4 wk later. Our results indicate that increasing noradrenergic activity during extinction augments extinction learning resulting in less recovery of responding at test. These results have important implications for models of relapse to drug seeking and the development of extinction-based therapies.     

The overjustification effect

After reviewing the literature on overjustification (defined as any situation in which an external reinforcer is tied to an activity that would have been performed without that reinforcer), it was concluded that reattribution cannot adequately account for response decrements following the removal of reinforcers. An alternative explanation was proposed involving the concept of “reward contrast,” in which it is argued that individuals compare their actual or expected outcomes following a response with previous outcomes. When this comparison is favorable, the response is strengthened; when unfavorable, the response is weakened. Based upon the notion of reward contrast, an experiment was performed in which 16 albino rats were permitted to wheel-run for 33 days. After 11 acquisition and 5 baseline days, rats in the experimental group were reinforced for running at their baseline rate, while control rats continued to run for no reinforcement. Following 5 days of reinforcement, experimental animals were put into extinction. As hypothesized, albino rats displayed typical over-justification effects during extinction. As also hypothesized, these effects were temporary, with a gradual return to baseline over repeated extinction trials.     

The timing of bar-pressing behaviour

The time spent by a rat in a bar-pressing situation is made up of active time spent n pressing, eating time, and extra time spent in other activities. With a well trained rat, active time and extra time are small, and eating time mainly determines the rate of reward delivery. Active time is affected by a change of weight on the bar, the time between reward deliveries is affected by the amount of reward, and the extra time is affected by extinction conditions.

There is not a one-to-one correspondence between periods of activity at the knob and rewards.

The term “response” and some variables based on it are given empirical referents, which show that much research and theorizing on bar-pressing behaviour has been concerned with only a small selection of the rat's bar-pressing activities. Some reasons for this restriction are the use of the simple weighted bar, the lack of a rationale for bar-pressing research, and the practice of not watching the rat during an experiment.     

Transfer of approach responding between punishment, frustrative nonreward, and the combination of punishment and nonreward

Rats trained to make an approach response with either partial reward, intermittent punishment, or a combination of partial reward and intermittent punishment, were tested for persistence to extinction, punishment with reward, or punishment during extinction. Partial reward, alone or with punishment, produced greates resistance to extinction, while intermittent punishment, alone or with partial reward, produced greatest persistence to punishment with reward. Transfer of persistence from partial reward to punishment with reward and intermittent punishment to extinction was also demonstrated. However, partial reward alone did not increase persistence to punishment during extinction, whereas intermittent punishment and partial reward combined with intermittent punishment did increase such persistence. These results were interpreted in Amsel's (1958, 1962) conditioning-model theory by extending the hypothesized similarity of frustrative nonreward and punishment.     

Persistent behaviour in extinction after partial deprivation in training

Four groups of rats were trained to run an alleyway with one trial per day. Two groups were always deprived when trained while the other two received a partial deprivation schedule. One group of each pair received a continuous reward in the goal box while the other received partial reward. A partial reinforcement effect was found during extinction. The partially deprived groups also showed persistence in extinction. This result extends parallels between the effects of satiation and nonreward upon behaviour.     

Probability-reward preferences of rhesus monkeys

The relative preferences of rhesus monkeys for reward probability versus amount were investigated with procedures which contrasted general experience with specific instructions, and evaluated the relationship between probability-amount combinations and preference strength. Four stimulus objects, each signifying a different combination of reward frequency and amount (100% with one unit; 50% with two units; 33% with three units; or 25% with four units), were presented in pairs, one pair per daily session, with trial schedules providing the same amount of reward within each set of 12 trials. In Phase A, 4 monkeys (Group 1) were tested on the six choice-pairs with no preliminary training. In Phase B, Group 1 was joined by an additional 4 monkeys (Group 2), and each of the tasks was preceded by a demonstration of the relevant stimuli, one at a time, together with their associated probabilities and amounts. Group 1 animals developed preferences during Phase A for the more frequently rewarded objects, which persisted into Phase B, whereas Group 2 animals showed no preferences. This result indicates that preliminary instructions concerning the reward combinations associated with stimulus objects can prevent the development of a preference for greater probability over greater amount of reward, but cannot extinguish it once it has been formed or reestablished within the context of a particular task.     

Pavlovian processes in the motivational control of instrumental performance

Hungry rats were rewarded for pressing a lever on a multiple schedule. During one component the reward was a sucrose solution, whereas food pellets acted as the reward during another component. Lever pressing was never rewarded during the third component. When the drive state was switched from hunger to thirst and the animals tested in extinction, they pressed more in the presence of the component stimulus that had been associated with the sucrose reward during training. A similar effect was observed during the extinction test of a second study in which the component stimuli had signalled non-contingent presentations of either the sucrose or pellet rewards in the absence of the lever. This suggests that the instrumental irrelevant incentive effect observed in the first experiment was due, at least in part, to the Pavlovian relationship between the component stimuli and the reinforcers during training. In fact, when the size of the effects controlled by purely Pavlovian and supposedly instrumental contingencies was compared directly in the final study, no difference could be detected.     

A COMPARATIVE STUDY OF INSTRUMENTAL LEARNING BY PRESCHOOL CHILDREN IN PAPUA NEW GUINEA AND AUSTRALIA1

Instrumental learning of preschool children in Papua New Guinea (PNG) and Australia (AUST) was compared using two tasks (imitative and nonimitative) and two rewards (social and nonsocial). There were no differences between the two groups in the rate of acquisition measure of trials to criterion. PNG children made more late responses during acquisition and, for nil responses, there were group x task and group x reward x task effects. In the extinction phase, there were two main effects for trials to criterion: PNG children were more resistant to extinction than AUST children, and there was greater persistence in responding after social reward regardless of nationality. Reward x group, reward x task, and reward x group x task interactions also were observed in the extinction trials to criterion. In addition, there were three main task effects during extinction for other responses: on the imitative task, more wrong responses were made, and on the nonimitative task, more extra responses and more paired responses were made. A subsidiary analysis compared the two culturally different but educationally similar groups comprising the PNG sample: no major differences were isolated in acquisition or extinction.     

Effects of expectancies of different reward magnitudes in transfer from noncontingent pairings to instrumental performance

In two experiments, rats received noncontingent pairings of two stimuli with food reward, one paired with small reward and the other with large reward, and received bar press training with large reward or with small reward. When the noncontingent stimuli (NS) were presented for test during subsequent rewarded bar pressing and during early extinction of bar pressing, responding for each group was faster in the presence of the NS which was paired with the same reward magnitude that group received in bar press training than to the NS which had been paired with a different reward magnitude. As extinction progressed, all groups responded more slowly in the presence of the NS which had been paired with the large reward than in the presence of the NS which had been paired with small reward. These results were interpreted as indicating that responding in the presence of an NS depends on: (i) whether the reward expectancy elicited by the NS has been conditioned to the instrumental response, and (ii) the relationship between the reward expected in the presence of the NS and that received in test.     

Effect of interpolated extinction after partial delay of reward training on subsequent reacquisition and extinction

Four groups of albino rats were run four trials a day in a straight runway for 44 days. On the first 15 days, two groups were given continuous immediate reward (IR) and two groups a 50 per cent, schedule of 30-sec. partial delay of reward (PDR). On the next 15 days, one IR group and one PDR group were extinguished, while the other IR and PDR groups remained on their original schedules. In the third phase, all groups received 8 days of training on IR. Finally, all groups were given 6 days of extinction training. In the first extinction, PDR produced greater resistance to extinction than IR. In the second extinction period, the PDR group which had previously been given extinction and the two IR groups extinguished relatively rapidly and at approximately the same rates, while the PDR group which had not been extinguished was significantly more resistant to extinction than the other three groups.     

d-Serine facilitates the effects of extinction to reduce cocaine-primed reinstatement of drug-seeking behavior

Male Sprague Dawley rats were allowed to self-administer cocaine (0.5 mg/kg) during 90 min sessions for a period of 15 days. On day 16, rats were either held abstinent in their home cage environment or experienced an extinction session in which the active lever had no programmed consequences. Facilitating N-methyl-d-aspartate (NMDA) receptor activity with the coagonist d-serine (100 mg/kg i.p.) before or following the extinction session significantly reduced the subsequent cocaine-primed reinstatement of drug-seeking behavior tested on day 17. d-Serine significantly reduced drug-primed reinstatement only when combined with extinction, and its effectiveness when administered following the training session suggested that an enhancement of consolidation of extinction learning had occurred. In contrast, d-serine treatment did not reduce sucrose-primed reinstatement, indicating that the beneficial effects of this adjunct pharmacotherapy with extinction training were specific to an addictive substance (cocaine) and did not generalize to a natural reward (sucrose).     

Communication between domestic dogs and humans: effects of shelter housing upon the gaze to the human

It is widely known that gaze plays an essential role in communicative interactions. Domestic dogs tend to look at the human face in situations of conflict and uncertainty. This study compares the gaze of shelter and pet dogs during acquisition and extinction phases in a situation involving a reward in sight but out of reach. Even though no significant differences between the groups were recorded during acquisition, gaze duration decreased in both groups during extinction, with shelter dogs showing a significant shorter duration. This could be related to their different living conditions and to the fact that through their ordinary everyday interactions, pet dogs have more opportunities to learn to persist in their communicative responses when they do not get what they want. These results highlight the relevance of learning experiences during ontogeny, which would therefore modulate communicative responses.     

Runway acquisition and extinction as a joint function of magnitude of reward and percentage of rewarded acquisition trials

To determine the joint effects of partial reward and reward magnitude on acquisition and extinction rates, and on acquisition and extinction asymptotes, 215 Wistar albino rats were trained in a Hunter straight runway. The experimental design was a 4 × 4 × 2 factorial combining four reward magnitudes, four reward percentages, and two experimenters. The data revealed that the acquisition rate was an increasing function of both percentage and magnitude of reward and that neither reward magnitude nor percentage of reward significantly affected acquisition asymptote. For extinction, it was found that, for continuous schedules, the larger the reward magnitude the less the resistance to extinction and, for partial schedules, the larger the reward magnitude the greater the resistance to extinction. These results were interpreted within the framework of the sequential effects hypothesis (Capaldi, 1966).     

Effects of reinforcement schedules on rats' choice behavior in extinction.

The effects of reinforcement schedules on rats' choice behavior in extinction were studied. In a free-operant chamber equipped with two retractable bars, the experimental animals were trained to press the bars separately for a food reward. One bar delivered the reward on a continuous reinforcement (CRF) schedule, and the other delivered the reward on a partial reinforcement (PRF) schedule. Control animals earned the reward from both bars with the same reinforcement schedule, either a CRF or a PRF. When both bars were simultaneously available during extinction, the experimental animals responded more frequently to the CRF than to the PRF alternative, demonstrating a reversed within-subjects partial reinforcement extinction effect (PREE). A conventional between-subjects PREE was replicated in the control subjects. The results of this study were inconsistent with both Amsel's (1962, 1967) frustration hypothesis and Capaldi's (1966, 1967) sequential hypotheses.     

Reward schedule effects in extinction: Intertrial interval, memory, and memory retrieval

Rats were trained in a runway such that partial reward occurred on Trial 1 of the day and consistent reward on subsequent massed trials (Group PRT1), or consistent reward occurred on Trial 1 of the day and partial reward on subsequent massed trials (Group PRTM). Under spaced (24-hr) extinction, Group PRT1 was more resistant to extinction than Group PRTM and under massed (1-min) extinction, Group PRTM was more resistant to extinction than Group PRT1. These findings suggest that (a) distinctive stimuli are associated with Trial 1 of the day and with subsequent massed trials, (b) these distinctive stimuli function as retrieval cues for memories, memory retrieval being independent of intertrial interval, and (c) behavior in extinction is controlled by a stimulus compound consisting of the memory of nonreward plus stimuli which accompany the memory of nonreward on rewarded acquisition trials.     

Accumbens shell AMPA receptors mediate expression of extinguished reward seeking through interactions with basolateral amygdala

Extinction is the reduction in drug seeking when the contingency between drug seeking behavior and the delivery of drug reward is broken. Here, we investigated a role for the nucleus accumbens shell (AcbSh). Rats were trained to respond for 4% (v/v) alcoholic beer in one context (Context A) followed by extinction in a second context (Context B). Rats were subsequently tested in the training context, A (ABA), or the extinction context, B (ABB). Pre-test injections of the glutamate AMPA receptor antagonist, NBQX (1 μg) into AcbSh had no effect on renewal of alcoholic beer seeking when rats were returned to the training context (ABA). However, NBQX increased responding when rats were tested in the extinction context (ABB). In a second experiment, rats received training, extinction, and test in the same context. Pre-test injections of NBQX (0, 0.3, and 1 μg) into the AcbSh dose-dependently attenuated expression of extinction. We also found that NBQX in the AcbSh had no effect on initial acquisition of extinction or the motivation to respond for reward as measured by break point on a progressive ratio schedule. Finally, we show that pharmacological disconnection of a basolateral amygdala (BLA) → AcbSh pathway via NBQX in AcbSh combined with reversible inactivation of the contralateral BLA attenuates expression of extinction. Together, these results suggest that AcbSh AMPA receptors mediate expression of extinguished reward seeking through glutamatergic inputs from the BLA.