Muscimol Induces Retrograde Amnesia for Changes in Reward Magnitude

These experiments examined the effect of the GABA_A, agonist, muscimol (MUS), on memory for changes in reward magnitude. In Experiment 1 rats were trained to run a straight alley for either a large or small food reward. After reaching asymptotic performance rats in the high reward group were shifted to the small food reward. Half the animals received 1.0 or 3.0 mg/kg (ip) of MUS or the equivalent volume of saline immediately after training. Shifted training continued for 3 more days and no further injections were given. Shifted saline animals displayed an increase in response latencies compared to unshifted controls with a sharp peak on the day after the shift. Shifted MUS receiving 1.0 mg/kg performed comparably to shifted saline animals. In contrast, shifted MUS animals receiving 3.0 mg/kg displayed performance comparable to shifted saline animals on the day of the shift but displayed a sharp increase in response latencies on the second day after the shift. These findings indicate that post-training systemic MUS injections delay the peak increase in response latencies and suggest that MUS induces retrograde amnesia for reward reduction. Experiment 2 examined the effect of MUS on the memory of a reward increase. Rats were first trained as in Experiment 1 and rats under the high reward condition were then shifted to the small reward. On the next training session, the large food reward was reinstated. Immediately after the session all animals were injected with saline or 3.0 mg/kg of MUS. The large food reward was continued for the remainder of training and no further injections were given. On the following session, the performance of the shifted saline animals was comparable to that of the unshifted controls while shifted MUS animals displayed significantly higher response latencies. The findings that MUS prevented the reduction in response latencies seen in saline-injected animals suggest that MUS also induces retrograde amnesia for reward increases.     

Resistance to extinction and punishment following training with shock and non-reinforcement: Failure to obtain cross-tolerance

Two experiments investigated the phenomenon of cross-tolerance between the partial reinforcement extinction effect (PREE) and the partial punishment effect (PPE). Three groups of rats were trained in acquisition to run in a straight alley. The continuously reinforced (CRF) group received a reward on every trial. The partially reinforced (PRF) group was rewarded on a quasi-random 50% schedule. The partially punished (PP) group received food reward on every trial but, in addition, received foot shocks of gradually increasing intensity in the goal box on a random 50% of the trials. In the test stage, half of the animals in each training condition were tested in extinction, where no reward was given on any of the trials, and the other half were tested in punishment, with both food and shock presented on each trial. Experiment 1 used a 1-trial/day procedure; Experiment 2 used a multi-trial procedure. In both procedures, clear PREE and PPE were obtained. In the 1-trial/day procedure, no cross-tolerance was evident: animals trained on a PRF or PP schedule did not show increased resistance to punishment and extinction, respectively. In the multi-trial procedure, only weak cross-tolerance was obtained in animals trained on partial reinforcement and tested in punishment.     

Double drive learning in rats without previous selective reinforcement

Can rats which have never been selectively satiated by food or water respond appropriately by choosing the relevant reward when they are made hungry or thirsty ? A group of 21 rats was put on a diet of wet mash and trained both hungry and thirsty in a T-maze with water on one side and food on the other. 8 out of 10 of the animals which were made thirsty on the test-run chose correctly in spite of an established preference for the other reward. Only 1 out of the 11 animals made hungry chose correctly. Possible explanations of this result are examined.     

Effects of reinforcement schedules on rats' choice behavior in extinction.

The effects of reinforcement schedules on rats' choice behavior in extinction were studied. In a free-operant chamber equipped with two retractable bars, the experimental animals were trained to press the bars separately for a food reward. One bar delivered the reward on a continuous reinforcement (CRF) schedule, and the other delivered the reward on a partial reinforcement (PRF) schedule. Control animals earned the reward from both bars with the same reinforcement schedule, either a CRF or a PRF. When both bars were simultaneously available during extinction, the experimental animals responded more frequently to the CRF than to the PRF alternative, demonstrating a reversed within-subjects partial reinforcement extinction effect (PREE). A conventional between-subjects PREE was replicated in the control subjects. The results of this study were inconsistent with both Amsel's (1962, 1967) frustration hypothesis and Capaldi's (1966, 1967) sequential hypotheses.     

Use of Vacuum Tube Amplification in Establishing Differential Motor Reactions

1. The electromyographic potentials arising from masticatory movements were picked up by electrodes chronically implanted in the brains of eight rats. The rats had been trained in a Skinner box to press a lever for food reinforcement on a 100 per cent schedule. Lever and food cup were separated by 180[ddot] in a cylindrical metal box one foot in diameter.

2. The problem was to determine whether thoroughly trained animals would show AGRs as determined by the presence of electromyographic potentials when, in the course of pressing, food was suddenly withheld. Only two of the eight animals gave an AGR during the extinction process, and in one of these the phenomenon was minimal.

3. In the discussion it is pointed out that the absence of AGRs in the animals was possibly related to the short time interval intervening between depression of the lever and securing the food-pellet reward.     

Means-means-end tool choice in cotton-top tamarins (Saguinus oedipus): finding the limits on primates’ knowledge of tools

Most studies of animal tool use require subjects to use one object to gain access to a food reward. In many real world situations, however, animals perform more than one action in sequence to achieve their goals. Of theoretical interest is whether animals have the cognitive capacity to recognize the relationship between consecutive action sequences in which there may be one overall goal and several subgoals. Here we ask if cotton-top tamarins, a species that in captivity uses tools to solve means-end problems, can go one step further and use a sequence of tools (means) to obtain food (end). We first trained subjects to use a pulling tool to obtain a food reward. After this initial training, subjects were presented with problems in which one tool had to be used in combination with a second in order to obtain food. Subjects showed great difficulty when two tools were required to obtain the food reward. Although subjects attended to the connection between the tool and food reward, they ignored the physical connection between the two tools. After training on a two-tool problem, we presented subjects with a series of transfer tests to explore if they would generalize to new types of connections between the tools. Subjects readily transferred to new connections. Our results therefore provide the first evidence to date that tamarins can learn to solve problems involving two tools, but that they do so only with sufficient training.     

Conservatism and “copy-if-better” in chimpanzees (<Emphasis Type="Italic">Pan troglodytes</Emphasis>)

Social learning is predicted to evolve in socially living animals provided the learning process is not random but biased by certain socio-ecological factors. One bias of particular interest for the emergence of (cumulative) culture is the tendency to forgo personal behaviour in favour of relatively better variants observed in others, also known as the “copy-if-better” strategy. We investigated whether chimpanzees employ copy-if-better in a simple token-exchange paradigm controlling for individual and random social learning. After being trained on one token-type, subjects were confronted with a conspecific demonstrator who either received the same food reward as the subject (control condition) or a higher value food reward than the subject (test condition) for exchanging another token-type. In general, the chimpanzees persisted in exchanging the token-type they were trained on individually, indicating a form of conservatism consistent with previous studies. However, the chimpanzees were more inclined to copy the demonstrator in the test compared to the control condition, indicating a tendency to employ a copy-if-better strategy. We discuss the validity of our results by considering alternative explanations and relate our findings to the emergence of cumulative culture.     

The effects of hippocampal lesions on learning, memory, and reward expectancies

The hippocampus appears to be critical for the formation of certain types of memories. Hippocampal-lesioned animals fail to exhibit some spatial, contextual, and relational associations. After aspiration lesions of the hippocampus and/or cortex, male rats were allowed to recover for three weeks before being trained on a matching-to-position task. The matching-to-position task was altered to influence the type of cognitive strategies a subject would use to solve the task. The main behavioral manipulation was the reinforcement contingency assignment: Use of a differential outcomes procedure (DOP) or a nondifferential outcomes procedure (NOP). The DOP involves correlating each to-be-remembered event with a distinct reward condition via Pavlovian trace conditioning, whereas the NOP results in random reward contingency. We found that hippocampal lesions did retard learning the matching rule, regardless of the reinforcement contingency assignment. However, when delay intervals were added to the task memory performance of subjects with hippocampal lesions was dramatically impaired--if subjects were not trained with the DOP. When subjects were trained with the DOP, the hippocampal lesion had a marginal effect on delayed memory performance. These findings demonstrate two important points regarding lesions of the hippocampus: (1) hippocampal lesions have a minimal effect on the on the ability of rats to use reward information to solve a delayed discrimination task; (2) rats with hippocampal lesions have the ability to learn about reward information using Pavlovian trace conditioning procedures.     

The effect of informational stimuli on instrumental response rate: Signalling reward versus signalling the availability of reward

Rats were trained to press a lever for food on an interval schedule and were given a brief cue (0.5 sec) between the operative response and the reward (C condition). Some control subjects in Experiments 1, 2 and 4 were given their cue either following the end of the temporal interval during which reward had been unavailable (SD condition), or randomly with respect to food (R condition). Other control subjects in Experiments 2 and 4 received both the food-correlated cue and the temporal-interval stimulus (B condition). In all experiments, rate of responding was lowest for the C subjects and for B animals when the two cues were from different modalities. Food-correlated and temporal-interval cues did not interact, suggesting that a reward-correlated signal does not affect response rate simply by enhancing the salience of the temporal interval offset.     

Stereotyping starlings are more ‘pessimistic’

Negative affect in humans and animals is known to cause individuals to interpret ambiguous stimuli pessimistically, a phenomenon termed ‘cognitive bias’. Here, we used captive European starlings (Sturnus vulgaris) to test the hypothesis that a reduction in environmental conditions, from enriched to non-enriched cages, would engender negative affect, and hence ‘pessimistic’ biases. We also explored whether individual differences in stereotypic behaviour (repetitive somersaulting) predicted ‘pessimism’. Eight birds were trained on a novel conditional discrimination task with differential rewards, in which background shade (light or dark) determined which of two covered dishes contained a food reward. The reward was small when the background was light, but large when the background was dark. We then presented background shades intermediate between those trained to assess the birds’ bias to choose the dish associated with the smaller food reward (a ‘pessimistic’ judgement) when the discriminative stimulus was ambiguous. Contrary to predictions, changes in the level of cage enrichment had no effect on ‘pessimism’. However, changes in the latency to choose and probability of expressing a choice suggested that birds learnt rapidly that trials with ambiguous stimuli were unreinforced. Individual differences in performance of stereotypies did predict ‘pessimism’. Specifically, birds that somersaulted were more likely to choose the dish associated with the smaller food reward in the presence of the most ambiguous discriminative stimulus. We propose that somersaulting is part of a wider suite of behavioural traits indicative of a stress response to captive conditions that is symptomatic of a negative affective state.     

Involvement of the rat anterior cingulate cortex in control of instrumental responses guided by reward expectancy

The anterior cingulate cortex (ACC) plays a critical role in stimulus-reinforcement learning and reward-guided selection of actions. Here we conducted a series of experiments to further elucidate the role of the ACC in instrumental behavior involving effort-based decision-making and instrumental learning guided by reward-predictive stimuli. In Experiment 1, rats were trained on a cost-benefit T-maze task in which they could either choose to climb a barrier to obtain a high reward (four pellets) in one arm or a low reward (two pellets) in the other with no barrier present. In line with previous studies, our data reveal that rats with quinolinic acid lesions of the ACC selected the response involving less work and smaller reward. Experiment 2 demonstrates that breaking points of instrumental performance under a progressive ratio schedule were similar in sham-lesioned and ACC-lesioned rats. Thus, lesions of the ACC did not interfere with the effort a rat is willing to expend to obtain a specific reward in this test. In a subsequent task, we examined effort-based decision-making in a lever-press task where rats had the choice between pressing a lever to receive preferred food pellets under a progressive ratio schedule, or free feeding on a less preferred food, i.e. lab chow. Results show that sham- and ACC-lesioned animals had similar breaking points and ingested comparable amounts of less-preferred food. Together, the results of Experiment 1 and 2 suggest that the ACC plays a role in evaluating how much effort to expend for reward; however, the ACC is not necessary in all situations requiring an assessment of costs and benefits. In Experiment 3 we investigated learning and reversal learning of instrumental responses guided by reward predictive stimuli. A reaction time (RT) task demanding conditioned lever release was used in which the upcoming reward magnitude (five vs. one food pellet) was signalled in advance by discriminative visual stimuli. Results revealed that rats with ACC lesions were able to discriminate reward magnitude-predictive stimuli and to adapt instrumental behavior to reversed stimulus-reward magnitude contingencies. Thus, in a simple discrimination task as used here, the ACC appears not to be required to discriminate reward magnitude-predictive stimuli and to use the learned significance of the stimuli to guide instrumental behavior.     

Extinction of a running response as a function of reinforcement magnitude

Four groups of rats were trained on different sucrose solutions in a straight runway. Terminal running speed was a monotonic function of reinforcement magnitude. After training each group was subdivided, one subgroup being extinguished under spaced, the other under massed conditions. In spaced extinction the animals trained on non extreme reward magnitudes showed most resistance to extinction. It was concluded that resistance to extinction is an inverted U-shaped function of reinforcement magnitude found in training. The massed extinction trials were conducted with a very short inter-trial interval. The animals showed an immediate drop in running speed followed by a gradual recovery and a subsequent decline. The number of trials taken to reach the peak recovery speed was a function of reinforcement magnitude found in training. Results on both massed and spaced extinction trials were interpreted in terms of the facilitatory and inhibitory effects of momentary and conditioned frustration.     

Idazoxan, an alpha-2 antagonist, facilitates memory retrieval in the rat

The role of the noradrenergic system in cognitive function was studied by using the alpha-2 adrenoceptor antagonist idazoxan to increase noradrenergic activity. Rats were trained in a complex maze task for food reward. They were left undisturbed for a 4-week "forgetting" period and were treated with idazoxan, just before the retention test. The dose of idazoxan used had previously been shown to enhance firing of units of the locus coeruleus and to increase noradrenaline (NE) turnover in the forebrain. This pharmacological treatment effectively alleviated forgetting, while control rats showed significant decrement compared to their performance at the last training trial. A control experiment showed that the facilitative effect was not on learning or on ongoing performance of the task, since there was no effect on simple acquisition. The results are taken as support for the notion that NE plays a role in memory retrieval processes.     

Reward schedule effects in extinction: Intertrial interval, memory, and memory retrieval

Rats were trained in a runway such that partial reward occurred on Trial 1 of the day and consistent reward on subsequent massed trials (Group PRT1), or consistent reward occurred on Trial 1 of the day and partial reward on subsequent massed trials (Group PRTM). Under spaced (24-hr) extinction, Group PRT1 was more resistant to extinction than Group PRTM and under massed (1-min) extinction, Group PRTM was more resistant to extinction than Group PRT1. These findings suggest that (a) distinctive stimuli are associated with Trial 1 of the day and with subsequent massed trials, (b) these distinctive stimuli function as retrieval cues for memories, memory retrieval being independent of intertrial interval, and (c) behavior in extinction is controlled by a stimulus compound consisting of the memory of nonreward plus stimuli which accompany the memory of nonreward on rewarded acquisition trials.     

Self-reinforcement in pigeons

Pigeons were trained to reward their own performances by eating from a freely available food source only after pecking a disc. The self-reinforcement pattern was established by fading in the work requirement and punishing noncontingent self-feeding by food withdrawal. The animals maintained faultless self-reinforcement for hundreds of trials after the punishment contingency was removed so that the birds could safely feed themselves without performing any pecking responses. In successive induction and extinction of the phenomenon, the number of responses per self-reward was observed to covary with self-reinforcement rate. By progressively raising the work requirements an animal was trained to adopt increasingly higher performance standards of self-reward. After maintaining a high response output for each self-reward the animal promptly discarded all self-imposed work contingencies and quickly resumed them again, though less durably, following additional training.     

The interaction between discriminative stimuli and outcomes during instrumental learning

Rats were trained on a biconditional discrimination in which the delivery of a food pellet stimulus signalled that pressing on one of two levers would be reinforced, whereas the delivery of a sucrose solution stimulus signalled that the reward was contingent on pressing the other lever. The outcome was the same food type as the discriminative stimulus in the congruent group but the other food type in the incongruent group. Both responses were rewarded with the same outcome in the same group. All the three groups learned the discrimination at statistically indistinguishable rates. Prefeeding one of the outcomes selectively reduced the associated response thereby demonstrating that responding was mediated by a representation of the outcome. Moreover, the outcome of one trial controlled responding on the next trial in accord with the stimulus function of the food type. These results are discussed in relation to the associative structures mediating the discriminative control of instrumental performance.     

Learning of object-object associations by monkeys

Monkeys were trained in tasks where stimulus objects were presented in pairs, one object of a pair concealing the other, and the monkeys had to displace the top object followed by the bottom object in order to obtain a food reward. In two experiments it was demonstrated that under these conditions the animals formed object-object associations such that the top object recalled in memory the bottom object that had previously been found underneath it. The monkeys were able to discriminate in recall between bottom objects that were identical in secondary reinforcing value and differed only in colour.     

Summation of symbols by pigeons (Columba livia): the importance of number and mass of reward items

In each of 3 experiments, different sets of 4 pigeons (Columba livia) were trained to discriminate between 2 visual symbols that covered wells containing food items that varied in number, mass, or both. In Experiment 1, the symbols were associated with 0, 1, 3, 5, 7, or 9 pieces of grain reward. The pigeons learned to choose the symbol corresponding to the larger reward, and on summation tests, they chose the pair of symbols that summed to the larger total reward. When number of food pellets was varied but mass of reward was held constant in Experiment 2, preference for the larger number symbols failed to appear. When number was held constant and mass was varied in Experiment 3, the pigeons showed a clear preference for the larger mass symbols on single-symbol and summation tests. These findings show that pigeons summate the value of symbols and are more likely to represent symbols by mass of food reward than by number of food items.     

Reverse-reward learning in squirrel monkeys (Saimiri sciureus): retesting after 5 years, and assessment on qualitative transfer

Seven squirrel monkeys (Saimiri sciureus) previously trained on reverse-reward tasks were presented with the original "1-versus-4" task after a 5-year interval without reverse-reward experience (Experiment 1). None of them reliably selected the smaller food array; however, at around chance level, their performance was superior to when they were first exposed to the task almost 6 years previously, suggesting some long-term memory retention. One naive monkey consistently selected the larger array, as expected. In Experiment 2, trials consisting of 1 versus 1 piece of two qualitatively different types of food were interspersed among familiar 1-versus-4 trials. None of five monkeys tested reliably selected the less-preferred food to get the more preferred food as the reward, and one monkey scored below chance. However, when one piece of low-preference food was paired with four pieces of high-preference food (Experiment 3), all four monkeys tested avoided reaching for the latter and thereby obtained it as the reward; two monkeys obtained perfect scores on these trials. These two monkeys were trained on a specific qualitative reverse-reward pairing and then again tested on new pairings (Experiment 4), but transfer was incomplete. Compound trials that pit quantity against quality in novel ways appear taxing for squirrel monkeys, despite competence in reverse-reward on both dimensions separately.     

Pavlovian counterconditioning is unchanged by chlordiazepoxide or by septal lesions

Two experiments were performed to test whether the minor tranquillizer chlordiaz-epoxide or septal lesions could interfere with Pavlovian counterconditioning. Rats were first trained to bar-press for food on a random interval schedule on which was superimposed the presentation of a light which predicted delivery of a shock. This training continued until all animals were showing stable suppression of responding to the light. Separate groups were then presented, while the manipulandum was retracted from the box, with either a shock paired with the delivery of food or shock and food delivery in a random relationship with one another. This type of intrusion period alternated with the presentation of the light and shock. Pavlovian counterconditioning developed, as shown by progressively less response suppression in the paired groups compared to the random groups. This Pavlovian counterconditioning was in no way reduced by either chlordiazepoxide (5 mg/kg) or by medial or lateral septal lesions. It is argued that these results rule out Pavlovian counterconditioning as a process which is affected by minor tranquillizers or by septal lesions when they interfere with the behavioural tolerance observed after partial punishment or partial reward schedules.