The role of exploration in win-shift and win-stay performance on a radial maze

Win-shift spatial memory tasks in a radial maze reinforce animals for avoiding previously visited rewarded arms; win-stay tasks reinforce them for returning to those arms. Win-shift tasks have generally been found much easier to perform, and this may be explained either in terms of foraging models which postulate avoidance of locations where food has been found, or in terms of the predominance of spontaneous alternation (exploration). Experiment 1 examined spontaneous alternation behavior in the radial maze as a function of whether the first visit to an arm had been rewarded or not, and showed that alternation was more probable after nonreward than after reward in both hungry and thirsty rats (a result which conflicts with the foraging account of the win-shift superiority). Experiment 2 replicated the finding that win-stay discrimination performance was inferior to win-shift. A manipulation (lengthening the delay between initial and test choices) which weakens spontaneous alternation, reduced, but did not reverse, the win-shift superiority. In Experiment 3, in order to eliminate the influence of spontaneous alternation, versions of the win-stay and win-shift tasks were devised in which, unlike the original task, all arms were familiar at the choice trial. Under those conditions win-stay was performed better than win-shift. It is concluded that spontaneous alternation plays a major role in many spatial memory tasks, and that the results can best be accounted for by combining principles of exploration and simple associative learning, without recourse to foraging models.     

Effects of opiate antagonists on spatial memory in young and aged rats

The effects of post-training opiate antagonist administration on spatial memory were assessed in young and aged male Long Evans rats. In Experiment I rats were trained to visit each arm of an eight-arm radial maze once in a session to obtain a food reward placed at the end of each arm. During training aged rats required significantly more trials to achieve criterion performance when compared to young mature rats. However, administration of the opiate antagonist naloxone (2.0 mg/kg) immediately after each training trial did not significantly alter the rate of achieving accurate performance in either age group. In Experiment II young and aged rats that were previously trained to a comparable criterion on the radial maze were tested on the same maze apparatus in novel spatial environments. When animals were exposed to novel spatial information, the effects of post-trial opiate antagonists were examined using a within-subjects counter-balanced design. In Experiment IIa naloxone (2 mg/kg) enhanced the performance of both young and aged rats. In Experiment IIB naltrexone (1.0 mg/kg) was found to have a comparable effect of enhancing the performance of both age groups. In addition, in Experiment IIb a significant age-related deficit was found in rats tested in novel spatial environments. These results indicate that opiate antagonists are capable of improving memory for new spatial information in both young and aged rats on a task that is sensitive to behavioral deficits during normal aging.     

Memory systems in the rat: effects of reward probability,context, and congruency between working and reference memory

The interaction of working and reference memory was studied in rats on an eight-arm radial maze. In two experiments, rats were trained to perform working memory and reference memory tasks. On working memory trials, they were allowed to enter four randomly chosen arms for reward in a study phase and then had to choose the unentered arms for reward in a test phase. On reference memory trials, they had to learn to visit the same four arms on the maze on every trial for reward. Retention was tested on working memory trials in which the interval between the study and test phase was 15 s, 15 min, or 30 min. At each retention interval, tests were performed in which the correct WM arms were either congruent or incongruent with the correct RM arms. Both experiments showed that congruency interacted with retention interval, yielding more forgetting at 30 min on incongruent trials than on congruent trials. The effect of reference memory strength on the congruency effect was examined in Experiment 1, and the effect of associating different contexts with working and reference memory on the congruency effect was studied in Experiment 2.     

Reward, novelty and spontaneous alternation

It has been suggested that rats' propensity for “win-shift” behaviour in spatial memory and spontaneous alternation tests reflects a species-specific foraging strategy which leads them to avoid places where they have recently found food. An alternative explanation is that they avoid places which are familiar. In three experiments using a T-maze, we evaluated these accounts by comparing the probability of avoiding or re-entering a recently visited arm, as a function of whether food had or had not been found on the previous visit. Each rat received a series of 16 exposure-test trial pairs over 8 days. Neither alternation nor repetition of the previous choice was differentially reinforced. Experiments I and II forced rats to enter a specific arm before a subsequent choice, and differed in the overall probability of reward; in Experiment III all choices were free. In all three experiments the probability of alternating was greater after nonreward than after reward. This effect occurred more reliably on later tests within a day, little difference appearing on earlier tests. It was concluded that there was no evidence for a spontaneous “win-shift” tendency as such, and that these and other results can be adequately accounted for by a combination of exploratory tendencies (spontaneous alternation) and the conventional effects of reward.     

Serial position curves in spatial memory of rats: Primacy and recency effects

Memory for lists of items was tested in rats (N = 18) in an 8-arm radial maze. In Experiment 1 trials consisted of a study phase, in which the rat could freely choose five arms to obtain a food reward, and a test phase in which the animal was presented with a choice between a novel and a previously visited arm. The rat received additional food reinforcement only when visiting the novel arm. The two phases of a trial were separated by a retention interval of 30 sec or of 4, 16 or 60 min. It was found that recall of the five free arm choices was related to the serial position of the previously visited arm. There was a significant recency effect at the 30-sec delay. With longer retention intervals this disappeared, and a significant primacy effect could be observed. In Experiment 2 the same animals were given forced arm entries during the study phase and delays of 30 sec or 4 or 16 min before the test phase. Again, there was a trend towards a recency effect after the shorter delays and a significant primacy effect after the 16-min interval. These results show that, in the recall of lists of spatial items, rats have serial position curves with primacy and recency effects, depending on the length of the retention interval.     

Spatial memory in rats under restricted viewing conditions

In a procedure devised by J. A. Walker and D. S. Olton (Learning and Motivation, 1979, 10, 73–84), rats were placed on two arms of a four-arm radial maze and then were placed in the center of the maze to test how accurately they could choose the alleys on which they had not been placed. In three experiments, the conditions under which animals viewed the environment from the arms were varied. In Experiment 1, both the extent of spatial view and the exposure time were varied factorially in a within-subjects design; animals viewed the environment down a tunnel or had a 180° or 360° view, and subjects were allowed to view the environment for either 2 or 20 sec. In Experiment 2, a between-subjects design was used, in which different groups of subjects were tested repeatedly under either the tunnel, 180°, or 360° conditions. Both experiments showed that animals could avoid the arms previously visited at no better than a chance level of accuracy in the tunnel viewing condition but could perform with progressively better accuracy at the 180 and 360° viewing conditions. Animals also were more accurate in Experiment 1 after viewing for 20 sec than after viewing for 2 sec. Experiment 3 involved a procedure in which restricted viewing conditions were used both during arm placements and testing. Animals tested under tunnel viewing eventually achieved above-chance performance with this procedure, but did not exceed chance as rapidly as groups tested with 45 and 90° views of the environment. These results suggest that animals can learn about their position in a spatial environment through observation and that an animal's ability to locate its position is directly related to the extent of the surrounding environment it can see and the length of time it is allowed to look. The implications of these findings for list and map hypotheses of spatial memory representation are discussed.     

Rats acquire spatial learning sets

This experiment was designed to examine the development of a spatial learning set in rats and some of the variables influencing the retention of individual problems. The apparatus was a plus maze. At the beginning of each test, the rat was put on two arms, each in a different place. Food was present in one of the arms, but not in the other. The rat was then given a choice between these two places; the correct response was to return to the place that previously contained food (win-stay, lose-shift, response-reinforcement contingency). Fifty different two-choice spatial discriminations were given, each in a different location. At the end of testing, the mean percentage of correct responding for the first choice between the two places was 83%. Control procedures showed that the discriminative stimuli were distal, extramaze spatial stimuli. Variations of the procedure examined the influence of proactive interference and temporal delay on the memory for each discrimination. These results demonstrate that rats can develop a spatial learning set and provide new information about the characteristics of the memory underlying learning sets.     

Hippocampal stimulation disrupts spatial working memory even 8 h after acquisition

The present experiment used hippocampal stimulation to determine the temporal gradient of consolidation of spatial working memory. Rats were trained to perform a spatial working memory task on a radial maze with 12 arms. Each rat went to the ends of 6 arms to obtain a food reward. After 8 h, the rat chose among all the arms to find the ones not previously chosen (and consequently still having food). During some test sessions, the hippocampus was stimulated electrically either at a current level just high enough to produce an electrophysiological seizure, or at a current level below this seizure threshold. Stimulation occurred at one of five intervals (0 to 8 h) following the completion of the first six choices. During other test sessions, the hippocampus was not stimulated. After seizure stimulation, the number of retroactive errors (returning to arms chosen prior to stimulation) increased at all delay intervals; the number of proactive errors (returning to arms chosen after stimulation) increased only with the delay of 8 h. Subthreshold stimulation had no influence on either type of error. These results indicate that normal hippocampal function is required for the maintenance of spatial information in working memory, and that the time course of consolidation of this information is significantly greater than that seen in other types of memory, or consolidation may not take place at all.     

Wistar rats with high versus low rearing activity differ in radial maze performance

Substantial work has shown that rats although identical in stock, sex, age, and housing conditions can differ considerably in terms of behavior and physiology. Such individual differences, which can be detected by specific behavioral screening tests, are rather stable, that is, they probably reflect a behavioral disposition or trait. Here, we asked whether and how such differences might affect performance in a task of spatial learning and memory, the radial maze. As in our previous work, we used the degree of rearing activity in a novel open field to assign male adult outbred Wistar rats into those with high versus low rearing activity (HRA/LRA rats). They were then tested in a plus-maze for possible differences in anxiety-related behavior. Finally, and most importantly, they were food deprived and underwent maze training using an 8-arm radial maze with four non-baited and four baited arms. One of these arms consistently contained a larger bait size than the other three. In the open field, HRA rats not only showed more rearing behavior, but also more locomotor activity than LRA rats. In the plus-maze, HRA rats again showed more locomotion, but did not differ in open arm time or percentage of open arm entries, that is, conventional measures of anxiety-related behavior. In the radial maze, HRA rats consistently needed less time to consume all pellets than LRA rats, which was due to faster locomotion on the arms and less time spent at the food pits (especially in baited arms) of HRA rats. During the initial days of training, they were also more efficient in obtaining all food pellets available. Furthermore, HRA rats visited more arms and made relatively less reference memory errors than LRA rats. This allowed them to forage food quickly, but was paralleled by more working memory errors than in LRA rats. In general, working memory errors were more frequent in the arm with the large bait size, but there were no indications that HRA and LRA rats responded differently dependent on reward size. Finally, LRA rats lost slightly more weight than HRA rats during the period of food deprivation. These results are discussed with respect to the role of cognitive and motivational mechanisms, which as subject-inherent factors can contribute substantially to inter-individual variability in the radial maze.     

Working memory theory of hippocampal function needs qualification

To compare the predictive value of "cognitive map" and "working memory" theories of hippocampal function, the performance of rats with dorsal hippocampal lesions was compared to that of control rats in a series of experiments. In Experiment I, experimental rats learned a spatial alternation task with normal ease, but in Experiment II, they were significantly impaired on an elevated 8-arm radial maze. In Experiment III, the performance of the same experimental and control rats was compared on two versions of a 16-arm enclosed radial maze. In the first version, carpet inserts served as cues to mark eight unbaited arms and each of the remaining arms contained one food pellet. While both experimental and control rats successfully avoided the set of cued arms, experimental rats reentered uncued baited arms more frequently than did control rats. In the second version no intramaze cues were provided, but the spatial distribution of baited and unbaited arms remained the same as that used in the first version. In this uncued version, experimental rats both entered unbaited arms and reentered baited arms more frequently than did control rats, i.e., they were impaired in both "reference" and "working" memory. These findings are compatible with the hypothesis that hippocampal lesions result in an impaired capacity to form cognitive maps but they are not compatible with the working memory hypothesis. Furthermore, twelve separate evaluators classed experimental rats as using fewer mapping and more orientation strategies than control rats in the 8-arm maze.     

Spatial memory for food hidden by rats (Rattus norvegicus) on the radial maze: studies of memory for where,what, and when

Rats (Rattus norvegicus) were allowed to hide food items on an 8-arm radial maze by carrying the items from the center to boxes at the end of each arm. Retrieval tests given after rats had hidden 4 items showed that they selectively returned to the maze arms where food had been hidden (Experiments 1 and 2). When rats were allowed to hide pieces of cheese (refed food) and pretzels (less preferred food) on different arms, they both hid and retrieved cheese before pretzels (Experiments 2-5). In Experiment 6, rats chose between arms where cheese and pretzels were hidden,with cheese degraded at one delay interval but not the other. Together, these experiments indicate memory for what and where but not when.     

Involvement of the rat anterior cingulate cortex in control of instrumental responses guided by reward expectancy

The anterior cingulate cortex (ACC) plays a critical role in stimulus-reinforcement learning and reward-guided selection of actions. Here we conducted a series of experiments to further elucidate the role of the ACC in instrumental behavior involving effort-based decision-making and instrumental learning guided by reward-predictive stimuli. In Experiment 1, rats were trained on a cost-benefit T-maze task in which they could either choose to climb a barrier to obtain a high reward (four pellets) in one arm or a low reward (two pellets) in the other with no barrier present. In line with previous studies, our data reveal that rats with quinolinic acid lesions of the ACC selected the response involving less work and smaller reward. Experiment 2 demonstrates that breaking points of instrumental performance under a progressive ratio schedule were similar in sham-lesioned and ACC-lesioned rats. Thus, lesions of the ACC did not interfere with the effort a rat is willing to expend to obtain a specific reward in this test. In a subsequent task, we examined effort-based decision-making in a lever-press task where rats had the choice between pressing a lever to receive preferred food pellets under a progressive ratio schedule, or free feeding on a less preferred food, i.e. lab chow. Results show that sham- and ACC-lesioned animals had similar breaking points and ingested comparable amounts of less-preferred food. Together, the results of Experiment 1 and 2 suggest that the ACC plays a role in evaluating how much effort to expend for reward; however, the ACC is not necessary in all situations requiring an assessment of costs and benefits. In Experiment 3 we investigated learning and reversal learning of instrumental responses guided by reward predictive stimuli. A reaction time (RT) task demanding conditioned lever release was used in which the upcoming reward magnitude (five vs. one food pellet) was signalled in advance by discriminative visual stimuli. Results revealed that rats with ACC lesions were able to discriminate reward magnitude-predictive stimuli and to adapt instrumental behavior to reversed stimulus-reward magnitude contingencies. Thus, in a simple discrimination task as used here, the ACC appears not to be required to discriminate reward magnitude-predictive stimuli and to use the learned significance of the stimuli to guide instrumental behavior.     

How rats perform spatial working memory tasks: Limitations in the use of egocentric and idiothetic working memory

Rats of the Dark Agouti strain were trained on delayed alternation under conditions that should encourage egocentric working memory. In two experiments a T-maze was set within a cross-maze so that different arms could be used for the sample and test runs. The maze had high opaque side-walls, and testing was conducted in low light levels so that distal visual cues might be eliminated. By rotating the maze 90° between the sample and choice run and by using two identical mazes set side by side it was possible to nullify other spatial strategies. Experiments 1 and 2 showed that rats preferentially used place information, intramaze cues, and direction cues, even though only egocentric or idiothetic (nonmatch-to-turn) working memory could successfully solve every trial. Rats were able to maintain an accurate sense of location within the maze even though distal cues were not visible and the animal was moved between the sample and choice runs. Experiment 2 confirmed that another rat strain (Long-Evans) shows the same learning profiles. Both experiments indicate that rats are very poor at using either egocentric or idiothetic information to alternate, and that retention delays as short as 10 s can eliminate the use of these forms of memory.     

Gerbils in space: performance on the 17-arm radial maze.

In Experiment 1 six hungry gerbils received six trials per day on a 17-arm radial maze. During each trial the subjects were allowed to choose freely among the arms, each of which contained a food pellet, until each arm had been visited once or until eight minutes had elapsed. An error was recorded when the subject entered a previously visited arm. The gerbils quickly learned not to re-enter previously visited arms and generally made errors on fewer than 15% of entries, performance comparable to that of the rat and superior to that of other species tested in the radial arm maze. The intertrial-interval duration did not affect accuracy of arm choices during acquisition but did influence asymptotic accuracy. Accuracy did not change systematically over the six trials. A high proportion of arm entries were to nearby arms. Errors occurred most often towards the end of a trial. Odor cues were not important. When the number of trials per day was reduced from six to one, accuracy deteriorated slightly. In Experiment 2 neither the transposition of extramaze cues nor the placement of the maze in a different room had large disruptive effects on accuracy. In Experiment 3 the addition of three explicit intramaze brightness cues aided accuracy, perhaps by permitting the subjects to decompose the large maze into three smaller mazes, although there was no direct evidence that this was the case. Implications of a number of these results for models of spatial maze performance were discussed.     

How rats perform spatial working memory tasks: limitations in the use of egocentric and idiothetic working memory

Rats of the Dark Agouti strain were trained on delayed alternation under conditions that should encourage egocentric working memory. In two experiments a T-maze was set within a cross-maze so that different arms could be used for the sample and test runs. The maze had high opaque side-walls, and testing was conducted in low light levels so that distal visual cues might be eliminated. By rotating the maze 90° between the sample and choice run and by using two identical mazes set side by side it was possible to nullify other spatial strategies. Experiments 1 and 2 showed that rats preferentially used place information, intramaze cues, and direction cues, even though only egocentric or idiothetic (nonmatch-to-turn) working memory could successfully solve every trial. Rats were able to maintain an accurate sense of location within the maze even though distal cues were not visible and the animal was moved between the sample and choice runs. Experiment 2 confirmed that another rat strain (Long-Evans) shows the same learning profiles. Both experiments indicate that rats are very poor at using either egocentric or idiothetic information to alternate, and that retention delays as short as 10 s can eliminate the use of these forms of memory.     

Visual discrimination learning with a 1-min delay of reward

Eight rats were successfully trained in a black-white discrimination with a 1-min delay of reward. The procedure was unusual in that the rat spent the delay outside the apparatus in its home cage. Immediately after the rat responded, whether correctly or incorrectly, it was removed from the choice compartment and placed in its home cage. When the delay ended, it was returned to the startbox. If the preceding response had been correct, the rat received a reward of sugar water; otherwise, it was allowed to make another choice response. Mediation by external cues was excluded because there was no difference in the way the rats were treated after a correct or an incorrect response until the delay interval ended. Mediation by proprioceptive stimuli was excluded because position was an irrelevant cue.     

Damage to the retrosplenial cortex produces specific impairments in spatial working memory

Mounting evidence indicates that the retrosplenial cortex (RSP) has a critical role in spatial navigation. The goal of the present study was to characterize the specific nature of spatial memory deficits that are observed following damage to RSP. Rats with RSP lesions or sham lesions were first trained in a working memory task using an 8-arm radial arm maze. Rats were allowed 5 min to visit each arm and retrieve food pellets and a 5-s delay was imposed between arm choices. Consistent with previous research, rats with RSP damage committed more errors than controls. In particular, RSP-lesioned rats committed more errors of omission (failing to visit an arm of the maze), but there were no lesion effects on errors of commission (revisiting an arm). Neither group of rats exhibited a turn bias (i.e., always turning a certain direction when choosing an arm). At the end of the training phase of the experiment, both groups had reached asymptote and committed very few errors. In the subsequent test phase, a longer delay (30-s) was imposed during some sessions. Both control and RSP-lesioned rats continued to make few errors during sessions with the standard 5-s delay, but RSP-lesioned rats were impaired at the 30-s delay and committed more errors of commission, consistent with an increase in taxing spatial working memory.     

Acute Ethanol Administration Impairs Spatial Performance While Facilitating Nonspatial Performance in Rats

Acute ethanol administration produces learning and memory impairments similar to those found following lesions to the hippocampal system in rats. For example, both ethanol and hippocampal lesions impair performance on spatial learning and memory tasks while sparing performance on many nonspatial learning and memory tasks. Lesions to the hippocampal system can also alter the nature of the information that the animal uses to guide its behavior, from using spatial information to using individual cues. In the present experiment, rats were trained, while sober, to navigate on an eight-arm radial arm maze to a specific arm for food reward. During training, the rewarded arm was always in the same specific location and contained well-defined cues. After the rat learned the task, a memory test was conducted under different doses of ethanol (0.0 g/kg [saline control], 1.0, 1.5, or 2.0 g/kg, intraperitoneal). On the test day the maze was rotated so that the cued arm was 90 degrees to the right of its original position. During testing, intact rats showed a significant bias to approach the place where they had been previously rewarded, even though the cue was no longer located there. Acute ethanol administration dose dependently reduced approaches to the rewarded place. However, ethanol administration did not result in increases in random choices; rather, it resulted in a dose-dependent increase in approaches to the cued arm, now in a new location. These results extend previous research showing that acute ethanol administration and lesions to the hippocampal system produce similar effects on learning and memory in rats.     

Food priming and short-term retention in the delayed alternation task

The delayed alternation task, with rats as subjects, was used to assess the effects of a priming food reward on subsequent retention of a reinforced target location in a T maze. Subjects received a pretrial feeding, followed by a forced-choice rewarded entry into one goal box of the maze (the “cuing” run). Following a delay interval (5–60 s), the subjects were allowed to choose either goal, with reward available only in the one not entered on the cuing run. Priming significantly reduced such alternation after 30- and 60-s delays (Experiment 1), whereas administering the reward following the cuing run had no effect on choice behavior (Experiment 2). In the third study, prefeeding had a decremental effect when the following cuing run was reinforced, but no decrement occurred when reinforcement was omitted on the cuing run. The results suggest that food reward enhances retention of spatial information, but priming reduces this effect.     

Prefrontal cortex and hippocampus subserve different components of working memory in rats

Both the medial prefrontal cortex (mPFC) and hippocampus are implicated in working memory tasks in rodents. Specifically, it has been hypothesized that the mPFC is primarily engaged in the temporary storage and processing of information lasting from a subsecond to several seconds, while the hippocampal function becomes more critical as the working memory demand extends into longer temporal scales. Although these structures may be engaged in a temporally separable manner, the extent of their contributions in the "informational content" of working memory remains unclear. To investigate this issue, the mPFC and dorsal hippocampus (dHPC) were temporarily inactivated via targeted infusions of the GABA(A) receptor agonist muscimol in rats prior to their performance on a delayed alternation task (DAT), employing an automated figure-eight maze that required the animals to make alternating arm choice responses after 3-, 30-, and 60-sec delays for water reward. We report that inactivation of either the mPFC or dHPC significantly reduced DAT at all delay intervals tested. However, there were key qualitative differences in the behavioral effects. Specifically, mPFC inactivation selectively impaired working memory (i.e., arm choice accuracy) without altering reference memory (i.e., the maze task rule) and arm choice response latencies. In contrast, dHPC inactivation increased both reference memory errors and arm choice response latencies. Moreover, dHPC, but not mPFC, inactivation increased the incidence of successive working memory errors. These results suggest that while both the mPFC and hippocampus are necessarily involved in DAT, they seem to process different informational components associated with the memory task.