The role of exploration in win-shift and win-stay performance on a radial maze

Win-shift spatial memory tasks in a radial maze reinforce animals for avoiding previously visited rewarded arms; win-stay tasks reinforce them for returning to those arms. Win-shift tasks have generally been found much easier to perform, and this may be explained either in terms of foraging models which postulate avoidance of locations where food has been found, or in terms of the predominance of spontaneous alternation (exploration). Experiment 1 examined spontaneous alternation behavior in the radial maze as a function of whether the first visit to an arm had been rewarded or not, and showed that alternation was more probable after nonreward than after reward in both hungry and thirsty rats (a result which conflicts with the foraging account of the win-shift superiority). Experiment 2 replicated the finding that win-stay discrimination performance was inferior to win-shift. A manipulation (lengthening the delay between initial and test choices) which weakens spontaneous alternation, reduced, but did not reverse, the win-shift superiority. In Experiment 3, in order to eliminate the influence of spontaneous alternation, versions of the win-stay and win-shift tasks were devised in which, unlike the original task, all arms were familiar at the choice trial. Under those conditions win-stay was performed better than win-shift. It is concluded that spontaneous alternation plays a major role in many spatial memory tasks, and that the results can best be accounted for by combining principles of exploration and simple associative learning, without recourse to foraging models.     

Sex differences in aversive memory in rats: Possible role of extinction and reactive emotional factors

Studies usually show better spatial learning in males and stronger emotional memory in females. Spatial memory differences could relate to diverse strategies, while dissimilar stress reactions could cause emotional memory differences. We compared male and female rats in two emotional (classical emotional conditioning and aversive discrimination memory) and two emotionally “neutral” tasks: (1) plus-maze discriminative avoidance, containing two open and two enclosed arms, one of which presenting aversive stimuli (light/noise). No differences were found in learning, retrieving, or basal emotional levels, while only male rats presented extinction of the task; (2) contextual fear conditioning – a cage was paired to mild foot shocks. Upon reexposure, freezing behavior was decreased in females; (3) spontaneous alternation – the animals were expected to alternate among the arms of a four-arm maze. No differences between genders were found and (4) open-field habituation was addressed in an arena which the rats were allowed to explore for 10 min. Habituation was similar between genders. Differences were found only in tasks with strong emotional contexts, where different fear responses and stress effects could be determinant. The lack of extinction of discriminative avoidance by females points out to stronger consolidation and/or impaired extinction of aversive memories.     

Extinction-induced "despair" in the water maze, exploratory behavior and fear: effects of chronic antidepressant treatment

In former studies, we found evidence for the hypothesis that withdrawal of negative reinforcement presents a major source for stress and despair. Specifically, the removal of a hidden platform in the water maze induced extinction of previously reinforced escape behavior and behavioral immobility, indicative of "despair", which also correlated with indices of fear. Here, we tested the effects of antidepressants on extinction in the water maze, and expected that such drugs would attenuate the rate of extinction of a conditioned place preference (CPP) and also any emotionally relevant behavior that is induced by the loss of reinforcement, such as immobility. Adult male Wistar rats were trained to escape onto a hidden platform for 10 days. Daily treatment with desipramine hydrochloride (DMI, 10mg/kg) or fluoxetine (FLX, 10 mg/kg) commenced 1 day before the first of 11 extinction trials without the platform, administered 48 h apart, and continued thereafter, as the rats were tested in an open field and elevated-plus maze. As compared to controls, DMI increased the resistance-to-extinction of CPP, attenuated immobility, and increased wall climbing behavior. In the open field, DMI reduced activity levels, but was without effect on traditional fear parameters in the elevated-plus maze. FLX, by contrast, increased immobility during the extinction trials and fear in the elevated-plus maze. The withdrawal of reinforcement induced "despair" that was alleviated by the noradrenaline reuptake inhibitor DMI. The effects of the selective serotonin reuptake inhibitor FLX on immobility and fear may be explained in terms of its side effect profile.     

Food priming and short-term retention in the delayed alternation task

The delayed alternation task, with rats as subjects, was used to assess the effects of a priming food reward on subsequent retention of a reinforced target location in a T maze. Subjects received a pretrial feeding, followed by a forced-choice rewarded entry into one goal box of the maze (the “cuing” run). Following a delay interval (5–60 s), the subjects were allowed to choose either goal, with reward available only in the one not entered on the cuing run. Priming significantly reduced such alternation after 30- and 60-s delays (Experiment 1), whereas administering the reward following the cuing run had no effect on choice behavior (Experiment 2). In the third study, prefeeding had a decremental effect when the following cuing run was reinforced, but no decrement occurred when reinforcement was omitted on the cuing run. The results suggest that food reward enhances retention of spatial information, but priming reduces this effect.     

Contributions of dorsal striatal subregions to spatial alternation behavior

Considerable evidence has shown a clear dissociation between the dorsomedial (DMS) and the dorsolateral (DLS) striatum in instrumental conditioning. In particular, DMS activity is necessary to form action-outcome associations, whereas the DLS is required for developing habitual behavior. However, few studies have investigated whether a similar dissociation exists in more complex goal-directed learning processes. The present study examined the role of the two structures in such complex learning by analyzing the effects of excitotoxic DMS and DLS lesions during the acquisition and extinction of spatial alternation behavior, in a continuous alternation T-maze task. We demonstrate that DMS and DLS lesions have opposite effects, the former impairing and the latter improving animal performance during learning and extinction. DMS lesions may impair the acquisition of spatial alternation behavior by disrupting the signal necessary to link a goal with a specific spatial sequence. In contrast, DLS lesions may accelerate goal-driven strategies by minimizing the influence of external stimuli on the response, thus increasing the impact of action-reward contingencies. Taken together, these results suggest that DMS- and DLS-mediated learning strategies develop in parallel and compete for the control of the behavioral response early in learning.     

Sound sequence discrimination learning motivated by reward requires dopaminergic D2 receptor activation in the rat auditory cortex

We have previously reported that sound sequence discrimination learning requires cholinergic inputs to the auditory cortex (AC) in rats. In that study, reward was used for motivating discrimination behavior in rats. Therefore, dopaminergic inputs mediating reward signals may have an important role in the learning. We tested the possibility in the present study. Rats were trained to discriminate sequences of two sound components, and licking behavior in response to one of the two sequences was rewarded with water. To identify the dopaminergic inputs responsible for the learning, dopaminergic afferents to the AC were lesioned with local injection of 6-hydroxydopamine (6-OHDA). The injection attenuated sound sequence discrimination learning, while it had no effect on discrimination between the sound components of the sequence stimuli. Local injection of 6-OHDA into the nucleus accumbens attenuated sound discrimination learning. However, not only discrimination learning of sound sequence but also that of the sound components were impaired. SCH23390 (0.2 mg/kg, i.p.), a D1 receptor antagonist, had no effect on sound sequence discrimination learning, while it attenuated the licking behavior to unfamiliar stimuli. Haloperidol (0.5 mg/kg, i.p.), a D2 family antagonist, attenuated sound sequence discrimination learning, while it had no clear suppressive effect on discrimination of two different sound components and licking. These results suggest that D2 family receptors activated by dopaminergic inputs to the AC are required for sound sequence discrimination learning.     

Rats use a sense of direction to alternate on T-mazes located in adjacent rooms

Lister hooded rats were trained on a forced-sample T-maze alternation task in an environment lacking spatial landmarks. An early study of spontaneous alternation on the T-maze had shown that rats use a "spatial sense" to select alternate maze arms across mazes. As this phenomenon may provide a useful tool for studying the neural substrates of a directional sense, we wished to confirm this finding on a different version of the T-maze task, with well-trained animals. We found that rats successfully selected the appropriate maze arm when the choice phase of the task was presented on a second maze, oriented in the same direction, and located in an adjacent room. However, choice performance fell to chance level when the second maze was oriented 90° relative to the first. This result suggests that the rats do not simply alternate turns across the two environments, but rather that they rely on a sense of direction that is carried across environments. Electronic Publication     

Response biases do not underlie the radial maze deficit in rats with mediodorsal thalamus lesions

Previous results indicating that radial maze performance in animals with mediodorsal thalamic lesions is deficient cannot exclude the possibility that these impairments are due to altered motor mechanisms (response biases). The present study sought to eliminate this potentially confounding variable by using a procedure which tests memory for serial position. This procedure involved experimenter-controlled arm entry (number and order) on a radial arm maze. Following this sequence, animals were presented with one previously entered arm along with an arm not yet visited on that trial. Avoidance of the previously entered arm constituted memory for the prior sequence. Thus, this task represents a form of a win-shift or nonmatching-to-sample task. Seven animals were given ibotenic acid lesions of the mediodorsal nucleus and nine others were given sham operations; 2 weeks later testing in the above procedure was conducted. Results indicated that, although control subjects could differentiate between entered and unentered arms without difficulty, animals with lesions were unable to exhibit this distinction. Much of their memory for arms previously entered in a sequence was at chance level, regardless of the placement of the tested arm in the sequence. Some tendency toward increased errors with longer sequences of arm entries was noted. This may indicate that animals with lesions were susceptible to proactive interference from previous choices. Regardless, even without the opportunity to develop or exhibit response biases, animals lesioned in the mediodorsal nucleus were unable to perform this win-shift task reliably. Thus, discrimination among maze arms is impaired after lesion of the mediodorsal nucleus and this impairment is independent of the motor response patterns which emerge during solution of a conventional radial maze task.     

"Despair" induced by extinction trials in the water maze: relationship with measures of anxiety in aged and adult rats

We have previously reported that extinction of escape behavior in the water maze due to the removal of the platform coincided with the development of behavioral "despair" in aged and adult rats, as assessed by immobility. The present study examines further predictions derived from the hypothesis that the withholding of reinforcement induces behaviors akin to depression. We tested for correlations between extinction performance and immobility, as well as between immobility and measures of anxiety in aged and adult rats. Age comparisons were also performed on these variables. Forty aged and 29 adult male Wistar rats (24 and 3 months old, respectively) were examined in the open field, black/white box and elevated-plus maze followed by 6 days of training in the water maze hidden platform task and 8 days of extinction without the platform. Indices of immobility increased over trials of extinction, with the aged showing higher levels, earlier onsets and larger slope increases of immobility than the adults. A lower resistance-to-extinction was predictive of more "despair" in both age groups. Between-group differences in the open field, black/white box and elevated-plus maze indicated that the aged showed more anxiety-like behavior than the adults and/or explored these environments less. Within the aged group, indicators of fearfulness in the three tests were predictive of higher levels of "despair". The extinction-despair model is held to provide the promise of a conceptual and empirical model of human depression that is the consequence of withdrawal of reinforcement.     

Longitudinal attentional engagement rescues mice from age-related cognitive declines and cognitive inflexibility

Learning, attentional, and perseverative deficits are characteristic of cognitive aging. In this study, genetically diverse CD-1 mice underwent longitudinal training in a task asserted to tax working memory capacity and its dependence on selective attention. Beginning at 3 mo of age, animals were trained for 12 d to perform in a dual radial-arm maze task that required the mice to remember and operate on two sets of overlapping guidance (spatial) cues. As previously reported, this training resulted in an immediate (at 4 mo of age) improvement in the animals' aggregate performance across a battery of five learning tasks. Subsequently, these animals received an additional 3 d of working memory training at 3-wk intervals for 15 mo (totaling 66 training sessions), and at 18 mo of age were assessed on a selective attention task, a second set of learning tasks, and variations of those tasks that required the animals to modify the previously learned response. Both attentional and learning abilities (on passive avoidance, active avoidance, and reinforced alternation tasks) were impaired in aged animals that had not received working memory training. Likewise, these aged animals exhibited consistent deficits when required to modify a previously instantiated learned response (in reinforced alternation, active avoidance, and spatial water maze). In contrast, these attentional, learning, and perseverative deficits were attenuated in aged animals that had undergone lifelong working memory exercise. These results suggest that general impairments of learning, attention, and cognitive flexibility may be mitigated by a cognitive exercise regimen that requires chronic attentional engagement.     

Knockdown of Nurr1 in the rat hippocampus: implications to spatial discrimination learning and memory

Nurr1 expression is up-regulated in the brain following associative learning experiences, but its relevance to cognitive processes remains unclear. In these studies, rats initially received bilateral hippocampal infusions of control or antisense oligodeoxynucleotides (ODNs) 1 h prior to training in a holeboard spatial discrimination task. Such pre-training infusions of nurr1 antisense ODNs caused a moderate effect in learning the task and also impaired LTM tested 7 d later. In a second experiment, ODN infusions were given immediately after the animals had received two sessions of training, during which all animals showed normal learning. Although antisense treated rats were significantly impaired during the post-infusion stages of acquisition of the task, no group differences were observed during the LTM test given 7 d later. These animals were subjected 3 d later to reversal training in the same maze in the absence of any additional treatments. Remarkably, rats previously treated with antisense ODNs displayed perseveration: The animals were fixated with the previously learned pattern of baited holes, causing them to be significantly impaired in the extinction of acquired spatial preferences and future learning. We postulate that Nurr1 function in the hippocampus is important for normal cognitive processes.     

Extinction of appetitive learning is disrupted by cycloheximide and propranolol in the sand maze in rats

The present study investigated whether memory for extinction in an appetitive task (the sand maze) could be attenuated by administration of cycloheximide (protein synthesis inhibitor) or propranolol (β-adrenergic receptor antagonist). Ninety-day-old male Long-Evans rats were trained to retrieve a sweet cereal reinforcer from an open container in the sand maze. One day following this non-spatial training, rats received three extinction trials in which they were placed in the maze with the reinforcer present, but unattainable. Thirty minutes prior to the first extinction trial, rats received an intraperitoneal injection of cycloheximide (1mg/kg), propranolol (25mg/kg), or vehicle (1mg/kg distilled water). Twenty-four hours later, rats were tested in the sand maze with the reinforcer again available. Results from the test trial showed that both cycloheximide and propranolol groups found the reinforcer more quickly than controls. Two weeks later, rats were trained on a spatial version of the sand maze in which they had to search for a buried reinforcer using extramaze cues. Cycloheximide and propranolol groups learned this task significantly faster than the control group, demonstrating the long-lasting effect of cycloheximide and propranolol on the blocking of memory for extinction.     

Evidence of a role for multiple memory systems in behavioral extinction

The acquisition of learned behavior involves multiple memory systems, and hippocampal system damage impairs cognitive learning while leaving stimulus-response habit learning intact. In view of evidence that extinction also involves new learning, the present experiments examined whether multiple memory systems theory may be applicable to the neural bases of extinction. Adult Long-Evans rats were trained to run in a straight-alley maze for food reward. Twenty-four hours later, rats matched for runway latencies during acquisition received extinction training. In a response extinction condition conducive to habit learning, rats performed a runway approach response to an empty food cup. In a latent extinction condition conducive to cognitive learning, rats were placed at an empty food cup without performing a runway approach response. Prior to daily extinction training, neural activity of the dorsal hippocampus was reversibly inactivated via infusion of bupivacaine (0.75%, 0.5 microl/side). Control rats receiving saline infusions displayed extinction behavior in both the response and latent training conditions. In contrast, rats receiving bupivacaine extinguished normally in the response condition, but did not display latent extinction. The findings (1) confirm that learning underlying extinction of the same overt behavior can occur with or without explicit performance of the previously acquired response, (2) indicate that extinction learning produced by response and latent training procedures can be neuroanatomically dissociated, and (3) suggest that similarly to initial task acquisition, the hippocampus may critically mediate extinction in situations requiring the use of cognitive learning, such as when performance of a previously acquired response habit is prevented.     

Stress during puberty boosts metabolic activation associated with fear-extinction learning in hippocampus, basal amygdala and cingulate cortex

Adolescence is characterized by major developmental changes that may render the individual vulnerable to stress and the development of psychopathologies in a sex-specific manner. Earlier we reported lower anxiety-like behavior and higher risk-taking and novelty seeking in rats previously exposed to peri-pubertal stress. Here we studied whether peri-pubertal stress affected the acquisition and extinction of fear memories and/or the associated functional engagement of various brain regions, as assessed with 2-deoxyglucose. We showed that while peri-pubertal stress reduced freezing during the acquisition of fear memories (training) in both sexes, it had a sex-specific effect on extinction of these memories. Moreover hippocampus, basal amygdala and cingulate and motor cortices showed higher metabolic rates during extinction in rats exposed to peri-pubertal stress. Interestingly, activation of the infralimbic cortex was negatively correlated with freezing during extinction only in control males, while only males stressed during puberty showed a significant correlation between behavior during extinction and metabolic activation of hippocampus, amygdala and paraventricular nucleus. No correlations between brain activation and behavior during extinction were observed in females (control or stress). These results indicate that exposure to peri-pubertal stress affects behavior and brain metabolism when the individual is exposed to an additional stressful challenge. Some of these effects are sex-specific.     

Fornix, medial prefrontal cortex, nucleus accumbens, and mediodorsal thalamic nucleus: roles in a fear-based context discrimination task

The goal of the present study was to evaluate the contributions of various brain structures anatomically and functionally linked to the hippocampus and amygdala in a fear-based context discrimination task. The brain areas of interest included the fornix, medial prefrontal cortex, mediodorsal (MD) thalamic nucleus, and nucleus accumbens. Damage to the MD thalamic nucleus and medial prefrontal cortex produced the largest impairment in context-specific fear responses. Damage to the fornix impaired some fear responses (freezing, ultrasonic vocalizations, defecation, and approach/avoidance) while leaving conditioned fear expression of heart rate and urination unaltered. Damage to the nucleus accumbens was also coupled with deficits in the discriminative expression of some (heart rate, urination, and ultrasonic vocalizations) but sparing of context-appropriate freezing, defecation, and approach/avoidance behaviors.     

Spatial Location Learning in Mice with Ibotenate Lesions of Entorhinal Cortex or Subiculum

This study examined the effects of ibotenate lesions of either the entorhinal cortex (EC) or the subiculum (SUB) on the ability of mice to memorize a single spatial location (initial discrimination), and on their capacity to switch to a new location (transfer) following the initial learning in an eight-arm radial maze. Results indicated that mice with ibotenate lesions of the EC or SUB were impaired in postoperative acquisition of the spatial discrimination task, making more reference, but not working memory, errors and displaying fewer first correct response trials than sham-operated control mice. Furthermore, additional damage to the ventral hippocampus exacerbated the impairment of performance induced by lesions of the SUB alone. In addition, all mice, except for the combined lesion group, exhibited similar performance levels when they were trained to choose another arm of the maze that had not previously been baited (transfer). These findings suggest that both the EC and the SUB play important roles in spatial information processing in mice.     

Extinction training regulates neuroadaptive responses to withdrawal from chronic cocaine self-administration

Cocaine produces multiple neuroadaptations with chronic repeated use. Many of these neuroadaptations can be reversed or normalized by extinction training during withdrawal from chronic cocaine self-administration in rats. This article reviews our past and present studies on extinction-induced modulation of the neuroadaptive response to chronic cocaine in the mesolimbic dopamine system, and the role of this modulation in addictive behavior in rats. Extinction training normalizes tyrosine hydroxylase levels in the nucleus accumbens (NAc) shell, an effect that could help ameliorate dysphoria and depression associated with withdrawal from chronic cocaine use. Extinction training also increases levels of GluR1 and GluR2/3 AMPA receptor subunits, while normalizing deficits in NR1 NMDA receptor subunits, in a manner consistent with long-term potentiation of excitatory synapses in the NAc shell. Our results suggest that extinction-induced increases in AMPA and NMDA receptors may restore deficits in cortico-accumbal neurotransmission in the NAc shell and facilitate inhibitory control over cocaine-seeking behavior. Other changes identified by gene expression profiling, including up-regulation in the AMPA receptor aggregating protein Narp, suggest that extinction training induces extensive synaptic reorganization. These studies highlight potential benefits for extinction training procedures in the treatment of drug addiction.     

Radial-arm-maze behavior of the red-footed tortoise (Geochelone carbonaria)

The radial-arm maze is an established method for testing an animal's spatial win-shift behavior. Research on mammals, birds, and fish has shown that the mastery of this task is commonly mediated, to different degrees, by two types of strategy: those based on external cues and those based on response stereotypy. In the present study we trained four red-footed tortoises (Geochelone carbonaria) to navigate an eight-arm radial maze while providing different levels of access to visual room cues. The results indicate that response stereotypy is the more prevalent mechanism in these tortoises, although navigation based on landmarks can also occur if learned initially. The findings suggest that tortoise spatial navigation may be more similar to that observed in mammals and birds than previously thought.     

Glucocorticoids, the hippocampus, and behavior: interactive relation between task activation and steroid hormone binding specificity

A previous investigation established a modulatory effect of adrenal glucocorticoids on a behavior generally believed to require participation of the hippocampus for normal expression. For further delineation of whether the observed effects on appetitive extinction are consistent with a glucocorticoid action on hippocampus, adrenalectomized rats were administered either corticosterone or dexamethasone, a synthetic glucocorticoid that is accumulated much less intensively by the hippocampus than is corticosterone. While corticosterone normalized extinction, dexamethasone did not. In two other hippocampal-dependent behaviors, alternation and exploration, there was no glucocorticoid modulation of behavior, which may indicate that factors such as the level of activation of the type of learning determine whether a glucocorticoid effect will occur. These findings of a glucocorticoid action on one of three tasks that involve hippocampal function reinforce the notion that in addition to their well-studied actions in protein and carbohydrate metabolism, the adrenal glucocorticoids may also be subtle modulators of behavior.     

Realization,explanation and the mind-body relation

The Morris water maze has been put forward in the philosophy of neuroscience as an example of an experimental arrangement that may be used to delineate the cognitive faculty of spatial memory (e.g., Craver and Darden, Theory and method in the neurosciences, University of Pittsburgh Press, Pittsburgh, 2001; Craver, Explaining the brain: Mechanisms and the mosaic unity of neuroscience, Oxford University Press, Oxford, 2007). However, in the experimental and review literature on the water maze throughout the history of its use, we encounter numerous responses to the question of “what” phenomenon it circumscribes ranging from cognitive functions (e.g., “spatial learning”, “spatial navigation”), to representational changes (e.g., “cognitive map formation”) to terms that appear to refer exclusively to observable changes in behavior (e.g., “water maze performance”). To date philosophical analyses of the water maze have not been directed at sorting out what phenomenon the device delineates nor the sources of the different answers to the question of what. I undertake both of these tasks in this paper. I begin with an analysis of Morris’s first published research study using the water maze and demonstrate that he emerged from it with an experimental learning paradigm that at best circumscribed a discrete set of observable changes in behavior. However, it delineated neither a discrete set of representational changes nor a discrete cognitive function. I cite this in combination with a reductionist-oriented research agenda in cellular and molecular neurobiology dating back to the 1980s as two sources of the lack of consistency across the history of the experimental and review literature as to what is under study in the water maze.