The sensitivity of response rate to the rate of variable-interval reinforcement for pigeons and rats: a review

The relation between the rate of a response (B) and the rate of its reinforcement (R) is well known to be approximately hyperbolic: B = kR/(R + Ro), where k represents the maximum response rate, and Ro indicates the rate of reinforcers that will engender a response rate equal to half its maximum value. A review of data reported in 17 published papers revealed that, under variable-interval schedules of reinforcement, Ro was usually lower when pigeons were the subjects than when rats were the subjects. The value of k, in contrast, did not differ consistently between pigeons and rats. Some accounts interpret Ro as the rate of alternative, unscheduled reinforcers in the situation, expressed in units of the scheduled reinforcer. So interpreted, the difference in Ro implies that less alternative reinforcement (relative to the scheduled reinforcement) typically is available to pigeons in their operant conditioning chambers than it is to rats in theirs. Whether or not that interpretation of Ro is valid, the pigeon-rat difference in Ro ensures that for reinforcer rates above about 10 per hour, response rate will be noticeably less sensitive to changes in reinforcer rate (and presumably to changes in other incentive and motivational operations) with pigeons than with rats as subjects, at least with the experimental conditions typically employed.     

Those cheating rats: male and female rats use odor trails in a water-escape "working memory" task.

Three-month-old Sprague-Dawley rats were trained on a working memory win-stay (spatial delayed matching-to-sample) water-escape task with the escape platform location the same for all subjects on a given trial, a procedure that maximizes the buildup of an odor trail to the escape platform. In subsequent tests during which the location of the escape platform varied randomly between subjects, the rats, especially the females, while continuing to perform above chance level, made increased errors. Varying the platform location between subjects eliminated odor trail as a nonambiguous cue for locating the escape platform. In a second experiment females performed better than males on a reference memory odor trail discrimination task which involved following the path of like-gender "pathmaker" rats to the escape platform. The relatively poor use of odor trails by the males was associated with a high frequency of choosing a preferred choice section or returning to the choice section selected first on the immediately preceding trial (perseveration). Collectively, the two experiments demonstrate that rats can use either working memory or odor trails to locate an escape platform in a water maze, and that they, especially females, will use odor trails in a working memory task if odor trails are available. Clearly, the location of the escape platform should be varied randomly between subjects in tests of working memory.     

Prior exposure to inescapable electric shock in rats affects extinction behavior after the successful acquisition of an escape response

In Expt 1, rats exposed to 64 inescapable electric shocks in a restrainer or merely restrained were later given either 0, 5, 15 or 30 escape/avoidance training trials with a two-way shuttlebox procedure that does not lead to interference with escape acquisition due to prior exposure to inescapable shock. After escape training all rats were given an escape/avoidance extinction procedure in which shock was inescapable. The rats which had received prior exposure to inescapable shock responded less often and with longer latencies in extinction than did the restrained rats. Experiment 2 demonstrated that this effect is caused by the inescapability of the initial shock treatment. These results were explained in terms of (a) associative interference which minimized the effect of shuttlebox escape training for the preshocked subjects, and (b) a stronger tendency to recognize the presence of an inescapable shock situation during extinction for the preshocked subjects. The relationship between these results and previous work demonstrating that exposure to the escape contingency mitigates the effects of inescapable shock exposure was also discussed.     

Endogenous-like orienting of visual attention in rats

This study investigated the orienting of visual attention in rats using a 3-hole nose-poke task analogous to Posner, Information processing in cognition: the Loyola Symposium, Erlbaum, Hillsdale, (1980) covert attention task for humans. The effects of non-predictive (50% valid and 50% invalid) and predictive (80% valid and 20% invalid) peripheral visual cues on reaction times and response accuracy to a target stimulus, using Stimuli-Onset Asynchronies (SOAs) varying between 200 and 1,200 ms, were investigated. The results showed shorter reaction times in valid trials relative to invalid trials for both subjects trained in the non-predictive and predictive conditions, particularly when the SOAs were 200 and 400 ms. However, the magnitude of this validity effect was significantly greater for subjects exposed to predictive cues, when the SOA was 800 ms. Subjects exposed to invalid predictive cues exhibited an increase in omission errors relative to subjects exposed to invalid non-predictive cues. In contrast, valid cues reduced the proportion of omission errors for subjects trained in the predictive condition relative to subjects trained in the non-predictive condition. These results are congruent with those usually reported for humans and indicate that, in addition to the exogenous capture of attention promoted by both predictive and non-predictive peripheral cues, rats exposed to predictive cues engaged an additional slower process equivalent to human's endogenous orienting of attention. To our knowledge, this is the first demonstration of an endogenous-like process of covert orienting of visual attention in rats.     

Response discriminative and reinforcement factors in stimulus control of performance on multiple and chained schedules of reinforcement

Discriminative control of the response rates of two groups of rats was equated by training them to cease bar pressing in light-out no-tone ( + ) and to respond during tone and light. Multiple-schedule subjects received food at the same rate for responding during tone or light as for nonresponding in + . For the chained-schedule subjects responding in tone or light only produced + where food was received for nonresponding. In extinction tests multiple-schedule subjects emitted approximately twice the responses to tone-plus-light as to tone or light presented individually (additive summation). The rats trained on the chained schedule, in which the tone and light each controlled substantial response rates but were never paired with food, showed no summation when the tone and light were presented together. The results indicate that discriminative control of response rates and reinforcement differences between schedule components determine stimulus control.     

Infantile handling and the extinction of responses acquired under food reward

A comparison was made of the runway behaviour of rats which had been handled from Days 1-21 and their non-handled litter mates. The training began on Day 70 after the animals were habituated to a restricted food schedule for 10 days. The subjects were given six trials each day in the runway and were rewarded with a 0.045 g. Noyes pellet. After 10 days of rewarded training trials, subjects were given 6 extinction trials a day for 10 days. Results showed that handled rats ran faster than non-handled rats during acquisition and during the first 3 days of extinction. The extinction data suggested that the relationship between emotionality and effects of frustrative non-reward should be re-evaluated.     

Direct transfer from a horizontal-vertical discrimination to a brightness discrimination in the rat

Eight rats were trained on a discrimination between horizontal and vertical striations, and subsequently between black and white vertical striations. Subjects originally trained with horizontal positive preferred black to white and subjects trained with vertical positive preferred white to black.

The present note reports briefly on a transfer effect in rats, observed incidentally in an experiment designed for a different purpose. The rats were trained in a jumping stand to discriminate between horizontal and vertical striations, and when training was complete, were tested on a brightness discrimination.     

Response induction during the acquisition and maintenance of lever pressing with delayed reinforcement

The acquisition of lever pressing by rats and the occurrence of unreinforced presses at a location different from that of the reinforced response were studied using different delays of reinforcement. An experimental chamber containing seven identical adjoining levers was used. Only presses on the central (operative) lever produced food pellets. Groups of 3 rats were exposed to one of seven different tandem random-interval (RI) fixed-time (FT) schedules. The average RI duration was the complement of the FT duration such that their sum yielded a nominal 32-s interreinforcement interval on average. Response rate on the operative lever decreased as the FT value was lengthened. The spatial distribution of responses on the seven levers converged on the operative lever when the FT was 0 or 2 s and spread across the seven levers as the FT value was lengthened to 16 or 32 s. Presses on the seven levers were infrequent during the FT schedule. Both operative- and inoperative-lever pressing intertwined in repetitive patterns that were consistent within subjects but differed between subjects. These findings suggest that reinforcer delay determined the response-induction gradient.     

A temporal parameter influencing choice between signalled and unsignalled shock schedules

The present study investigated whether choice of a signalled variable-time shock schedule over an unsignalled one was influenced by the average intershock interval. Eight rats were given a choice between signalled and unsignalled shock schedules in a series of conditions with average intershock intervals of 510, 270, 150, 90, 60, and 45 sec. Each test condition was preceded by a training-baseline condition, and schedule values were arranged in an ascending (four subjects) or descending (four subjects) order. Choice of the signalled conditions was directly related to the average intershock interval of the variable-time schedule for six of the eight subjects. The per cent of time in the signalled condition was highest when the average intershock interval was 150 sec or longer and lowest when the average intershock interval was 45 sec. The findings were interpreted as being due to changes in the safety features of the signalled schedule, rather than to changes in the average intershock interval per se.     

Partial reinforcement and breadth of learning

Two experiments on partial reinforcement were undertaken to test predictions made by a two process model of discrimination learning. In the first experiment rats were trained on a discrimination involving two relevant cues: one group (C) was trained on a 100: o schedule, the other (P) on a 50:0 schedule. Both groups were then given transfer tests with the two cues presented individually; finally all animals were extinguished on the original training stimuli and on the single cue stimuli. During extinction there was a negative correlation between the number of correct responses made by individual subjects of Group C to each single cue; whereas the correlation was positive for subjects of Group P. The second experiment employed basically the same design, but subjects were trained with seven relevant cues. The results of transfer tests showed that subjects of Group P learned to attach the correct response to many more cues than subjects of Group C. This suggests that the breadth of learning is greater under partial than under consistent reinforcement. The results were predicted by the model of discrimination learning under test.     

Thiamin deprivation in ventromedial hypothalamic hyperphagic rats: anorexia, specificity of food aversion, and a dietary consideration.

The anorexic consequence of thiamin deprivation was investigated in ventromedial hypothalamic (VMH) hyperphagic rats under either high-fat or low-fat thiamin-free diet conditions. The low-fat diet maintained feeding significantly longer in thiamin-deprived VMH rats than in intact rats, whereas the hig-fat diet sustained feeding in thiamin-deficient intact rats and accelerated anorexia onset in vitamin B1 deprived VMH rats. This effect was noted under both ad lib and pair-feeding conditions. Thiamin-deprived VMH rats subjected to weight control developed anorexia sooner than intact subjects regardless of the diet employed. The VMH rats fed a high-fat diet failed to resume feeding after thiamin readministration, which was interpreted as a permanent aversion to this diet. The relation between dietary intake and conditioned taste aversion is discussed with reference to the VMH and intact rat.     

Choice of longer or stronger signalled shock over shorter or weaker unsignalled shock

Unsignalled, inescapable shocks were presented to four albino rats in one study and to six rats in a second study. By pressing a lever, subjects could change the condition to signalled shock for 3 min after which unsignalled shock was automatically reinstated. All subjects changed frequently to the signalled shock schedule. After a minimum of three 6-hr sessions or after changeover responding stabilized at the previous values, higher values of signalled shock intensity or duration were introduced. In the first study, the duration of signalled shock was increased in increments of 0.5 sec. In the second study, the intensity of signalled shock was increased in increments of either 0.2 or 0.4 mA. Duration subjects chose signalled shock four (2.0 sec) to nine times (4.5 sec) longer than unsignalled shock (0.5 sec). Intensity subjects chose signalled shock two (2.0 mA) to three times (3.0 mA) more intense than unsignalled shock (1.0 mA).     

Ontogenetic changes in the modulation of taste aversion learning by home environmental cues in rats

Eight experiments using 611 rats as subjects were conducted to define and analyze an age-related phenomenon of conditioned taste aversion. When consumption of sucrose solution was followed by LiCl-induced illness in the animals' home, acquisition of the aversion to sucrose solution was retarded in preweanling (18-day-old) rats. This effect was not found in adults or in slightly older (21-day-old) rats. Place of testing had no effect in the younger two age-groups, but in adults manifestation of the acquired aversion was retarded when they were tested in the home. There was no interaction between place of conditioning and testing for any age. The locus of the environmental influence on conditioning in preweanling rats was found to be the place of tasting rather than place of illness, retention interval, or testing. Also, the effect was found to be invariant under minor variations in familiarization of the animal with the non-home environment. The principle emerging from these data and others is that the home environment can have a significant influence on learning and conditioning in the immature rat.     

Extinction of conditioned flavor preferences

In a series of 4 experiments, the effects of extinction on flavor preferences conditioned by mixing flavor cues with a nutrient were examined. In each experiment it was observed that rats preferred a flavor cue that had not undergone extinction to one that had. In addition, this preference was reversed in subjects trained thirsty (Experiments 1 and 2) if the associated nutrient had been devalued prior to the test or the preference for the nonextinguished cue was attenuated by nutrient devaluation in subjects trained hungry (Experiments 3 and 4). The results suggest that extinction may weaken associations between the flavor and the specific sensory properties of the nutrient and, for subjects trained hungry, between the flavor and the motivational components of the nutrient as well.     

Behavioral variability in SHR and WKY rats as a function of rearing environment and reinforcement contingency.

The spontaneously hypertensive rat (SHR) may model aspects of human attention deficit hyperactivity disorder (ADHD). For example, just as responses by children with ADHD tend to be variable, so too SHRs often respond more variably than do Wistar-Kyoto (WKY) control rats. The present study asked whether behavioral variability in the SHR strain is influenced by rearing environment, a question related to hypotheses concerning the etiology of human ADHD. Some rats from each strain were reared in an enriched environment (housed socially), and others were reared in an impoverished environment (housed in isolation). Four groups--enriched SHR, impoverished SHR, enriched WKY, and impoverished WKY--were studied under two reinforcement contingencies, one in which reinforcement was independent of response variability and the other in which reinforcement depended upon high variability. The main finding was that rearing environment did not influence response variability (enriched and impoverished subjects responded similarly throughout). However, rearing environment affected body weight (enriched subjects weighted more than impoverished subjects) and response rate (impoverished subjects generally responded faster than enriched subjects). In addition, SHRs tended to respond variably throughout the experiment, whereas WKYs were more sensitive to the variability contingencies. Thus, behavioral variability was affected by genetic strain and by reinforcement contingency but not by the environment in which the subjects were reared.     

Dextrorphan effects on cocaine and brainstem perturbation

Dextrorphan is a noncompetitive blocker of N-methyl-D-aspartate (NMDA) receptors. Since NMDA blockers are known to reduce the locomotor stimulatory and toxic effects of cocaine, it was speculated that dextrorphan would attenuate cocaine-induced behavioral excitatory motor activity associated with and without mechanical perturbation of the brainstem. Technique: Motor activity was recorded following dextrorphan and/or cocaine challenge in 25 SHR rats. Ten were naive subjects. Mini-osmotic pumps delivering cocaine (2.5 mg/0.49 ul/hr) were placed in 15 subjects, and infusion was halted after the third infusion day. On the fifth day either a dextrorphan (25 mg/kg, subcutaneous) or a dextrorphan and cocaine (40 mg/kg, intraperitoneal) challenge was done. Ten rats had bipolar electrode implants in the bilateral brainstem. Five were treated with DC current lesions in each of 12 days over a 3-week period. The effects of brainstem lesions on escape behavior were also evaluated in those five subjects. Results: In the naive subjects, dextrorphan reduced motor activity (P=.0001), whereas combined cocaine and dextrorphan increased motor activity (P=0.04). In lesioned subjects, dextrorphan decreased motor activity (P=0.0001). In electrode implant subjects, combined dextrorphan and cocaine challenge decreased the motor activity (P=0.04). Hyperactivity in the electrode implant group was greater than in the lesioned subjects. Midbrain electrolytic lesions attenuated escape behavior. A variety of behaviors were produced by brainstem lesions. Conclusions: Dextrorphan and brainstem lesions reduced motor hyperactivity and escape behavior. In electrode implant subjects dextrorphan counteracted the expected cocaine excitatory motor effects. Dextrorphan did not activate nor facilitate seizures.     

Discounting of food and water in rats shows trait- and state-like characteristics

Delay discounting is the loss in value of an outcome as a function of its delay. The present study focused on examining a trait-like characteristic of delay discounting in a preclinical animal model. Specifically, we were interested in whether there was a positive relation between discounting of 2 different outcomes in rats. That is, would rats that discount delayed food steeply also discount delayed water steeply? In addition, we examined how session-to-session variability in discounting could be attributed to differences between subjects (trait variability) and to differences within subjects (state variability). Finally, we measured discounting from early- to mid-adulthood, allowing us to examine changes in discounting as a function of age. Overall, we found a moderate, positive correlation between discounting of food and discounting of water in rats, providing further evidence that the relative consistency with which individuals discount different outcomes is a trait-like characteristic. In addition, we found a high degree of within-subject variability in discounting, indicating strong state-like differences from session to session. Finally, overall, discounting decreased as a function of age; however, individual-subject data showed variability in how discounting changed across time. Overall, our results show that differences in delay discounting between individuals reflect variability in both trait- and state-like characteristics.     

The effects of hippocampal lesions on learning, memory, and reward expectancies

The hippocampus appears to be critical for the formation of certain types of memories. Hippocampal-lesioned animals fail to exhibit some spatial, contextual, and relational associations. After aspiration lesions of the hippocampus and/or cortex, male rats were allowed to recover for three weeks before being trained on a matching-to-position task. The matching-to-position task was altered to influence the type of cognitive strategies a subject would use to solve the task. The main behavioral manipulation was the reinforcement contingency assignment: Use of a differential outcomes procedure (DOP) or a nondifferential outcomes procedure (NOP). The DOP involves correlating each to-be-remembered event with a distinct reward condition via Pavlovian trace conditioning, whereas the NOP results in random reward contingency. We found that hippocampal lesions did retard learning the matching rule, regardless of the reinforcement contingency assignment. However, when delay intervals were added to the task memory performance of subjects with hippocampal lesions was dramatically impaired--if subjects were not trained with the DOP. When subjects were trained with the DOP, the hippocampal lesion had a marginal effect on delayed memory performance. These findings demonstrate two important points regarding lesions of the hippocampus: (1) hippocampal lesions have a minimal effect on the on the ability of rats to use reward information to solve a delayed discrimination task; (2) rats with hippocampal lesions have the ability to learn about reward information using Pavlovian trace conditioning procedures.     

Role of fear in mediating shuttle escape learning deficit produced by inescapable shock

The relation between the shuttlebox escape deficit produced by prior inescapable shock (IS) and fear during shuttlebox testing as assessed by freezing was investigated in rats. IS rats learned to escape poorly and were more fearful than either escapably shocked subjects or controls, both before and after receiving shock in the shuttlebox. However, fear and poor escape performance did not covary with the manipulation of variables designed to modulate the amount of fear and the occurrence of the escape deficit. A 72-hr interval between IS and testing eliminated the escape deficit but did not reduce preshock freezing. Diazepam before testing reduced both preshock and postshock fear in the shuttlebox but had no effect on the escape deficit. Naltrexone had no effect on fear but eliminated the escape deficit. This independence of outcomes suggests that the shuttlebox escape deficit is not caused by high levels of fear in IS subjects.