Orbital prefrontal cortex and guidance of instrumental behavior of rats by visuospatial stimuli predicting reward magnitude

The orbital prefrontal cortex (OPFC) is part of a circuitry mediating the perception of reward and the initiation of adaptive behavioral responses. We investigated whether the OPFC is involved in guidance of the speed of instrumental behavior by visuospatial stimuli predictive of different reward magnitudes. Unoperated rats, sham-lesioned rats, and rats with bilateral lesions of the OPFC by N-methyl-D-aspartate (NMDA) were trained in a visuospatial discrimination task. The task required a lever press on the illuminated lever of two available to obtain a food reward. Different reward magnitudes were permanently assigned to lever presses to respective sides of the operant chamber; that is, responses to one lever (e.g., the left one) were always rewarded with one pellet and responses to the other lever with five pellets. On each trial, the position of the illuminated lever was pseudorandomly determined in advance. Results revealed that OPFC lesions did not impair acquisition of the task, as the speed of conditioned responses was significantly shorter with expectancy of a high reward magnitude. In addition, during reversal, shift and reshift of lever position–reward magnitude contingencies and under extinction conditions, performance of the OPFC-lesioned and control groups did not differ. It is concluded that the OPFC in rats might not be critical for adapting behavioral responses to changes of stimulus–reward magnitude contingencies signaled by visuospatial cues.     

Rats (Rattus norvegicus) find occupancy of a restraint tube rewarding

Two experiments evaluated whether rats' occupancy of a restraint tube is reinforcing. In Experiment 1, each rat in the 0-min group moved freely in a chamber where a wall blocked access to a restraint tube. After 10 min the wall was removed, permitting 15 min of chamber access and tube entry. The other 2 groups were locked in the tube for 10 and 20 min respectively before release into the chamber for 15 min. Across sessions, rats locked up for 10 and 20 min entered the tube more frequently than rats in the 0-min group, and during the first 2 sessions rats in the 20-min group stayed in the tube longer than the other groups. Over sessions this difference disappeared. However, for all groups and sessions the mean percentage of session time in the tube exceeded chance expectations. This result suggests tube occupation was reinforcing. In Experiment 2’s Phase 1, rats could enter an open tube. On exiting, the tube door closed. A lever press opened the door for the rest of the 1-hr session. In Phase 2, these rats were locked in the tube for 10 min before the door opened. Upon exiting, the door closed. As in Phase 1, a lever press opened the door for the rest of the session. The latency between pressing and tube entry decreased over sessions, indicating that tube entry reinforced lever pressing. These results are difficult to reconcile with accounts of rat empathy based on the thesis that tube restraint distresses occupants.     

Some effects of response-independent shocks after unsignaled avoidance conditioning in rats

After extended unsignaled avoidance training, the majority of rats continued frequent responding during long periods when only response-independent shocks were presented. Most responses were shock elicited, i.e., followed immediately after shock. Response rates were directly related to both the frequency and the intensity of response-independent shocks. Responding continued to approximately the same extent, whether shocks were presented at fixed or variable intervals. Few responses were directed toward a second lever in the test chamber, even when the lever previously associated with avoidance was removed. When avoidance was scheduled on the second lever, the rats learned to avoid by operating it. Meanwhile, responding on the first lever became infrequent. Subsequently, when only response-independent shocks were presented, almost all responses occurred upon the lever initially associated with avoidance. Responding that was elicited by response-independent shocks was suppressed by response-dependent (punishment)_shocks of the same intensity. When punishment was withdrawn, recovery of responding occurred. An explanation of the results based upon shock-elicited behaviors is preferred to one stressing unextinguished avoidance responding.     

Automaintenance in guinea pigs: effects of feeding regimen and omission training

Behavior maintained by stimulus-reinforcer pairings was examined. Guinea pigs maintained at 85 per cent of free-feeding weights reliably contacted a retractable lever presented before delivery of a single piece of guinea-pig chow or a 45-milligram guinea-pig pellet. When animals were given free access to one food and received the second food preceded by the lever, contact responses persisted. Such responses seldom occurred when a single food was freely available and was also delivered after lever presentation. Introduction of an omission training (negative automaintenance) procedure, in which lever contacts resulted in lever retraction and prevented food delivery, strongly reduced lever contacts. Observation indicated that mouthing the food cup, instead of the lever, became the prominent behavior during the prefood stimulus under the omission training procedure.     

Lever for recording fish target bumping

A method for constructing a lever to record fish target bumping is described. The lever has a pivot assembly that is submerged in a fish chamber. A photoelectric system is activated when a target is moved. The lever requires very little force. Force requirements and the distance that the lever travels can be changed easily.     

Group behavior of rats under schedules of reinforcement

Groups of three rats were placed in a chamber containing one response lever and one water dispenser. A variety of schedule conditions were explored including fixed ratio, extinction, satiation, fixed interval, fixed time, differential reinforcement of low rates, and discrimination learning. Each group was treated as a single unit, with the collective lever responses emitted by the three rats being the main dependent variable. Group responding was found to be controlled by the reinforcement schedules in an orderly and consistent manner. However, the groups often paused less and responded faster than individual rats working under identical conditions.     

Conditioned suppression of counting behavior in rats

Three rats were trained on a schedule in which a response on lever B was reinforced only if it was preceded by a minimum number of consecutive responses on lever A. The minimum requirement was 27 A responses for Rat 1, and 20 A responses for Rats 2 and 3. The schedule maintained high rates of responding on lever A, and a slow, spaced pattern of responding on lever B. The mean number of consecutive responses on lever A was slightly greater than the minimum required. The effect of superimposing on this behavior a stimulus that ended with an unavoidable shock was the suppression of responding on both levers during the pre-shock stimulus. Responses on lever A were more suppressed, and the proportion of relatively short response runs on lever A during the pre-shock stimulus increased. With all three rats, the mean number of consecutive responses on lever A during the pre-shock stimulus decreased to a value below the minimum requirement for reinforcement of the subsequent B response.     

Temporal control in a complex environment: An analysis of schedule-related behavior

Three experiments conducted in an automated ten-compartment chamber recorded collateral activities of rats reinforced for lever pressing on differential-reinforcement-of-low-rate schedules. In Experiment 1, the rate of lever pressing increased when stimulus support for collateral activities was removed, thus confirming earlier findings. However, there were no temporal or sequential patterns of collateral activities that predicted operant responding. In Experiment 2, the rate of lever pressing increased only if (a) access to all stimulus support for collateral activities was simultaneously prevented, and (b) the rat was forced to remain in the presence of the lever and food tray. The availability of any of the stimuli related to collateral activity was sufficient to keep lever-pressing rates from increasing. Experiment 3 examined collateral activities under a signaled differential-reinforcement-of-low-rate schedule. Preventing access to stimuli supporting collateral activities had little effect on stable lever pressing when the signal was maintained. When the signal was removed, collateral activities continued, but lever-pressing rates increased in three of the four rats and rates of food presentation declined in all rats. Hypotheses that collateral activities have (a) a timekeeping or discriminative function, or (b) directly inhibit operant responding were not supported. The results suggest that collateral activities may facilitate operant responding by simply removing the subject from the presence of reinforcement-related stimuli.     

Post-Conditioning Devaluation of an Instrumental Reinforcer has no Effect on Extinction Performance

The extent to which a representation of the reinforcer controls an instrumental response can be assessed by studying the effect of post-conditioning changes in the reinforcer value. In the first experiment rats were trained to press a lever for sucrose pellets on a variable-interval (VI) schedule. The sucrose was subsequently devalued by pairing with Lithium Chloride (LiCl). This had no effect on lever pressing in extinction, although it profoundly reduced reacquisition responding and consumption. In Experiment II rats were trained to shuttle between the two distinctive chambers of a choice-box, in which lever pressing was reinforced in one chamber by sucrose and in the other chamber by food pellets programmed on independent VI schedules. A LiCl-induced taste-aversion was conditioned to the sucrose, and although this markedly affected reacquisition, extinction responding in the sucrose chamber and chamber preference were unaffected. These results indicate that instrumental performance can be independent of the current value of the reinforcer, and are discussed with reference to stimulus-response theory and second-order Pavlovian conditioning.     

Roles of nucleus accumbens and basolateral amygdala in autoshaped lever pressing

Initially-neutral cues paired with rewards are thought to acquire motivational significance, as if the incentive motivational value of the reward is transferred to the cue. Such cues may serve as secondary reinforcers to establish new learning, modulate the performance of instrumental action (Pavlovian-instrumental transfer, PIT), and be the targets of approach and other cue-directed behaviors. Here we examined the effects of lesions of the ventral striatal nucleus accumbens (ACb) and the basolateral amygdala (BLA) on the acquisition of discriminative autoshaped lever-pressing in rats. Insertion of one lever into the experimental chamber was reinforced by sucrose delivery, but insertion of another lever was not reinforced. Although sucrose was delivered independently of the rats' behavior, sham-lesioned rats rapidly came to press the reinforced but not the nonreinforced lever. Bilateral ACb lesions impaired the initial acquisition of sign-tracking but not its terminal levels. In contrast, BLA lesions produced substantial deficits in terminal levels of sign-tracking. Furthermore, whereas ACb lesions primarily affected the probability of lever press responses, BLA lesions mostly affected the rate of responding once it occurred. Finally, disconnection lesions that disrupted communication between ACb and BLA produced both sets of deficits. We suggest that ACb is important for initial acquisition of consummatory-like responses that incorporate hedonic aspects of the reward, while BLA serves to enhance such incentive salience once it is acquired.     

Do rats in a two-action test encode movement egocentrically or allocentrically?

Two-action tests of imitation compare groups that observe topographically different responses to a common manipulandum. The general aim of the two experiments reported here was to find a demonstrator-consistent responding effect in a procedure that could be elaborated to investigate aspects of what was learned about the demonstrated lever response. Experiment 1 was a pilot study with rats of a variant of the two-action method of investigating social learning about observed responses. Groups of observer rats (Rattus norvegicus) saw a demonstrator push a lever up or down for a food reward. When these observers were subsequently given access to the lever and rewarded for responses in both directions, their directional preferences were compared with two 'screen control' groups that were unable to see their demonstrators' behaviour. Demonstrator-consistent responding was found to be restricted to observers that were able to see demonstrator performance, suggesting that scent cues alone were insufficient to cue a preference for the demonstrators' response direction and thereby that the rats learned by observation about body movements (imitation) or lever movement (emulation). Experiment 2 assessed responding on two levers, one that had been manipulated by the demonstrator, and a second, transposed lever positioned some distance away. Demonstrator-consistent responding was abolished when actions were observed and performed in different parts of the apparatus, suggesting that observed movement was encoded allocentrically with respect to the apparatus rather than egocentrically with respect to the actor's body. With particular reference to the influence of scent cues, the results are discussed in relation to the strengths and weaknesses of this and other varieties of the two-action procedure as tests of imitation in animals and human infants. Electronic Publication     

A new technique for measuring thermoregulatory behavior in the rat

A tracking procedure was used to investigate the ability of rats to regulate their ambient temperature. Rats were placed in a chamber with two levers; depressions of one lever controlled a cold-water flow (11 degrees C) and the other controlled the flow of hot water (57 +/- 1 degrees C). If it alternated responses, the rat could regulate temperature within these two extremes. With training, this regulatory behavior resulted in a narrow environmental temperature range that approximated normal body temperature.     

The repeated acquisition of behavioral chains

Monkeys were trained with food reinforcement in a chamber containing four groups of three levers. For each session the monkey's task was to learn a new four-response chain by pressing the correct lever in each group. A stable pattern of learning resulted, and the number of errors reached a steady state from session to session. The technique was then used to determine how various durations of timeouts, following errors, affected the acquisition of new chains. With no timeout, the monkeys made a great many errors, due in large part to superstitious responses within the reinforced chain. Timeout durations ranging from 1 sec to 4 min reduced the number of errors substantially. A second experiment investigated the effects upon acquisition errors of presenting a single light (an “instruction” stimulus) over the correct lever. When this light did not influence the monkeys' responses to the three alternatives, the chains were learned as without it. When the light did control responding, the monkey pressed the appropriate sequence of levers but did not learn the sequence. Thus, when the light was removed, the monkey performed as if learning that sequence for the first time.     

Transfer of persistence to the acquisition of a new behaviour

Two experiments tested whether the degree of effort required for the reinforcement of one behaviour would affect the acquisition of a second behaviour. In the first experiment, rats were placed in a conditioning chamber and: (a) were required to press a lever for food pellets on a fixed ratio schedule, (b) received free presentation of the pellets, or (c) did not receive pellets. Next, all rats were rewarded for a new behaviour, round trips across the length of a runway. As predicted, the fixed-ratio group had the greatest shuttle rate. In the second experiment, two groups were required to press a lever, and the number of presses per pellet was varied. For two other groups not required to press the lever, the amount of food presented per approach to the feeder was varied. The greater required number of lever presses and the lesser number of pellets per approach to the feeder produced the higher subsequent shuttle rates. Two alternative explanations were compared: the degree of accustomed effort per reinforcer becomes a learned component of behaviour, or high effort increases the habituation of frustration-produced disruptive responses.     

Measurement of directional lever response reaction time with the Commodore 64

Routines are described for timing directional lever responses with the Commodore 64 micro-computer. Millisecond reaction timing is carried out with on-board hardware clocks, and lever responses are detected by monitoring the position of a joystick interfaced to a controller port. In a demonstration program, a machine code subroutine is used to measure the reaction times of lever responses in a spatial relations task. In addition, modifications to the demonstration program are suggested to adapt it for use with other tasks.     

Maintenance of responding by squirrel monkeys under a concurrent shock-postponement, fixed-interval shock-presentation schedule.

A chain-pulling response was initially developed under a shock-postponement (avoidance) schedule with two squirrel monkeys. Few responses occurred on a lever where responding initially had no scheduled consequence or, subsequently, when a 3-minute fixed-interval shock-presentation schedule was concurrently arranged for lever responses. Appropriate rates and patterns of lever responding developed and were later maintained under the fixed-interval 3-minute shock-presentation schedule alone when the chain and shock-postponement schedule were removed. When both the shock-postponement and shock-presentation schedules were again simultaneously in effect, steady rates of chain pulling were maintained by the shock-postponement schedule and positively accelerated rates and patterns were maintained on the lever by the shock-presentation schedule. Response rates under both schedules were directly related to shock intensity. A history of exposure to a shock-postponement schedule, even though with a topographically different response and manipulandum, was sufficient for the development and eventual maintenance of responding by the presentation of shock. Further, differential performances can be maintained simultaneously by the presentation and postponement of electric shock.     

Response induction during the acquisition and maintenance of lever pressing with delayed reinforcement

The acquisition of lever pressing by rats and the occurrence of unreinforced presses at a location different from that of the reinforced response were studied using different delays of reinforcement. An experimental chamber containing seven identical adjoining levers was used. Only presses on the central (operative) lever produced food pellets. Groups of 3 rats were exposed to one of seven different tandem random-interval (RI) fixed-time (FT) schedules. The average RI duration was the complement of the FT duration such that their sum yielded a nominal 32-s interreinforcement interval on average. Response rate on the operative lever decreased as the FT value was lengthened. The spatial distribution of responses on the seven levers converged on the operative lever when the FT was 0 or 2 s and spread across the seven levers as the FT value was lengthened to 16 or 32 s. Presses on the seven levers were infrequent during the FT schedule. Both operative- and inoperative-lever pressing intertwined in repetitive patterns that were consistent within subjects but differed between subjects. These findings suggest that reinforcer delay determined the response-induction gradient.     

Techniques for establishing schedules with wheel running as reinforcement in rats.

In three experiments, access to wheel running was contingent on lever pressing. In each experiment, the duration of access to running was reduced gradually to 4, 5, or 6 s, and the schedule parameters were expanded gradually. The sessions lasted 2 hr. In Experiment 1, a fixed-ratio 20 schedule controlled a typical break-and-run pattern of lever pressing that was maintained throughout the session for 3 rats. In Experiment 2, a fixed-interval schedule of 6 min maintained lever pressing throughout the session for 3 rats, and for 1 rat, the rate of lever pressing was positively accelerated between reinforcements. In Experiment 3, a variable-ratio schedule of 20 or 35 was in effect and maintained lever pressing at a very stable pace throughout the session for 2 of 3 rats; for 1 rat, lever pressing was maintained at an irregular rate. When the session duration was extended to successive 24-hr periods, with food and water accessible in Experiment 3, lever pressing settled into a periodic pattern occurring at a high rate at approximately the same time each day. In each experiment, the rats that developed the highest local rates of running during wheel access also maintained the most stable and highest rates of lever pressing.     

Contingency tracking during unsignaled delayed reinforcement

Three experiments were conducted with rats in which responses on one lever (labeled the functional lever) produced reinforcers after an unsignaled delay period that reset with each response during the delay. Responses on a second, nonfunctional, lever did not initiate delays, but, in the first and third experiments, such responses during the last 10 s of a delay did postpone food delivery another 10 s. In the first experiment, the location of the two levers was reversed several times. Responding generally was higher on the functional lever, though the magnitude of the difference diminished with successive reversals. In the second experiment, once a delay was initiated by a response on the functional lever, in different conditions responses on the nonfunctional lever either had no effect or postponed food delivery by 30 s. The latter contingency typically lowered response rates on the nonfunctional lever. In the first two experiments, both the functional and nonfunctional levers were identical except for their location; in the third experiment, initially, a vertically mounted, pole-push lever defined the functional response and a horizontally mounted lever defined the nonfunctional response. Higher response rates occurred on the functional lever. These results taken together suggest that responding generally tracked the response-reinforcer contingency. The results further show how nonfunctional-operanda responses are controlled by a prior history of direct reinforcement of such responses, by the temporal delay between such responses and food delivery, and as simple generalization between the two operanda.     

Vocal and nonvocal discriminative performance in monkeys

Discriminative performance of a lever press and a vocal call was compared in five macaques. The animals exhibited similar scores for Response Rate, Efficiency, and Error Index. Total vocal responses were slightly higher than total lever responses. Latency for vocal responses was significantly greater than lever press. Similar scores in measures of performance of the two behaviors indicate that discriminative vocalization is not exceedingly difficult. Failure by others to achieve discriminative vocalization in studies employing weak positive-reinforcement contingencies suggest that certain experimental paradigms are unsatisfactory for vocal conditioning.