Choice and delay of reinforcement: Effects of terminal-link stimulus and response conditions

In two experiments, pigeons were exposed to concurrent-chains schedules in which a single initial-link variable-interval schedule led to access to terminal links composed of fixed-interval or fixed-delay schedules. In Experiment 1, an 8-s (or 16-s) delay to reinforcement was associated with the standard key, while reinforcer delay values associated with the experimental key were varied from 4 to 32 s. The results of Experiment 1 showed undermatching of response ratios to delay ratios with terminal-link fixed-delay schedules, whereas in some pigeons matching or overmatching was evident with the fixed-interval schedules. In Experiment 2, one pair of reinforcer delay values, either 8 versus 16 s or 16 versus 32 s, was used. In the first condition of Experiment 2, different delays were associated with different keylight stimuli (cued condition). In the second condition, different terminal-link delays were associated with the same stimulus, either a blackout (uncued-blackout condition) or a white key (uncued-white condition). To examine the role of responses emitted during delays, the keys were retracted during a delay (key-absent condition) in the third condition and responses were required by a fixed-interval schedule in the fourth condition. Experiment 2 demonstrated that the choice proportions for the shorter delay were more extreme in the cued condition than in the uncued-blackout condition, and that the response requirement imposed by the fixed-interval schedules did not affect choice of the shorter delay, nor did the key-absent and key-present conditions. These results indicate that the keylight-stimulus conditions affected preference for the shorter of two delays and that the findings obtained in Experiment 1 depended mainly on the keylight-stimulus conditions of the terminal links (i.e., the conditioned reinforcing value of the terminal-link stimuli).     

Techniques for establishing schedules with wheel running as reinforcement in rats.

In three experiments, access to wheel running was contingent on lever pressing. In each experiment, the duration of access to running was reduced gradually to 4, 5, or 6 s, and the schedule parameters were expanded gradually. The sessions lasted 2 hr. In Experiment 1, a fixed-ratio 20 schedule controlled a typical break-and-run pattern of lever pressing that was maintained throughout the session for 3 rats. In Experiment 2, a fixed-interval schedule of 6 min maintained lever pressing throughout the session for 3 rats, and for 1 rat, the rate of lever pressing was positively accelerated between reinforcements. In Experiment 3, a variable-ratio schedule of 20 or 35 was in effect and maintained lever pressing at a very stable pace throughout the session for 2 of 3 rats; for 1 rat, lever pressing was maintained at an irregular rate. When the session duration was extended to successive 24-hr periods, with food and water accessible in Experiment 3, lever pressing settled into a periodic pattern occurring at a high rate at approximately the same time each day. In each experiment, the rats that developed the highest local rates of running during wheel access also maintained the most stable and highest rates of lever pressing.     

Tolerance to effects of cocaine on behavior under a response-initiated fixed-interval schedule

Tolerance to effects of cocaine can be modulated by schedules of reinforcement. With multiple ratio schedules, research has shown an inverse relationship between ratio requirement and amount of tolerance that resulted from daily administration of the drug. In contrast, tolerance to the effects of cocaine on behavior under multiple interval schedules generally has developed regardless of interval value. Under interval schedules reinforcement depends on the animal making one response following a time interval. Thus, as time to respond increases, the time to reinforcement decreases. On the other hand, fixed ratio schedules require a specified number of responses to be made prior to reinforcement. Therefore, delaying the initiation of responding does not coincide with a significant decrease in the time to reinforcement. In the current experiment, 6 pigeons were trained to respond under a three-component multiple schedule, with a different tandem fixed-ratio 1 fixed-interval schedule in each component. The multiple schedule required one response, which was followed by one of three fixed-interval values (5, 15, or 60 s). Thus, the multiple schedule was interval-like because after the fixed-ratio 1, only one more response was required for reinforcement, but it was also ratio-like because the length of the pause at the beginning of each interreinforcer interval affected the time until the next reinforcer. Acute administration of cocaine generally resulted in dose-dependent decreases in responding. Chronic (i.e., daily) administration of a rate-decreasing dose resulted in tolerance patterns similar to those usually obtained with multiple ratio schedules. That is, the magnitude of tolerance was related inversely to schedule size. These results suggest that delay to reinforcement from the initial response may play a role in the development of schedule-parameter-related tolerance.     

Marc Richelle (1930–2021) and the Study of Temporal Regulation of Behaviour in Animals

This article discusses the contribution of Marc Richelle to the study of temporal regulation of behaviour in animals. Richelle was a pioneer of behavioural pharmacology in Europe in the 1960s, and some of his early pharmacological experiments, particular those involving chlordiazepoxide, are discussed. Richelle frequently tested drug effects on performance on fixed-interval (FI) and differential reinforcement of low rate (DRL) schedules. Much of his later work, conducted with Helga Lejeune, involved cross-species comparisons of performance on FI and DRL, and often focused on potential differences between “timing competence” and “timing performance”. His work provided an unrivalled body of research on operant behaviour in different species, involving research on animals as different as cats and fish. Much of the work was reviewed in Richelle and Lejeune’s 1980 book Time in Animal Behaviour, which contained particularly influential accounts of collateral behaviour and inter-species comparisons.     

Molecular analyses of the effects of d-amphetamine on fixed-interval schedule performances of rats.

A series of doses (0.5 to 2.0 mg/kg) of d-amphetamine was administered to rats whose lever pressing was maintained by fixed-interval 30-s, 60-s, or 120-s schedules of reinforcement by sucrose delivery. Under both saline and d-amphetamine conditions, molecular features of responding were reliably described in terms of the distribution of postreinforcement pauses and local response rate following the onset of responding. Postreinforcement pause always varied from interval to interval but, on average, shortened under the drug. Local response rate (response rate exclusive of pause time) tended to decrease under the drug, and where acceleration occurred within runs of responses, it was reduced by the drug. All of these effects were dose-related. These findings suggest that fixed-interval behavior can be analyzed effectively at a molecular level, and that the effects of d-amphetamine are best described as disruption of temporal discrimination.     

Effects of chlorpromazine on fixed-ratio responding: modification by fixed-interval discriminative stimuli.

Effects of chlorpromazine (1 to 100 mg/kg) were assessed on two pigeons' responding under various modifications of a multiple schedule of food delivery. During a fixed-interval component, the first response after 5 min produced food; during the subsequent, fixed-ratio component, the 30th response produced food. Modifications of the schedule entailed changes in stimulus conditions imposed during the fixed-ratio component that did not systematically alter characteristics of performance under non-drug conditions. In the first phase of the experiment, distinctive visual stimuli were correlated with each schedule component (conventional multiple schedule); chlorpromazine produced small decreases in fixed-ratio responding (20% at 30 mg/kg). When each response during the fixed-ratio component produced the stimulus correlated with the fixed-interval schedule (fixed-interval discriminative stimulus) for 1.2 s, effects of chlorpromazine were not different from those under the conventional multiple schedule. Chlorpromazine produced greater decreases in fixed-ratio responding (55% at 30 mg/kg) when either the first response of each fixed ratio changed the stimulus correlated with the fixed-ratio schedule to the fixed-interval discriminative stimulus for the remainder of the fixed-ratio component, or when the fixed-interval discriminative stimulus was presented independently of responding according to a matched temporal sequence. When the fixed-interval discriminative stimulus was present continuously during the fixed-ratio component (mixed schedule), chlorpromazine produced even more substantial decreases in fixed-ratio responding (greater than 80% at 30 mg/kg). Effects of chlorpromazine on fixed-interval responding were also modified by the schedules of fixed-interval discriminative stimulus presentation. The effects of chlorpromazine were a joint function of the stimuli prevailing during the multiple schedule and the degree to which responding influenced these stimuli.     

Variable-ratio conditioning history produces high- and low-rate fixed-interval performance in rats

Four rats were exposed to an A-B-A-B series of 30 sessions each of variable-ratio 20 (A) and fixed-interval 30-s (B) schedules. Four other rats received 120 sessions of fixed-interval 30 s. The rats with a history of variable-ratio responding subsequently showed primarily high or low response-rate patterns on the fixed-interval schedule without evidence of classical scalloping (i.e., increased rates of responding throughout the interreinforcement interval), except infrequently in 1 rat. The rats exposed to only the fixed-interval 30-s schedule displayed the expected sequence of scalloping giving way to lower rate break-run or simply low-rate responding over time. This experiment shows that when naive rats are exposed to even a simple history of reinforcement (in this case, a variable-ratio 20), their subsequent fixed-interval performance is very different from comparable performance in naive rats, and might be said to be more similar to the responding of adult humans. The argument is made that care should be taken in comparing the fixed-interval performance of humans and nonhumans because humans have a complex history of reinforcement, whereas laboratory nonhumans are typically naive.     

Remote effects of aversive contingencies: Disruption of appetitive behavior by adjacent avoidance sessions

Disruption of ongoing appetitive behavior before and after daily avoidance sessions was examined. After baselines of appetitive responding were established under a fixed-interval 180-s schedule of food presentation, 4 rats were exposed to 40-min sessions of the appetitive schedule just prior to 100-min sessions of electric shock postponement, while another 4 rats received the 40-min appetitive sessions just following daily sessions of shock postponement. In all 8 subjects, fixed-interval response rates decreased relative to baseline levels, the effect being somewhat more pronounced when the avoidance sessions immediately followed. The disruption of fixed-interval responding was only partially reversed when avoidance sessions were discontinued. During the initial exposure to the avoidance sessions, patterns of responding under the fixed-interval schedule were differentially sensitive to disruption, with high baseline response rates generally more disturbed than low rates. These disruptions were not systematically related to changes in reinforcement frequency, which remained fairly high and invariant across all conditions of the experiment; they were also not systematically related to the response rates or to the shock rates of the adjacent avoidance sessions. The results, while qualitatively resembling patterns of conditioned suppression as typically studied, occurred on a greatly expanded time scale. As disruption of behavior extending over time, the present data suggest that some forms of conditioned suppression are perhaps best viewed within a larger temporal context.     

The effects of chlorpromazine and imipramine on rate and stimulus control of matching to sample.

Pigeons were trained to perform simultaneous, two-color matching to sample under a multiple fixed-ratio fixed-interval schedule of food presentation. The sequence terminating with a peck on the matching key (a "match") was treated as a unit, analogous to a single key peck in conventional schedules. Except for intermittent reinforcement of matches, no consequent stimulus distinguished matches from mismatches (sequences terminating with pecks on the nonmatching key). The pattern of matches during nondrug sessions resembled that of simpler operants maintained by similar schedules. Matches increased in rate toward the end of both components; mismatch rates increased more slowly. Imipramine increased the rate of mismatches, disrupted schedule patterning, and lowered accuracy in a dose-dependent fashion. Chlorpromazine lowered the overall rate of matches but affected schedule patterns and accuracy less than imipramine. The types of errors during drug sessions were not systematically related to the types of errors that appeared during nondrug sessions. Stimulus control was evaluated for each of the four possible color configurations and was found to be by the entire configuration of colors, not simply by the color of the sample.