Brain stimulation as a reinforcer: intermittent schedules

Rats with chronically implanted, bipolar electrodes in the septal and medial forebrain bundle areas, in addition to the region of the mammillary bodies of the posterior hypothalamus, were trained to press a permanently mounted lever in order to produce a second, retractable lever. Rewarding brain stimulation was programmed on the retractable lever; following completion of the programmed number of CRF response-stimulations, that lever was retracted from the box. Responding on the permanent lever could reintroduce the retractable lever. Fixed interval, fixed ratio, DRL, and variable interval schedules were programmed on the permanent lever in the range of schedule parameters often used with conventional reinforcers. Typical effects are described, and it is concluded that there are no striking differences between brain-stimulation reinforcement and the conventional reinforcers.     

Timeout and concurrent fixed-ratio schedules with human subjects

Human subjects given choices among 10 different pairs of concurrent fixed-ratio schedules preferred the smaller ratio. After a preference had been determined, timeout of increasing duration followed the completion of the preferred schedule. The larger the fixed-ratio difference, the longer the timeout necessary to produce the shift to the previously nonpreferred ratio. Responses by two of three subjects were unaffected by changes from response-dependent to response-independent pay.     

Timeout from a stimulus correlated with the extinction component of a multiple schedule

Four groups of pigeons were trained to discriminate between green and red. Pecks on a second key produced a timeout from the schedule in effect for 30 sec. For two of the groups, this timeout response turned off all the lights in the chamber (Blackout), while for the other two, only the key-lights were turned off (No Blackout). For one of the Blackout groups and one of the No Blackout groups, responses on the discrimination key during the extinction component (S^?) also resulted in a mild electric shock. Blackout groups produced more timeouts during S^? than did No Blackout groups, but electric shock punishment suppressed, rather than enhanced, timeout responding. These findings suggest a need for reevaluation of the hypothesis that the timeout response is an escape from an aversive S^?.     

Inhibitory stimulus control in concurrent schedules

Six pigeons were exposed to two keys, a main key and a changeover key. Pecking the main key was reinforced on a variable-interval 5-min schedule when the key was blue and never reinforced when the key displayed a vertical line on a blue background. Each peck on the changeover key changed the stimulus displayed on the main key. Each subject was given two generalization tests, consisting of presentations on the main key of six orientations of the line on the blue background, with no reinforcements being given. In one test changeover-key pecks changed the stimulus; in the other test the changeover key was covered and the experimenter controlled stimulus changes. Both responses to the six stimuli and time spent in the presence of the stimuli gave U-shaped gradients when the changeover key was operative. With most subjects, absolute rates of responding to each stimulus produced unsystematic gradients, whether or not the changeover key was operative.     

Molecular contingencies in schedules of intermittent punishment

In two experiments, key pecking of pigeons was maintained by a variable-interval 180-s schedule of food presentation. Conjointly, a second schedule delivered response-dependent electric shock. In the first experiment, shocks were presented according to either a variable-interval or a nondifferential interval-percentile schedule. The variable-interval shock schedule differentially delivered shocks following long interresponse times. Although the nondifferential shock schedules delivered shocks less differentially with respect to interresponse times, the two shock schedules equally reduced the relative frequency of long interresponse times. The second experiment differentially shocked long or short interresponse times in different conditions, with resulting decreases in the relative frequency of the targeted interresponse times. These experiments highlight the importance of selecting the appropriate level of analysis for the interaction of behavior and environment. Orderly relations present at one level of analysis (e.g., interresponse times) may not be revealed at other levels of analysis (e.g., overall response rate).     

Contingency and stimulus change in chained schedules of reinforcement

Higher rates of pecking were maintained by pigeons in the middle component of three-component chained fixed-interval schedules than in that component of corresponding multiple schedules (two extinction components followed by a fixed-interval component). This rate difference did not occur in equivalent tandem and mixed schedules, in which a single stimulus was correlated with the three components. The higher rates in components of chained schedules demonstrate a reinforcing effect of the stimulus correlated with the next component; the acquired functions of this stimulus make the vocabulary of conditioned reinforcement appropriate. Problems in defining conditioned reinforcement arise not from difficulties in demonstrating reinforcing effects but from disagreements about which experimental operations allow such reinforcing effects to be called conditioned.     

Timeout as a reinforcer for errors in a serial position task

After learning to press keys in a predetermined serial position sequence, with timeouts scheduled as a consequence of errors, monkeys developed stereotyped errors. As soon as a new trial started, the animals would make an error. On trial after trial, they pressed the same incorrect key at the first member of the sequence, even though they had previously learned the sequence. First-member errors occurred even when sequences of fully bright keys marking correct choices were presented. When timeout was eliminated as a consequence of one first-member error, subjects switched to an error that did produce the timeout. When all first-member errors failed to produce timeout the monkeys ceased responding. Both prefeeding and reduction in reinforcement density resulted in stereotyped errors occurring earlier in the session.     

Interresponse-time analysis of stimulus control in multiple schedules

Interresponse-time distributions were recorded in two components of multiple variable-interval schedules that were varied over several conditions. Values of the exponent for power functions relating ratios of interresponse times emitted per opportunity to ratios of reinforcers obtained in the two components varied with interresponse-time class interval. The exponent (sensitivity to reinforcement) afforded a measure of stimulus control exerted by the discriminative stimuli. Exponents were near zero for short interresponse times, consistent with previous conclusions that responses following short interresponse times are controlled by response-produced or proprioceptive stimuli. Values of exponents increased with longer interresponse times, indicating strong control by exteroceptive stimuli over responses following interresponse times of approximately one second or longer.     

Acquisition of a simple running response as a function of partial and continuous schedules of reinforcement

The Psychological Record -     

Control of a continuous response dimension by a continuous stimulus dimension

Pigeons were trained to respond to stimuli from a continuous stimulus dimension (tonal frequency) with response values from a continuous response dimension. Both the number of points of correspondence and problem difficulty were varied. After training, subjects were tested with stimulus values intermediate to those trained. During these test tones, subjects emitted only those response values reinforced during training. The study suggested that if there are fast and efficient methods to obtain control of a continuous response dimension by a continuous stimulus dimension, these methods must depend on factors other than simple generalization.     

Concurrent schedules of response-independent reinforcement: duration of a reinforcing stimulus

Presentations of grain to three pigeons were determined by two response-independent schedules. Interpresentation intervals varied with a mean interval of 1.5 min for each schedule. Both were concurrently operative, but grain was presented by one only when the chamber was illuminated with blue light and by the other only during amber illumination. A response on a white key, the only key in the chamber, alternated the stimulus conditions and the effective schedule. Grain presentation durations associated with the illumination conditions were varied from 1.5 to 4.5 sec. The proportion of the total session time spent in an illumination condition closely approximated the relative grain presentation duration provided in that illumination. For two of the birds, the proportion of the total number of grain presentations obtained in an illumination condition was an increasng function of the presentation duration in that illumination.     

Combined stimulus control over time allocation in response-independent schedules

Rats were exposed to concurrent schedules in which reinforcers occurred independently of behavior. In Experiment 1, rats could control time spent in the following conditions: (1) a light, (2) white noise, and (3) the absence of both light and noise. Response-independent reinforcers occurred at the same rate during the light and the noise and at either a higher rate or not at all in the absence of both stimuli. In subsequent tests, the rats spent more time in a light and noise compound than in either light or noise alone after the absence of both stimuli had signaled no reinforcers. When the absence of both stimuli had signaled a higher rate of reinforcement, however, the rats typically spent less time in the compound than in light or noise alone. In Experiment 2, rats could control time spent in the presence of a light and of a buzzer. The reinforcement rate in the light was twice that in the buzzer. In a later test, the rats spent more time in a light and buzzer compound than in the buzzer, but less time in the compound than in the light. The results show that additive summation, suppressive summation, and stimulus averaging of time allocation occur and that response rate differences between training stimuli are not necessary for these phenomena.     

Fixed versus variable sequences of food and stimulus presentation in second-order schedules

Three pigeons were exposed to a second-order schedule in which the behavior specified by a fixed-interval component schedule was reinforced according to a ratio overall schedule. The completion of components not followed by food was signalled by a brief stimulus never paired with food. Food and the stimulus occurred in a random sequence or in fixed alternation, but the overall schedules (variable ratio 2 or fixed ratio 2) ensured that an equal number of food and brief-stimulus presentations occurred in each session. The control exerted by the food and by the brief stimulus was measured by overall response rates, mean pauses, frequency distributions of pauses, and response patterning across components. In general, the stimulus controlled patterns of behavior more similar to those controlled by food when food and the stimulus occurred in a random sequence than when they occurred in fixed alternation.     

Responding in the squirrel monkey under fixed-ratio schedules of stimulus termination

Key pressing was maintained under a fixed-ratio schedule in which electric shock was scheduled for delivery at a fixed time (t seconds) after each stimulus onset, and every n^th response terminated the stimulus and initiated a timeout from shock. Under this procedure, the higher the rate of responding, the briefer the duration of the stimulus presentation and the lower the frequency of shock delivery. The effects of several schedule parameters were studied to determine whether the maintenance of responding was dependent on an inverse relation between response rate and shock frequency. Shock rate and shock frequency were made independent of response rate by decreasing the value of t to 0.5 second and delivering shock only during the first presentation of the stimulus after a fixed time, including stimulus and timeout durations, had elapsed since the previous shock. The experiments showed that shock frequency and response rate are inversely related when t is of relatively long duration compared to the value of the fixed-ratio parameter, but that a decrease in shock rate or frequency due to a high rate of responding is not necessary for the maintenance of responding under a fixed-ratio schedule of stimulus termination.     

Discriminability of frequency of food or stimulus presentations in variable-time schedules

Pigeons responded in a two-alternative forced-choice task in which reinforcement was dependent upon the frequency of events that occurred in an immediately preceding schedule sample. On a given trial the events were either brief food presentations or brief visual and auditory stimulus changes. High levels of stimulus control were obtained by food-presentation schedules only. Discriminative control by frequency or stimulus change was absent. Stimulus control by food frequency was decreased by the imposition of a delay period between the schedule sample and the choice. Moreover, stimulus control by food frequency was related to the ratio of food-presentation schedule pairs when novel schedules were presented in a transfer test.     

Conjunctive schedules of reinforcement II: response requirements and stimulus effects

Responding of three pigeons was maintained under conjunctive fixed-ratio, fixed-interval schedules where a key peck produced food after both schedule requirements were completed. The individual schedule requirements were then successively removed and reinstated with responding maintained under the following conditions: conjunctive fixed-ratio, fixed-time; fixed-time; and fixed-interval schedules. Patterns of responding changed in accord with the successive removal of the schedule requirements. Compared to the conjunctive fixed-ratio, fixed-interval schedule, pause duration increased and response rate decreased under conjunctive fixed-ratio, fixed-time schedules and under fixed-time schedules alone. Overall mean rates of responding were highest and pause duration lowest under fixed-interval schedules. When changes in the keylight colors were correlated with completion of the fixed-ratio, the end of the fixed-interval, or both of these conditions, the pattern of responding was modified and indicated a greater degree of control by the individual schedules. Although two birds showed large increases in interreinforcement time when they were initially exposed to the conjunctive schedule, when responding stabilized this measure was largely invariant for all birds across most schedule conditions.