Conjunctive schedules of response-dependent and response-independent reinforcement

Pigeons received food when they emitted the number of responses specified by a fixed-ratio schedule, and the time specified by a fixed-time schedule had elapsed. The order of meeting the response and time requirements was irrelevant. In different conditions, stimuli signalled completion of one, both, or neither requirement. Ratio size interacted with stimulus condition to determine performance. When a stimulus signalled the end of the fixed-time period, under all ratios the birds tended to respond after the stimulus appeared. When stimuli followed both components, small ratios produced responding during the fixed-time period, and other ratios resulted in responses after the time period had elapsed. With either no stimulus changes, or with a stimulus correlated with completion of the ratio alone, responding first increased and then decreased as the ratio increased. Low and high ratios produced stable response frequencies and patterns in successive intervals. Intermediate ratios resulted in two types of performance. Intervals with long initial pauses and few responses during the fixed-time period were followed by intervals with short pauses and numerous responses and vice versa. The source of these dynamic effects was hypothesized to be number of responses per reinforcer in one condition and response-reinforcer contiguity in the other.     

Preference and discrimination between response-dependent and response-independent schedules of reinforcement

Four Asian quail (Coturnix coturnix japonica) were exposed to concurrent-chain schedules, the terminal links of which were either variable-interval 30 sec and variable-time 30 sec, or fixed-interval 30 sec and fixed-time 30 sec. Except for one bird that exhibited a preference for the variable-interval schedule over the variable-time schedule, no consistent preferences were demonstrated for response-dependent or response-independent schedules. However, response rates were three times greater on response-dependent than on response-independent schedules. The discrimination between terminal-link schedules was rapidly recovered after the schedule positions were reversed. Casual observations revealed that the birds engaged in stereotypic circling and pecking while the response-independent schedules were operative.     

Responding under schedules combining response-dependent and response-independent shock delivery

Lever pressing of three squirrel monkeys with experience under continuous avoidance schedules was maintained by response-produced shock under a 5-minute variable-interval schedule. Responding decreased when half of the scheduled shocks were delivered independently of lever pressing and decreased further when all shocks were independent of lever pressing. Responding was lowest when all shocks were eliminated. When the proportion of response-dependent shocks increased, responding increased. This relation occurred even though the frequency and temporal distribution of shock delivery remained the same. Responding of two monkeys increased in a graded fashion as the frequency of shock was increased by arranging variable-time 5-minute, 2-minute, and 1-minute schedules jointly with the variable-interval 5-minute schedule. Thus, increasing the proportion of response-independent shocks decreased responding when the overall frequency of shocks stayed the same, but increased responding when the overall frequency of shock delivery increased.     

Combinations of response-dependent and response-independent schedule-correlated stimulus presentation in an observing procedure

Pigeons pecked a response key on a variable-interval (VI) schedule, in which responses produced food every 40 s, on average. These VI periods, or components, alternated in irregular fashion with extinction components in which food was unavailable. Pecks on a second (observing) key briefly produced exteroceptive stimuli (houselight flashes) correlated with the component schedule currently in effect. Across conditions within a phase, the dependency between observing and presentation of the stimuli was decreased systematically while the density of stimulus presentation was held constant. Across phases, the proportion of session time spent in the VI component was adjusted from 0.5 to 0.25, and then to 0.75. Results indicate that rate of observing decreased as the dependency between responses and stimulus presentations was decreased. Further, discriminative control by the schedule-correlated stimuli was systematically weakened as dependency was decreased. Increasing the proportion of session time spent in VI decreased the rate of observing. This effect was additive with the manipulation of the dependency between observing and presentation of the stimuli. Overall, these results show that conditioned reinforcers function similarly to unconditioned reinforcers with respect to response-consequence dependencies, and that stimulus control is enhanced under conditions in which the relevant stimuli are produced by an organism's behavior.     

The effects of clozapine on delayed spatial alternation deficits in rats with hippocampal damage

Clozapine is an atypical antipsychotic drug that has been shown to improve spatial memory in some animal models; however its efficacy in reversing spatial memory impairment in rats with hippocampal lesions is unknown. To address this issue, we tested the effects of clozapine on delayed spatial alternation deficits in rats with hippocampal damage in three separate experiments. In each experiment, adult male rats received sham surgery or direct stereotaxic infusions of the excitotoxin, NMDA, into the hippocampus. In the first study, seven days after surgery, the sham control animals received daily saline injections while the lesioned animals were split into two groups that received daily saline or clozapine (2.0 mg/kg, sc) injections. During the fifth week of injections, all animals were tested in a food-motivated delayed spatial alternation task. Saline-treated rats with excitotoxic hippocampal damage displayed significant deficits in delayed spatial alternation. Daily clozapine injections completely reversed this deficit. In a second experiment, it was found that clozapine treatment limited to the testing days only did not improve alternation performance in lesioned rats. Finally, in a third experiment, chronic clozapine treatment did not improve alternation performance in lesioned rats that were pre-trained in the alternation task prior to surgery. These results suggest that chronic, but not acute, clozapine treatment enables rats with hippocampal damage to develop new spatial learning, but can not rescue old spatial learning established prior to damage. These results may have implications for the treatment of cognitive deficits caused by hippocampal dysfunction in disorders such as schizophrenia, Alzheimer's disease, and others.     

The effects of stimuli followed by response-independent shock on shock-avoidance behavior

Four rhesus monkeys were trained on a non-discriminated shock-avoidance schedule (baseline). Stimuli followed by response-independent shock were then presented with the avoidance baseline no longer in effect. The main portion of the experiment consisted of superimposing (independently of responding) the stimuli followed by response-independent shock on the avoidance baseline. Different temporal values of stimulus duration and delay of shock (produced by an avoidance response) were presented successively, using each subject as his own control. When the stimulus duration was short or the delay of shock was long, so that avoidance rate during the stimulus could assume any value without resulting in baseline (avoidable) shocks during the stimulus, a lowered or "suppressed" rate of responding developed during the stimulus. When the stimulus duration was long or the delay of shock was brief, so that avoidable shocks resulted from a response decrement during the stimulus, high or "facilitated" rates of responding developed for a large proportion of the time that the stimulus was present.     

The effects of a high-probability instruction sequence and response-independent reinforcer delivery on child compliance

We compared the effects of a high-probability (high-p) instruction sequence and a fixed-time (FT) schedule of reinforcement on the compliance of 2 typically developing children. A multielement experimental design with a reversal component was implemented according to a multiple baseline across participants arrangement. Both the high-p and FT conditions resulted in increased compliance for both participants during the multielement sessions. These results suggest that it may be possible to increase compliance without a response requirement of the type arranged in the high-p instruction sequence.     

Some effects of response-independent reinforcement on auditory generalization gradients.

Two groups of six rats received discrimination training with two auditory stimuli differing in intensity. During one stimulus, the schedule was variable interval; during the other, it was either variable time or extinction. Both the variable time and extinction schedules resulted in differential rates of responding in the presence of the two stimuli. Extinction resulted in an earlier and more stable difference. Stimulus generalization gradients obtained along the noise-intensity dimension revealed peak shift with both procedures. In addition, a secondary peak to stimuli in between the two training stimuli occurred with the variable-time schedule.     

The effects of interval duration on temporal tracking and alternation learning

On cyclic-interval reinforcement schedules, animals typically show a postreinforcement pause that is a function of the immediately preceding time interval (temporal tracking). Animals, however, do not track single-alternation schedules-when two different intervals are presented in strict alternation on successive trials. In this experiment, pigeons were first trained with a cyclic schedule consisting of alternating blocks of 12 short intervals (5 s or 30 s) and 12 long intervals (180 s), followed by three different single-alternation interval schedules: (a) 30 s and 180 s, (b) 5 s and 180 s, and (c) 5 s and 30 s. Pigeons tracked both schedules with alternating blocks of 12 intervals. With the single-alternation schedules, when the short interval duration was 5 s, regardless of the duration of the longer interval, pigeons learned the alternation pattern, and their pause anticipated the upcoming interval. When the shorter interval was 30 s, even when the ratio of short to long intervals was kept at 6:1, pigeons did not initially show anticipatory pausing-a violation of the principle of timescale invariance.     

Food priming and short-term retention in the delayed alternation task

The delayed alternation task, with rats as subjects, was used to assess the effects of a priming food reward on subsequent retention of a reinforced target location in a T maze. Subjects received a pretrial feeding, followed by a forced-choice rewarded entry into one goal box of the maze (the “cuing” run). Following a delay interval (5–60 s), the subjects were allowed to choose either goal, with reward available only in the one not entered on the cuing run. Priming significantly reduced such alternation after 30- and 60-s delays (Experiment 1), whereas administering the reward following the cuing run had no effect on choice behavior (Experiment 2). In the third study, prefeeding had a decremental effect when the following cuing run was reinforced, but no decrement occurred when reinforcement was omitted on the cuing run. The results suggest that food reward enhances retention of spatial information, but priming reduces this effect.     

Some effects of response-dependent clock stimuli in a fixed-interval schedule

Two experiments studied the effects of brief response-dependent clock stimuli in fixed-interval schedules of reinforcement. In the first experiment, two pigeons were exposed to a fixed-interval schedule. Two conditions were compared. In both conditions each peck on the key produced a brief stimulus. In one condition, pecks produced a different stimulus in successive sixths of the interval. This was the clock condition. In the other condition, the same stimulus was produced throughout the interval. Response rates were lower and the pause after reinforcement was longer in the clock condition. In the second experiment, a two-key optional clock procedure was used. Responding on the clock key produced one of three stimuli correlated with the three successive minutes of a fixed-interval schedule. A response on the other key produced grain at the end of the 3 min. When the final stimulus was removed from the situation and pecking produced nothing during the third minute, responding to the clock key declined to a very low rate. When the first two stimuli were removed and the third one replaced, responding to the clock key was resumed.     

The effects upon food-reinforced pecking and treadle-pressing of auditory and visual signals for response-independent food

Seven pigeons whose key-pecking was maintained by food reinforcement on a differential-reinforcement-of-low-rates 12-sec limited-hold 4-sec schedule and 12 other pigeons whose treadle-pressing was maintained by the same schedule received appetitive Pavlovian conditioning trials superimposed upon the instrumental baseline. Half the birds in each group received a tone as the CS, and the other half received a stimulus change on the key. Each CS was 20 sec long, and was immediately followed by 10-sec access to grain. The visual CS markedly facilitated the rate of pecking on the key for the birds whose baseline response was pecking. The visual CS produced auto-shaping of the key-peck and tended to produce suppression of treadle-pressing for the birds whose baseline response was treadle-pressing. The auditory CS produced inconsistent effects across birds regardless of the baseline response. In all cases the conditioned effects extinguished when response-independent food was omitted.     

The effects of delayed reinforcement on free-operant responding

In previous studies of delayed reinforcement, response rate has been found to vary inversely with the response-reinforcer interval. However, in all of these studies the independent variable, response-reinforcer time, was confounded with the number of reinforcers presented in a fixed period of time (reinforcer frequency). In the present study, the frequency of available reinforcers was held constant, while temporal separation between response and reinforcer was independently manipulated. A repeating time cycle, T, was divided into two alternating time periods, t^D and t^Δ. The first response in t^D was reinforced at the end of the prevailing T cycle and extinction prevailed in t^Δ. Two placements for t^D were defined, an early t^D placement in which t^D precedes t^Δ and a late t^D placement in which t^D follows t^Δ. The duration of the early and late t^D was systematically decreased from 30 seconds (i.e., t^D = T) to 0.1 second. Manipulation of t^D placement and duration controlled the temporal separation between response and reinforcement, but it did not affect the frequency of programmed reinforcers, which was 1/T. The results show that early and late t^D placements of equal duration have similar overall effects upon response rate, reinforcer frequency, responses per reinforcer, and obtained response-reinforcer temporal separation. A stepwise regression analysis using log response rate as the dependent variable showed that the obtained delay was a significant first-step variable for six of eight subjects, with obtained reinforcer frequency significant for the remaining two subjects.     

Avoidance of timeout from response-independent food: effects of delivery rate and quality

In three experiments, a rat's lever presses could postpone timeouts from food pellets delivered on response-independent schedules. In Experiment 1, the pellets were delivered at variable-time (VT) rates ranging from VT 0.5 to VT 8 min. Experiment 2 replicated the VT 1 min and VT 8 min conditions of Experiment 1 with new subjects. Finally, subjects in Experiment 3 could postpone timeouts from delivery of pellets that differed in quality rather than quantity (unsweetened versus sweetened pellets). In general, response rates and success in avoiding increased as a function of the rate and quality of the pellets. Also, performance efficiency increased as the experiments progressed, that is, the avoidance response occurred later and later in the response-timeout interval. The results support the conclusion that timeout from reinforcement has functional properties similar to those of more commonly studied aversive stimuli (e.g., shock).     

Epinephrine fails to enhance performance of food-deprived rats on a delayed spontaneous alternation task

Increases in blood glucose levels after epinephrine injection appear to contribute to the hormone's effects on learning and memory. The present experiment evaluated whether epinephrine-induced enhancement of spontaneous alternation performance would be attenuated in fasted rats that had blunted increases in circulating glucose levels after injections of epinephrine. Rats deprived of food for 24 h prior to injection of epinephrine exhibited significant attenuation of the increase in blood glucose levels seen in fed rats. When the rats were tested on a delayed spontaneous alternation task, epinephrine enhanced performance in fed rats but not in rats deprived of food for 24 h. These findings are consistent with the view that hyperglycemia subsequent to epinephrine injections contributes to the memory-enhancing effects of epinephrine.     

Some effects of response-independent shocks after unsignaled avoidance conditioning in rats

After extended unsignaled avoidance training, the majority of rats continued frequent responding during long periods when only response-independent shocks were presented. Most responses were shock elicited, i.e., followed immediately after shock. Response rates were directly related to both the frequency and the intensity of response-independent shocks. Responding continued to approximately the same extent, whether shocks were presented at fixed or variable intervals. Few responses were directed toward a second lever in the test chamber, even when the lever previously associated with avoidance was removed. When avoidance was scheduled on the second lever, the rats learned to avoid by operating it. Meanwhile, responding on the first lever became infrequent. Subsequently, when only response-independent shocks were presented, almost all responses occurred upon the lever initially associated with avoidance. Responding that was elicited by response-independent shocks was suppressed by response-dependent (punishment)_shocks of the same intensity. When punishment was withdrawn, recovery of responding occurred. An explanation of the results based upon shock-elicited behaviors is preferred to one stressing unextinguished avoidance responding.