Behavior of rats under fixed consecutive number schedules: effects of drugs of abuse.

Four rats responded under a simple fixed consecutive number schedule in which eight or more consecutive responses on the run lever, followed by a single response on the reinforcement lever, produced the food reinforcer. Under this simple schedule, dose-response curves were determined for diazepam, morphine, pentobarbital, and phencyclidine. The rats were then trained to respond under a multiple fixed consecutive number schedule in which a discriminative stimulus signaled when the response requirement on the run lever had been completed in one of the two fixed consecutive number component schedules. Under control conditions, the percentage of reinforced runs under the multiple-schedule component with the discriminative stimulus added was much higher than the percentage of reinforced runs under the multiple-schedule component without the discriminative stimulus. All of the drugs decreased the percentage of reinforced runs under each of the fixed consecutive number schedules by increasing the conditional probability of short run lengths. This effect was most consistently produced by morphine. The drugs produced few differences in responding between the multiple fixed consecutive number components. Responding under the simple fixed consecutive number schedule, however, was affected at lower doses of the drugs than was responding under the same fixed consecutive number schedule when it was a component of the multiple schedule. This result may be due to the difference in schedule context or, perhaps, to the order of the experiments.     

A tracking procedure for determining the cat's frequency discrimination

A tracking procedure was used to investigate the time required to train cats to discriminate between a frequency-modulated tone and a steady tone. The animal was reinforced with food on a VR schedule only when the steady tone was present and the animal pressed the correct bar (one of two). After reinforcement, the steady tone usually changed to a frequency-modulated signal; by pressing the other bar, the tone could be changed to its steady state and the reinforcement then obtained as before. A major difficulty was the lack of control by the auditory stimulus on the cat's responses. This problem was solved by introducing interpress time outs which forced the animal to hesitate after every press. The use of light cues to signal the time outs and the correct bar to press accelerated the rate at which the training progressed.

With cats, this conditioning procedure apparently requires a much longer training period before the actual threshold determinations than the more commonly used avoidance conditioning procedures. However, when animals are to be tested repeatedly over a period of several months or longer, the procedure may prove the more desirable one because it reduces experimental neurosis.

     

Response acquisition under targeted percentile schedules: a continuing quandary for molar models of operant behavior.

The number of responses rats made in a "run" of consecutive left-lever presses, prior to a trial-ending right-lever press, was differentiated using a targeted percentile procedure. Under the nondifferential baseline, reinforcement was provided with a probability of .33 at the end of a trial, irrespective of the run on that trial. Most of the 30 subjects made short runs under these conditions, with the mean for the group around three. A targeted percentile schedule was next used to differentiate run length around the target value of 12. The current run was reinforced if it was nearer the target than 67% of those runs in the last 24 trials that were on the same side of the target as the current run. Programming reinforcement in this way held overall reinforcement probability per trial constant at .33 while providing reinforcement differentially with respect to runs more closely approximating the target of 12. The mean run for the group under this procedure increased to approximately 10. Runs approaching the target length were acquired even though differentiated responding produced the same probability of reinforcement per trial, decreased the probability of reinforcement per response, did not increase overall reinforcement rate, and generally substantially reduced it (i.e., in only a few instances did response rate increase sufficiently to compensate for the increase in the number of responses per trial). Models of behavior predicated solely on molar reinforcement contingencies all predict that runs should remain short throughout this experiment, because such runs promote both the most frequent reinforcement and the greatest reinforcement per press. To the contrary, 29 of 30 subjects emitted runs in the vicinity of the target, driving down reinforcement rate while greatly increasing the number of presses per pellet. These results illustrate the powerful effects of local reinforcement contingencies in changing behavior, and in doing so underscore a need for more dynamic quantitative formulations of operant behavior to supplement or supplant the currently prevalent static ones.     

Control over response number by a targeted percentile schedule: reinforcement loss and the acute effects of d-amphetamine.

Two fixed-consecutive-number-like procedures were used to examine effects of acute d-amphetamine administration on control over response number. In both procedures, rats were required to press the left lever at least once and then press the right lever to complete a trial. The consecutive left-lever presses on each trial comprised a "run." Under the targeted percentile schedule, reinforcement was provided if the current run length was closer to the target length (16) than half of the most recent 24 runs. This differentially reinforced run length while holding reinforcement probability constant at .5. A second group acquired the differentiation under the targeted percentile schedule, but were then shifted to a procedure that yoked reinforcement probability by subject and run length to that obtained under the targeted percentile schedule. The two procedures generated practically identical control run lengths, response rates, reinforcement probabilities, and reinforcement rates. Administration of d-amphetamine disrupted percentile responding to a greater degree than yoked control responding. This disruption decreased reinforcement frequency less in the former than the latter procedure. The similar baseline responding under these two procedures suggests that this difference in sensitivity was due to behavioral adjustments to drug prompted by reduction of reinforcement density in the yoked control but not the percentile schedule. These adjustments attenuate the drug's effects under the former, but not the latter, procedure.     

A choice technique to assess the effects of selective punishment on fixed-ratio performance

The emission of a fixed number of responses by rats was followed by food reinforcement. This fixed number could be accumulated in any way from two continuously available but mutually incompatible response classes, bar pressing, and not bar pressing for a fixed time period. A preference for one response class was arranged by specifying different maximum reinforcement rates for the two classes. Under selective punishment conditions, the preferred response occasionally led to both food and electric shock, while the non-preferred response led only to food. Selective punishment effects were measured through changes in the preference to the two responses in the sequence. The actions of shock intensity, deprivation, the specification of the non-preferred response, and three drugs were investigated. The results were broadly similar to the work reported by Dardano and Sauerbrunn (1964), who found localized increases in interresponse times before punished responses in fixed-ratio schedules. Performance under this procedure was found to be stable and sensitive to each of the experimental variables examined.     

Conditioned suppression of counting behavior in rats

Three rats were trained on a schedule in which a response on lever B was reinforced only if it was preceded by a minimum number of consecutive responses on lever A. The minimum requirement was 27 A responses for Rat 1, and 20 A responses for Rats 2 and 3. The schedule maintained high rates of responding on lever A, and a slow, spaced pattern of responding on lever B. The mean number of consecutive responses on lever A was slightly greater than the minimum required. The effect of superimposing on this behavior a stimulus that ended with an unavoidable shock was the suppression of responding on both levers during the pre-shock stimulus. Responses on lever A were more suppressed, and the proportion of relatively short response runs on lever A during the pre-shock stimulus increased. With all three rats, the mean number of consecutive responses on lever A during the pre-shock stimulus decreased to a value below the minimum requirement for reinforcement of the subsequent B response.     

Effects of reinforcement scheduling on simultaneous discrimination performance

Pigeons were trained on a discrete-trials, simultaneous discrimination procedure, with confusable stimuli such that asymptotic performance was about 85% correct. Trials were terminated if no response occurred within 2 sec of stimulus onset, so that probability of responding was free to vary. The schedule of reinforcement for correct responses was varied, with the following results: (1) there was no relation between frequency of reinforcement and accuracy of responding. (2) In extinction, the probability of responding fell to low levels, but accuracy remained roughly constant. (3) When reinforcement was available after a fixed number of trials or after a fixed number of correct responses, the probability of responding increased with successive trials after reinforcement, but accuracy was generally constant. (4) When every fifth correct response was reinforced, accuracy decreased immediately after reinforcement if the birds were required to respond on every trial.     

Acquisition of lever-press responding in rats with delayed reinforcement: A comparison of three procedures

The present study examined the acquisition of lever pressing in rats under three procedures in which food delivery was delayed by 4, 8, and 16 seconds relative to the response. Under the nonresetting delay procedure, food followed the response selected for reinforcement after a specified interval elapsed; responses during this interval had no programmed effect. Under the resetting procedure, the response selected for reinforcement initiated an interval to food delivery that was reset by each subsequent response. Under the stacked delay procedure, every response programmed delivery of food t seconds after its occurrence. Two control groups were studied, one that received food immediately after each lever press and another that never received food. With the exception of the group that did not receive food, responding was established with every procedure at every delay value without autoshaping or shaping. Although responding was established under the resetting delay procedure, response rates were generally not as high as under the other two procedures. These findings support the results of other recent investigations in demonstrating that a response not previously reinforced can be brought to strength by delayed reinforcement in the absence of explicit training.     

Behavioral effects of caffeine, methamphetamine, and methylphenidate in the rat

It was possible to distinguish three closely-related psychomotor stimulants, caffeine, methamphetamine, and methylphenidate, by means of two operant behavior procedures, fixed interval and fixed number. Under the fixed interval procedure, the percentage change in the number of R_Bs per reinforcement was significantly smaller with caffeine than with methamphetamine or methylphenidate (p < .001). Under the fixed number procedure, the percentage change was significantly smaller with methamphetamine than with caffeine or methylphenidate (p < .001). Thus, methylphenidate had a methamphetamine-like effect under fixed interval and a caffeine-like effect under fixed number.     

Reinforcement probability and ordinal position of response in fixed-interval schedules

Four rats pressed levers and received food pellets under fixed-interval reinforcement schedules of 20, 60, and 180 seconds. The number of responses in each interval was recorded. From these data, the probability of reinforcement was determined as a function of response count. These functions were generally increasing. This finding is consistent with previous suggestions that increasing response rates within fixed intervals may be a function of response count in addition to or instead of elapsed or remaining time.     

Effects of different delay of reinforcement procedures on variable-interval responding

Two experiments studied responding in the rat when the first bar press after a variable period of time produced a cue light that remained on for either 10, 30, or 100 sec and terminated with the delivery of food. In Experiment I, response rate decreased and time to the first response after reinforcement increased as the delay of reinforcement increased. Similar results were obtained whether the delay consisted of retracting the lever during the delay, a fixed delay with no scheduled consequence for responding, or every response during the delay restarted the delay interval. In Experiment II, fixed-delay and fixed-interval schedules of the same duration during the delay period had no differential effect on either response rate or time to the first response after reinforcement, but differentially controlled responding during the delay periods.     

Signalled reinforcement in differential-reinforcement-of-low rate schedules

At several fixed and variable minimum reinforced interresponse times, a stimulus was added to differential-reinforcement-of-low-rate schedules to signal the availability or nonavailability of reinforcement. As the minimum reinforced interresponse time increased, the rate of unreinforced responding decreased. Changing from fixed to variable minimum interresponse time in the basic differential-reinforcement-of-low-rate schedule further decreased the rate of unreinforced responding. Both effects were to some degree reversible. For fixed minimum reinforced interresponse times of 30 sec or shorter, most unreinforced responses terminated interresponse times just short of that required for reinforcement. The minimum reinforced interresponse time and the number of short response latencies (≤0.5 sec) to the onset of the signal were negatively correlated. Both of these analyses suggested that at values of 30 sec or shorter, the subjects discriminated the availability of the reinforcer more on the basis of time than on the basis of presence or absence of the signal.     

The roles of stimulus control and reinforcement frequency in modulating the behavioral effects of d-amphetamine in the rat.

The behavioral effects of d-amphetamine have been shown to be modulated by stimulus control, with less impairment of performance occurring when control is great. When the fixed-consecutive-number schedule is used (on which at least a specified consecutive number of responses must be made on one operandum before a single response on another will produce a reinforcer), response rate tends to be invariant but reinforcement frequency is not. This study asks whether the differences in reinforcement frequency that usually accompany changes in stimulus control could themselves be responsible for the performance differences. Two versions of the fixed-consecutive-number schedule of reinforcement were combined into a multiple schedule within which stimulus control was varied but differences in reinforcement frequency were minimized by omitting some reinforcer deliveries during the component that usually had the higher reinforcement frequency. In one component, a compound discriminative stimulus was added with the eighth consecutive response on the first lever; a single response on the second lever was then reinforced. In the other component, no such stimulus was presented. With no added stimulus, large decreases occurred in the number of runs satisfying the minimum requirement for reinforcement at doses of drug that produced only minimal changes when an added stimulus controlled behavior. Thus, increased stimulus control diminishes the behavioral changes produced by d-amphetamine even when the possible contribution by baseline reinforcement rate is minimized.     

Choice and behavioral patterning

Ten pigeons pecked left and right keys in a discrete-trials experiment in which access to food was contingent upon changeovers to the right key after particular runs of left-key pecks. In each of three sets of conditions, two run lengths were reinforced according to a concurrent variable-interval schedule: reinforcement followed runs of either 1 or 2, 1 or 4, or 2 or 4 left-key pecks preceding changeovers. The intertrial interval separating successive pecks was varied from .5 to 10.0 sec, and the relative frequency of reinforcement for the shorter of the two reinforced runs was varied from 0 to .75. The contingencies established local behavioral patterning that roughly approximated that required for reinforcement. For a fixed pair of reinforced run lengths, preference for the shorter of the two frequently increased as the intertrial interval increased and therefore as the minimum temporal durations of both reinforced runs increased. Preference for the shorter of the two also increased as its corresponding relative frequency of reinforcement increased. Both of these effects on preference were qualitatively similar to corresponding effects in previous research with two different kinds of reinforced behavioral patterns, interresponse times and interchangeover times. In all these experiments, analytical units were found in the temporal patterns of behavior, not in the behavior immediately contiguous with a reinforcer. It is suggested that a particular local temporal pattern of behavior is established to the extent to which it is repeatedly remembered when reinforcers are delivered, regardless of whether the delivery of a reinforcer is explicitly contingent upon that pattern.     

Partial reinforcement effects and type of reward

In two experiments 68 rats were trained to bar press or run down a straight runway for food or for water under conditions of either continuous reinforcement or partial reinforcement. In both experiments, there was greater persistence of behavior which had been reinforced with food than with water. In Expt 2, the partial reinforcement extinction effect (PREE) was observed with food reward, but not with water.Within the context of the experimental procedures used, it can be concluded that the rat has mechanisms for developing persistence which are dependent on the specific motivational system involved. This conclusion is related to theories of partial reinforcement effects and to possible biological origins of the mechanisms.     

Stimulus change as a factor in response maintenance with free food available.

Rats bar pressed for food on a reinforcement schedule in which every response was reinforced, even though a dish of pellets was present. Initially, auditory and visual stimuli accompanied response-produced food presentation. With stimulus feedback as an added consequence of bar pressing, responding was maintained in the presence of free food; without stimulus feedback, responding decreased to a low level. Auditory feedback maintained slightly more responding than did visual feedback, and both together maintained more responding than did either separately. Almost no responding occurred when the only consequence of bar pressing was stimulus feedback. The data indicated conditioned and sensory reinforcement effects of response-produced stimulus feedback.     

Differential reinforcement of lever-press durations: Effects of deprivation level and reward magnitude

Three experiments investigated the performance of rats on a task involving differential reinforcement of lever-press durations. Experiment 1, which employed a discrete-trials procedure, manipulated deprivation level between subjects and reward magnitude within subjects. The minimum lever-press duration which would result in reward was varied from .4 to 6.4 sec. It was found that low deprivation resulted in longer mean durations and less response variability at the higher criterial values than did high deprivation. The magnitude of reward was not found to affect performance. Experiment 2 manipulated reward magnitude between subjects while holding deprivation level constant, and used the same general procedures as in Experiment 1. Small reward resulted in longer mean lever-press durations and less variability in responding than did large reward at the higher criterial values. The intertrial intervals were omitted in Experiment 3 in which deprivation level was varied between subjects and reinforcement was delivered only for response durations extending between 6.0 and 7.6 sec. Low deprivation resulted in longer mean lever-press durations and less response variability than did high deprivation, but the probability of a rewarded press duration did not differ between groups. The results overall are consistent with the hypothesis that low deprivation and small reward magnitude lead to weaker goal-approach responses and, hence, to less competition with lever holding. The deprivation and reward magnitude manipulations did not appear to influence lever holding performance by affecting the ability of animals to form temporal discriminations.     

Characteristics and response-displacement effects of shock-generated responding during negative reinforcement procedures: pre-shock responding and post-shock aggressive responding

Bar-pressing (Experiment I) or key-pressing (Experiments II and III) responses of monkeys were reinforced according to a fixed-interval schedule of negative reinforcement: the first response after a fixed interval of time terminated regularly spaced shocks for a fixed time designated as the reinforcement period. During extinction, shocks continued during the reinforcement period. That there were two types of responding generated by shock alone was indicated by (1) the level of responding maintained during extinction relative to conditions without shock, (2) the stability of two between-shock response patterns across reinforcement and extinction conditions, and (3) the development of these two between-shock patterns without a history of reinforcement. Subjects developed either a pre-shock or a post-shock response pattern when only the bar was available. However, when both a bite tube, an operandum requiring an aggressive topography, and a recessed key, an operandum that did not require an aggressive topography, were provided, the post-shock pattern was observed in tube biting and the pre-shock pattern was observed in key pressing. Removal of the bite tube produced post-shock key responding similar to that observed when only the bar was available. The displacement of post-shock, aggression-motivated responding confirmed the confounding effect of shock-generated responding in negative reinforcement procedures, and suggests that the use of concurrent response alternatives would reduce such confounding.     

Continuous, fixed-ratio, and fixed-interval reinforcement in honey bees

Bees learned to enter a Plexiglas tube and to suck small portions of sugar solution; every entry or every fifth entry was reinforced. During an extinction phase, the bees on the fixed-ratio schedule emitted twice as many responses as did those given continuous reinforcement. Bees on a fixed-interval schedule of reinforcement emitted lower response rates than did those given fixed-ratio reinforcement. By extending the conditioning procedure for several days, it was possible to maintain responding with fixed-ratio schedules requiring 30 responses per reinforcement and with fixed-interval values up to 90 sec. Under fixed-interval schedules, response rates did not increase toward the end of the reinforcement intervals.     

The operant control of vocalization in the dog

Control over the vocal responses of three dogs was established using operant-conditioning procedures. Several points of interest were observed in the data. First, fixed-ratio schedules of reinforcement generated a vocal response topography which was similar in detail to that of a “motor” bar-nosing response. Second, vocal responding was brought under the control of external visual stimuli as a result of differential reinforcement. Third, good stimulus control was maintained on a multiple schedule containing a vocal-response component and a bar-response component. Fourth, the stimulus control on the multiple schedule transferred with minimal disruption to a chain schedule requiring a sequence of 10 bar responses followed by 10 vocal responses. Fifth, because vocal and bar responses are not mutually exclusive, concurrent responding tended to develop on the chain schedule.

These results were discussed with reference to the advisability of applying the terms operant and respondent to unconditioned behavior, and, particularly, to unconditioned verbal behavior.