Choice of a terminating over a non-terminating signal in free-operant avoidance

Rats chose between two signalled avoidance schedules. Under one schedule, responses in the presence of the signal terminated it and resulted in avoidance of shock; in the other, responses in the presence of the signal resulted in shock avoidance but signal termination was delayed for 10 sec. Pressing a second (changeover) lever produced change from one schedule to the other for 1 min. Once this 1-min period timed out, subjects could remain under the schedule in effect or could reinstate the other schedule for another 1-min period. All four subjects continuously changed over from the non-terminating to the terminating signal schedule. Changeover responding was not maintained when the termination contingency was removed. When changeover responding resulted in a change from a terminating to a non-terminating signal schedule, changeover responding did not occur.     

Concurrent schedules of reinforcement: effects of gradual and abrupt increases in changeover delay

Pigeons' pecks were maintained on concurrent variable-interval 1-min variable-interval 3-min schedules of reinforcement, with a changeover delay of 2 sec. When changeover delay was increased successively to 5.0, 7.5, and 12.5 sec (Exp. I) the actual relative rate of reinforcement for the variable-interval 3-min key decreased progressively for two birds, abruptly for two other birds, and the subjects devoted proportionately less of their time and responding to that key. However, the relative performance measures (relative time and relative responding) approximated the actual relative rate of reinforcement, with a maximum discrepancy of 11%, over all changeover delay values investigated. Experiment II attempted to lengthen response-run durations on the variable-interval 3-min key so that they were long enough to meet the changeover delay requirement at each new changeover delay value, by progressively increasing the changeover delay by 0.5-sec increments. With this procedure the actual relative rate of reinforcement approximated more closely the scheduled relative rate as changeover delay increased. As in Exp. I, relative performance measures approximated the actual relative reinforcement rate (maximum discrepancy 17%).     

Changeover delay and concurrent schedules: some effects on relative performance measures

The pigeon and the rat partition total response output between both schedules of a concurrent variable-interval pair. The quantitative nature of a partition seems critically dependent on the relative rates with which the two schedules provide reinforcements for responding, in addition to the changeover delay. The manner in which the changeover delay controls the partition was studied by varying the duration of the changeover delay from 0 to 20 sec with each of two pairs of concurrent variable-interval schedules, viz., Conc VI 1.5-min VI 1.5-min and Conc VI 1-min VI 3-min. Rats served as the subjects and brain stimulation was employed as the reinforcer. When the schedules were Conc VI 1.5-min VI 1.5-min, relative response rate approximated 0.50 at all values of the changeover delay. When the schedules were Conc VI 1-min VI 3-min, relative response rate, computed with respect to the VI 1-min schedule, increased when the duration of the changeover delay increased. Changeover rate decreased when the duration of the changeover delay increased. The decrease was the same for both VI schedules of the Conc VI 1.5-min VI 1.5-min pair but was more rapid for the VI 3-min schedule of the Conc VI 1-min VI 3-min pair.     

Concurrent performances: a baseline for the study of conditioned anxiety

Three rats were trained to lever press on concurrent random interval 2-min random interval 2-min schedules of milk reinforcement. With a 5-sec changeover delay, relative response rate matched the relative reinforcement duration associated with each lever. A stimulus, during which unavoidable shocks occurred at random intervals, was superimposed on this concurrent baseline, and shifts in preference were found. However, data from this procedure were ambiguous, apparently confounded by shock-elicited response bursts. Termination of the shocks during the stimulus resulted in a rapid recovery of matching, which was preceded by a brief facilitation of responding on the less-preferred lever. The procedure was then changed to a conventional conditioned anxiety paradigm with a variable duration pre-shock stimulus. A marked shift in relative response rate towards the preferred lever was found in all three rats; that is, responding on the preferred lever was far less suppressed during the pre-shock stimulus than responding on the less-preferred lever.     

EFFECTS OF SPACED RESPONDING DRL ON THE STEREOTYPED BEHAVIOR OF PROFOUNDLY RETARDED PERSONS

Stereotypic responding and social behaviors of three profoundly retarded children were measured before and during application of a DRL contingency for stereotypic responding. A variant of the standard DRL procedure, spaced responding DRL, was used, in which reinforcement is delivered following a response if that response has been separated from the previous response by at least a fixed minimum time interval. Three children were treated by using a reversal design. Results showed that: (a) during baseline sessions, the children engaged in high rates of stereotypic responding and very low rates of appropriate social behavior; and (b) during DRL sessions, appropriate behavior increased markedly as stereotypic responding was reduced. The data suggest that spaced responding DRL may be effective in increasing appropriate social behavior as well as in reducing stereotypic responding.     

A COMPARISON OF TWO PROCEDURES FOR PROGRAMMING THE DIFFERENTIAL REINFORCEMENT OF OTHER BEHAVIORS

The relative effectiveness of two methods of programming DRO schedules of reinforcement was examined in two experiments. In these two methods, reinforcement is delivered if inappropriate responding is not occurring (a) at the end of an interval (momentary DRO), or (b) throughout the entire interval (whole-interval DRO). In Experiment 1, the effects of these schedules on disruptive responding of three retarded students were assessed in a multiple-baseline design. For two students, the momentary schedule occurred first and was ineffective, whereas the whole interval that followed was effective; for the third student, the whole-interval schedule occurred first and was effective, and reduced responding was maintained under the momentary schedule. In Experiment 2, baseline and the two DRO schedules were each presented in random order each day to one student in an alternating treatments design. The momentary DRO schedule reduced responding, but the whole-interval schedule was more effective.     

Preference for less frequent shock under fixed-interval schedules of electric-shock presentation.

Lever pressing by 2 squirrel monkeys was maintained under fixed-interval 6-min and fixed-interval 2-min schedules of electric-shock presentation. Preference for these schedules was assessed during three experimental phases. In all phases, responses on one lever produced shock according to one or the other fixed-interval schedule, and responses on a second, changeover, lever switched between schedules. The opportunity to change over was presented during separate choice periods (during which the fixed-interval schedules did not operate) that followed the first through fourth shocks in each schedule. If no changeover occurred during those choice periods, a changeover automatically occurred following the fifth shock. In Phase I, durations of the choice periods were fixed. In Phase II, the choice periods equaled a proportion of their respective fixed interval. During Phase III (completed with 1 monkey) a response on the changeover lever during a given choice period reinstated the most recent fixed interval, and a failure to respond resulted in a changeover. During each of these phases, distinct preferences developed for the 6-min schedule. These results suggest that the maintenance of lever pressing by fixed-interval presentation of electric shock may not be an example of positive reinforcement, and that the response-maintaining characteristics of shock presentation may derive from other properties of the schedule.     

Some Effects Of Intertrial Interval Duration On Discrete-trial Choice

Pigeons were trained in Experiment 1 on a discrete-trial concurrent variable-interval (VI) 1-min VI 3-min schedule, and in Experiment 2 on a discrete-trial concurrent VI 1.5-min VI 1.5-min schedule. In each experiment, the intertrial-interval durations (ITIs) were 0 s, 6 s, 22 s, and 120 s, and the schedules were both independent and interdependent. The purpose of the research was to determine whether lengthening the ITI would disrupt any local control that existed, measured with respect to relative response rate and changeover probability. In Experiment 1, with the independent schedules, both preference and obtained relative reinforcement rate approximated .75 at short ITIs, but then decreased toward .50 with longer ITIs. With interdependent schedules, both preference and obtained relative reinforcement rate approximated .75 at all ITIs. In both experiments, with both independent and interdependent schedules, changeover probabilities for each response in a sequence of up to five successive responses to a given schedule were variable for individual birds. The average changeover probabilities for all birds suggested perseveration rather than a systematic increase within a given ITI or a systematic trend toward chance responding as ITI duration increased. Finally, the changeover functions did not differ when a sequence of responses was calculated to begin anew after reinforcement rather than with the first response on a schedule. Taken together, the data were inconsistent with a theory holding that only local processes underlie choice in discrete-trial procedures.     

Punished and Nonpunished Responding in A Multiple Schedule in Humans: A Brief Report

Four male subjects responded on a multiple schedule in which responding was maintained by a random interval 20-sec (RI20) schedule of point presentation. Responding was suppressed in alternating components by an added variable ratio 30 (VR30) schedule of point subtractions. Each component was accompanied by distinctive stimulus lights. Subjects were exposed to the multiple schedule from the initial session. Two subjects experienced four 50-min sessions daily (Experiment 1) and the other two subjects participated in one 50-min session daily (Experiment 2). Once responding in the punished components had stabilized, responding in the nonpunished components continued to increase across sessions. Nonpunished responding did not stabilize even after as many as 36 sessions. These results are discussed in the context of previous studies using animals which employed multiple schedules with punished and nonpunished response contingencies.

     

Some determinants of inhibitory stimulus control

Interspersed reinforcement and extinction during discrimination learning generate a U-shaped gradient of inhibition about the stimulus correlated with extinction. The present work showed that extinction is not a necessary determinant of inhibitory stimulus control. In Exp. I, a reduction in the rate of reinforcement, through a shift from a multiple variable-interval 1-min variable-interval 1-min schedule to a multiple variable-interval 1-min variable-interval 5-min schedule, resulted in a post-discrimination line orientation gradient of inhibition about the stimulus correlated with the variable-interval 5-min schedule. In Exp. II, the rates of reinforcement, correlated with a pair of stimuli, were held constant during a shift from a multiple variable-interval 1-min variable-interval 1-min schedule to a multiple variable-interval 1-min differential-reinforcement-of-low-rate schedule. Inhibitory stimulus control about the stimulus correlated with the differential reinforcement of low rate was obtained. In both experiments, a reduction in the rate of responding during one stimulus and behavioral contrast during the other stimulus preceded the observation of inhibitory stimulus control.     

Prospective factors contribute little to within-session changes in responding

Five rats pressed levers for food delivered by a multiple variable interval 1-min variable interval 1-min schedule. In theunpredictable conditions, sessions were 20, 40, 60, 80, or 100 min long, determined randomly at the beginning of each session. In thepredictable conditions, each of these session durations was presented for 15 consecutive sessions. Rate of responding changed systematically within the session even when the end of the session was unpredictable. This implies that prospective factors related to anticipation of the end of the session are not necessary for producing withinsession changes in responding. Within-session patterns of responding were also similar for the predictable and unpredictable conditions. This suggests that prospective factors contributed little to the form of the within-session patterns under the present conditions.     

CONCURRENT RESURGENCE AND BEHAVIORAL HISTORY

The contribution of past experiences to concurrent resurgence was investigated in three experiments. In Experiment 1, resurgence was related to the length of reinforcement history as well as the reinforcement schedule that previously maintained responding. Specifically, more resurgence occurred when key pecks had been reinforced on a variable-interval 1-min schedule than a variable-interval 6-min schedule, but this effect may have been due either to the differential reinforcement rates or differential response rates under the two schedules. When reinforcement rates were similar (Experiment 2), there was more resurgence of high-rate than low-rate responding. When response rates were similar (Experiment 3), resurgence was not related systematically to prior reinforcement rates. Taken together, these three experimental tests of concurrent resurgence illustrate that prior response rates are better predictors of resurgence than are prior reinforcement rates.     

Inhibitory stimulus control in concurrent schedules

Six pigeons were exposed to two keys, a main key and a changeover key. Pecking the main key was reinforced on a variable-interval 5-min schedule when the key was blue and never reinforced when the key displayed a vertical line on a blue background. Each peck on the changeover key changed the stimulus displayed on the main key. Each subject was given two generalization tests, consisting of presentations on the main key of six orientations of the line on the blue background, with no reinforcements being given. In one test changeover-key pecks changed the stimulus; in the other test the changeover key was covered and the experimenter controlled stimulus changes. Both responses to the six stimuli and time spent in the presence of the stimuli gave U-shaped gradients when the changeover key was operative. With most subjects, absolute rates of responding to each stimulus produced unsystematic gradients, whether or not the changeover key was operative.     

Choice: Effects of changeover schedules on concurrent performance

The components of concurrent schedules were separated temporally by placing interval schedules on the changeover key. The rates of responding on both the main and changeover keys were examined as a function of the reinforcement rates. In the first experiment, the sensitivity of main-key performance to changing reinforcement rates was inversely related to the temporal separation of components, and changeover performance was monotonically related to the ratio of the reinforcement rates. In the second experiment, when the ratio of the reinforcement rates was scheduled to remain constant while the frequency of reinforcement was varied, changeover performance did not remain constant. A “sampling” interpretation of changeover responding was proposed and subsequently tested in a third experiment where extinction was always scheduled in one component and the frequency of reinforcement was varied in the second component. It was concluded that changeover performance can be interpreted using molar measures of reinforcement and that animals sample activities available to them at rates which are controlled by relative reinforcement rates.     

The relative aversiveness of signalled vs unsignalled avoidance

Subjects avoided shock by pressing on one lever under an unsignalled condition, but by pressing a separate lever they changed the condition to signalled avoidance for 1-min periods. Signalled avoidance periods were identified by a correlated stimulus. All eight subjects responded to change the unsignalled schedule to a signalled one. Once contact with signalled avoidance was made, subjects continued responding to remain in that condition. Other tests showed that changeover responding was greater when the correlated stimulus was presented without the signal than when the signal was presented without the correlated stimulus. An analysis based upon shock and shock-free periods is presented.     

Behavioral parameters of drug action: signalled and response-independent reinforcement

Four pigeons were initially trained under a multiple variable-interval 1-min variable-interval 1-min schedule of food reinforcement. For two of the pigeons, a signal was then presented whenever the reinforcer was available in one component; this resulted in positive contrast. For the other two pigeons, the reinforcer was presented independently of responding on a variable-time schedule in one component; this resulted in negative induction. After 30 to 50 sessions, however, a similar degree of differential responding occurred under both multiple schedules, i.e., high rates in the variable-interval component and low rates in the other component. Reinforcement frequency remained about the same in each of the schedule components. The stable performances then served as baselines for studying drug effects. In the high-rate component of both multiple schedules, small doses of d-amphetamine increased responding, whereas larger doses decreased responding. In the low-rate component of both multiple schedules, there was no rate-increasing effect at any dose of d-amphetamine; such an effect was found, however, with phenobarbital at a dose that decreased responding in the high-rate component. The drug effects thus depended on the interaction of pharmacologic variables (specific drug and dose) with behavioral variables (schedule components).     

The effects of brief stimuli presented under a multiple schedule of second-order schedules

The effects of briefly presented stimuli paired or not paired with food reinforcement were investigated in the pigeon on a multiple schedule containing second-order schedules. A stimulus paired with food reinforcement was presented on a variable-interval schedule in one unit of the multiple schedule and either a stimulus not paired with food reinforcement or no stimuli were scheduled in the other unit. Response rates were highest when behavior was followed by the food-paired stimulus. Presentation of the food-paired stimulus at completion of each 1-min variable-interval component maintained a steady rate of responding between consecutive food presentations. Pausing following food reinforcement was greatest in the second-order schedule not containing the paired stimulus. Reversing the stimulus pairings led to a reversal of the relative response rates and patterns of responding for each stimulus.     

A preliminary evaluation of the emergence of novel mand forms

Strategies that produce generalized responding are valuable, especially with regard to language acquisition, because relatively little training may result in large behavior changes. Conditions that result in generalized manding were analyzed in the current study. We demonstrated in reversal designs that undesirable or single-word responses were the predominant mand forms of 3 preschool children. Multiple baseline designs with 2 participants and a reversal design with 1 participant were then used to demonstrate the extent to which differential reinforcement of single-word mands (e.g., "cars") or framed mands (e.g., "I want the cars, please") would result in the emergence of other single-word and framed mands for different items (e.g., mands for music, puppets, or puzzles). Results showed that prompting and differential reinforcement of one or two mand frames resulted in the emergence of other framed mands for all participants.     

Observational learning in monkeys

Observer monkeys were housed next to demonstrator monkeys that were conditioned to respond on a multiple reinforcement schedule whose components were fixed-ratio 32, variable-interval 3-min, and extinction 5-min followed by an additional 30 sec of extinction during which every response started a new 30-sec interval. After observational periods from 113 to 210 hr long, during which observers could not perform the response and were given no extrinsic reinforcers, their first-response latencies to fixed ratio and variable interval were as short as the demonstrators; and their rates of responding were well above pre-observational baseline levels. About 8 hr later, a temporal pattern of responding appropriate to the multiple schedule emerged, including non-emission of responses during extinction. Controls lacking the chance to observe did not develop typically patterned responding after 60 hr in one case and, in two other cases, after 80 hr during which, on two occasions, every one of 50 responses was reinforced. In a second experiment, the stimulus lights associated with fixed ratio and variable interval were presented simultaneously. Subjects chose one of the schedules by responding to one of the levers beneath the lights. All subjects initially chose fixed ratio. Seeing the demonstrators switch to variable interval, due to increases in the fixed-ratio requirement, had no effect upon observers, which continued to choose fixed ratio.     

Transfer of a response controlling stimulus duration across discrimination problems

In the acquisition phase, pigeons learned to peck at a changeover key to shorten the duration of S? but not of S+ presented on the food key in a discrimination problem. In the transfer phase, the significance of S+ and S? was changed through extinction of both, equal reinforcement, or discrimination reversal, while the changeover key was not available. Transfer tests then showed appropriate modification of the changeover response. Similar transfer was demonstrated across orthogonal stimulus dimensions. Further analytic studies showed that this transfer of the changeover response did not depend upon mediation due to differential response rates to the food key. This research strategy enriches the study of the “second learning process” by providing an indicator of stimulus control in all phases of the procedure. Direct transfer between different problems also indicates that discriminative stimuli, although physically dissimilar, have the same “psychological value” for the subject.