Effects of chlorpromazine on fixed-ratio responding: modification by fixed-interval discriminative stimuli.

Effects of chlorpromazine (1 to 100 mg/kg) were assessed on two pigeons' responding under various modifications of a multiple schedule of food delivery. During a fixed-interval component, the first response after 5 min produced food; during the subsequent, fixed-ratio component, the 30th response produced food. Modifications of the schedule entailed changes in stimulus conditions imposed during the fixed-ratio component that did not systematically alter characteristics of performance under non-drug conditions. In the first phase of the experiment, distinctive visual stimuli were correlated with each schedule component (conventional multiple schedule); chlorpromazine produced small decreases in fixed-ratio responding (20% at 30 mg/kg). When each response during the fixed-ratio component produced the stimulus correlated with the fixed-interval schedule (fixed-interval discriminative stimulus) for 1.2 s, effects of chlorpromazine were not different from those under the conventional multiple schedule. Chlorpromazine produced greater decreases in fixed-ratio responding (55% at 30 mg/kg) when either the first response of each fixed ratio changed the stimulus correlated with the fixed-ratio schedule to the fixed-interval discriminative stimulus for the remainder of the fixed-ratio component, or when the fixed-interval discriminative stimulus was presented independently of responding according to a matched temporal sequence. When the fixed-interval discriminative stimulus was present continuously during the fixed-ratio component (mixed schedule), chlorpromazine produced even more substantial decreases in fixed-ratio responding (greater than 80% at 30 mg/kg). Effects of chlorpromazine on fixed-interval responding were also modified by the schedules of fixed-interval discriminative stimulus presentation. The effects of chlorpromazine were a joint function of the stimuli prevailing during the multiple schedule and the degree to which responding influenced these stimuli.     

Inverse relations between preference and contrast

Pigeons were trained on a multiple schedule in which two target components with identical reinforcement schedules were followed by either the same-valued schedule or by extinction. Response rate increased in both target components but was higher in the target component followed by extinction, replicating previous findings of positive anticipatory contrast. A similar design was used to study negative contrast, in that the two target components were followed either by the same-valued schedule or by a higher valued schedule. Negative contrast occurred equally, on average, in both target components, thus failing to demonstrate negative contrast that is specifically anticipatory in nature. When the stimuli correlated with the two target components were paired in choice tests, the pattern of preference was in the opposite direction. For the positive contrast procedure, no significant preference between the two target stimuli was evident. But for the negative contrast procedure, preference favored the stimulus followed by the higher valued schedule. The results demonstrate a functional dissociation between positive and negative contrast in relation to stimulus value. More generally, the results demonstrate an inverse relation between response rate and preference and challenge existing accounts of contrast in terms of the concept of relative value.     

Stimulus control of behavioral history.

Pigeons were exposed to two different reinforcement schedules under different stimulus conditions in each of two daily sessions separated by 6 hr (Experiments 1 and 2) or in a single session (Experiment 3). Following this, either a fixed-interval (Experiment 1) or a variable-interval schedule (Experiments 2 and 3) was effected in both stimulus conditions. In the first two experiments, exposure to fixed-ratio or differential-reinforcement-of-low-rate schedules led to response-rate, but not pattern, differences in subsequent performance on fixed- or variable-interval schedules that persisted for up to 60 sessions. The effects of reinforcement-schedule history on fixed-interval schedule performance generally were more persistent. In Experiment 3, a history of high and low response rates in different components of a multiple schedule resulted in subsequent response-rate differences under identical variable-interval schedules. Higher response rates initially occurred in the component previously correlated with high response rates. For 3 of 4 subjects, the differences persisted for 20 or more sessions. Previous demonstrations of behavioral history effects have been confined largely to between-subject comparisons. By contrast, the present results demonstrate strong behavioral effects of schedule histories under stimulus control within individual subjects.     

The effects of brief-stimulus presentations in fixed-ratio second-order schedules

Pigeons' responses were reinforced according to a three-component multiple schedule. In Component 1, key pecks produced food according to a fixed-ratio second-order schedule with fixed-ratio units. Here, a fixed number of fixed-ratio units produced food, and the brief stimulus terminating each unit also accompanied food. Responses in Component 2 produced food on an identical schedule except that the brief stimulus was not paired with food. Component 3 contained a simple fixed-ratio schedule whose response requirement equaled that of Components 1 and 2. Across conditions the size of the fixed-ratio unit (five, ten, twenty, forty, and eighty responses) and the total number of responses per reinforcement were parametrically manipulated. The highest response rates and shortest preratio pauses were observed in Component 3 (no brief stimulus). The lowest rates and longest pauses were found in the component with paired brief-stimulus presentations, indicating that the food-paired brief stimulus suppressed responding. The suppressive effects were greatest when the fixed-ratio units were small (e.g., fixed-ratio 5) and the total fixed-ratio requirement was large (e.g., fixed-ratio 160). Under no conditions did the paired brief stimulus facilitate responding. The nonpaired brief stimulus also suppressed responding but to a lesser extent. The suppressive effects of nonpaired brief stimuli were greatest when the fixed-ratio units were small and the total response requirement was large. These data suggest that the suppressive effects of the brief stimuli may have masked the conditioned-reinforcing effects reported in other studies, and that conditions that maximize suppression in second-order schedules involve the use of fixed-ratio schedule units and the presentation of many brief stimuli per reinforcer.     

Positive conditioned suppression: Transfer of performance between contingent and noncontingent reinforcement situations

Five homing pigeons were trained on concurrent variable-interval schedules. A fixed-duration stimulus was occasionally presented on one key; and, in various conditions, this stimulus terminated (a) without reinforcement, (b) in noncontingent reinforcement, (c) with reinforcement contingent on a response on the key on which the stimulus was presented, and (d) with reinforcement contingent on a response on the key on which the stimulus was not presented. Initially, a stimulus terminating in noncontingent reinforcement generally produced decreased response rates on both keys during the stimulus. Contingencies, however, reliably produced increased rates during the stimulus on the key on which the contingency was arranged, relative to the rate on the concurrently available key. Contingency conditions were followed by noncontingency conditions in which the separation of rates caused by contingencies was maintained. When rates during the stimulus were compared with response rates on the same keys in the absence of the stimulus, contingency-caused rate increases and decreases were again found, but only the rate decreases were maintained in subsequent noncontingency conditions. Further data suggested that the contingency-caused rate changes were not maintained when the stimulus terminated without reinforcement, and that they were unaffected by a threefold decrease in the reinforcement rate provided by the baseline schedules. The results support the suggestion that performance in the positive conditioned suppression procedure results from concurrent and multiple schedule interactions. They further suggest that the production of either acceleration or suppression is dependent on adventitious and historical contingencies.     

Testing for inhibitory stimulus control with S- superimposed on S+

Pigeons learned a successive discrimination between a positive stimulus (red) correlated with a variable-interval 1-min reinforcement schedule and a negative stimulus (vertical line) correlated with either a variable-interval 5-min schedule or extinction. Transfer tests measured the rate of responding to the positive stimulus alone, to various orientations of the negative stimulus, and to the same line orientations superimposed on the positive stimulus. Although there were no gradients with minima at the training value for the negative stimulus dimension, the addition of the negative stimulus dimension to the positive stimulus always resulted in a lower response rate than that for the positive stimulus alone. The results demonstrate that an operational definition of inhibitory stimulus control that requires increased responding to stimuli more distant from a negative stimulus (along some dimension) is not always consistent with a definition that requires the suppression of responding in the presence of one stimulus, the positive stimulus, by the simultaneous presentation of another, the negative stimulus.     

Variable-interval schedules of timeout from avoidance

Rats were trained on concurrent schedules in which pressing one lever postponed shock and pressing the other occasionally produced a 2-min timeout during which the shock-postponement schedule was suspended and its correlated stimuli were removed. Throughout, the shock-postponement schedule maintained proficient levels of avoidance. Nevertheless, in Experiment 1 responding on the timeout lever was established rapidly, was maintained at stable levels on variable-interval schedules, was extinguished by withholding timeout, was reestablished when timeout was reintroduced, and was brought under discriminative control with a multiple variable-interval extinction schedule of timeout. These results are in contrast with Verhave's (1962) conclusion that timeout is an ineffective reinforcer when presented to rats on intermittent schedules. In Experiment 2 the consequence of responding on the timeout lever was altered so that the shock-postponement schedule remained in effect even though the stimulus conditions associated with timeout were produced for 2 min. Responding extinguished, indicating that suspension of the shock-postponement schedule, not stimulus change, was the source of reinforcement. By establishing the reinforcing efficacy of timeout with standard variable-interval schedules, these experiments illustrate a procedure for studying negative reinforcement in the same way as positive reinforcement.     

Behavior simultaneously maintained by both presentation and termination of noxious stimuli

Lever pressing by two squirrel monkeys was maintained under a 3-minute variable-interval schedule of response-produced electric-shock presentation. At the same time, responding on a second lever was maintained under a 3-minute fixed-interval schedule of termination of the shock-presentation schedule and shock-correlated stimuli. Under the termination schedule, the first response after a 3-minute period produced a 1-minute timeout, during which no events occurred and responding had no scheduled consequence. Relatively high and constant rates of responding were maintained on the lever where responding produced shock. Lower rates and positively accelerated patterns of responding occurred on the lever where responding terminated the shock schedule. Thus, responding was simultaneously maintained by presentation of an event and by termination of a stimulus associated with that event. Rates and patterns of responding on each lever were reversed when the schedules arranged on each lever were reversed on two occasions. When shock intensity was increased from 0 to 10 mA, responding maintained both by presentation of shock and by termination of the shock schedule increased, but responding maintained by shock presentation increased to a greater extent. Positive and negative reinforcement, usually regarded as separate behavioral processes involving different events, can coexist when behavior is controlled by different contingencies involving the same event.     

A comparison of responding maintained under second-order schedules of intramuscular cocaine injection or food presentation in squirrel monkeys.

Key pressing by squirrel monkeys was maintained under second-order schedules of either intramuscular cocaine injection or food presentation. Under one schedule, each completion of a 10-response fixed-ratio unit produced a brief visual stimulus; the first fixed-ratio unit completed after 30 minutes elapsed produced the stimulus paired with either cocaine injection or food presentation. Generally, short pauses followed by high rates of responding were maintained within the fixed-ratio units, and responding was positively accelerated over the 30-minute interval. Under another schedule, each completion of a 3-minute fixed-interval unit produced the brief stimulus; completion of the 10th fixed-interval unit produced the stimulus paired with either cocaine injection or food presentation. Generally, short pauses followed by high rates of responding were maintained within the fixed-ratio units, and responding was positively accelerated over the 30-minute interval. Under another schedule, each completion of a 3-minute fixed-interval unit produced the brief stimulus; completion of the 10th fixed-interval unit produced the stimulus paired with either cocaine injection or food presentation. Rates of responding increased within the fixed-interval units, and to a greater extent over the entire 10 fixed-interval units. Patterns of responding depended more on the schedule of reinforcement than on whether cocaine or food maintained responding. Omitting the brief stimuli following all but the last fixed-ratio or fixed-interval units decreased average rates and altered the patterns of responding. Substituting a visual stimulus that was never paired with cocaine or food following all but the last fixed-ratio or fixed-interval units decreased response rates to a lesser extent and did not substantially alter patterns of responding. When the duration of the paired stimulus was varied from .3 to 30.0 seconds, the highest response rates occurred at intermediate durations (1.0 to 10.0 seconds). The manner in which the stimulus changes affected performances depended more on the schedule of reinforcement than on whether cocaine injection or food presentation maintained responding.     

Effects of d-amphetamine on responding under second-order schedules of reinforcement with paired and nonpaired brief stimuli.

Three pigeons were studied under a multiple schedule in which pecks in each component were reinforced according to a variable-interval 120-s second-order schedule with fixed-interval 60-s units. In the first component of the multiple schedule, the completion of a fixed interval produced either food or a 4-s change in key color plus houselight illumination. In the second component an identical schedule was in effect, but the stimulus was a 0.3-s change in key color. Both long and short brief stimuli were not paired with food presentations in Conditions 1 and 3 and were paired with food in Condition 2. There were no consistent differences in response patterns under paired and nonpaired brief-stimulus conditions when the stimulus was a 4-s change in key color accompanied by houselight illumination. However, pairing the 0.3-s key-color change with food presentations resulted in higher indices of curvature and lower response rates in the early segments of the fixed interval than when the stimulus was not paired with food presentations. Low doses of d-amphetamine (0.3 and 1 mg/kg) produced small and inconsistent increases in overall response rates, and higher doses (3 and 10 mg/kg) decreased overall response rates. d-Amphetamine altered response patterns within fixed intervals by decreasing the indices of curvature and increasing response rates in the early segments of the fixed interval. Response rates and patterns under paired and nonpaired brief-stimulus conditions were not differentially affected by d-amphetamine. Thus, evidence for the enhancement of the conditioned reinforcement effects of psychomotor stimulant drugs was not found with the second-order schedules used in the present study.     

Choice between response units: The rate constancy model

In a conjoint schedule, reinforcement is available simultaneously on two or more schedules for the same response. The present experiments provided food for key pecking on both a random-interval and a differential-reinforcement-of-low-rate (DRL) schedule. Experiment 1 involved ordinary DRL schedules; Experiment 2 added an external stimulus to indicate when the required interresponse time had elapsed. In both experiments, the potential reinforcer frequency from each component was varied by means of a second-order fixed-ratio schedule, and the DRL time parameter was changed as well. Response rates were described by a model stating that time allocation to each component matches the relative frequency of reinforcement for that component. When spending time in a given component, the subject is assumed to respond at the rate characteristic of baseline performance. This model appeared preferable to the absolute-rate version of the matching law. The model was shown to be applicable to multiple-response concurrent schedules as well as to conjoint schedules, and it described some of the necessary conditions for response matching, undermatching, and bias. In addition, the pigeons did not optimize reinforcer frequency.     

Separating the reinforcing and discriminative properties of brief-stimulus presentations in second-order schedules.

Pigeons' responses were maintained under multiple schedules to study properties of briefly presented stimuli. Responses in one component produced food according to a second-order schedule with fixed-interval components in which food or a brief stimulus occurred with equal probability. In the second component responses produced only the brief stimulus under a fixed-ratio schedule. Under various conditions the brief stimulus in the first component was (a) paired with food, (b) not paired with food, (c) partially omitted, or (d) scheduled simultaneously with the second-order schedule under an independent variable-interval schedule. Paired and nonpaired brief stimuli maintained similar response patterning in the second-order schedule. However, only paired stimuli maintained responses in the second component. The data suggest that nonpaired brief stimuli engender response patterning in second-order schedules as a result of their discriminative properties. When the stimulus is paired with food, these discriminative properties sometime mask a reinforcement effect, and no change in response patterning is observed. When the discriminative properties of the brief stimulus are absent, the reinforcing effects of pairing the brief stimulus with food may be observed.     

An analysis of reinforcement history effects

Four pigeons were exposed to two tandem variable-interval differential-reinforcement-of-low-rate schedules under different stimulus conditions. The values of the tandem schedules were adjusted so that reinforcement rates in one stimulus condition were higher than those in the other, even though response rates in the two conditions were nearly identical. Following this, a fixed-interval schedule of either shorter or longer values than, or equal to the baseline schedule, was introduced in the two stimulus conditions respectively. Response rates during those fixed-interval schedules typically were higher in the presence of the stimuli previously correlated with the lower reinforcement rates than were those in the presence of the stimuli previously correlated with the higher reinforcement rates. Such effects of the reinforcement history were most prominent when the value of the fixed-interval schedule was shorter. The results are consistent with both incentive contrast and response strength conceptualizations of related effects. They also suggest methods for disentangling the effects of reinforcement rate on subsequent responding, from the response rate with which it is confounded in many conventional schedules of reinforcement.     

Facilitation of food-reinforced responding by a signal for response-independent food

Five pigeons whose key pecking was maintained by 4-sec access to grain on a variable-interval 2-min schedule received Pavlovian differential conditioning trials superimposed upon the instrumental baseline. The conditioned stimuli were changes in the stimulus on the key from white to red, or to a white horizontal line against a dark background. The positive conditioned stimulus was 20 sec long, and was followed immediately by 8-sec access to grain. The negative conditioned stimulus, also 20 sec long, was never paired with response-independent food. All pigeons responded more rapidly in the presence of the positive conditioned stimulus than in the presence of the negative one. The positive conditioned stimulus produced an increase in response rate over the pre-conditioned stimulus period. The negative conditioned stimulus had no marked effect upon response rate. When the roles of the positive and negative stimuli were reversed, and the duration of the response-independent reinforcement was reduced to 4 sec, the new positive conditioned stimulus came to facilitate responding, and the new negative conditioned stimulus no longer produced facilitation. A second discrimination reversal produced similar outcomes. When a third reversal was initiated, and the duration of response-independent reinforcement was reduced to 2 sec, the difference between the effects of the positive and negative stimuli diminished.     

The effects of signaling stimulus presentation during noncontingent reinforcement

The effects of signaling the return of items or attention during treatment with noncontingent reinforcement were examined. First, functional analyses showed that the problem behavior exhibited by 2 teenagers with developmental disabilities was sensitive to social positive reinforcement. Next, delivery of the stimulus that maintained problem behavior on a fixed-time (FT) schedule was compared to a condition in which the removal of the stimulus during the same FT schedule was immediately preceded by a statement indicating that the stimulus would be returned and the initiation of a digital timer. Results show that the FT schedule reduced problem behavior, and the addition of an informative statement and a timer further decreased problem behavior.     

Separating the effects of salience and disparity on the rate of observing

Pigeons producing deliveries of grain on a mixed variable-interval, extinction schedule by pecking a center key could also produce discriminative stimuli on concurrent variable-interval schedules by pecking the left or right observing key. The stimuli produced by each observing key were varied independently. In the first experiment, the negative discriminative stimulus was at the far end of the spectrum from the key illumination accompanying the mixed schedule and from the positive discriminative stimulus. When the magnitude of the difference between the latter two stimuli (salience) was varied, more pecks occurred on the observing key producing the larger of the two differences than on the key producing the smaller difference. In the second experiment, the stimulus accompanying the mixed schedule was at the far end of the spectrum, and the magnitude of the difference between the two discriminative stimuli (disparity) was varied. The proportion of pecks occurring on each observing key shifted systematically in the direction of the key producing the larger difference. The salience of the discriminative stimuli and their disparity each has an independent influence on the frequency of observing when the other is controlled, but the effect of the salience appears to be the more substantial.     

The role of discriminative stimuli in concurrent performances

Key pecking in pigeons was examined under concurrent and parallel arrangements of two independent and simultaneously available variable-interval schedules. Pecks on the changeover key alternated the schedule of reinforcement for responses on the main key. Under concurrent schedules, discriminative stimuli were paired with the reinforcement schedule arranged in each component and changeover responses also alternated these stimuli. Under parallel schedules, changeover responses alternated the effective reinforcement schedule, but did not change the discriminative stimulus. On concurrent procedures, changeover response rate was inversely related to the difference in reinforcement rate between the two components, whereas on parallel schedules no consistent relationship was found. With both schedules, absolute response and reinforcement rates were positively related, although for a given set of reinforcement frequencies, rates were often higher on the concurrent schedules. On concurrent schedules, relative response rates and relative times were equal to relative reinforcement rates. On parallel schedules these ratios were positively related, but response and time ratios were much smaller than were obtained with comparable concurrent schedules. This inequality was most pronounced when absolute reinforcement frequencies were lowest.     

Conditioned reinforcement versus time to reinforcement in chain schedules.

Pigeons were trained on three-component chain schedules in which the initial component was either a fixed-interval or variable-interval schedule. The middle and terminal components were varied among fixed-interval fixed-interval, variable-interval variable-interval, and an interdependent variable-interval variable-interval schedule in which the sum of the durations of the two variable-interval components was always equal to the sum of the fixed-interval fixed-interval components. At issue was whether the response rate in the initial component was controlled by its time to primary reinforcement or by the temporal parameters of the stimulus correlated with the middle terminal link. The fixed-interval initial-link schedule maintained much lower response rates than the variable-interval initial-link schedule regardless of the schedules in the middle and terminal links. Nevertheless, the intervening schedules played some role: With fixed-interval schedules in the initial links, response rates were consistently highest with independent variable-interval schedules in the middle and terminal links and intermediate with the interdependent variable-interval schedules; these initial-link differences were predicted by the response rates in the middle link of the chain. With variable-interval schedules in the initial links, response rates were lowest with the fixed-interval fixed-interval schedules following the initial link and were not systematically different for the two types of variable-interval variable-interval schedules. The results suggest that time to reinforcement itself accounts for little if any variance in initial-link responding.     

Food-paired stimuli as conditioned reinforcers: effects of d-amphetamine.

Seven pigeons were studied in two experiments in which key pecks were reinforced under a second-order schedule wherein satisfaction of variable-interval schedule requirements produced food or a brief stimulus. In the second part of each session, responses produced only the brief stimulus according to a variable-interval schedule (food extinction). For the 4 pigeons in Experiment 1, the response key was red throughout the session. In separate phases, the brief stimulus was either paired with food, not paired with food, or not presented during extinction. d-Amphetamine (0.3 to 10.0 mg/kg) dose-dependently reduced food-maintained responding during the first part of the session and, at intermediate dosages, increased responding during the extinction portion of the session. The magnitude of these increases, however, did not consistently depend on whether the brief stimulus was paired, not paired, or not presented. It was also true that under nondrug conditions, response rates during extinction did not differ reliably depending on pairing operations for the brief stimulus. In Experiment 2, 3 different pigeons responded under a procedure wherein the key was red in the component with food presentations and blue in the extinction component (i.e., multiple schedule). Again, d-amphetamine produced dose-related decreases in responding during the first part of a session and increases in responding in the second part of the session. These increases, however, were related to the pairing operations; larger increases were observed when the brief stimulus was paired with food than when it was not or when it was not presented at all. Under nondrug conditions, the paired brief stimulus controlled higher response rates during extinction than did a nonpaired stimulus or no stimulus. These findings suggest that d-amphetamine can enhance the efficacy of conditioned reinforcers, and that this effect may be more robust if conditioned reinforcers occur in the context of a signaled period of extinction.     

Signalled reinforcement and multiple schedules

The responses of four pigeons were first reinforced in the presence of two different wave-lengths (green and red) on a two-ply multiple schedule with identical variable-interval 3-min schedules of reinforcement associated with each component. While the constant-component reinforcement schedule remained unchanged during the experiment, the schedule associated with the variable component was changed to (1) signalled variable time, (2) unsignalled variable time, or (3) signalled variable interval. The probability with which the availability of the reinforcer was signalled in the variable-interval schedules was either 0.5 or 1.0. Positive contrast occurred in both signalled variable-interval and variable-time schedules, but only when the availability of all the variable-component reinforcers was signalled. Signalling the availability of only 50% of the reinforcers in signalled variable-interval schedules resulted in negative induction. The present data suggest that positive behavioral contrast resulting from signalled reinforcer availability is due to the presence of an extinction-correlated stimulus.