Multiple baseline investigation of stimulus functions in an FR chained schedule,

The performance of pigeons was studied on a multiple schedule in which a three-member FR chained schedule alternated with a three-member FR tandem schedule. The chain and tandem schedules contained identical response requirements. In the chained schedule, more pausing and lower response rates occurred in the first and second components than occurred in the tandem control, in which the same exteroceptive stimulus was associated with all components. Because the reinforcement and response contingencies were identical in the chain and tandem schedules, differences in performances can be attributed to stimulus control.     

Second-order schedules: a comparison of chained, brief-stimulus, and tandem procedures

Pigeons were exposed to seven types of two-component schedules, each component a 2-min fixed-interval schedule. Food presentation occurred at the completion of the second component under all conditions. The seven types of schedules were: (1) a chained schedule in which completion of the first component produced the discriminative stimulus associated with the second component; (2) a chained schedule to which was added the brief presentation of a food-paired stimulus at the completion of the first component; (3) a chained schedule to which was added the brief presentation of a stimulus not paired with food at the completion of the first component; (4) a multiple schedule in which food presentation occurred at the completion of both components; (5) a tandem schedule in which completion of the first component initiated the second component, with no changes in exteroceptive stimuli; (6) a food-paired brief-stimulus schedule in which the brief presentation of a food-paired stimulus was made at the completion of the first component and no other changes in stimuli occurred; and (7) a brief-stimulus schedule in which the brief presentation of a stimulus not paired with food was made at the completion of the first component and no other changes in stimuli occurred. Positively accelerated patterns of responding developed in the first component under three conditions: (1) the chained schedule with the added food-paired brief stimulus; (2) the multiple schedule; and (3) the food-paired brief-stimulus schedule. Response rates were low in the first component, with few instances of positively accelerated patterns, under two conditions: (1) the chained schedule; and (2) the chained schedule with the added nonpaired brief stimulus. The results suggest that a briefly presented food-paired stimulus may function as a more effective conditioned reinforcer than does the presentation of a discriminative stimulus.     

Stimulus functions in chained fixed-interval schedules

Pigeons were required to complete three successive fixed-interval components to obtain food. When the same exteroceptive stimulus was correlated with the three components, responding was positively accelerated between food deliveries. When different exteroceptive stimuli were correlated with each component in a fixed sequence, prolonged pauses developed in the first component; low response rates developed in the second component; and responding was positively accelerated in the second and third components. When different exteroceptive stimuli were correlated with each component in a variable sequence, responding was positively accelerated in each component. Because the response and reinforcement contingencies were the same in all three procedures, the differences in performances must be due to the changes in the sequence of stimuli.     

Responding under chained and tandem fixed-ratio schedules

The role of stimuli in chained fixed-ratio schedules of reinforcement was examined. At various ratio values, responding on schedules consisting of three or five equal components, with a different colored light in each component (“block counter”) was compared with responding on tandem or simple fixed-ratio schedules having the same color present throughout the entire ratio. At all ratio values except the smallest, the chain stimuli resulted in longer pauses after reinforcement. The magnitude of this effect became greater as the size of the ratio was increased. Post-reinforcement pause durations were longer under five-component schedules than under three-component schedules. Running rates in the first component were lower on the chained schedules than on the tandem schedules; on both kinds of schedule, rates were lower in the first component than in the rest of the ratio. When the sequence of stimuli was reversed, the duration of the post-reinforcement pause dropped markedly and the running rate in the initial component increased, but these effects gradually disappeared after the first reversal session. When the final chain stimulus was substituted for the first component stimulus but continued to appear in the final chain component as well, the pause duration dropped and remained at this lower level during subsequent sessions.     

Extinction of a heterogeneous chain after several reinforcement schedules

Rats were conditioned to emit the following two-member chain of responses on two different operanda always available: responses on a vertical bar produced a discriminative stimulus for food-reinforced responding on a horizontal bar. Responses on the vertical bar produced a discriminative stimulus on a variable-interval 1-min schedule, and the horizontal bar produced food on a variable ratio of 10 responses. Control conditions were included in which vertical bar responses were also food-reinforced simultaneous with the onset of the discriminative stimulus for the horizontal bar response and a tandem schedule which had the same response requirements but without different exteroceptive stimuli associated with the separate components of the response chain. The latter condition greatly retarded acquisition of the response chain compared to the other schedules studied here and compared to reports in the literature on homogeneous (single operandum) response chains. Intermittent reinforcement of the chain led to greater resistance to extinction of both members and the chain remained intact longer in the sense that stimulus control was maintained.     

Conditioned reinforcement and discrimination in second-order schedules

Pigeons were exposed to multiple second-order schedules in which responding on the “main key” was reinforced according to either a variable-interval or fixed-interval schedule by production of a brief stimulus on the “brief-stimulus key”. A response was required to the brief stimulus during its fourth (final) presentation to produce food; responses to the earlier brief stimuli indicated the extent to which the final brief stimulus was discriminated from preceding ones. Main-key response rates were higher in early components of paired brief-stimulus schedules, in which each brief stimulus was the same as that paired with reinforcement, than in comparable unpaired brief-stimulus or tandem schedules. Poor discrimination occurred between paired brief stimuli (Experiment I). When chain stimuli on the main key induced a discrimination between the first two and second two brief stimuli, the response-rate enhancement in the paired brief-stimulus schedule persisted (Experiment II). Rate enhancement diminished when the initial link of the chain included the first three components (Experiment IV). Eliminating the contingency between responding and brief-stimulus production also diminished rate enhancement (Experiment III). The results show that the discriminative and conditioned reinforcing effects of food-paired brief stimuli may be selectively manipulated and suggest that the reinforcing effects are modulated by other reinforcers in the situation.     

An analysis of reinforcement history effects

Four pigeons were exposed to two tandem variable-interval differential-reinforcement-of-low-rate schedules under different stimulus conditions. The values of the tandem schedules were adjusted so that reinforcement rates in one stimulus condition were higher than those in the other, even though response rates in the two conditions were nearly identical. Following this, a fixed-interval schedule of either shorter or longer values than, or equal to the baseline schedule, was introduced in the two stimulus conditions respectively. Response rates during those fixed-interval schedules typically were higher in the presence of the stimuli previously correlated with the lower reinforcement rates than were those in the presence of the stimuli previously correlated with the higher reinforcement rates. Such effects of the reinforcement history were most prominent when the value of the fixed-interval schedule was shorter. The results are consistent with both incentive contrast and response strength conceptualizations of related effects. They also suggest methods for disentangling the effects of reinforcement rate on subsequent responding, from the response rate with which it is confounded in many conventional schedules of reinforcement.     

Second-order schedules: discrimination of components

Pigeons were exposed to a series of second-order schedules in which the completion of a fixed number of fixed-interval components produced food. In Experiment 1, brief (2 sec) stimulus presentations occurred as each fixed-interval component was completed. During the brief-stimulus presentation terminating the last fixed-interval component, a response was required on a second key, the brief-stimulus key, to produce food. Responses on the brief-stimulus key before the last brief-stimulus presentation had no scheduled consequences, but served as a measure of the extent to which the final component was discriminated from preceding components. Whether there were one, two, four, or eight fixed-interval components, responses on the brief-stimulus key occurred during virtually every brief-stimulus presentation. In Experiment 2, an attempt was made to punish unnecessary responses on the brief-stimulus key, i.e., responses on the brief-stimulus key that occurred before the last component. None of the pigeons learned to withhold these responses, even though they produced a 15-sec timeout and loss of primary reinforcement. In Experiment 3, different key colors were associated with each component of a second-order schedule (a chain schedule). In contrast to Experiment 1, brief-stimulus key responses were confined to the last component. It was concluded that pigeons do not discriminate well between components of second-order schedules unless a unique exteroceptive cue is provided for each component. The relative discriminability of the components may account for the observed differences in initial-component response rates between comparable brief-stimulus, tandem, and chain schedules.     

Effects on concurrent performances of a stimulus correlated with reinforcer availability

A multiple schedule was arranged in which each component consisted of two, concurrent variable-interval schedules of reinforcement. A changeover-key procedure was used, and the components of the multiple schedule were distinguished (initially) by the color of the changeover key. During one component of the multiple schedule, the availability of a reinforcer arranged by one of the variable-interval schedules was marked by an exteroceptive stimulus, provided that that variable-interval schedule was not at the time assigned to the main key. During the other component of the multiple schedule, no reinforcer-correlated stimuli were ever presented. During the latter component of the multiple schedule, the distribution of responses and time for the concurrent variable-interval schedules suggested control by the distribution of reinforcements. During the former component, most main-key responses were emitted on the key in the presence of which reinforcer-correlated stimuli were presented. Changeover rate in the presence of that key color was depressed. The discriminative control over the changeover was easily established and was reversible.     

Observing behavior in squirrel monkeys under a multiple schedule of reinforcement availability

Observing behavior of two squirrel monkeys was examined under a multiple schedule of four components. Lever (observing) responses produced either a stimulus indicating the availability of food or another stimulus indicating food was not available. Key responses in the presence of the food-available stimulus produced food on a continuous reinforcement schedule. In the absence of food-available stimuli, responding on the key had no scheduled consequences. Observing responses produced food-available stimuli according to three different random-interval schedules with mean interstimulus availability times of 1, 2, and 4 min. In the fourth component of the multiple schedule (observing extinction) food-available stimuli never occurred. Each component of the schedule was correlated with a distinctive auditory stimulus. Observing rates decreased with decreasing frequency of the food-available stimulus. Observing rates during extinction continued decreasing when the brief stimulus indicating food unavailability was no longer produced by lever pressing. When the brief stimulus was reinstated response rates increased abruptly.     

Second-order schedules: brief shock at the completion of each component

Pigeons worked on second-order schedules in which completion of fixed-interval component schedules was reinforced with food according to a variable-interval schedule of reinforcement. The completion of each fixed-interval component resulted in the presentation of a brief electric shock. In one condition (shock-paired), the completion of every fixed-interval component, including those that ended in food, resulted in the shock. In another condition (shock-nonpaired), completion resulted in shock except for those components that ended in food. Shock presentations resulted in a positively accelerated rate within fixed-interval components. This patterning within components was similar whether the shock was intermittently paired with food or not. Response rates tended to decrease as shock intensity increased. The characteristic fixed-interval response pattern within components did not occur when shock presentations were omitted at the end of each component (tandem schedule). When shocks were scheduled but food was no longer presented (extinction) response rates declined to a near-zero level. The performance under shock conditions is similar to that in other studies in which visual and auditory stimuli are presented at the completion of component schedules.     

RESISTANCE TO CHANGE AND FREQUENCY OF RESPONSE‐DEPENDENT STIMULI UNCORRELATED WITH REINFORCEMENT

Stimuli uncorrelated with reinforcement have been shown to enhance response rates and resistance to disruption; however, the effects of different rates of stimulus presentations have not been assessed. In two experiments, we assessed the effects of adding different rates of response‐dependent brief stimuli uncorrelated with primary reinforcement on relative response rates and resistance to change. In both experiments, pigeons responded on variable‐interval 60‐s schedules of food reinforcement in two components of a multiple schedule, and brief response‐dependent keylight‐color changes were added to one or both components. Although relative response rates were not systematically affected in either experiment, relative resistance to presession feeding and extinction were. In Experiment 1, adding stimuli on a variable‐interval schedule to one component of a multiple schedule either at a low rate (1 per min) for one group or at a high rate (4 per min) for another group similarly increased resistance to disruption in the components with added stimuli. When high and low rates of stimuli were presented across components (i.e., within subjects) in Experiment 2, however, relative resistance to disruption was greater in the component presenting stimuli at a lower rate. These results suggest that stimuli uncorrelated with food reinforcement do not strengthen responding in the same way as primary reinforcers.     

The role of discriminative stimuli in concurrent performances

Key pecking in pigeons was examined under concurrent and parallel arrangements of two independent and simultaneously available variable-interval schedules. Pecks on the changeover key alternated the schedule of reinforcement for responses on the main key. Under concurrent schedules, discriminative stimuli were paired with the reinforcement schedule arranged in each component and changeover responses also alternated these stimuli. Under parallel schedules, changeover responses alternated the effective reinforcement schedule, but did not change the discriminative stimulus. On concurrent procedures, changeover response rate was inversely related to the difference in reinforcement rate between the two components, whereas on parallel schedules no consistent relationship was found. With both schedules, absolute response and reinforcement rates were positively related, although for a given set of reinforcement frequencies, rates were often higher on the concurrent schedules. On concurrent schedules, relative response rates and relative times were equal to relative reinforcement rates. On parallel schedules these ratios were positively related, but response and time ratios were much smaller than were obtained with comparable concurrent schedules. This inequality was most pronounced when absolute reinforcement frequencies were lowest.     

Separating the reinforcing and discriminative properties of brief-stimulus presentations in second-order schedules.

Pigeons' responses were maintained under multiple schedules to study properties of briefly presented stimuli. Responses in one component produced food according to a second-order schedule with fixed-interval components in which food or a brief stimulus occurred with equal probability. In the second component responses produced only the brief stimulus under a fixed-ratio schedule. Under various conditions the brief stimulus in the first component was (a) paired with food, (b) not paired with food, (c) partially omitted, or (d) scheduled simultaneously with the second-order schedule under an independent variable-interval schedule. Paired and nonpaired brief stimuli maintained similar response patterning in the second-order schedule. However, only paired stimuli maintained responses in the second component. The data suggest that nonpaired brief stimuli engender response patterning in second-order schedules as a result of their discriminative properties. When the stimulus is paired with food, these discriminative properties sometime mask a reinforcement effect, and no change in response patterning is observed. When the discriminative properties of the brief stimulus are absent, the reinforcing effects of pairing the brief stimulus with food may be observed.     

Second-order schedules and the problem of conditioned reinforcement

Thirteen pigeons were exposed to a variety of second-order schedules in which responding under a component schedule was reinforced according to a schedule of reinforcement. Under different conditions, completion of each component resulted in either (1) the brief presentation of a stimulus also present during reinforcement (pairing operation), (2) the brief presentation of a stimulus not present during reinforcement (nonpairing operation), or (3) no brief stimulus presentation (tandem). Brief-stimulus presentations engendered a pattern of responding within components similar to that engendered by food. Patterning was observed when fixed-interval and fixed-ratio components were maintained under fixed- and variable-ratio and fixed- and variable-interval schedules. There were no apparent differences in performance under pairing and nonpairing conditions in any study. The properties of the stimuli presented in brief-stimulus operations produced different effects on response patterning. In one study, similar effects on performance were found whether brief-stimulus presentations were response-produced or delivered independently of responding. Response patterning did not occur when the component schedule under which a nonpaired stimulus was produced occurred independently of the food schedule. The results suggest a reevaluation of the role of conditioned reinforcement in second-order schedule performance. The similarity of behavior under pairing and nonpairing operations is consistent with two hypotheses: (1) the major effect is due to the discriminative properties of the brief stimulus; (2) the scheduling operation under which the paired or nonpaired stimulus is presented can establish it as a reinforcer.     

Effects of chlorpromazine on fixed-ratio responding: modification by fixed-interval discriminative stimuli.

Effects of chlorpromazine (1 to 100 mg/kg) were assessed on two pigeons' responding under various modifications of a multiple schedule of food delivery. During a fixed-interval component, the first response after 5 min produced food; during the subsequent, fixed-ratio component, the 30th response produced food. Modifications of the schedule entailed changes in stimulus conditions imposed during the fixed-ratio component that did not systematically alter characteristics of performance under non-drug conditions. In the first phase of the experiment, distinctive visual stimuli were correlated with each schedule component (conventional multiple schedule); chlorpromazine produced small decreases in fixed-ratio responding (20% at 30 mg/kg). When each response during the fixed-ratio component produced the stimulus correlated with the fixed-interval schedule (fixed-interval discriminative stimulus) for 1.2 s, effects of chlorpromazine were not different from those under the conventional multiple schedule. Chlorpromazine produced greater decreases in fixed-ratio responding (55% at 30 mg/kg) when either the first response of each fixed ratio changed the stimulus correlated with the fixed-ratio schedule to the fixed-interval discriminative stimulus for the remainder of the fixed-ratio component, or when the fixed-interval discriminative stimulus was presented independently of responding according to a matched temporal sequence. When the fixed-interval discriminative stimulus was present continuously during the fixed-ratio component (mixed schedule), chlorpromazine produced even more substantial decreases in fixed-ratio responding (greater than 80% at 30 mg/kg). Effects of chlorpromazine on fixed-interval responding were also modified by the schedules of fixed-interval discriminative stimulus presentation. The effects of chlorpromazine were a joint function of the stimuli prevailing during the multiple schedule and the degree to which responding influenced these stimuli.     

Marking effects in instrumental performance on DRH schedules

Three experiments investigated the effect of presenting a brief stimulus after a response sequence on the rate of lever-pressing by rats on differential reinforcement of high rate (DRH) schedules. In Experiment 1 enhanced responding was produced by a visual stimulus presented during a 500-msec delay of reinforcement compared to a condition in which no stimulus was presented. In Experiment 2 rats responded on a multiple DRH DRH schedule in which the DRH contingency was reinforced on a 50% schedule in each component. Equivalent levels of responding occurred in the components when reinforcement was signalled in one component and when the signal was presented following the non-reinforced schedules in the other components. A further group of rats received the stimulus presented after non-reinforced schedules in one component but not at all in the other component; responding was enhanced in the former component relative to the latter component. In Experiment 3 brief stimuli presented after the completion of DRH components on a second-order VR (DRH) schedule elevated response rates irrespective of whether the signal was presented paired or unpaired with reinforcement. The present data support the view that a brief signal may serve to mark a response sequence in memory and facilitate instrumental performance.     

Brief-stimulus presentations on multiform tandem schedules

Three experiments examined the influence of a brief stimulus (a light) on the behavior of food-deprived rats whose lever pressing on tandem schedules comprising components of different schedule types resulted in food presentation. In Experiment 1, either a tandem variable-ratio variable-interval or a tandem variable-interval variable-ratio schedule was used. The variable-interval requirement in the tandem variable-ratio variable-interval schedule was yoked to the time taken to complete the variable-ratio component in the tandem variable-interval variable-ratio schedule, and the length of the variable-interval component in the latter schedule was yoked to the variable-ratio component in the former schedule. If a brief stimulus occurred following completion of the first component, then behavior was differentiated in the two components; subjects responded more quickly in the variable-ratio than in the variable-interval component. If the stimulus was removed, then response rate was determined by the nature of the final component. Similar results were obtained in Experiments 2 and 3 with the use of a three-component tandem variable-ratio variable-interval variable-ratio schedule or tandem variable-interval variable-ratio variable-interval schedule. Thus, a brief stimulus that was not explicitly paired with reinforcement engendered behavior typical of the component schedule preceding its presentation.     

Behavioral momentum: the effects of the temporal separation of rates of reinforcement.

In Part 1 of the experiment, rats responded under a variable-interval (VI) 30-s schedule and a VI 120-s schedule, with each in effect for a block of consecutive sessions. That is, the two VI schedules were presented in successive conditions. In Part 2 the VI schedules alternated each day, and in Part 3 the schedules alternated within the session as a multiple schedule. For half of the rats in Parts 1 and 2, the VI schedule alternated every few minutes within the session with a stimulus that signaled extinction. For each part, once response rates had stabilized, resistance to change was measured by prefeeding and extinction. When the schedules were examined in successive conditions (Part 1), resistance to extinction was greater under the VI 120-s schedule of reinforcement than under the VI 30-s schedule, but no consistent differences in resistance to prefeeding were observed between the two VI schedules. When the VI schedules alternated each day (Part 2), resistance to extinction was greater under the VI 120-s schedule. However, no consistent differences in resistance to prefeeding were observed between the VI schedules without extinction in Group A, but resistance to prefeeding was greater under the VI 30-s schedule for rats with the added extinction component in Group B. When the VI schedules alternated within the session as a multiple schedule (Part 3), resistance to extinction and resistance to prefeeding were greater under the VI 30-s schedule. The data suggest that different rates of reinforcement, and their accompanying discriminative stimuli, must be compared within the same session (or at least on alternate days) to produce data consistent with the behavioral momentum model.     

Stimulus control of behavioral history.

Pigeons were exposed to two different reinforcement schedules under different stimulus conditions in each of two daily sessions separated by 6 hr (Experiments 1 and 2) or in a single session (Experiment 3). Following this, either a fixed-interval (Experiment 1) or a variable-interval schedule (Experiments 2 and 3) was effected in both stimulus conditions. In the first two experiments, exposure to fixed-ratio or differential-reinforcement-of-low-rate schedules led to response-rate, but not pattern, differences in subsequent performance on fixed- or variable-interval schedules that persisted for up to 60 sessions. The effects of reinforcement-schedule history on fixed-interval schedule performance generally were more persistent. In Experiment 3, a history of high and low response rates in different components of a multiple schedule resulted in subsequent response-rate differences under identical variable-interval schedules. Higher response rates initially occurred in the component previously correlated with high response rates. For 3 of 4 subjects, the differences persisted for 20 or more sessions. Previous demonstrations of behavioral history effects have been confined largely to between-subject comparisons. By contrast, the present results demonstrate strong behavioral effects of schedule histories under stimulus control within individual subjects.