Conditioned reinforcement and discrimination in second-order schedules

Pigeons were exposed to multiple second-order schedules in which responding on the “main key” was reinforced according to either a variable-interval or fixed-interval schedule by production of a brief stimulus on the “brief-stimulus key”. A response was required to the brief stimulus during its fourth (final) presentation to produce food; responses to the earlier brief stimuli indicated the extent to which the final brief stimulus was discriminated from preceding ones. Main-key response rates were higher in early components of paired brief-stimulus schedules, in which each brief stimulus was the same as that paired with reinforcement, than in comparable unpaired brief-stimulus or tandem schedules. Poor discrimination occurred between paired brief stimuli (Experiment I). When chain stimuli on the main key induced a discrimination between the first two and second two brief stimuli, the response-rate enhancement in the paired brief-stimulus schedule persisted (Experiment II). Rate enhancement diminished when the initial link of the chain included the first three components (Experiment IV). Eliminating the contingency between responding and brief-stimulus production also diminished rate enhancement (Experiment III). The results show that the discriminative and conditioned reinforcing effects of food-paired brief stimuli may be selectively manipulated and suggest that the reinforcing effects are modulated by other reinforcers in the situation.     

Conditioned reinforcement in second-order schedules

Pigeons responded under a schedule in which food was presented only after a fixed number of fixed-interval components were completed. Two such second-order schedules were studied: under one, 30 consecutive 2-min fixed-interval components were required; under the other, 15 consecutive 4-min fixed-interval components were required. Under both schedules, when a 0.7-sec stimulus light was presented at completion of each fixed interval, positively accelerated responding developed in each component. When no stimulus change occurred at completion of each fixed interval, relatively low and constant rates of responding prevailed in each component; a similar result was obtained when a 0.7-sec stimulus change occurred at completion of each fixed interval except the one which terminated with primary reinforcement. The 0.7-sec stimulus correlated with food delivery was an effective conditioned reinforcer in maintaining patterns of responding in fixed-interval components despite low average frequencies of food reinforcement.     

Conditioned reinforcement versus time to reinforcement in chain schedules.

Pigeons were trained on three-component chain schedules in which the initial component was either a fixed-interval or variable-interval schedule. The middle and terminal components were varied among fixed-interval fixed-interval, variable-interval variable-interval, and an interdependent variable-interval variable-interval schedule in which the sum of the durations of the two variable-interval components was always equal to the sum of the fixed-interval fixed-interval components. At issue was whether the response rate in the initial component was controlled by its time to primary reinforcement or by the temporal parameters of the stimulus correlated with the middle terminal link. The fixed-interval initial-link schedule maintained much lower response rates than the variable-interval initial-link schedule regardless of the schedules in the middle and terminal links. Nevertheless, the intervening schedules played some role: With fixed-interval schedules in the initial links, response rates were consistently highest with independent variable-interval schedules in the middle and terminal links and intermediate with the interdependent variable-interval schedules; these initial-link differences were predicted by the response rates in the middle link of the chain. With variable-interval schedules in the initial links, response rates were lowest with the fixed-interval fixed-interval schedules following the initial link and were not systematically different for the two types of variable-interval variable-interval schedules. The results suggest that time to reinforcement itself accounts for little if any variance in initial-link responding.     

The reinforcing effects of houselight illumination during chained schedules of food presentation

Pigeons' keypecking was maintained under two- and three-component chained schedules of food presentation. The component schedules were all fixed-interval schedules of either 1- or 2-min duration. Across conditions the presence of houselight illumination within each component schedule was manipulated. For each pigeon, first-component response rates increased significantly when the houselight was extinguished in the first component and illuminated in the second. The results suggest that the increase was not the result of disinhibition or modification of stimulus control by component stimuli, but appears to result from the reinforcement of responding by the onset of illumination in the second component. Additionally, the apparent reinforcing properties of houselight illumination resulted neither from association of the houselight with the terminal component of the chained schedule nor through generalization of the hopper illumination present during food presentation. The results of the present series of experiments are related to previous demonstrations of illumination-reinforced responding and to the interpretation of data from experiments employing houselight illumination as stimuli associated with timeout or brief stimuli in second-order schedules.     

A pharmacological examination of the resistance-to-change hypothesis of response strength.

The effects of d-amphetamine sulfate, sodium pentobarbital, haloperidol, and cholecystokinin-octapeptide were examined within the context of Nevin's (1974, 1979) resistance-to-change hypothesis of response strength. In three experiments, rats' responding was reinforced by delivery of food under chained random-interval 30-s random-interval 30-s, multiple fixed-interval 30-s fixed-interval 120-s, or multiple random-interval 30-s random-interval 120-s schedules. Each rat received several doses of each drug and changes in response rate were measured. The resistance-to-change hypothesis predicts greater disruption of response rate relative to baseline in the initial component of the chained schedule and in the 120-s component of the multiple schedules. In the chained schedule cholecystokinin-octapeptide produced greater reductions in response rate relative to baseline in the initial component. However, no differences between components were observed with haloperidol or sodium pentobarbital, and high doses of d-amphetamine reduced response rate in the terminal component relatively more than in the initial component. In the multiple schedules either no differences were observed between components or response rate was reduced more relative to baseline in the 30-s component. The data fail to support the notion that drugs may be viewed within the same context as other response disruptors such as extinction, satiation, and the presentation of alternative reinforcement.     

Separating the reinforcing and discriminative properties of brief-stimulus presentations in second-order schedules.

Pigeons' responses were maintained under multiple schedules to study properties of briefly presented stimuli. Responses in one component produced food according to a second-order schedule with fixed-interval components in which food or a brief stimulus occurred with equal probability. In the second component responses produced only the brief stimulus under a fixed-ratio schedule. Under various conditions the brief stimulus in the first component was (a) paired with food, (b) not paired with food, (c) partially omitted, or (d) scheduled simultaneously with the second-order schedule under an independent variable-interval schedule. Paired and nonpaired brief stimuli maintained similar response patterning in the second-order schedule. However, only paired stimuli maintained responses in the second component. The data suggest that nonpaired brief stimuli engender response patterning in second-order schedules as a result of their discriminative properties. When the stimulus is paired with food, these discriminative properties sometime mask a reinforcement effect, and no change in response patterning is observed. When the discriminative properties of the brief stimulus are absent, the reinforcing effects of pairing the brief stimulus with food may be observed.     

Accuracy of performance on a matching-to-sample procedure under interval schedules

Correct matches on a matching-to-sample procedure were reinforced under fixed-interval, chained fixed-interval, and fixed-interval schedules with exteroceptive stimulus changes correlated with time since the last reinforcer (an added clock). For all four pigeons, accuracy changed within the fixed-interval and fixed-interval schedules with added clock, decreasing from the beginning of the interval to some point in the middle. The performance then became increasingly more accurate until the end of the interval. Under the chained schedules, accuracy also changed within the components. During the initial component, accuracy decreased from the beginning of the fixed interval to some point in the middle or at the end. During the middle component, the performance usually remained at an intermediate level of accuracy. During the terminal component, the initially inaccurate performance became increasingly more accurate throughout the interval. Systematic relationships between response rate and per cent error showed that all four pigeons performed most accurately at high rates. The accuracy of the performance at low rates was also quite high. These relationships held for all three types of schedules through an eight-fold variation in scheduled interreinforcement time.     

Second-order schedules: discrimination of components

Pigeons were exposed to a series of second-order schedules in which the completion of a fixed number of fixed-interval components produced food. In Experiment 1, brief (2 sec) stimulus presentations occurred as each fixed-interval component was completed. During the brief-stimulus presentation terminating the last fixed-interval component, a response was required on a second key, the brief-stimulus key, to produce food. Responses on the brief-stimulus key before the last brief-stimulus presentation had no scheduled consequences, but served as a measure of the extent to which the final component was discriminated from preceding components. Whether there were one, two, four, or eight fixed-interval components, responses on the brief-stimulus key occurred during virtually every brief-stimulus presentation. In Experiment 2, an attempt was made to punish unnecessary responses on the brief-stimulus key, i.e., responses on the brief-stimulus key that occurred before the last component. None of the pigeons learned to withhold these responses, even though they produced a 15-sec timeout and loss of primary reinforcement. In Experiment 3, different key colors were associated with each component of a second-order schedule (a chain schedule). In contrast to Experiment 1, brief-stimulus key responses were confined to the last component. It was concluded that pigeons do not discriminate well between components of second-order schedules unless a unique exteroceptive cue is provided for each component. The relative discriminability of the components may account for the observed differences in initial-component response rates between comparable brief-stimulus, tandem, and chain schedules.     

Concurrent second-order schedules of reinforcement

Responses on one key (the main key) of a two-key chamber produced food according to a second-order variable-interval schedule with fixed-interval schedule components. A response on a second key (the changeover key) alternated colors on the main key and provided a second independent second-order variable-interval schedule with fixed-interval components. The fixed-interval component on one variable-interval schedule was held constant at 8 sec, while the fixed interval on the other variable-interval schedule was varied from 0 to 32 sec. Under some conditions, a brief stimulus terminated each fixed interval and generated fixed-interval patterns; in other conditions, the brief stimulus was omitted. Relative response rate and relative time deviated substantially from scheduled relative reinforcement rate and, to a lesser extent, from obtained relative reinforcement rate under both brief-stimulus and no-stimulus conditions. Matching was observed with equal components on both schedules; with unequal components, increasingly greater proportions of time and responses than the matching relation would predict were spent on the variable-interval schedule containing the shorter component. Preference for the shorter fixed interval was typically more extreme under brief-stimulus than under no-stimulus schedules. The results limit the extension of the matching relation typically observed under simple concurrent variable-interval schedules to concurrent second-order variable-interval schedules.     

Second-order schedules: a comparison of chained, brief-stimulus, and tandem procedures

Pigeons were exposed to seven types of two-component schedules, each component a 2-min fixed-interval schedule. Food presentation occurred at the completion of the second component under all conditions. The seven types of schedules were: (1) a chained schedule in which completion of the first component produced the discriminative stimulus associated with the second component; (2) a chained schedule to which was added the brief presentation of a food-paired stimulus at the completion of the first component; (3) a chained schedule to which was added the brief presentation of a stimulus not paired with food at the completion of the first component; (4) a multiple schedule in which food presentation occurred at the completion of both components; (5) a tandem schedule in which completion of the first component initiated the second component, with no changes in exteroceptive stimuli; (6) a food-paired brief-stimulus schedule in which the brief presentation of a food-paired stimulus was made at the completion of the first component and no other changes in stimuli occurred; and (7) a brief-stimulus schedule in which the brief presentation of a stimulus not paired with food was made at the completion of the first component and no other changes in stimuli occurred. Positively accelerated patterns of responding developed in the first component under three conditions: (1) the chained schedule with the added food-paired brief stimulus; (2) the multiple schedule; and (3) the food-paired brief-stimulus schedule. Response rates were low in the first component, with few instances of positively accelerated patterns, under two conditions: (1) the chained schedule; and (2) the chained schedule with the added nonpaired brief stimulus. The results suggest that a briefly presented food-paired stimulus may function as a more effective conditioned reinforcer than does the presentation of a discriminative stimulus.     

An analysis of reinforcement history effects

Four pigeons were exposed to two tandem variable-interval differential-reinforcement-of-low-rate schedules under different stimulus conditions. The values of the tandem schedules were adjusted so that reinforcement rates in one stimulus condition were higher than those in the other, even though response rates in the two conditions were nearly identical. Following this, a fixed-interval schedule of either shorter or longer values than, or equal to the baseline schedule, was introduced in the two stimulus conditions respectively. Response rates during those fixed-interval schedules typically were higher in the presence of the stimuli previously correlated with the lower reinforcement rates than were those in the presence of the stimuli previously correlated with the higher reinforcement rates. Such effects of the reinforcement history were most prominent when the value of the fixed-interval schedule was shorter. The results are consistent with both incentive contrast and response strength conceptualizations of related effects. They also suggest methods for disentangling the effects of reinforcement rate on subsequent responding, from the response rate with which it is confounded in many conventional schedules of reinforcement.     

Preference for less segmented fixed-time components in concurrent-chain schedules of reinforcement

A concurrent-chain procedure was used to examine choice between segmented and less segmented response-independent schedules of reinforcement. A pair of independent, concurrent variable-interval 60-s schedules were presented in the initial link, along with a 1.5-s changeover delay. A chained fixed-interval fixed-time and its corresponding tandem schedule constituted the terminal links. The length of the fixed-interval schedule in the terminal link was varied between 5 s and 30 s while that of the fixed-time schedule was kept at 5 s over conditions. The first components of both terminal-link schedules were accompanied by the same stimulus. Except in the baseline condition, the onset of the second component of the terminal-link chained schedule was accompanied by either a localized (key color) or a nonlocalized (dark houselight) stimulus change. Stimulus conditions were constant during the terminal-link tandem schedule. With three exceptions, pigeons demonstrated a slight preference for the tandem over the chained schedule in the terminal link. Furthermore, this preference varied inversely with the length of the first component. In general, these results are consistent with previous studies that reported an adverse effect on choice by segmenting an interval schedule into two or more components, but they are inconsistent with studies that reported preference for signaled over unsignaled delay of reinforcement.     

Sensory superstition on multiple interval schedules.

Pigeons were exposed to multiple schedules in which an irregular repeating sequence of five stimulus components was correlated with the same reinforcement schedule throughout. Stable, idiosyncratic, response-rate differences developed across components. Components were rank-ordered by response rate; an approximately linear relation was found between rank order and the deviation of mean response rate from the overall mean rate. Nonzero slopes of this line were found for multiple fixed-interval and variable-time schedules and for multiple variable-interval schedules both when number of reinforcements was the same in all components and when it varied. The steepest function slopes were found in the variable schedules with relatively long interfood intervals and relatively short component durations. When just one stimulus was correlated with all components of a multiple variable-interval schedule, the slope of the line was close to zero. The results suggest that food-rate differences may be induced initially by different reactions to the stimuli and subsequently maintained by food.     

Chained and tandem fixed-interval schedules of punishment

Pigeons' key pecks were both punished with electric shock on four-component chained and tandem fixed-interval schedules and reinforced on a variable-interval schedule of food presentation. Pecking was suppressed less in the early components of the chained schedule than in the early components of the tandem schedule. Related multiple and mixed schedules of punishment were also presented; these schedules were identical to the chained and tandem schedules, respectively, except that components changed independently of responding. Similar effects were obtained, in that responding was suppressed in all components of the mixed schedule and only in the fourth component of the multiple schedule of punishment. The performances maintained on the chained and tandem schedules of punishment were generally symmetrical to those found in analogous chained and tandem schedules of food reinforcement.     

The delay-reduction hypothesis of conditioned reinforcement and punishment: Observing behavior

Pigeons responded in an observing-response procedure in which three fixed-interval components alternated. Pecking one response key produced food reinforcement according to a mixed schedule. Pecking the second (observing) key occasionally replaced the mixed-schedule stimulus with the stimulus correlated with the fixed-interval component then in effect. In Experiment 1, observing was best maintained by stimuli correlated with a reduction in mean time to reinforcement. That finding was consistent with the conditioned-reinforcement hypothesis of observing behavior. However, low rates of observing were also maintained by stimuli not representing delay reduction. Experiment 2 assessed the role of sensory reinforcement. It showed that response rate was higher when maintained by stimuli uncorrelated with reinforcement delay than when the stimuli were correlated with a delay increase. This latter result supports a symmetrical version of the conditioned-reinforcement hypothesis that requires suppression by stimuli correlated with an increase in time to reinforcement. The results were inconsistent with hypotheses stressing the reinforcing potency of uncertainty reduction.     

Stimulus control of behavioral history.

Pigeons were exposed to two different reinforcement schedules under different stimulus conditions in each of two daily sessions separated by 6 hr (Experiments 1 and 2) or in a single session (Experiment 3). Following this, either a fixed-interval (Experiment 1) or a variable-interval schedule (Experiments 2 and 3) was effected in both stimulus conditions. In the first two experiments, exposure to fixed-ratio or differential-reinforcement-of-low-rate schedules led to response-rate, but not pattern, differences in subsequent performance on fixed- or variable-interval schedules that persisted for up to 60 sessions. The effects of reinforcement-schedule history on fixed-interval schedule performance generally were more persistent. In Experiment 3, a history of high and low response rates in different components of a multiple schedule resulted in subsequent response-rate differences under identical variable-interval schedules. Higher response rates initially occurred in the component previously correlated with high response rates. For 3 of 4 subjects, the differences persisted for 20 or more sessions. Previous demonstrations of behavioral history effects have been confined largely to between-subject comparisons. By contrast, the present results demonstrate strong behavioral effects of schedule histories under stimulus control within individual subjects.     

Factors influencing responding under multiple schedules of conditioned and unconditioned reinforcement

Two experiments examined pigeons' responses under multiple schedules of conditioned and unconditioned reinforcement. In one component, responses produced food according to a fixed-interval schedule; in a second component, responses produced brief stimuli according to a fixed-ratio schedule. When brief-stimulus presentations were paired with food in the first component, rates in the second component were usually higher than 10 responses per minute. When pairing in the first component was eliminated, responding continued to be maintained in the second component. Elimination of food presentation from the first component substantially decreased responding in the second component, even though the brief stimulus had not been paired with food. Experiment II demonstrated that response rate was affected by the duration of both the second component and the brief stimulus. The results suggest that three conditions are important in maintaining responding with brief-stimulus presentations: (1) pairing the brief stimulus, at least initially, with food, (2) maintaining unconditioned reinforcement in one component, and (3) employing optimal brief-stimulus and component durations.     

Fixed-ratio and variable-ratio schedules of brief stimuli in second-order schedules of matching to sample

Eight pigeons matched to sample under second-order schedules of food reinforcement. Under fixed-interval unit schedules, the first correct match to occur after a given period of time was followed by the presentation of a brief stimulus. The termination of the last fixed-interval unit schedule was followed by food according to second-order fixed-ratio and variable-ratio schedules. In Experiment 1, as the number of fixed-interval unit schedules increased, long pauses occurred under the second-order fixed-ratio schedules, but not under the variable-ratio schedules. The similarity of performance measures such as local rate and accuracy indicated that the differences engendered by these two types of schedule are in the duration of the periods of not-responding. In Experiment 2, the addition of a brief stimulus at the end of each unit schedule in chained schedules that had different discriminative stimuli present for the duration of each unit did not substantially affect the performance, and long pauses continued to occur. However, few long pauses occurred under schedules with brief stimulus presentations alone. The most inaccurate performances were engendered by chained schedules without brief stimuli.     

Uncertainty reduction, conditioned reinforcement, and observing

In a concurrent-chains procedure, pigeons chose between equivalent mixed and multiple fixed-interval schedules of reinforcement. In the first experiment, preference for the multiple schedule was higher when the probability of the shorter fixed interval was less than .50 than for complementary points, an outcome consistent with the delay-reduction hypothesis of conditioned reinforcement and observing, but inconsistent with the uncertainty-reduction hypothesis which requires symmetrical preferences with a maximum when the two intervals are equiprobable. A second experiment assessed preference for equivalent mixed and multiple schedules when each choice outcome resulted in two reinforcements, one on the longer and one on the shorter fixed interval. The order of the two fixed intervals was determined probabilistically. Pigeons again preferred multiple to mixed schedules, although multiple-schedule preference did not vary systematically with the likelihood of the shorter fixed interval occurring first. The results from these choice procedures are consistent with those from the observing-response literature in suggesting that the strength of a stimulus cannot be well described as a function of the degree of uncertainty reduction the stimulus provides about reinforcement.     

Elicited responding in chain schedules.

An omission procedure was employed to study elicited pecking in the first component of a two-component chain schedule. Both components were fixed-interval schedules correlated with colored keylights. The first response following the initial-link schedule produced a second fixed-interval schedule. We studied several fixed-interval lengths in two conditions: a standard response-dependent condition and an omission-contingent condition. The omission-contingent condition differed from the response-dependent condition in that responses during the initial fixed interval terminated the trial (omitting the terminal component and grain). If the terminal component was not omitted, a response following the terminal link's requirement produced 4-s access to grain. Pigeons responded during more than 70% of the initial links in the omission-contingent condition and responded during more than 90% of the initial links in the response-dependent condition. In general, rates of responding were consistent with the percentage data. The responding in the omission condition suggests that there may be elicited pecking, in chain schedules using pigeons, that is not the result of contingent conditioned reinforcement.