The effects of brief-stimulus presentations in fixed-ratio second-order schedules

Pigeons' responses were reinforced according to a three-component multiple schedule. In Component 1, key pecks produced food according to a fixed-ratio second-order schedule with fixed-ratio units. Here, a fixed number of fixed-ratio units produced food, and the brief stimulus terminating each unit also accompanied food. Responses in Component 2 produced food on an identical schedule except that the brief stimulus was not paired with food. Component 3 contained a simple fixed-ratio schedule whose response requirement equaled that of Components 1 and 2. Across conditions the size of the fixed-ratio unit (five, ten, twenty, forty, and eighty responses) and the total number of responses per reinforcement were parametrically manipulated. The highest response rates and shortest preratio pauses were observed in Component 3 (no brief stimulus). The lowest rates and longest pauses were found in the component with paired brief-stimulus presentations, indicating that the food-paired brief stimulus suppressed responding. The suppressive effects were greatest when the fixed-ratio units were small (e.g., fixed-ratio 5) and the total fixed-ratio requirement was large (e.g., fixed-ratio 160). Under no conditions did the paired brief stimulus facilitate responding. The nonpaired brief stimulus also suppressed responding but to a lesser extent. The suppressive effects of nonpaired brief stimuli were greatest when the fixed-ratio units were small and the total response requirement was large. These data suggest that the suppressive effects of the brief stimuli may have masked the conditioned-reinforcing effects reported in other studies, and that conditions that maximize suppression in second-order schedules involve the use of fixed-ratio schedule units and the presentation of many brief stimuli per reinforcer.     

Second-order schedules with fixed-ratio components: variation of component size

Key pecking by pigeons was reinforced with food under second-order schedules with fixed-ratio units. A constant total number of key pecks was required for reinforcement under each condition, but the size and, inversely, number of fixed-ratio components were varied. The total response requirement of 256 pecks was divided into fixed-ratio units of 128, 64, 32, 8, and 2 responses. A brief stimulus, which always preceded food reinforcement, was presented upon completion of each fixed-ratio unit. Under most conditions, the pattern of within-unit responding was typical of that under simple fixed-ratio schedules. Overall response rate was an inverted U-shaped function of component size. That is, response rates were highest under moderate sized units (fixed ratio 128 and 64). This relationship is consistent with previous determinations of rate as a function of fixed-ratio value for simple fixed-ratio schedules.     

Factors influencing responding under multiple schedules of conditioned and unconditioned reinforcement

Two experiments examined pigeons' responses under multiple schedules of conditioned and unconditioned reinforcement. In one component, responses produced food according to a fixed-interval schedule; in a second component, responses produced brief stimuli according to a fixed-ratio schedule. When brief-stimulus presentations were paired with food in the first component, rates in the second component were usually higher than 10 responses per minute. When pairing in the first component was eliminated, responding continued to be maintained in the second component. Elimination of food presentation from the first component substantially decreased responding in the second component, even though the brief stimulus had not been paired with food. Experiment II demonstrated that response rate was affected by the duration of both the second component and the brief stimulus. The results suggest that three conditions are important in maintaining responding with brief-stimulus presentations: (1) pairing the brief stimulus, at least initially, with food, (2) maintaining unconditioned reinforcement in one component, and (3) employing optimal brief-stimulus and component durations.     

Some factors controlling preference between fixed-ratio and variable-ratio schedules of reinforcement

A multiple schedule of food reinforcement for key-pecking was arranged which consisted of nine fixed-ratios, each of which operated in the presence of a different stimulus. Pigeons could complete a given fixed-ratio within the multiple schedule or, by pecking a second key, could switch from the fixed-ratio schedule to a variable-ratio schedule consisting of the same nine ratios. Stable switching behavior was established which did not maximize simple probability or rate of reinforcement. Instead, the subjects showed a stable preference for the variable-ratio schedule of food reinforcement. Increasing the number of responses required to switch, and removing the occasions on which reinforcement was delivered after a single response in the variable schedule, decreased the number of switches to the variable schedule. Periods of delay interposed between a completed switch and the availability of reinforcement after one response in the variable schedule also decreased switching to the variable schedule, particularly at long delay intervals.     

Separating the reinforcing and discriminative properties of brief-stimulus presentations in second-order schedules.

Pigeons' responses were maintained under multiple schedules to study properties of briefly presented stimuli. Responses in one component produced food according to a second-order schedule with fixed-interval components in which food or a brief stimulus occurred with equal probability. In the second component responses produced only the brief stimulus under a fixed-ratio schedule. Under various conditions the brief stimulus in the first component was (a) paired with food, (b) not paired with food, (c) partially omitted, or (d) scheduled simultaneously with the second-order schedule under an independent variable-interval schedule. Paired and nonpaired brief stimuli maintained similar response patterning in the second-order schedule. However, only paired stimuli maintained responses in the second component. The data suggest that nonpaired brief stimuli engender response patterning in second-order schedules as a result of their discriminative properties. When the stimulus is paired with food, these discriminative properties sometime mask a reinforcement effect, and no change in response patterning is observed. When the discriminative properties of the brief stimulus are absent, the reinforcing effects of pairing the brief stimulus with food may be observed.     

"Turning back the clock" on serial-stimulus sign tracking.

Two experiments examined the effects of a negative (setback) response contingency on key pecking engendered by a changing light-intensity stimulus clock (ramp stimulus) signaling fixed-time 30-s food deliveries. The response contingency specified that responses would immediately decrease the light-intensity value, and, because food was delivered only after the highest intensity value was presented, would delay food delivery by 1 s for each response. The first experiment examined the acquisition and maintenance of responding for a group trained with the contingency in effect and for a group trained on a response-independent schedule with the ramp stimulus prior to introduction of the contingency. The first group acquired low rates of key pecking, and, after considerable exposure to the contingency, those rates were reduced to low levels. The rates of responding for the second group were reduced very rapidly (within four to five trials) after introduction of the setback contingency. For both groups, rates of responding increased for all but 1 bird when the contingency was removed. A second experiment compared the separate effects of each part of the response contingency. One group was exposed only to the stimulus setback (stimulus only), and a second group was exposed only to the delay of the reinforcer (delay only). The stimulus-only group's rates of responding were immediately reduced to moderate levels, but for most of the birds, these rates recovered quickly when the contingency was removed. The delay-only groups's rates decreased after several trials, to very low levels, and recovery of responding took several sessions once the contingency was removed. The results suggest that (a) sign-tracking behavior elicited by an added clock stimulus may be reduced rapidly and persistently when a setback contingency is imposed, and (b) the success of the contingency is due both to response-dependent stimulus change and response-dependent alterations in the frequency of food delivery. The operation of the contingency is compared with the effects of secondary reinforcement and punishment procedures.     

Response acquisition with delayed reinforcement

Discrete responses of experimentally naive, food-deprived White Carneaux pigeons (key pecks) or Sprague-Dawley rats (bar or omnidirectional lever presses) initiated unsignaled delay periods that terminated with food delivery. Each subject first was trained to eat from the food source, but no attempt was made to shape or to otherwise train the response. In both species, the response developed and was maintained. Control procedures excluded the simple passage of time, response elicitation or induction by food presentation, type of operandum, food delivery device location, and adventitious immediate reinforcement of responding as the basis for the effects. Results revealed that neither training nor immediate reinforcement is necessary to establish new behavior. The conditions that give rise to both the first and second response are discussed, and the results are related to other studies of the delay of reinforcement and to explanations of behavior based on contingency or correlation and contiguity.     

Concurrent performances: stimulus-control gradients during schedules of signalled and unsignalled concurrent reinforcement

On one key, pigeons' pecks were reinforced according to a variable-interval schedule in the presence of vertical lines, and were not reinforced in the presence of oblique lines. On a second key, pecks were reinforced according to a variable-interval schedule in the presence of blue, according to a signalled variable-interval schedule in the presence of red, and were not reinforced in the presence of white. Subsequently, during extinction, stimulus-control gradients were obtained by presenting eight different line orientations on the first key concurrent with each of the three colors on the second key. On the first key, line-orientation gradients tended to be lower, narrower, and less shifted in peak or area when the second-key stimulus was blue or red, the stimuli respectively correlated with unsignalled and signalled reinforcement, than when it was white, the stimulus correlated with extinction. Thus, the effect on first-key line-orientation gradients depended on second-key stimuli correlated with concurrent reinforcement, whether or not these stimuli were also correlated with concurrent responding. As a function of first-key line orientation, an inverted gradient was obtained on the second key during blue; during both red and white, rates of pecking on the second key were near zero.     

Quantification of rats' behavior during reinforcement periods

What is treated as a single unit of reinforcement often involves what could be called a reinforcement period during which two or more acts of ingestion may occur, and each of these may have associated with it a series of responses, some reflexive, some learned, that lead up to ingestion. Food-tray presentation to a pigeon is an example of such a “reinforcement period.” In order to quantify this behavior, a continuous-reinforcement schedule was used as the reinforcement period and was chained to a fixed-ratio schedule. Both fixed-ratio size and reinforcement-period duration were manipulated. Rats were used as subjects, food as reinforcement, and a lever press as the operant. Major findings included (a) a rapid decline in response rates during the first 15 to 20 seconds of the reinforcement periods, and (b) a strong positive relationship between these response rates and the size of the fixed ratio. Also revealed was a short scallop not normally found in fixed-ratio response patterns, whose length was a function of fixed-ratio size and reinforcement-period duration. It is suggested that rapidly fluctuating excitatory processes can account for many of these findings and that such processes are functionally significant in terms of behavioral compensation.     

Concurrent performances: synthesizing rate constancies by manipulating contingencies for a single response

An earlier experiment scheduled variable-interval reinforcement for pigeons' pecks on one key, and variable-interval reinforcement alternating with extinction, in a multiple schedule, for pecks on a second key. During the second key's extinction component, first-key pecking was relatively slow and continuous, rarely interrupted by second-key pecking; during the variable-interval component, first-key pecking was frequently interrupted by second-key pecking. When changeover delays operated, so that reinforced pecks on one key could not follow closely upon changeovers from the other key, rapid first-key pecking between interruptions compensated sufficiently for the time lost in second-key pecking that the overall rate of first-key pecking remained roughly constant across the alternating multiple-schedule components. The present experiments duplicated, on a single key, the temporal pattern of first-key pecking generated in the earlier experiments: components of continuous key availability were alternated with components of interrupted key availability. Approximately constant overall rates of responding were observed with a single-key equivalent of a changeover delay scheduled after interruptions and with manipulations of the on-off durations of the interruption cycle. Rate constancies in the original concurrent situation presumably depended on analogous contingencies that operated upon the concurrent responses, rather than on any constant “reserve” of responses.     

ALTERNATIVE REINFORCEMENT INCREASES RESISTANCE TO CHANGE: PAVLOVIAN OR OPERANT CONTINGENCIES?

Two multiple-schedule experiments with pigeons examined the effect of adding food reinforcement from an alternative source on the resistance of the reinforced response (target response) to the decremental effects of satiation and extinction. In Experiment 1, key pecks were reinforced by food in two components according to variable-interval schedules and, in some conditions, food was delivered according to variable-time schedules in one of the components. The rate of key pecking in a component was negatively related to the proportion of reinforcers from the alternative (variable-time) source. Resistance to satiation and extinction, in contrast, was positively related to the overall rate of reinforcement in the component. Experiment 2 was conceptually similar except that the alternative reinforcers were contingent on a specific concurrent response. Again, the rate of the target response varied as a function of its relative reinforcement, but its resistance to satiation and extinction varied directly with the overall rate of reinforcement in the component stimulus regardless of its relative reinforcement. Together the results of the two experiments suggest that the relative reinforcement of a response (the operant contingency) determines its rate, whereas the stimulus-reinforcement contingency (a Pavlovian contingency) determines its resistance to change.     

Choice with a fixed requirement for food, and the generality of the matching relation

Pigeons were trained on choice procedures in which responses on each of two keys were reinforced probabilistically, but only after a schedule requirement had been met. Under one arrangement, a fixed-interval choice procedure was used in which responses were not reinforced until the interval was over; then a response on one key would be reinforced, with the effective key changing irregularly from interval to interval. Under a second, fixed-ratio choice procedure, responses on either key counted towards completion of the ratio and then, once the ratio had been completed, a response on the probabilistically selected key would produce food. In one experiment, the schedule requirements were varied for both fixed-interval and fixed-ratio schedules. In the second experiment, relative reinforcement rate was varied. And in a third experiment, the duration of an intertrial interval separating choices was varied. The results for 11 pigeons across all three experiments indicate that there were often large deviations between relative response rates and relative reinforcement rates. Overall performance measures were characterized by a great deal of variability across conditions. More detailed measures of choice across the schedule requirement were also quite variable across conditions. In spite of this variability, performance was consistent across conditions in its efficiency of producing food. The absence of matching of behavior allocation to reinforcement rate indicates an important difference between the present procedures and other choice procedures; that difference raises questions about the specific conditions that lead to matching as an outcome.     

Drug discrimination: stimulus control during repeated testing in extinction

Rats were trained, under a two-lever drug-discrimination procedure, to respond differentially depending upon whether lorazepam (1.0 mg/kg) or no injection had been administered before the session. Responses on the appropriate lever produced a food pellet under a modified fixed-ratio (FR) 10 schedule, in which the 10 responses had to be emitted consecutively. In reinforcement tests, completing an FR 10 on either lever produced a pellet. In extinction tests, stimulus changes paired with reinforcement occurred but no pellet was delivered. Training sessions were conducted between test sessions. Each of four extinction phases consisted of six tests preceded by one stimulus (e.g., lorazepam). Repeated exposures to extinction reduced response rates for all rats, but stimulus control, as inferred from either percentage of total responses or percentage of total FR 10s on the drug-appropriate lever, remained high. The percentage of total FR 10s measure was less subject to skewing under low-rate conditions than was the percentage of total responses measure and provided an evaluation of stimulus control in terms of meeting the consecutive response contingency. These results demonstrate a level of independence between response rate and stimulus control in drug discrimination, which has positive implications for the validity of interpreting discriminative effects of novel test conditions in well-trained animals, even when overall response rates are low.     

Concurrent responding with fixed relative rate of reinforcement

Responding by pigeons on one key of a two-key chamber alternated the color of the second key, on which responding produced food according to a variable-interval schedule of reinforcement. From time to time, reinforcement would be available for a response, but in the presence of a particular stimulus, either red or green light on the key. Red or green was chosen irregularly from reinforcement to reinforcement, so that a proportion of the total number of reinforcements could be specified for each color. Experimental manipulations involved variations of (1) the proportions for each color, (2) changeover delay, or, alternatively, (3) a fixed-ratio changeover requirement. The main findings were: (1) relative overall rates of responding and relative times in the presence of a key color approximated the proportions of reinforcements obtained in the presence of that color, while relative local rates of responding changed little; (2) changeover rate decreased as the proportions diverged from 0.50; (3) relative overall rate of responding and relative time remained constant as the changeover delay was increased from 2 to 32 sec, with reinforcement proportions for red and green of 0.75 and 0.25, but they increased above 0.90 when a fixed-ratio changeover of 20 responses replaced the changeover delay; (4) changeover rate decreased as the delay or fixed-ratio was increased.     

Reversal of preference under progressive-ratio schedules by punishment

A progressive-ratio reinforcement schedule, in which successive reinforcements required an additional 50 responses, was programmed on one key. A response on a second key reset the progressive-ratio schedule to the first step. Before punishment, all pigeons consistently reset the schedule after reinforcement on the first step, thereby minimizing the number of responses required for reinforcement. Punishment was a brief electric shock contingent upon each response on the reset key. The first effect of punishment was to change the frequency of extra responses on the reset key. Under higher intensities of punishment, the pigeons completed the advanced steps of the progressive-ratio schedule before resetting to the first step. Completions of advanced steps were accompanied by decreases in the overall rate of responding and the rate of reinforcement. When the punishment contingency was removed, the major features of pre-punishment performance were recovered.     

Preference for mixed- versus fixed-ratio schedules

Pigeons' pecks on one key produced a stimulus correlated with a mixed-ratio schedule of food reinforcement. Pecks on a second key produced a stimulus correlated with a fixed-ratio schedule. When the arithmetic mean of the mixed ratios equaled the fixed ratio, the former stimulus maintained a higher rate of pecking. When the fixed ratio was sufficiently smaller, preference shifted to it. The pigeons' relative preference for the schedules could be described by comparing the geometric mean of the reinforcement rates in the several mixed-ratio components with the reinforcement rates in the fixed-ratio components.     

Conditioned reinforcement and discrimination in second-order schedules

Pigeons were exposed to multiple second-order schedules in which responding on the “main key” was reinforced according to either a variable-interval or fixed-interval schedule by production of a brief stimulus on the “brief-stimulus key”. A response was required to the brief stimulus during its fourth (final) presentation to produce food; responses to the earlier brief stimuli indicated the extent to which the final brief stimulus was discriminated from preceding ones. Main-key response rates were higher in early components of paired brief-stimulus schedules, in which each brief stimulus was the same as that paired with reinforcement, than in comparable unpaired brief-stimulus or tandem schedules. Poor discrimination occurred between paired brief stimuli (Experiment I). When chain stimuli on the main key induced a discrimination between the first two and second two brief stimuli, the response-rate enhancement in the paired brief-stimulus schedule persisted (Experiment II). Rate enhancement diminished when the initial link of the chain included the first three components (Experiment IV). Eliminating the contingency between responding and brief-stimulus production also diminished rate enhancement (Experiment III). The results show that the discriminative and conditioned reinforcing effects of food-paired brief stimuli may be selectively manipulated and suggest that the reinforcing effects are modulated by other reinforcers in the situation.     

Second-order schedules of token reinforcement with pigeons: implications for unit price

Four pigeons were exposed to second-order schedules of token reinforcement, with stimulus lights serving as token reinforcers. Tokens were earned according to a fixed-ratio (token-production) schedule, with the opportunity to exchange tokens for food (exchange period) occurring after a fixed number had been produced (exchange-production ratio). The token-production and exchange-production ratios were manipulated systematically across conditions. Response rates varied inversely with the token-production ratio at each exchange-production ratio. Response rates also varied inversely with the exchange-production ratio at each token-production ratio, particularly at the higher token-production ratios. At higher token-production and exchange-production ratios, response rates increased in token-production segments closer to exchange periods and food. Some conditions were conducted in a closed economy, in which the pigeons earned all their daily ration of food within the session. Relative to comparable open-economy conditions, response rates in the closed economy were less affected by changes in token-production ratio, resulting in higher levels of food intake and body weight. Some of the results are consistent with the economic concept of unit price, a cost-benefit ratio comprised of responses per unit of food delivery, but most are well accounted for by a consideration of the number of responses required to produce exchange periods, without regard to the amount of reinforcement available during those exchange periods.     

Discriminability of fixed-ratio schedules for pigeons: effects of payoff values

Three pigeons, previously trained to discriminate different numbers of responses (fixed ratios), were tested under different reinforcement contingencies (payoff matrices) at two levels of sensitivity. For one subject, relative reinforcement magnitude was varied—at first, across sessions and then, at midsession by reversing values—without exteroceptive cues. For another, relative reinforcement magnitude and/or probability was varied every 50 trials with cues by correlating different payoff matrices with different key colors. For the third subject, relative reinforcement probability was varied more frequently with cues—in the limit, at random—to demonstrate stimulus control of response bias on a trial-by-trial basis. A signal-detection analysis showed that bias changed with payoffs, for as many as seven different matrices, while sensitivity remained unchanged. The obtained functions (receiver operating characteristics) were similar under different payoff conditions, which suggests that a single mechanism controls bias. However, they differed enough in slope to require a relatively complex account (e.g., the general Gaussian model of detection theory).     

On the discriminability of stimulus duration.

The performance of pigeons trained to detect differences in the duration of stimuli was analysed using a matching model of signal detection. Two white stimuli, S1 and S2, differing in duration, were arranged with equal probability on the center key of a three-key chamber. S1 was systematically varied from 5 seconds to 25 seconds while S2 remained constant at 30 seconds. On completion of the center-key stimulus, a peck on the center key turned on the two red side keys. A left-key response was "correct" when S1 had been in effect on the center key and a right-key response was "correct" on S2 trials. A correct response produced a 3-second magazine light accompanied intermittently by food. Incorrect responses produced 3-second blackouts. Detection performance was measured under two procedures. In the first, the obtained reinforcement ratio was uncontrolled by allowing the number of food reinforcements obtained for correct left- and right-key responses to vary as the stimuli were changed. In the second procedure, the presentation of food reinforcement was controlled by holding the obtained reinforcement ratio constant. Discriminability changed as a function of stimulus differences under both procedures. No such trend was found in response bias.