Resistance to change of operant variation and repetition

A multiple chained schedule was used to compare the relative resistance to change of variable and fixed four-peck response sequences in pigeons. In one terminal link, a response sequence produced food only if it occurred infrequently relative to 15 other response sequences (vary). In the other terminal link, a single response sequence produced food (repeat). Identical variable-interval schedules operated in the initial links. During baseline, lower response rates generally occurred in the vary initial link, and similar response and reinforcement rates occurred in each terminal link. Resistance of responding to prefeeding and three rates of response-independent food delivered during the intercomponent intervals then was compared between components. During each disruption condition, initial- and terminal-link response rates generally were more resistant in the vary component than in the repeat component. During the response-independent food conditions, terminal-link response rates were more resistant than initial-link response rates in each component, but this did not occur during prefeeding. Variation (in vary) and repetition (in repeat) both decreased during the response-independent food conditions in the respective components, but with relatively greater disruption in repeat. These results extend earlier findings demonstrating that operant variation is more resistant to disruption than is operant repetition and suggest that theories of response strength, such as behavioral momentum theory, must consider factors other than reinforcement rate. The implications of the results for understanding operant response classes are discussed.     

Effects of D-amphetamine and ethanol on variable and repetitive key-peck sequences in pigeons

This experiment assessed the effects of d-amphetamine and ethanol on reinforced variable and repetitive key-peck sequences in pigeons. Pigeons responded on two keys under a multiple schedule of Repeat and Vary components. In the Repeat component, completion of a target sequence of right, right, left, left resulted in food. In the Vary component, 4-peck sequences differing from the previous 10 produced food. d-Amphetamine (0.1-3.0 mg/kg, i.m.) was administered in two separate phases, separated by ethanol administration (1.0-2.0 g/kg, i.g.). Under control conditions, measures of variability were high in the Vary component, and lower in the Repeat component. Following administration of the highest dose of d-amphetamine, but not ethanol, response rates decreased in both components. d-Amphetamine and ethanol tended to increase overall sequence variability in the Repeat component, and had less of an effect in the Vary component. Performance in the Repeat component during Phase 2 of d-amphetamine administration was more disrupted than during Phase 1. Measures of variability and repetition based on shifts in the relative frequency distributions of the 16 possible keypeck sequences differed from those based on the overall measure of variability, highlighting the importance of considering both molar and molecular measures when assessing the effects of drugs on reinforced variability and repetition. In addition, the shifts in the relative frequency distribution of response sequences suggest that d-amphetamine produced decrements in repeat performance by decreasing discriminative control within response sequences, whereas ethanol decreased repeat performance by decreasing discriminability between components as well as discriminative control within response sequences.     

Persistence and relapse of reinforced behavioral variability

The present study examined persistence and relapse of reinforced behavioral variability in pigeons. Pigeons emitted four‐response sequences across two keys. Sequences produced food according to a lag schedule, in which a response sequence was followed by food if it differed from a certain number of previous sequences. In Experiment 1, food was delivered for sequences that satisfied a lag schedule in both components of a multiple schedule. When reinforcement was removed for one component (i.e., extinction), levels of behavioral variability decreased for only that component. In Experiment 2, food was delivered for sequences satisfying a lag schedule in one component of a multiple schedule. In the other component, food was delivered at the same rate, but without the lag variability requirement (i.e., yoked). Following extinction, levels of behavioral variability returned to baseline for both components after response‐independent food delivery (i.e., reinstatement). In Experiment 3, one group of pigeons responded on a lag variability schedule, and the other group responded on a lag repetition schedule. For both groups, levels of behavioral variability increased when alternative reinforcement was suspended (i.e., resurgence). In each experiment, we observed some evidence for extinction‐induced response variability and for variability as an operant dimension of behavior.     

Effects of a brief stimulus accompanying reinforcement on instrumental responding in pigeons

The responding of pigeons on a variable interval schedule of reinforcement was investigated in four experiments. In some conditions in each experiment reinforced keypecks were accompanied by a brief (0.5-sec) flash of the houselight. This procedure resulted in a low rate of response in comparison with that found in conditions when response-contingent light flashes occurred uncorrelated with reinforcement (Experiments 1 and 2) or when no light flash was presented (Experiment 3). Experiment 4 allowed a comparison between the effects of a signal accompanying the reinforced response and one accompanying the delivery of “free” food. Signaling the delivery of earned food produced a lower rate of response than did signaling the delivery of free food. The role of stimulus-reinforcer and response-reinforcer associations in producing these effects is discussed.     

Some effects of lick-contingent delays on the development of schedule-induced polydipsia

Experiments 1 and 2 examined the effects of various durations of lickcontingent delays of reinforcement on the development of schedule-induced polydipsia. Food was provided every 1 min in the absence of licking. Delays of 1 min or more effectively disrupted the development of the licking behavior. In Experiments 3 and 4 a lick-contingent stimulus was provided in addition to the delay of food delivery. A light, which was turned on by a lick and turned off by food, greatly enhanced the development of polydipsia under delays that had previously prevented such development. A tone proved less effective than the light in promoting the development of polydipsia. Adventitious reinforcement and conditioned punishment explanations of these results were considered. A distinction was drawn between the mechanism responsible for the development of schedule-induced behavior and that responsible for its maintenance.     

Resistance to change of forgetting functions and response rates

This experiment examined the effects of reinforcement probability on resistance to change of remembering and response rate. Pigeons responded on a two-component multiple schedule in which completion of a variable-interval 20-s schedule produced delayed matching-to-sample trials in both components. Each session included four delays (0.1 s, 2 s, 4 s, and 8 s) between sample termination and presentation of comparison stimuli in both components. The two components differed in the probability of reinforcement arranged for correct matches (i.e., rich, p = .9; lean, p = .1). Response rates during the variable-interval portion of the procedure were higher in the rich component during baseline and more resistant to the disruptive effects of intercomponent food and extinction. Forgetting functions were constructed by examining matching accuracy as a function of delay duration. Baseline accuracy was higher in the rich component than in the lean component as measured by differences in the gamma-intercept of the forgetting functions (i.e., initial discrimination), rather than from differences in the slope of the forgetting function (i.e., rate of forgetting). Intercomponent food increased the rate of forgetting relatively more in the lean component than in the rich component, but initial discrimination was not systematically affected. Extinction reduced initial discrimination relatively more in the lean component than in the rich component, but did not systematically affect rate of forgetting. These results are consistent with our previous data suggesting that, as for response rate, accuracy and resistance to change of discriminating are positively related to rate of reinforcement. These data also suggest that the disruptability of remembering depends on the conditions of reinforcement, but the way in which remembering is disrupted depends on the nature of the disruptor.     

Effects of ethanol on reinforced variations and repetitions by rats under a multiple schedule.

Response sequences emitted by five Long-Evans rats were reinforced under a two-component multiple schedule. In the REPEAT component, food pellets were contingent upon completion of a left-left-right-right (LLRR) sequence on two levers. In the VARY component, pellets were contingent upon variable sequences (i.e., a sequence was reinforced only if it differed from each of the previous five sequences). The rats learned to emit LLRR sequences in the REPEAT component and variable sequences in VARY. Intraperitoneal injections of ethanol (1.25, 1.75, and 2.25 g/kg) significantly increased sequence variability in REPEAT, thereby lowering reinforcement probability, but had little effect on sequence variability in the VARY component. These results extend previous findings that alcohol impairs the performance of reinforced repetitions but not of reinforced variations in response sequences.     

Brief-stimulus presentations on multiform tandem schedules

Three experiments examined the influence of a brief stimulus (a light) on the behavior of food-deprived rats whose lever pressing on tandem schedules comprising components of different schedule types resulted in food presentation. In Experiment 1, either a tandem variable-ratio variable-interval or a tandem variable-interval variable-ratio schedule was used. The variable-interval requirement in the tandem variable-ratio variable-interval schedule was yoked to the time taken to complete the variable-ratio component in the tandem variable-interval variable-ratio schedule, and the length of the variable-interval component in the latter schedule was yoked to the variable-ratio component in the former schedule. If a brief stimulus occurred following completion of the first component, then behavior was differentiated in the two components; subjects responded more quickly in the variable-ratio than in the variable-interval component. If the stimulus was removed, then response rate was determined by the nature of the final component. Similar results were obtained in Experiments 2 and 3 with the use of a three-component tandem variable-ratio variable-interval variable-ratio schedule or tandem variable-interval variable-ratio variable-interval schedule. Thus, a brief stimulus that was not explicitly paired with reinforcement engendered behavior typical of the component schedule preceding its presentation.     

Resistance to change of responding maintained by unsignaled delays to reinforcement: a response-bout analysis

Previous experiments have shown that unsignaled delayed reinforcement decreases response rates and resistance to change. However, the effects of different delays to reinforcement on underlying response structure have not been investigated in conjunction with tests of resistance to change. In the present experiment, pigeons responded on a three-component multiple variable-interval schedule for food presented immediately, following brief (0.5 s), or following long (3 s) unsignaled delays of reinforcement. Baseline response rates were lowest in the component with the longest delay; they were about equal with immediate and briefly delayed reinforcers. Resistance to disruption by presession feeding, response-independent food during the intercomponent interval, and extinction was slightly but consistently lower as delays increased. Because log survivor functions of interresponse times (IRTs) deviated from simple modes of bout initiations and within-bout responding, an IRT-cutoff method was used to examine underlying response structure. These analyses suggested that baseline rates of initiating bouts of responding decreased as scheduled delays increased, and within-bout response rates tended to be lower in the component with immediate reinforcers. The number of responses per bout was not reliably affected by reinforcer delay, but tended to be highest with brief delays when total response rates were higher in that component. Consistent with previous findings, resistance to change of overall response rate was highly correlated with resistance to change of bout-initiation rates but not with within-bout responding. These results suggest that unsignaled delays to reinforcement affect resistance to change through changes in the probability of initiating a response bout rather than through changes in the underlying response structure.     

Matching, contrast, and equalizing in the concurrent lever-press responding of rats

Five rats pressed levers for food reinforces delivered by several concurrent variable-interval variable-interval schedules. The rate of reinforcement available for responding on one component schedule was held constant at 60 reinforcers per hour. The rate of reinforcement available for responding on the other schedule varied from 30 to 240 reinforcers per hour. The behavior of the rats resembled the behavior of pigeons pecking keys for food reinforcers. The ratio of the overall rates of responding emitted under, and the ratio of the time spent responding under, the two components of each concurrent schedule were approximately equal to the ratio of the overall rates of reinforcement obtained from the components. The overall rate of responding emitted under, and the time spent responding under, the variable component schedule varied directly with the overall rate of reinforcement from that schedule. The overall rate of responding emitted under, and the time spent responding under, the constant component schedule varied inversely with the overall rate of reinforcement obtained from the variable component. The local rates of responding emitted under, and the local rates of reinforcement obtained from, the two components did not differ consistently across subjects. But they were not exactly equal either.     

Quasi-reinforcement: control of responding by a percentage-reinforcement schedule

When a variable-interval schedule of reinforcement was segmented into small fixed-interval components, with reinforcements following some components and brief blackouts following the others, rate of responding doubled and a positively accelerated pattern within each component was obtained. Presented according to this percentage reinforcement paradigm, the blackouts approximated the functions of a food reinforcer. These effects occurred only when the behavior sequence required to produce reinforcement was identical to that required to produce blackout. The quasi-reinforcing effects of these blackout stimuli suggest that a neutral stimulus need not occasion or accompany a primary reinforcer to acquire reinforcing properties.     

Unsignaled delay of reinforcement, relative time, and resistance to change

Two experiments with pigeons examined the effects of unsignaled, nonresetting delays of reinforcement on responding maintained by different reinforcement rates. In Experiment 1, 3-s unsignaled delays were introduced into each component of a multiple variable-interval (VI) 15-s VI 90-s VI 540-s schedule. When considered as a proportion of the preceding immediate reinforcement baseline, responding was decreased similarly for the three multiple-schedule components in both the first six and last six sessions of exposure to the delay. In addition, the relation between response rates and reinforcement rates was altered such that both parameters of the single-response version of the matching law (i.e., k and Re) were decreased. Experiment 2 examined the effects of unsignaled delays ranging from 0.5 s to 8.0 s on responding maintained by a multiple VI 20-s VI 120-s schedule of reinforcement. Response rates in both components increased with brief unsignaled delays and decreased with longer delays. As in Experiment 1, response rates as a proportion of baseline were affected similarly for the two components in both the first six and last six sessions of exposure to the delay. Unlike delays imposed between two stimulus events, the effects of delays between responses and reinforcers do not appear to be attenuated when the average time between reinforcers is longer. In addition, the disruptions produced by unsignaled delays appear to be inconsistent with the general finding that responding maintained by higher rates of reinforcement is less resistant to change.     

Rate of conditioned reinforcement affects observing rate but not resistance to change

The effects of rate of conditioned reinforcement on the resistance to change of operant behavior have not been examined. In addition, the effects of rate of conditioned reinforcement on the rate of observing have not been adequately examined. In two experiments, a multiple schedule of observing-response procedures was used to examine the effects of rate of conditioned reinforcement on observing rates and resistance to change. In a rich component, observing responses produced a higher frequency of stimuli correlated with alternating periods of random-interval schedule primary reinforcement or extinction. In a lean component, observing responses produced similar schedule-correlated stimuli but at a lower frequency. The rate of primary reinforcement in both components was the same. In Experiment 1, a 4:1 ratio of stimulus production was arranged by the rich and lean components. In Experiment 2, the ratio of stimulus production rates was increased to 6:1. In both experiments, observing rates were higher in the rich component than in the lean component. Disruptions in observing produced by presession feeding, extinction of observing responses, and response-independent food deliveries during intercomponent intervals usually were similar in the rich and lean components. When differences in resistance to change did occur, observing tended to be more resistant to change in the lean component. If resistance to change is accepted as a more appropriate measure of response strength than absolute response rates, then the present results provide no evidence that higher rates of stimuli generally considered to function as conditioned reinforcers engender greater response strength.     

Preference and resistance to change with constant- and variable-duration terminal links: independence of reinforcement rate and magnitude

Pigeons responded in a three-component multiple concurrent-chains procedure in which the variable-interval reinforcement schedules were the same across components but magnitudes differed across components. The terminal links were arranged either as a variable delay followed by presentation of a reinforcer ("variable duration") or as a fixed period of access to the schedule during which a variable number of reinforcers could be earned ("constant duration"). Relative reinforcement rate was varied parametrically across both types of conditions. After baseline training in each condition, resistance to change of terminal-link responding was assessed by delivering food during the initial links according to a variable-time schedule. Both preference and resistance to change were more sensitive to reinforcement-rate differences in the constant-duration conditions. Sensitivities of preference and resistance to change to relative reinforcement rate did not change depending on relative reinforcement magnitude. Taken together, these results confirm and extend those of prior studies, and suggest that reinforcement rate and magnitude combine additively to determine preference and resistance to change. A single structural relation linking preference and resistance to change describes all the data from this and several related studies.     

Performance of rats under concurrent variable-interval schedules of negative reinforcement

The behavior of rats under concurrent variable-interval schedules of negative reinforcement was examined. A single one-minute variable-interval programmer determined the availability of 30-second timeouts from electric shock. These were assigned to one or the other of the two component schedules with a probability of 0 to 1.0. The response requirement for the component schedules was standing to the right or left of the center of the experimental chamber. With a six-second changeover delay, the relative time spent under one component schedule varied directly and linearly with the relative number of timeouts earned under that component schedule. The absolute number of changeovers was highest when a similar number of timeouts was earned under each component schedule, and lowest when all or nearly all timeouts were earned under one component schedule. In general, these relations are similar to those reported with concurrent variable-interval schedules of positive reinforcement.     

Response additivity: effects of superimposed free reinforcement on a variable-interval baseline

Three experiments examined the effects of superimposing free reinforcement (Free VI 30-sec) on behavior maintained by a response dependent mult VI 2-min VI 2-min schedule of reinforcement. Experiment I used pigeons as subjects, key pecking as the response, and colors of response key as the stimuli associated with the multiple-schedule components. When free reinforcement was added during only one component (Differential condition) a large and highly significant increase in response rate developed in this component. Adding free reinforcement during both components (Nondifferential condition) produced smaller and far less-consistent effects. An entirely different pattern of results was obtained in two subsequent experiments, where similar procedures and reinforcement conditions were used with rats as subjects and bar pressing as the response. In both Experiments II and III, response rates decreased to the stimulus associated with added free reinforcement in the Differential condition. These findings are interpreted as the result of interactions between behavior maintained by response-reinforcer contingencies and behavior maintained by stimulus-reinforcer contingencies. As such, they support the main assumption of an autoshaping theory of behavioral contrast, that additivity of responding generated by the two kinds of contingency can occur only in situations favorable to autoshaping.     

Quasireinforcement: Control of behavior by second-order interval schedules

Pigeons were trained on a variable-interval 66-sec schedule of reinforcement that was segmented into either fixed- or variable-interval 10-sec components. Three-second access to food followed some components according to the overall VI 66-sec schedule, but 3-sec periods of nonreinforcement followed the other components. With both FI 10-sec and VI 10-sec segments, overall response rates were generally higher when the completion of unreinforced segments was signaled by a red key (never paired with food) than when it was unsignaled. Response rates during the red-key periods dropped to zero. Brief presentations of the red key engendered the distinctive (FI or VI) patterns of responding which would be expected if each segment were followed by food. These data demonstrate behavioral control by brief stimuli which are not paired with primary reinforcement and show that such control may develop even when the sequences of behavior required to produce food or brief stimuli are variable in duration.     

The effects of a signal for free or for earned reward: Implications for the role of response-reinforcer associations in instrumental performance

Pearce and Hall (1978) investigated the effects of making a brief flash of light contingent upon response in rats lever-pressing for food on a variable-interval (VI) schedule. When this signal occurred in conjunction only with reinforced responses the response rate was lowered with respect to a condition in which an equal number of light flashes occurred uncorrelated with reinforcement. The experiments reported here compared these effects with those produced by signalling “free” food deliveries in a similar way. Experiments I and II compared the effects of presenting correlated and uncorrelated schedules of light and food to rats given no opportunity to lever-press. The different schedules did not produce differences in response rate when the levers were made available. In Experiment III, free food was delivered to rats responding on a VI schedule. Signalling the delivery of earned food pellets produced a low response rate in comparison with a condition in which the free pellets were signalled. It is concluded that signalling food delivery is effective only when the rat must respond to earn the food and it is argued that the signal has its effect by overshadowing a response-reinforcer association.     

Contrast, component duration, and the following schedule of reinforcement

Experiment 1 investigated component duration in a four-component multiple schedule designed to separate the effects of the preceding schedule of reinforcement from those of the following schedule of reinforcement. The preceding schedule in the preceding component had no consistent effect regardless of component duration. The following schedule was a powerful determinant of behavior, however, with higher response rates resulting from a following period of extinction. Moreover, the effect of the following schedule was greater with short component durations, which strongly suggests that component-duration effects previously found with multiple schedules are due generally to variation in the degree of control by the following schedule. Experiments 2--4 investigated the basis of the effects of the following schedule. In some situations differential responding in the following component was the controlling variable, but in others it was differential reinforcement in the following component. The results are consistent with the view that response rate during a stimulus is inversely related to the "value" of the following component, where the calculation of value must include both primary and conditioned reinforcement.