Within-subject testing of the signaled-reinforcement effect on operant responding as measured by response rate and resistance to change

Response rates under random-interval schedules are lower when a brief (500 ms) signal accompanies reinforcement than when there is no signal. The present study examined this signaled-reinforcement effect and its relation to resistance to change. In Experiment 1, rats responded on a multiple random-interval 60-s random-interval 60-s schedule, with signaled reinforcement in only one component. Response resistance to alternative reinforcement, prefeeding, and extinction was compared between these components. Lower response rates, and greater resistance to change, occurred in the component with the reinforcement signal. In Experiment 2, response rates and resistance to change were compared after training on a multiple random-interval 60-s random-interval 60-s schedule in which reinforcer delivery was unsignaled in one component and a response-produced uncorrelated stimulus was presented in the other component. Higher response rates and greater resistance to change occurred with the uncorrelated stimulus. These results highlight the significance of considering the effects of an uncorrelated signal when used as a control condition, and challenge accounts of resistance to change that depend solely on reinforcer rate.     

Assessing potential reinforcement-like effects of brief stimuli unrelated to food reinforcers

It is widely assumed that reinforcers are biologically relevant stimuli, or stimuli that have been associated with biologically relevant stimuli. However, brief, arbitrary stimuli have also been reported to have reinforcement-like effects, despite being unrelated to biologically relevant stimuli like food. The present study explored the potential reinforcement-like effects of brief stimuli across 5 experiments. In Experiments 1 through 4, pigeon subjects responded for food reinforcement and brief stimulus presentations in a 2-component multiple schedule. Neither baseline response rates nor resistance to change during disruption tests were systematically greater in a component with versus without brief stimulus presentations. Increasing the rate and duration of brief stimulus presentations in Experiment 4 did not reveal reinforcement-like effects when compared directly with food. In Experiment 5, pigeons chose between independent terminal links in a concurrent-chains procedure. Across conditions, varying the location, duration, and rate of brief stimulus presentations in the terminal links had no systematic effects on preference. In contrast, varying rates of food reinforcers resulted in large and reliable shifts in preference. Therefore, the present study found no systematic evidence that brief stimuli unrelated to food reliably increase response rates, resistance to change, or preference. These data demonstrate the value of systematic replication, and a behavioral momentum approach to assessing potential reinforcement-like effects.     

Conditioned reinforcement value and resistance to change

Three experiments examined the effects of conditioned reinforcement value and primary reinforcement rate on resistance to change using a multiple schedule of observing-response procedures with pigeons. In the absence of observing responses in both components, unsignaled periods of variable-interval (VI) schedule food reinforcement alternated with extinction. Observing responses in both components intermittently produced 15 s of a stimulus associated with the VI schedule (i.e., S+). In the first experiment, a lower-valued conditioned reinforcer and a higher rate of primary reinforcement were arranged in one component by adding response-independent food deliveries uncorrelated with S+. In the second experiment, one component arranged a lower valued conditioned reinforcer but a higher rate of primary reinforcement by increasing the probability of VI schedule periods relative to extinction periods. In the third experiment, the two observing-response components provided similar rates of primary reinforcement but arranged different valued conditioned reinforcers. Across the three experiments, observing-response rates were typically higher in the component associated with the higher valued conditioned reinforcer. Resistance to change was not affected by conditioned reinforcement value, but was an orderly function of the rate of primary reinforcement obtained in the two components. One interpretation of these results is that S+ value does not affect response strength and that S+ deliveries increase response rates through a mechanism other than reinforcement. Alternatively, because resistance to change depends on the discriminative stimulus-reinforcer relation, the failure of S+ value to impact resistance to change could have resulted from a lack of transfer of S+ value to the broader discriminative context.     

The delay-reduction hypothesis of conditioned reinforcement and punishment: Observing behavior

Pigeons responded in an observing-response procedure in which three fixed-interval components alternated. Pecking one response key produced food reinforcement according to a mixed schedule. Pecking the second (observing) key occasionally replaced the mixed-schedule stimulus with the stimulus correlated with the fixed-interval component then in effect. In Experiment 1, observing was best maintained by stimuli correlated with a reduction in mean time to reinforcement. That finding was consistent with the conditioned-reinforcement hypothesis of observing behavior. However, low rates of observing were also maintained by stimuli not representing delay reduction. Experiment 2 assessed the role of sensory reinforcement. It showed that response rate was higher when maintained by stimuli uncorrelated with reinforcement delay than when the stimuli were correlated with a delay increase. This latter result supports a symmetrical version of the conditioned-reinforcement hypothesis that requires suppression by stimuli correlated with an increase in time to reinforcement. The results were inconsistent with hypotheses stressing the reinforcing potency of uncertainty reduction.     

Baseline response rates affect resistance to change

The effect of response rates on resistance to change, measured as resistance to extinction, was examined in two experiments. In Experiment 1, responding in transition from a variable‐ratio schedule and its yoked‐interval counterpart to extinction was compared with pigeons. Following training on a multiple variable‐ratio yoked‐interval schedule of reinforcement, in which response rates were higher in the former component, reinforcement was removed from both components during a single extended extinction session. Resistance to extinction in the yoked‐interval component was always either greater or equal to that in the variable‐ratio component. In Experiment 2, resistance to extinction was compared for two groups of rats that exhibited either high or low response rates when maintained on identical variable‐interval schedules. Resistance to extinction was greater for the lower‐response‐rate group. These results suggest that baseline response rate can contribute to resistance to change. Such effects, however, can only be revealed when baseline response rate and reinforcement rate are disentangled (Experiments 1 and 2) from the more usual circumstance where the two covary. Furthermore, they are more cleanly revealed when the programmed contingencies controlling high and low response rates are identical, as in Experiment 2.     

Marking effects in instrumental performance on DRH schedules

Three experiments investigated the effect of presenting a brief stimulus after a response sequence on the rate of lever-pressing by rats on differential reinforcement of high rate (DRH) schedules. In Experiment 1 enhanced responding was produced by a visual stimulus presented during a 500-msec delay of reinforcement compared to a condition in which no stimulus was presented. In Experiment 2 rats responded on a multiple DRH DRH schedule in which the DRH contingency was reinforced on a 50% schedule in each component. Equivalent levels of responding occurred in the components when reinforcement was signalled in one component and when the signal was presented following the non-reinforced schedules in the other components. A further group of rats received the stimulus presented after non-reinforced schedules in one component but not at all in the other component; responding was enhanced in the former component relative to the latter component. In Experiment 3 brief stimuli presented after the completion of DRH components on a second-order VR (DRH) schedule elevated response rates irrespective of whether the signal was presented paired or unpaired with reinforcement. The present data support the view that a brief signal may serve to mark a response sequence in memory and facilitate instrumental performance.     

On the form of the relation between response rates in a multiple schedule

Three pigeons received training on multiple variable-interval schedules with brief alternating components, concurrently with a fixed-interval schedule of food reinforcement on a second key. Fixed-interval performance exhibited typical increases in rate within the interval, and was independent of multiple-schedule responding. Responding on the multiple-schedule key decreased as a function of proximity to reinforcement on the fixed-interval key. The overall relative rate of responding in one component of the multiple schedule roughly matched the overall relative rate of reinforcement. Within the fixed interval, response rate during one multiple-schedule component was a monotonic, negatively accelerated function of response rate during the other component. To a first approximation, the data were described by a power function, where the exponent depended on the relative rate of reinforcement obtained in the two components. The relative rate of responding in one component of the multiple schedule increased as a function of proximity to fixed-interval reinforcement, and often exceeded the overall obtained relative rate of reinforcement. The form of the function relating response rates is discussed in relation to findings on rate-dependent effects of drugs, chaining, and the relation between response rate and reinforcement rate in single-schedule conditions.     

Effects of a brief stimulus accompanying reinforcement on instrumental responding in pigeons

The responding of pigeons on a variable interval schedule of reinforcement was investigated in four experiments. In some conditions in each experiment reinforced keypecks were accompanied by a brief (0.5-sec) flash of the houselight. This procedure resulted in a low rate of response in comparison with that found in conditions when response-contingent light flashes occurred uncorrelated with reinforcement (Experiments 1 and 2) or when no light flash was presented (Experiment 3). Experiment 4 allowed a comparison between the effects of a signal accompanying the reinforced response and one accompanying the delivery of “free” food. Signaling the delivery of earned food produced a lower rate of response than did signaling the delivery of free food. The role of stimulus-reinforcer and response-reinforcer associations in producing these effects is discussed.     

Rate of conditioned reinforcement affects observing rate but not resistance to change

The effects of rate of conditioned reinforcement on the resistance to change of operant behavior have not been examined. In addition, the effects of rate of conditioned reinforcement on the rate of observing have not been adequately examined. In two experiments, a multiple schedule of observing-response procedures was used to examine the effects of rate of conditioned reinforcement on observing rates and resistance to change. In a rich component, observing responses produced a higher frequency of stimuli correlated with alternating periods of random-interval schedule primary reinforcement or extinction. In a lean component, observing responses produced similar schedule-correlated stimuli but at a lower frequency. The rate of primary reinforcement in both components was the same. In Experiment 1, a 4:1 ratio of stimulus production was arranged by the rich and lean components. In Experiment 2, the ratio of stimulus production rates was increased to 6:1. In both experiments, observing rates were higher in the rich component than in the lean component. Disruptions in observing produced by presession feeding, extinction of observing responses, and response-independent food deliveries during intercomponent intervals usually were similar in the rich and lean components. When differences in resistance to change did occur, observing tended to be more resistant to change in the lean component. If resistance to change is accepted as a more appropriate measure of response strength than absolute response rates, then the present results provide no evidence that higher rates of stimuli generally considered to function as conditioned reinforcers engender greater response strength.     

Discriminative properties of briefly presented stimuli

In Experiment I, pigeons' responses produced food according to a fixed-interval schedule while responses on the key also produced brief stimuli according to a variable-interval schedule. Each brief stimulus reset the fixed interval. Thus, a brief stimulus occurred irregularly but a fixed minimum time separated the occurrence of food from a brief stimulus. Pauses followed brief stimuli and were followed by an accelerated response rate until another brief stimulus or food occurred. In Experiment II, four control procedures were examined. (1) Brief-stimulus presentations were omitted, producing a loss of response patterning. (2) A second-order schedule was studied with fixed-interval components. This schedule produced patterning following brief stimuli similar in kind and degree to that found in Experiment I. (3) A conjoint schedule was arranged in which food was no longer separated from the stimulus by a fixed time; pauses following the stimulus no longer resulted. (4) A brief food reinforcer replaced the brief visual stimulus, resulting in a constant response rate with no pausing following the brief food stimulus. The results suggest that the brief-stimulus effects were due to discriminative functions produced by the fixed temporal relation separating the stimulus from food.     

The schedule dependency of the signaled reinforcement effect

In Experiment 1, rats leverpressed for food reinforcement on either a variable ratio (VR) 30 schedule or a variable interval (VI) 15-s schedule. One group in each condition received a signal filling a 500-ms delay of reinforcement. This treatment enhanced rates on the VR schedule, and attenuated rates on the VI schedule, relative to the rate seen in an unsignaled control condition. In Experiment 2 there was no delay of reinforcement and the signal and food were presented simultaneously. Attenuated rates of responding were observed on VI schedules with a range of mean interval values (15 to 300 s). Experiment 3 used a range of VR schedules (10 to 150) with simultaneous presentations of signal and food. A signal-induced enhancement of response rate was found at all VR values. In Experiment 4, a signal elevated response rates on a tandem VI VR schedule, but depressed rates on a tandem VR VI schedule, compared to control conditions receiving unsignaled delayed reinforcement. These results are taken to show that the effect of a signal accompanying reinforcement depends upon the nature of the behavior that is reinforced during exposure to a given schedule.     

Variable-ratio versus variable-interval schedules: response rate, resistance to change, and preference

Two experiments asked whether resistance to change depended on variable-ratio as opposed to variable-interval contingencies of reinforcement and the different response rates they establish. In Experiment 1, pigeons were trained on multiple random-ratio random-interval schedules with equated reinforcer rates. Baseline response rates were disrupted by intercomponent food, extinction, and prefeeding. Resistance to change relative to baseline was greater in the interval component, and the difference was correlated with the extent to which baseline response rates were higher in the ratio component. In Experiment 2, pigeons were trained on multiple variable-ratio variable-interval schedules in one half of each session and on concurrent chains in the other half in which the terminal links corresponded to the multiple-schedule components. The schedules were varied over six conditions, including two with equated reinforcer rates. In concurrent chains, preference strongly overmatched the ratio of obtained reinforcer rates. In multiple schedules, relative resistance to response-independent food during intercomponent intervals, extinction, and intercomponent food plus extinction depended on the ratio of obtained reinforcer rates but was less sensitive than was preference. When reinforcer rates were similar, both preference and relative resistance were greater for the variable-interval schedule, and the differences were correlated with the extent to which baseline response rates were higher on the variable-ratio schedule, confirming the results of Experiment 1. These results demonstrate that resistance to change and preference depend in part on response rate as well as obtained reinforcer rate, and challenge the independence of resistance to change and preference with respect to response rate proposed by behavioral momentum theory.     

Conditioned reinforcement and discrimination performance

Pigeons were trained in a three-key chamber to peck one side key in the presence of a vertical line on the center key and to peck the other side key in the presence of a horizontal line. Correct choice responses were reinforced with food according to fixed- and variable-ratio, fixed-interval, and differential-reinforcement-of-long-latency schedules of reinforcement. For each schedule, the birds performed under each of two conditions: (1) each correct choice response produced a brief presentation of stimuli intermittently paired with food, then the next trial; (2) each correct choice response produced an intertrial interval only. For all schedules except one long latency schedule, response rates were higher under the condition of brief stimulus presentation than under the comparable control condition. Presentation of brief magazine stimuli increased choice accuracy. The amount of change in accuracy was correlated with the rate of food presentation. Performance under the schedules with highest food reinforcement rates showed no enhancement; performance under the schedules with the lowest reinforcement rates showed the greatest enhancement.     

Changeover behavior and preference in concurrent schedules

Pigeons were trained on a multiple schedule of reinforcement in which separate concurrent schedules occurred in each of two components. Key pecking was reinforced with milo. During one component, a variable-interval 40-s schedule was concurrent with a variable-interval 20-s schedule; during the other component, a variable-interval 40-s schedule was concurrent with a variable-interval 80-s schedule. During probe tests, the stimuli correlated with the two variable-interval 40-s schedules were presented simultaneously to assess preference, measured by the relative response rates to the two stimuli. In Experiment 1, the concurrently available variable-interval 20-s schedule operated normally; that is, reinforcer availability was not signaled. Following this baseline training, relative response rate during the probes favored the variable-interval 40-s alternative that had been paired with the lower valued schedule (i.e., with the variable-interval 80-s schedule). In Experiment 2, a signal for reinforcer availability was added to the high-value alternative (i.e., to the variable-interval 20-s schedule), thus reducing the rate of key pecking maintained by that schedule but leaving the reinforcement rate unchanged. Following that baseline training, relative response rates during probes favored the variable-interval 40-s alternative that had been paired with the higher valued schedule. The reversal in the pattern of preference implies that the pattern of changeover behavior established during training, and not reinforcement rate, determined the preference patterns obtained on the probe tests.     

Conditioned reinforcement and discrimination in second-order schedules

Pigeons were exposed to multiple second-order schedules in which responding on the “main key” was reinforced according to either a variable-interval or fixed-interval schedule by production of a brief stimulus on the “brief-stimulus key”. A response was required to the brief stimulus during its fourth (final) presentation to produce food; responses to the earlier brief stimuli indicated the extent to which the final brief stimulus was discriminated from preceding ones. Main-key response rates were higher in early components of paired brief-stimulus schedules, in which each brief stimulus was the same as that paired with reinforcement, than in comparable unpaired brief-stimulus or tandem schedules. Poor discrimination occurred between paired brief stimuli (Experiment I). When chain stimuli on the main key induced a discrimination between the first two and second two brief stimuli, the response-rate enhancement in the paired brief-stimulus schedule persisted (Experiment II). Rate enhancement diminished when the initial link of the chain included the first three components (Experiment IV). Eliminating the contingency between responding and brief-stimulus production also diminished rate enhancement (Experiment III). The results show that the discriminative and conditioned reinforcing effects of food-paired brief stimuli may be selectively manipulated and suggest that the reinforcing effects are modulated by other reinforcers in the situation.     

Resistance to change and relapse of observing

Four experiments examined relapse of extinguished observing behavior of pigeons using a two-component multiple schedule of observing-response procedures. In both components, unsignaled periods of variable-interval (VI) food reinforcement alternated with extinction and observing responses produced stimuli associated with the availability of the VI schedule (i.e., S+). The components differed in the rate of food arranged (Rich = VI 30 s; Lean = VI 120 s). In Experiment 1, following baseline training, extinction of observing involved removal of both food and S+ deliveries, and reinstatement was examined by presenting either response-independent food or S+ deliveries. In Experiment 2, extinction involved removal of only food deliveries while observing responses continued to produce S+. Reinstatement was examined by delivering food contingent upon the first two food-key responses occurring in the presence of the S+. Experiment 3 assessed ABA renewal of observing by extinguishing food-key and observing responses in the presence of one contextual stimulus (i.e., B) and then returning to the original training context (i.e., A) during continued extinction. Experiment 4 examined resurgence by introducing food reinforcement for an alternative response during extinction, and subsequently removing that alternative source of food. Across experiments, relative resistance to extinction and relapse of observing tended to be greater in the component previously associated with the higher rate of primary reinforcement. Relapse of observing or attending to stimuli associated with primary reinforcement appears to be impacted by frequency of primary reinforcement in a manner similar to responding maintained directly by primary reinforcement.     

Visual Reinforcement Signals Interfere with the Effects of Reinforcer Magnitude Manipulations

Three experiments examined the effect of reinforcement magnitude on free-operant response rates. In Experiment 1, rats that received four food pellets responded faster than rats that received one pellet on a variable ratio 30 schedule. However, when the food hopper was illuminated during reinforcer delivery, there was no difference between the rates of response produced by the two magnitudes of reward. In Experiment 2, there was no difference in response rates emitted by rats receiving either one or four pellets of food as reward on a random interval (RI) 60-s schedule. In Experiment 3, rats responding on an RI 30-s schedule did so at a lower rate with four pellets as reinforcement than with one pellet. This effect was abolished by the illumination of the food hopper during reinforcement delivery. These results indicate that the influence of magnitude is obscured by manipulations which signal the delivery of reinforcement.     

The effect of signaled reinforcement on rats' fixed-interval responding

Four experiments examined the effect on rats' response rate of presenting a brief (500 ms) stimulus simultaneously with the delivery of food on fixed-interval (FI) schedules. In Experiment 1, reinforcement signals that were spatially diffuse (both tones and lights) elevated rates of responding, but responding was attenuated by localized visual stimuli. The remaining experiments examined the signal-induced potentiation of responding. In Experiment 2, a tone reinforcement signal potentiated response rates on an FI schedule, but attenuated response rates on a variable-interval (VI) schedule. This difference was obtained even though the overall rate of responding was equated on the two schedules before the introduction of the signal. Signal-induced potentiation of responding occurred over a range of FI values employed in Experiment 3. In Experiment 4, presenting a reinforcement signal when high local rates of response had occurred immediately before reinforcement resulted in potentiated rates of responding on an FI schedule. The opposite effect on response rate occurred when the reinforcement signal followed only low local rates of response. These results indicate that a variety of factors influence the effects of a reinforcement signal. They imply, however, that the local rate of response at the time of reinforcement is a key factor in establishing the nature of the signaling effect.     

Concurrent second-order schedules of reinforcement

Responses on one key (the main key) of a two-key chamber produced food according to a second-order variable-interval schedule with fixed-interval schedule components. A response on a second key (the changeover key) alternated colors on the main key and provided a second independent second-order variable-interval schedule with fixed-interval components. The fixed-interval component on one variable-interval schedule was held constant at 8 sec, while the fixed interval on the other variable-interval schedule was varied from 0 to 32 sec. Under some conditions, a brief stimulus terminated each fixed interval and generated fixed-interval patterns; in other conditions, the brief stimulus was omitted. Relative response rate and relative time deviated substantially from scheduled relative reinforcement rate and, to a lesser extent, from obtained relative reinforcement rate under both brief-stimulus and no-stimulus conditions. Matching was observed with equal components on both schedules; with unequal components, increasingly greater proportions of time and responses than the matching relation would predict were spent on the variable-interval schedule containing the shorter component. Preference for the shorter fixed interval was typically more extreme under brief-stimulus than under no-stimulus schedules. The results limit the extension of the matching relation typically observed under simple concurrent variable-interval schedules to concurrent second-order variable-interval schedules.     

Responding of pigeons under variable-interval schedules of unsignaled, briefly signaled, and completely signaled delays to reinforcement

In Experiment 1, three pigeons' key pecking was maintained under a variable-interval 60-s schedule of food reinforcement. A 1-s unsignaled nonresetting delay to reinforcement was then added. Rates decreased and stabilized at values below those observed under immediate-reinforcement conditions. A brief stimulus change (key lit red for 0.5 s) was then arranged to follow immediately the peck that began the delay. Response rates quickly returned to baseline levels. Subsequently, rates near baseline levels were maintained with briefly signaled delays of 3 and 9 s. When a 27-s briefly signaled delay was instituted, response rates decreased to low levels. In Experiment 2, four pigeons' responding was first maintained under a multiple variable-interval 60-s (green key) variable-interval 60-s (red key) schedule. Response rates in both components fell to low levels when a 3-s unsignaled delay was added. In the first component delays were then briefly signaled in the same manner as Experiment 1, and in the second component they were signaled with a change in key color that remained until food was delivered. Response rates increased to near baseline levels in both components, and remained near baseline when the delays in both components were lengthened to 9 s. When delays were lengthened to 27 s, response rates fell to low levels in the briefly signaled delay component for three of four pigeons while remaining at or near baseline in the completely signaled delay component. In Experiment 3, low response rates under a 9-s unsignaled delay to reinforcement (tandem variable-interval 60 s fixed-time 9 s) increased when the delay was briefly signaled. The role of the brief stimulus as conditioned reinforcement may be a function of its temporal relation to food, and thus may be related to the eliciting function of the stimulus.